Method and device for determining the condition of a road surface of a lane to be used by a vehicle

Abstract

A method for determining the condition of a road surface of a lane to be traveled by a vehicle (10), in which lane areas located in front of the vehicle (10) are measured using a 3D camera (1) to generate 3D image data (K3D) and the 3D image data (K3D) are fused with sensor data from at least one chassis sensor (4), by: a) creating a height profile (HProfile) of the road surface from the 3D image data (K3D), b) generating an expected vehicle excitation pattern (SPpred) for the created height profile (HProfile) of the road surface using vehicle and tire modeling, c) detecting the actual vehicle excitation pattern (SPreal) of the vehicle (10) generated when driving over the previously measured lane area using the at least one chassis sensor (4) and comparing it with the expected vehicle excitation pattern (SPpred), and d) if the actual and expected vehicle excitation patterns do not match, Vehicle excitation pattern (SPpred,SPreal) detects a friction-reducing condition of the road surface.

Description

The invention relates to a method and a device for determining the condition of a road surface of a lane to be traveled by a vehicle.The detection of the condition of the road surface (e.g. dry, wet, snow covered or covered), from which a coefficient of friction group can be derived, is an important prerequisite for assisting the driver in his driving task.The accident-avoiding effect of driver assistance systems, such as, for example. Emergency braking and driving dynamics systems are essentially dependent on the condition of the road surface and thus on the coefficient of friction. Wet, snow, snowmat, ice or dirt constitute an intermediate medium which reduces the coefficient of friction and thus considerably reduces the coefficient of friction available between the tire and the roadway compared to a roadway without intermediate medium.In driver assistance systems, different sensors, including video cameras, provide environmental information which, if appropriate, provides information about the current condition of the road surface by fusion with other sensor data and driving dynamic information of the vehicle.DE 10 2004 018 088 A1 discloses a roadway recognition system having a temperature sensor, an ultrasonic sensor and a camera, in which the temperature, roughness and image data (roadway data) obtained from the sensors are filtered and compared with reference data. On the basis of the comparison of the filtered roadway data with the reference data, the state of the roadway surface is determined. The roadway surface (e.g., concrete, asphalt, dirt, grass, sand, or gravel) and its condition (e.g., dry, icy, snowy, wet) may be classified in this manner.WO 2012 / 110030 A2 describes a method and a device for estimating the coefficient of friction by means of a 3D camera, for example a stereo camera, in which at least one image of the surroundings of the vehicle is recorded and a height profile of the road surface lying in front of the vehicle is created from the image data. From the height profile, the expected local coefficient of friction of the road surface in the vehicle front area is estimated. From specific height profiles determined, in the individual case, a classification of the road surface can be carried out, for example as a snow cover or a sludgey field path.In connection with the analysis of tire vibrations of a vehicle wheel of a vehicle, a method for estimating the coefficient of friction is described, on the one hand, from DE 10 2008 047 750 A1, in which the tire vibrations of a vehicle wheel of a vehicle are analyzed and an estimated value of a coefficient of friction is calculated on the basis of the analysis. On the other hand, indirectly measuring tire pressure monitoring systems (DDS, Deflection Detection System) are known which detect the tire pressure of individual wheels of a vehicle on the basis of wheel speed signals via wheel circumference changes and / or which additionally carry out a vibration analysis of the wheels via the wheel speed signals and infer the tire air pressure from the vibration behavior.In US 2002 / 0087251A1, a possibility for estimating a friction coefficient of a road has been described, in which movement parameters of the vehicle are detected and a road friction value is estimated on the basis of the movement parameters, wherein the estimated friction value is approximated to a predetermined road friction value of an asphalted road surface when the vehicle is travelling at high speed.US4712635A relates to prediction of a friction coefficient for an all-wheel vehicle, and correction of the prediction from a calculated friction coefficient if control of all-wheel drive based on the prediction is incorrect. In this way, a control for switching between two-wheel drive and all-wheel drive and for blocking a center differential mechanism can be effected in such a way that it adapts itself appropriately to the respective roadway condition.It is known from US6526804B2 to determine a first road friction coefficient based on a reference yaw rate with high friction and one with low friction, which are each calculated by means of a motion model, and a calculated current yaw rate. The final road friction coefficient is then determined from weighting a previously determined road friction coefficient and the first road friction coefficient by means of a suitable weighting function variable with respect to a detected road roughness. Finally, US6597980B2 discloses an apparatus for estimating a road coefficient of friction of a road surface, which apparatus has three road coefficient of friction estimation devices for calculating three road coefficients of friction on the basis of vehicle behavior in the lateral direction, in the longitudinal direction and on the basis of road conditions. One of the road friction values is then selected as the final value.The known methods for detecting the roadway condition place high demands on the required sensor system and often do not have the required robustness with regard to the reliable determination of the condition of a roadway surface lying in front of the vehicle.Proceeding from this prior art, it is an object of the invention to specify a method for determining the nature of a road surface of a lane to be traveled by a vehicle, which method requires only a few sensors and has a high degree of robustness. It is a further object of the invention to specify an apparatus for carrying out the method according to the invention.The first-mentioned object is achieved by a method having the features of claim 1.In such a method for determining the condition of a road surface of a lane to be traveled by a vehicle, lane areas lying in front of the vehicle are measured by means of a 3D camera to generate 3D image data, and the 3D image data is fused with sensor data of at least one chassis sensor by a) creating a height profile of the road surface from the 3D image data, b) creating an expected vehicle excitation pattern for the created height profile of the road surface by means of vehicle and tire modelling, c) detecting the actual vehicle excitation pattern of the vehicle generated when traveling over the previously measured lane area by means of the at least one chassis sensor and comparing it with the expected vehicle excitation pattern, and d) detecting a friction-value-reducing condition of the road surface if the actual condition does not match the expected vehicle excitation pattern.This method according to the invention uses only a small number of sensors, which are generally already available in a modern vehicle. The invention essentially consists in that an expected vehicle excitation pattern is generated on the basis of a height profile of the lane to be traveled by the vehicle, which height profile is detected by a predictive 3D sensor system and should be triggered by this detected height profile. This expected vehicle excitation pattern is then compared with the actually generated vehicle excitation pattern, which is phase-shifted with respect to the detection time of the detected height profile and is generated when driving over the previously measured lane region. If these two vehicle excitation patterns do not match, the lane region with the detected height profile is classified as friction-reducing, specifically preferably in the form of an intermediate medium, that is to say the roadway surface of this lane region is covered, for example, with water, snow, snow slush, trut / slush, leaves and / or sand as intermediate medium. In the other case, if a match of these two vehicle excitation patterns is determined, a road surface is detected without the listed intermediate media of this lane region which reduce the coefficient of friction.This achieves a high degree of robustness in the differentiation of a driving lane without intermediate media that reduce the coefficient of friction from a driving lane with a characteristic of the road surface that reduces the coefficient of friction.According to an advantageous embodiment of the invention, the comparison of the two vehicle excitation patterns can be carried out in a simple manner, since essentially the amplitudes of the expected vehicle excitation pattern and of the actual vehicle excitation pattern are compared with one another. The phase is shifted according to the distance of the forward-looking 3D camera from the vehicle axes and the travel speed of the vehicle.According to a further advantageous embodiment of the invention, it is particularly advantageous if, when a friction-reducing characteristic of the road surface is detected by means of the 3D image data, a classification of its friction-reducing characteristic is carried out. Thus, the condition of the roadway surface can be further classified, in particular with respect to a wet, snow-covered, icy or soiled lane.This is preferably done by carrying out the classification of the road surface by means of a classifier to which at least one image section of the road surface recorded by the 3D camera is supplied and which assigns at least one friction-reducing characteristic class to this image section, such as "wet", "snow covered", "snow-like", "covered" or "soiled", to the road surface.In the other case, if a match is established between the expected vehicle excitation pattern and the actual vehicle excitation pattern, a road surface having a road roughness corresponding to the created height profile is assumed according to the development and the road surface is classified with the quality class "without intermediate medium".It is particularly advantageous to use as chassis sensors according to a development an inertial sensor system and / or at least one suspension travel or height sensor on the suspension struts of the chassis of the vehicle. Since such chassis sensors are already present in modern vehicles, no additional sensors are required for carrying out the method according to the invention.According to a particularly preferred embodiment of the invention, at least one wheel speed sensor is used as the chassis sensor and the actual vehicle excitation pattern of the vehicle generated when driving over the previously measured lane region is determined with the tire vibration pattern detected from the wheel speed sensor signal. In modern vehicles, wheel speed sensors of this type are likewise already present, the sensor signals of which are made available to different vehicle systems.According to a further advantageous embodiment, the tire pressure of the vehicle wheels of the vehicle is used to determine the expected vehicle excitation pattern according to method step b). By using such tire pressure, prediction of the expected vehicle excitation pattern can be made with improved accuracy. This tire pressure is provided by an indirectly or directly measuring tire pressure monitoring system.The second object mentioned is achieved by a device having the features of claim 9.Such a device for determining the condition of a road surface of a lane to be traveled by a vehicle is characterized in thata 3D camera is provided for generating 3D image data of a roadway region lying in front of the vehicle,an image processing unit for creating a height profile of the road surface is provided,a data processing unit is provided which is designed to generate an expected vehicle excitation pattern from the created height profile of the road surface by means of a vehicle and tire modelling algorithm,at least one chassis sensor is provided, which is designed to detect the actual vehicle excitation pattern of the vehicle generated when driving over the previously measured lane region, anda comparison unit is provided which is designed to compare the vehicle excitation pattern of the vehicle detected when driving over the previously measured lane region with the expected vehicle excitation pattern and to output a classification signal indicating the friction-reducing nature of the road surface in the event of a mismatch.This apparatus according to the invention also has the advantages of the method according to the invention carried out with this apparatus.According to an advantageous embodiment of the invention, a classifier is provided which is designed to determine at least one friction-reducing characteristic class of the road surface from the classification signal indicating the friction-reducing characteristic of the road surface and the 3D image data.Furthermore, according to a further preferred refinement of the invention, an inertial sensor system and / or at least one suspension travel or height sensor is / are provided as chassis sensors on the suspension struts of the chassis.Finally, according to a last advantageous embodiment of the invention, at least one wheel speed sensor is provided as the chassis sensor and the data processing unit is designed to determine the vehicle excitation pattern of the vehicle generated when driving over the previously measured lane region using the tire vibration pattern detected from the wheel speed sensor signal.The information about the condition of the road surface of the lane areas being measured and traveled over can be made available to the driver of the vehicle via a display unit or can also be used by control systems of the vehicle, in particular driver assistance systems.The invention will be explained in more detail below with reference to the single enclosed figure. FIG. 1 shows a block diagram of a device of a vehicle for carrying out the method according to the invention.According to this FIG. 1, a vehicle 10 comprises a device for determining the condition of a road surface of a lane to be traveled by a vehicle 10 (not shown in FIG. 1 ).This device comprises a 3D camera 1, which is designed as a stereo camera and which is followed by a plurality of components for evaluating the image data and will be explained below.With this stereo camera 1, all 3D points of the road surface lying in front of the vehicle 10 of a lane to be traveled by this vehicle 10 are detected and a height profile H profil of the road surface is created from this image data K 3D by means of an image processing unit 2. This height profile H profil is calculated by means of this image processing unit 2 above all in the driving path of the vehicle 10 and in particular in front of the right and left vehicle wheels of the vehicle 10.This height profile information H profil obtained in this way is fed to a data processing unit 3 and an expected vehicle excitation pattern SP präd is calculated by means of a vehicle and tire modelling algorithm stored there, including the tire pressure p r of the vehicle wheels. It is therefore predictively calculated how the acquired height profile H profil will excite the vehicle 10 to oscillations or which excitation spectrum will occur on the vehicle 10. The tire pressure p r is provided by a tire pressure monitoring system of the vehicle 10. This predictive vehicle excitation pattern SP präd is supplied to a downstream comparison unit 6.When the vehicle 10 travels over the measured lane region for which the predictive vehicle excitation pattern SP präd has been calculated, the vehicle excitation pattern SP real actually generated by the measured lane region is determined by an evaluation unit 5 by evaluating the sensor signals of at least one chassis sensor 4. The chassis sensors 4 used are initial sensors 4.1, chassis height sensors 4.2, i.e. suspension travel or height sensors on the suspension struts of the chassis of the vehicle 10 and wheel speed sensors 4.3. When wheel speed sensors 4.3 are used, the tire vibrations are evaluated in order to determine the actual vehicle excitation pattern SP real. This actual vehicle excitation pattern SP real is also supplied to the comparison unit 6.In this comparison unit 6, the predictive vehicle excitation pattern SP präd is now compared with the actual vehicle excitation pattern SP real and a condition signal S k1 or S k2 indicating the condition of the road surface is generated as a function of the comparison result, which is fed to a classifier 7 for further processing. The comparison of these two vehicle excitation patterns SP präd and SP real is performed mainly in terms of their amplitudes. The phase is corrected according to the prediction width of SP präd and the running speed.If the two vehicle excitation patterns SP präd and SP real are the same within the scope of a measurement and prediction accuracy and can be brought into coincidence, the detected height profile H profil is actual unevenness of the road surface, or unevenness of the road surface, which originates from the road surface. Therefore, the road surface is classified with the condition class "without intermediate medium", and the classification signal S k1 is output from the comparison unit 6.If, on the other hand, a non-coincidence of the two vehicle excitation patterns SP präd and SP real is determined by the comparison unit 6, i.e. these two vehicle excitation patterns SP präd and SP real cannot be made coincident, it is assumed that a friction-reducing characteristic of the road surface is present in the form of an intermediate medium, i.e. the surface of the measured lane region is covered with snow, dirt / slush, leaf or sand etc. as intermediate medium. It is assumed that the predictive vehicle excitation pattern SP präd has higher amplitudes than the actually measured vehicle excitation pattern SP real. The comparison unit 6 generates a classification signal S k2, which indicates the friction-reducing characteristic of the road surface, i.e. the presence of an intermediate medium between tire and road surface, and is likewise supplied to the classifier 7.The classification signal S k2 indicates that the coefficient of friction available between a tire of the vehicle wheel of the vehicle 10 and the road surface of the lane region covered or measured is significantly reduced by an intermediate medium.The classifier 7 now serves to plausibilize the comparison result in the form of the two classification signals S k1 and S k2 on the one hand by means of further image data of the 3D camera 1 and / or, by a suitable combination with other methods of digital image processing on the basis of image data of the 3D camera 1, to display a further classification of the friction-reducing nature of the road surface displayed with the classification signal S k2 and / or by means of a suitable combination with other methods of digital image processing on the basis of image data of the 3D camera 1. For this purpose, the image data of the 3D camera 1 according to FIG. 1 are also supplied to this classifier 7.A classification signal S k1 checked for plausibility in this way is fed both to a driver assistance system 8 and to a display unit 9 for informing a driver of the vehicle 10. On the basis of the classification signal S k2 a further classification of the friction-reducing nature of the road surface with respect to moisture, snow, snowmat, ice or dirt is carried out by means of the image data of the 3D camera 1 and here at least one monoimage. The classification signal S k21 indicates the condition class "wet", the classification signal S k22 the condition class "snow" or "snow-like", the classification signal S k23 the condition class "ice" and the classification signal S k24 the condition class "dirt". These classification signals S k21, S k22, S k23 and S k24 are also supplied to both the driver assistance system 8 and the display unit 9.Suitable methods for plausibility checking and further classifying the detected characteristic classes by means of the image data of the stereo camera 1 are methods of digital image processing, such as classification or the interpretation of gray value images, in order to be able to reliably infer the type of road surface and thus also the coefficient of friction.Thus, the classifier can be trained, for example, on the basis of example data in order to detect the correct condition class on the basis of assignments of example image regions to known roadway states or to be able to assign unknown image contents or image regions to at least one condition class.Furthermore, it is also possible to extract one feature or a plurality of features from the relevant image region and use them for classification. This feature or features are such that they are suitable for detecting the different appearance of the road surface in the camera image as a function of the road condition. A plurality of individual features can form a feature vector which combines different information from the relevant image region in order to be able to decide more robust and accurate road condition during classification. Different feature types for an image area yield a set of feature vectors. The resulting set of feature vectors for an image region is referred to as a feature descriptor. The feature descriptor is then assigned by a classification system to at least one of the condition classes. As a feature extracted from the relevant image area and used for classification, the average grayscale value or the average color value (RGB) of the relevant image area may be used.The result from the comparison of the two vehicle excitation patterns SP präd and SP real can also be used to be able to detect non-uniformities of the tires of the vehicle wheels of the vehicle 10 or chassis wear (e.g. deployed bearings) in a robust manner.Reference numerals denote reference numerals1 3D camera, stereo camera 2 image processing unit 3 data processing unit 4 chassis sensor 4.1 inertial sensor 4.2 chassis height sensor 4.3 wheel speed sensor 5 evaluation unit 6 comparison unit 7 classifier 8 driver assistance system 9 display unit 10 vehicle

Claims

Method for determining the condition of a road surface of a lane to be traveled by a vehicle (10), in which, in order to generate 3D image data (K 3D) by means of a 3D camera (1), lane regions lying in front of the vehicle (10) are measured and the 3D image data (K 3D) are fused with sensor data of at least one chassis sensor (4), in that a) a height profile (H Profil) of the road surface is created from the 3D image data (K 3D) b) an expected vehicle excitation pattern (SP präd) is generated by means of vehicle and tire modelling for the created height profile (H Profil) of the road surface, c) the actual vehicle excitation pattern (SP real) of the vehicle (10) generated when driving over the previously measured driving lane region is detected by means of the at least one chassis sensor (4) and compared with the expected vehicle excitation pattern (SP präd) and d) a friction-reducing condition of the road surface is detected if the actual vehicle excitation pattern does not match the expected vehicle excitation pattern (SP präd, SP real).Method according to Claim 1, characterized in that the comparison of the detected vehicle excitation pattern (SP real) with the expected vehicle excitation pattern (SP präd) is carried out substantially with regard to the amplitudes thereof.Method according to Claim 1 or 2, characterized in that, when a friction-reducing characteristic of the road surface is detected by means of the 3D image data (K 3D) a classification of its friction-reducing characteristic is carried out.Method according to Claim 3, characterized in that the classification of the road surface is carried out by means of a classifier (7), to which at least one image section of the road surface recorded by the 3D camera (1) is fed and which assigns at least one coefficient of friction-reducing characteristic class to the road surface to this image section.Method according to one of the preceding claims, characterized in that, if the actual vehicle excitation pattern matches the expected vehicle excitation pattern (SP präd, SP real) a road surface is detected with a road roughness corresponding to the created height profile (H Profil) and the road surface is classified with the quality class "without intermediate medium".Method according to one of the preceding claims, characterized in that the chassis sensors (4) used are an inertial sensor system (4.1) and / or at least one suspension travel or height sensor (4.2) on the suspension struts of the chassis of the vehicle (10).Method according to one of the preceding claims, characterized in that at least one wheel speed sensor (4.3) is used as the chassis sensor (4) and the actual vehicle excitation pattern (SP real) of the vehicle (10) generated when driving over the previously measured lane region is determined with the tire vibration pattern detected from the wheel speed sensor signal.Method according to one of the preceding claims, characterized in that the tyre pressure (p r) of the vehicle wheels of the vehicle (10) is used to determine the expected vehicle excitation pattern (SP präd) according to method step b).Device for determining the condition of a road surface of a lane to be traveled by a vehicle (10), in particular for carrying out the method according to one of the preceding claims, characterized in that - a 3D camera (1) is provided for generating 3D image data (K 3D) of a road region lying in front of the vehicle, - an image processing unit (2) is provided for creating a height profile (H profil) of the road surface, - a data processing unit (3) is provided which is designed to generate an expected vehicle excitation pattern (SP präd) from the created height profile (H profil) of the road surface by means of a vehicle and tire modelling algorithm, - at least one chassis sensor (4) is provided which is designed to generate a vehicle excitation pattern, detecting the actual vehicle excitation pattern (SP real) of the vehicle generated when driving over the previously measured lane region, a comparison unit (6) is provided which is designed to compare the actual vehicle excitation pattern (SP real) of the vehicle (10) detected when driving over the previously measured lane region with the expected vehicle excitation pattern (SP präd) and to output a classification signal (S k2) indicating the friction-reducing nature of the road surface when there is no agreement.Device according to Claim 9, characterized in that a classifier (7) is provided which is designed to determine at least one coefficient-of-friction-reducing characteristic class of the road surface from the classification signal (S k2) indicating the coefficient-of-friction-reducing characteristic of the road surface and the 3D image data (K 3D).Device according to Claim 9 or 10, characterized in that the chassis sensors (4) provided are / are an inertial sensor system (4.1) and / or at least one suspension travel or height sensor (4.2) on the suspension struts of the chassis of the vehicle (10).Device according to one of Claims 9 to 11, characterized in that at least one wheel rotational speed sensor (4.3) is provided as the chassis sensor (4), and the data processing unit (3) is designed to determine the vehicle excitation pattern (SP real) of the vehicle (10) which is generated when driving over the previously measured lane region using the tire oscillation pattern detected from the wheel rotational speed sensor signal (4.3).

Images