Method for operating an assistance system of a motor vehicle and assistance system

Abstract

The invention relates to a method for operating an assistance system (2) of a motor vehicle (1), wherein shoulder driving (8) of the motor vehicle (1) on a road shoulder (9) is detected by means of at least one detection device (4, 5, 6) of the assistance system (2), and wherein, as a function of the detected shoulder driving (8), a control signal (10) for intervening in a functional unit (11) of the motor vehicle (1) is generated by an electronic computing device (3) of the assistance system (2), wherein the detected road shoulder (9) is evaluated with respect to the shoulder type, and as a function of the specific shoulder type, a determination is made as to whether the shoulder driving (8) is dangerous or not dangerous for the motor vehicle (1), and the control signal (10) is generated only in the case of dangerous shoulder driving (8). The invention also relates to an assistance system (2).

Description

Technical Field

[0001] The invention relates to a method for operating an assistance system of a motor vehicle, wherein shoulder driving of the motor vehicle on a road shoulder is detected by means of at least one detection device of the assistance system, and wherein, as a function of the detected shoulder driving, a control signal for intervening in a functional unit of the motor vehicle is generated by an electronic computing device of the assistance system. The invention also relates to an assistance system. Background Art

[0002] Assistance systems are already known from the prior art, which have a curb function and prevent skidding when a motor vehicle is driven onto an unpaved curb beside a road, wherein an unpaved curb can also be referred to as a curb. The functional mode is that if driving on a curb is detected and the driver attempts to steer away from the curb with a rapid steering movement, the motor vehicle is prevented from skidding by selectively slowing down the wheels that are driving on the curb. The physical effect of a skidding accident when driving on a curb is based on the transition of the wheels from a surface with a low friction value (e.g. a curb with grass or sand) to a surface with a high friction value (e.g. a road with asphalt or concrete). In the case of a sudden change in the friction value, combined with a strong steering reaction of the driver, the high lateral forces of the wheels that were previously driving on the curb are suddenly transferred to the asphalt. This leads to a yaw moment, which leads to oversteering of the motor vehicle. If the driver of the motor vehicle does not control the oversteering vehicle, a skidding accident occurs.

[0003] US2020 / 0066146 A1 discloses an information system including a controller, which is configured to detect a screen image of a vehicle traveling on a road, wherein the screen image is captured by an image recording unit. A dangerous driving state of the traveling vehicle is identified in accordance with the temporal change process of the screen image. Information related to the vehicle that identifies the vehicle in a dangerous driving state is displayed. Summary of the invention

[0004] The object of the present invention is to provide a method and an assistance system with which a user of a motor vehicle can be better supported in assistance functions.

[0005] This technical problem is solved by the method and the auxiliary system of the present invention.

[0006] One aspect of the invention relates to a method for operating an assistance system of a motor vehicle, wherein shoulder driving of the motor vehicle on a road shoulder is detected by means of at least one detection device of the assistance system, and wherein, as a function of the detected shoulder driving, a control signal for intervening in a functional unit of the motor vehicle is generated by an electronic computing device of the assistance system.

[0007] Provision is made here to evaluate the detected road shoulder with regard to the shoulder type and to determine, based on the specific shoulder type, whether driving on the shoulder is dangerous or not for the motor vehicle, and to generate a control signal only in the case of driving on a dangerous shoulder.

[0008] In particular, the user of the motor vehicle (who may in particular be the driver of the motor vehicle) can therefore be reliably supported in the assistance function. In particular, the so-called shoulder function of the motor vehicle is thus triggered only when, for example, the right wheel or the left wheel is traveling on a shoulder whose surface has a low friction value (e.g. grass, sand or gravel) and the respective other wheel is traveling on a surface with a high friction value (e.g. asphalt or concrete). On the contrary, if the left and right wheels are in the shoulder, on a shoulder whose surface has a high friction value (e.g. cobblestones, blue basalt, paving bricks, lawn plastic grids or vibrating strips) and the respective other wheel is traveling on a surface with a high friction value (e.g. asphalt or concrete), the shoulder function should not be triggered. In addition, unnecessary erroneous triggering of the shoulder function, for example due to unevenness of the road surface or specific structures on the road surface, should be prevented. The shoulder function is to be understood in particular as preventing the motor vehicle from skidding by selective wheel deceleration of the wheels traveling on the shoulder (which corresponds in particular to an intervention in the functional unit) if driving on the shoulder is detected and the driver attempts to steer away from the shoulder with a rapid steering movement.

[0009] In particular, therefore, triggering of the shoulder function can be prevented if, for example, there is no sharp change in friction value between the shoulder and the road. Unnecessary triggering in this case is thus prevented. The sensor for detecting shoulder driving will therefore distinguish between the above-mentioned scenarios or the above-mentioned different shoulder types.

[0010] Therefore, it is particularly proposed to classify whether there is shoulder driving. If there is shoulder driving, it is evaluated what kind of shoulder driving it is. If a shoulder with a low friction value (e.g. composed of grass, sand or gravel) is determined, it is particularly dangerous shoulder driving. On the other hand, if a shoulder with a high friction value (e.g. composed of pebbles, blue basalt, paving bricks, lawn plastic grids or vibration strips) is detected, it is not dangerous shoulder driving.

[0011] According to an advantageous design, the road shoulder is evaluated in terms of the shoulder type by means of machine learning of an electronic computing device. Therefore, in particular, the classification of the shoulder type is performed via pattern recognition, by evaluating the corresponding sensor measurement data of the detection device by means of sensor-specific evaluation. The evaluation software is trained in particular for the classification of the above-mentioned shoulder types by means of learning data. The learning data is the time-varying process of the corresponding sensor measurement data, which is recorded during the measurement driving when passing the above-mentioned different shoulder types. For example, machine learning also involves neural networks. Here, for example, information on whether there is shoulder driving and the shoulder type can be transmitted via a data bus to the corresponding evaluation module of the electronic computing device of the corresponding sensor, that is, the data merging algorithm (or data fusion algorithm) provided in particular in the electronic computing device. Based on the input data of the detection device, the data merging algorithm then determines whether there is shoulder driving and whether there is a dangerous shoulder type determined by a low friction value. If both are true, the data merging algorithm sends the information to the triggering algorithm. If a strong steering reaction of the driver to leave the road shoulder is detected at the same time, the triggering algorithm activates the road shoulder function again, in particular, for example, by means of an electronic stability program (ESP) to carry out wheel-selective braking of the wheels travelling on the road shoulder.

[0012] In a further advantageous embodiment, the road shoulder is detected by means of a camera as the detection device. The camera is in particular a camera arranged at the front of the motor vehicle. In particular, a sequence of images can be recorded by means of the camera and the road shoulder can be detected accordingly, for example by means of an image processing algorithm. The camera is in particular a detection device that is already built into the motor vehicle, so that it can be used reliably without having to incur additional costs in manufacturing the motor vehicle.

[0013] It is also advantageous if an assessment with respect to the type of shoulder is carried out based on different image patterns of the types of shoulder. In particular, when training the image data evaluation software, it can be used that different types of shoulder produce specific image patterns in the image sequence of the camera data, which can be assigned to the respective type of shoulder and thus distinguished from one another. After the training, the image data evaluation software can assign the measurement data acquired during driving with the forward-pointing camera to the different types of shoulder or classify them therein.

[0014] In another advantageous embodiment, the road shoulder is detected by means of at least two wheel speed sensors on different wheels of the motor vehicle as detection devices. In particular, the road shoulder can be detected by means of four wheel speed sensors on different wheels of the motor vehicle. The motor vehicle has in particular four wheels, wherein a corresponding wheel speed sensor is located on each wheel of the motor vehicle. Based on the corresponding evaluation of the wheel speed sensors or the detected wheel speeds of the individual wheels, it can be reliably determined whether the motor vehicle is located on the road shoulder.

[0015] It has also been found to be advantageous to implement an assessment in terms of the type of road shoulder based on the time course of at least two wheel speed sensors and by comparing the time course of the change. In particular, the sensor measurement data of the wheel speed sensor can detect the time course of the wheel speed of the wheels of the motor vehicle. When driving on a rough surface, as found, for example, when driving on a road shoulder, the time course of the wheel speed sensor measurement data fluctuates greatly, although the vehicle speed remains unchanged during the road shoulder driving. When training the wheel speed data evaluation software, it is used that different road shoulder types produce specific patterns or specific fluctuations in the time course of the wheel speed sensor measurement data, which can be assigned to the corresponding road shoulder type and thus distinguish them from each other. A plurality of road shoulder surfaces with high friction values, such as vibration strips made of concrete slabs, lane markings, grass paving bricks, etc., also have a regular structure, which is reflected in the sensor measurement data of the wheel speed sensor. After training, the wheel speed data evaluation software can assign the measurement data detected by the wheel speed sensor during driving to different road shoulder types, or classify them into different road shoulder types.

[0016] According to another advantageous design, the road shoulder is detected by means of at least two height sensors as detection devices on different wheels of the motor vehicle. The motor vehicle can have four height sensors on the corresponding wheels of the motor vehicle in particular. The sensor measurement data of the height sensor is the time-varying process of the height of the four wheels of the motor vehicle. The time-varying process of the sensor measurement data fluctuates on the rough ground. When training the height data evaluation software, it is used that different shoulder types produce specific patterns or specific fluctuations of the time-varying process of the height sensor measurement data, and the specific patterns or specific fluctuations can be assigned to the corresponding shoulder types and thus distinguish them from each other. A plurality of shoulder surfaces with high friction values, such as vibration strips made of concrete slabs, lane markings, grass bricks, etc., also have regular structures, which are reflected in the sensor measurement data of the height sensor. After training, the height data evaluation software can assign the measurement data detected by the height sensor during driving to different shoulder types, or classify them into different shoulder types.

[0017] It is also advantageous to detect the road shoulder by means of at least two height sensors and / or two wheel speed sensors as detection devices on the opposing wheels of the motor vehicle. "Opposite" is particularly understood to mean that the motor vehicle should have four wheels, for example, so that two wheels are on the left and two wheels are on the right. At least two height sensors can be arranged on one of the left wheels and one of the right wheels, and the wheel speed sensor can be arranged on one of the left wheels and one of the right wheels. In order to classify whether there is a shoulder, the measurement data of the wheel speed sensors of the left and right wheels and / or the measurement data of the height sensor are also compared with each other. If the measurement data of the wheel speed sensors of the left and right wheels and / or the measurement data of the height sensor indicate rough ground, then the motor vehicle is likely not to be driven on the shoulder, but to be driven on cobblestones, dirt roads or other bad sections. If the measurement data of the wheel speed sensors of the left or right wheels and / or the measurement data of the height sensor indicate rough ground, then the vehicle may either use the left wheel or the right wheel to drive in the shoulder. As mentioned above, whether this is dangerous must be determined with the help of corresponding evaluation software.

[0018] In another advantageous embodiment, the road surface of the road on which the motor vehicle is located is detected and taken into account when evaluating the shoulder driving. Therefore, dangerous special cases, such as roads made of pebbles with high friction values ​​and rough surfaces, and shoulders made of grass or sand with low friction values ​​and less rough surfaces compared to the road, can also be classified simultaneously with the described method. In addition, specific structures on the road surface, which are generated, for example, by reflections, shadows and tracks, can also lead to incorrect recognition of shoulder driving by the camera. Since learning data can also be generated for these specific structures on the road surface, the image data evaluation software of the camera can be trained with the help of the learning data so that these specific structures are classified as non-hazardous. In this way, incorrect recognition of shoulder driving can be prevented in specific structures on the road surface. In addition, unevenness of the road surface, such as manhole covers, road surface damage, such as potholes, asphalt patches, asphalt seams, strong bumps in the asphalt or spots embedded in the asphalt can also lead to incorrect recognition of shoulder driving by the wheel speed sensor. Since these irregularities also generate measurement data of a specific pattern, the wheel speed evaluation software and / or the height evaluation software can be trained with the aid of the measurement data to classify them as not dangerous. Incorrect recognition of shoulder driving in the case of road irregularities can thus be prevented.

[0019] The proposed method is in particular a computer-implemented method. For this purpose, another aspect of the invention relates to a computer program product having program code means which, when executed on an electronic computing device, cause the electronic computing device to carry out the method according to the invention. The computer program product can be stored on a computer-readable storage medium, wherein the computer-readable storage medium can in turn be configured on the electronic computing device.

[0020] Another aspect of the invention relates to an assistance system for a motor vehicle, which has at least one electronic computing device and a detection device, wherein the assistance system is designed to carry out the method according to the above-mentioned aspect. In particular, the method is carried out by the assistance system.

[0021] Yet another aspect of the invention relates to a motor vehicle having an assistance system according to the preceding aspect.

[0022] The invention also includes an extension of the assistance system according to the invention and of the motor vehicle according to the invention, which has the features already described in conjunction with the extension of the method according to the invention. For this reason, the corresponding extension of the assistance system according to the invention and of the motor vehicle according to the invention will not be described here.

[0023] The invention also includes combinations of features of the described embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The embodiments of the present invention are described below. Figure 1 A schematic top view shows an embodiment of a motor vehicle having an embodiment of an assistance system. DETAILED DESCRIPTION

[0025] The embodiments described below are preferred embodiments of the present invention. In the embodiments, the components described represent features of the present invention that can be considered independently of one another, they also expand the present invention independently of one another, and therefore can also be considered as components of the present invention individually or in a combination different from that shown. In addition, the described embodiments can also be supplemented by other features of the present invention that have already been described.

[0026] In the figures, functionally identical elements are in each case provided with the same reference symbols.

[0027] Figure 1A schematic top view of an embodiment of a motor vehicle 1 with an embodiment of an assistance system 2 is shown. The assistance system 2 has an electronic computer 3. In addition, the assistance system 2 has at least one detection device 4, 5, 6. For example, the assistance system 2 can have a camera 4 as a detection device 4, 5, 6. Furthermore, it can be provided that a corresponding wheel speed sensor 5 can be provided as a detection device 4, 5, 6 and / or a corresponding height sensor 6 can be provided as a detection device 4, 5, 6 on a corresponding wheel 7 of the motor vehicle 1.

[0028] In particular, shoulder driving 8 of the motor vehicle 1 on a road shoulder 9 can be detected by the assistance system 2. In particular, shoulder driving 8 on the road shoulder 9 is detected by at least one detection device 4, 5, 6, and, as a function of the detected shoulder driving 8, a control signal 10 for intervening in a functional unit 11 of the motor vehicle 1 is generated by the electronic computing device 3.

[0029] Provision is made here to evaluate detected road shoulder 9 with regard to its shoulder type and to determine, based on the determined shoulder type, whether a shoulder 8 is hazardous or not hazardous for motor vehicle 1 and to generate control signal 10 only during driving on hazardous shoulders 8 .

[0030] In particular, provision can be made for the road shoulder 9 to be evaluated with regard to the shoulder type by means of machine learning of the electronic computing device 3 .

[0031] By means of the current assistance system 2, it is therefore particularly classified whether there is a shoulder driving 8, and if so, which type of shoulder is present. For example, a shoulder with a low friction value formed by grass, sand, gravel, etc. can be determined, which shoulder can be particularly referred to as dangerous. In addition, a shoulder with a high friction value formed by pebbles, blue basalt, grass bricks, lawn plastic grids or vibration strips can be detected, which shoulder is particularly not dangerous. The classification of shoulder types by pattern recognition is achieved here in particular by evaluating the corresponding sensor measurement data with the aid of sensor-specific evaluation software. Here, the detection devices 4, 5, 6 can be regarded as sensors in particular. The evaluation software is trained with the aid of learning data so that the shoulder types are classified as described above. The learning data is the time-dependent change process of the corresponding sensor measurement data, which is recorded during the measurement driving and when passing the above-mentioned different shoulder types. The training of the evaluation software is carried out with the aid of known methods, in particular with the aid of machine learning.

[0032] The information about whether there is a shoulder 8 and the type of shoulder is input data for the corresponding sensor or the corresponding evaluation software of the corresponding detection device 4, 5, 6, which is transmitted via a data bus to a data merging algorithm, which is particularly constructed in the electronic computing device 3. The data merging algorithm determines from the input data of the detection device 4, 5, 6 whether there is a shoulder 8 and whether there is a dangerous shoulder type with low friction value. If both are true, the data merging algorithm sends this information to the triggering algorithm. If a strong steering reaction of the driver for leaving the road shoulder 9 is detected at the same time, the triggering algorithm activates the shoulder function, which is particularly shown by the control signal 10. In particular, wheel-selective braking of the wheels 7 traveling on the road shoulder 8 can be implemented by an electronic stability system, wherein the electronic stability system (ESP) corresponds in particular to the functional unit 11.

[0033] In this case, the shoulder type can be implemented in particular on the basis of different detection devices 4, 5, 6. For example, with the aid of the camera 4 pointing forward, it is possible to detect a crossing of a lane marking 12 or a crossing of a road edge, which corresponds in particular to a transition from asphalt to a road shoulder 9. For example, these data are already used in so-called lane departure warning assistance systems or lane keeping assistance systems. In this case, in particular, image sequences are recorded. When training the image data evaluation software, it is utilized that different shoulder types produce specific image patterns in the image sequences of the camera data, which can be assigned to the respective shoulder types and thus distinguish them from one another. After the training, the image processing software can, as already described, assign the measurement data detected during driving with the camera 4 pointing forward to different shoulder types or classify them therein.

[0034] In addition, the roughness of the ground can be derived with the aid of the wheel speed sensors 5. In addition, these data can be used to support anti-lock braking systems or dynamic stability systems, or to avoid rattles and hums in the transmission. The wheel speed sensors 5 record sensor measurement data, which correspond to the time course of the wheel speed of the four wheels 7 of the motor vehicle 1. When driving on a rough surface, as occurs, for example, when driving on a shoulder 8, the time course of the wheel speed sensor measurement data fluctuates greatly, although the vehicle speed remains virtually constant during the shoulder driving 8. When training the wheel speed data evaluation software, it is used that different shoulder types produce specific patterns or specific fluctuations in the time course of the wheel speed sensor measurement data, which specific patterns or specific fluctuations can be assigned to the corresponding shoulder types and thus distinguish them from each other. In addition, a plurality of shoulder surfaces with high friction values, such as vibration strips made of concrete slabs, lane markings 12, paving bricks, etc., have a regular structure, which is reflected in the sensor measurement data of the wheel speed sensors. After the training, the wheel speed data evaluation software can, as described above, assign the measurement data detected during driving with the wheel speed sensor 5 to different road shoulder types or classify them therein.

[0035] In addition, it is possible to detect the wheel movement in the vertical direction with the aid of a height sensor 6, for example, from the vertical adjustment system of the motor vehicle 1, and to deduce the roughness of the ground with the aid of it. The sensor measurement data of the height sensor 6 is the time-dependent variation of the height of the four wheels 7 of the motor vehicle 1. The time-dependent variation of the sensor measurement data fluctuates on the rough ground. When training the height data evaluation software, it is utilized that different shoulder types produce specific patterns or specific fluctuations of the time-dependent variation of the height sensor measurement data, which can be assigned to the corresponding shoulder types and thus distinguish them from each other. A plurality of shoulder surfaces with high friction values, such as vibration strips made of concrete slabs, lane markings 12, grass bricks, etc., also have a regular structure, which is reflected in the sensor measurement data of the height sensor. After training, the height data evaluation software can assign the measurement data detected by the height sensor 6 during driving to different shoulder types, or classify them into them, as described above.

[0036] In order to classify whether there is a road shoulder 8, the measurement data of the wheel speed sensors 5 and / or the measurement data of the height sensors 6 of the left and right wheels 7 are compared with each other. If the measurement data of the wheel speed sensors 5 and / or the measurement data of the height sensors 6 of the left and right wheels 7 indicate rough ground, then the motor vehicle 1 is most likely not driving on the road shoulder 9, but on cobblestones, dirt roads or other bad roads. If the measurement data of the wheel speed sensors 5 and / or the measurement data of the height sensors 6 of the left and right wheels 7 7 indicate rough ground, then the motor vehicle 1 is probably driving on the road shoulder 9 either with the left wheel 7 or with the right wheel 7. As mentioned above, whether this is dangerous must be determined with the help of corresponding evaluation software.

[0037] Special situations that may be dangerous, such as a road 13 made of cobblestones and a road shoulder 9 made of grass or sand, can be addressed with the aid of the method described above by simultaneously classifying the surfaces of the road 13 and the road shoulder 9 .

[0038] Specific structures on the road surface, which are produced, for example, by reflections, shadows and tracks, can also lead to an erroneous recognition of a road shoulder 8 by the camera 4. Since learning data can also be generated for these specific structures on the road surface, the image data evaluation software of the camera 4 can be trained with the aid of the learning data so that these specific structures are classified as not dangerous. In this way, an erroneous recognition of a road shoulder 8 can be prevented in the case of specific structures on the road surface.

[0039] Unevenness of the road surface, such as manhole covers, road damage, asphalt seams, strong bumps in the asphalt or spots embedded in the asphalt, can also lead to erroneous detection of shoulder driving 8 by the wheel speed sensor 5. Since these unevennesses also generate measurement data with a specific pattern, the wheel speed evaluation software and / or the height evaluation software can be trained with the aid of the measurement data so that they are classified as not dangerous. In this way, erroneous detection of shoulder driving 8 in the case of unevenness of the road surface can be prevented.

[0040] Generally, the present invention describes a method for classifying road shoulder types.

[0041] Reference numerals list

[0042] 1Motor Vehicle

[0043] 2Assistance systems

[0044] 3 Electronic computing devices

[0045] 4 Camera

[0046] 5. Wheel speed sensor

[0047] 6Height sensor

[0048] 7 wheels

[0049] 8. Driving on the shoulder

[0050] 9 road shoulder

[0051] 10 Control signal

[0052] 11 Functional Units

[0053] 12 lane markings

[0054] 13Road

Claims

1. A method for operating an assistance system (2) of a motor vehicle (1), wherein: By means of at least one detection device (4, 5, 6) of the assistance system (2), shoulder driving (8) of the motor vehicle (1) on a road shoulder (9) is detected, and, depending on the detected shoulder driving (8), a control signal (10) for intervening in a functional unit (11) of the motor vehicle (1) is generated by an electronic computing device (3) of the assistance system (2), characterized in that the detected road shoulder (9) is evaluated with respect to the shoulder type, and depending on the specific shoulder type, a determination is made as to whether the shoulder driving (8) is dangerous or not dangerous for the motor vehicle (1), and the control signal (10) is generated only in the case of dangerous shoulder driving (8), wherein triggering of the shoulder function is prevented if there is no jump in the friction value between the shoulder and the road, wherein the road shoulder (9) is detected by means of at least two height sensors (6) as detection devices (4, 5, 6) on different wheels (7) of the motor vehicle (1).

2. The method according to claim 1, characterized in that A road shoulder (9) is evaluated with respect to shoulder type using machine learning of the electronic computing device (3).

3. The method according to claim 1 or 2, characterized in that: The road shoulder (9) is detected by means of a camera (4) as a detection device (4, 5, 6).

4. The method according to claim 3, characterized in that An assessment with regard to the road shoulder type is carried out based on the different image modes of the road shoulder type.

5. The method according to claim 1, characterized in that The road shoulder (9) is detected by means of at least two wheel speed sensors (5) as detection devices (4, 5, 6) on different wheels (7) of the motor vehicle (1).

6. The method according to claim 5, characterized in that An assessment with respect to the type of road shoulder is carried out based on the temporal profile of at least two wheel speed sensors (5) and by comparing the temporal profiles.

7. The method according to claim 1, characterized in that The road shoulder (9) is detected by means of at least two height sensors (6) and / or two wheel speed sensors (5) as detection devices (4, 5, 6) on opposite wheels (7) of the motor vehicle (1).

8. The method according to claim 1, characterized in that The road surface of a road (13) on which the motor vehicle (1) is located is detected and taken into account when evaluating shoulder driving (8).

9. An assistance system (2) for a motor vehicle (1), comprising at least one electronic computing device (3) and a detection device (4, 5, 6), wherein: The assistance system (2) is designed to carry out a method according to any one of claims 1 to 8.

Images