Method for detecting a vehicle crossing a lane marking

Abstract

The invention relates to a method for detecting when a vehicle (1) crosses a lane marking (M1 to M4) that defines a lane (F1 to F3) by means of detected rolling noise from the vehicle's tires (1). According to the invention, it is provided that - sensor signals acquired by a plurality of sensors designed as microphones (2) and signals acquired for determining vehicle parameters and environmental conditions of the vehicle (1) are fed to a model (M) pre-trained by machine learning and - using the model (M) to determine, based on the recorded sensor signals and the recorded signals, whether a vehicle tire of the vehicle (1) assigned to a microphone (2) crosses a lane marking (M1 to M4).

Description

[0001] The invention relates to a method for detecting when a vehicle crosses a lane marking delimiting its lane, based on the recorded rolling noise of the vehicle's tires. The invention also relates to a vehicle designed to carry out the method.

[0002] From DE 10 2014 115 057 A1, a driver assistance system for supporting a driver of a motor vehicle, a motor vehicle, and a method for supporting a driver of a motor vehicle on a roadway by means of a driver assistance system are known. The method provides that a sensor device detects a sensor signal describing when the motor vehicle leaves its lane, and a control unit recognizes the lane departure by the motor vehicle based on the sensor signal and outputs a control signal upon detection. Alternatively, the sensor device can detect the rolling noise of a tire of the motor vehicle on a roadway surface as the sensor signal, and the control unit can detect the motor vehicle leaving its lane based on the detected rolling noise.

[0003] The invention is based on the objective of providing a method for detecting a crossing of a lane marking that limits a vehicle's lane and a vehicle.

[0004] The problem is solved according to the invention by a method which has the features specified in claim 1 and by a vehicle which has the features specified in claim 10.

[0005] Advantageous embodiments of the invention are the subject of the dependent claims.

[0006] Method for detecting when a vehicle crosses a lane marking delimiting its lane, based on the recorded rolling noise of the vehicle's tires, characterized in that - Sensor signals captured by a plurality of sensors designed as microphones and signals captured to determine vehicle settings and environmental conditions of the vehicle are fed to a model pre-trained by machine learning and - using the model, based on the recorded sensor signals and the recorded signals, it is determined whether a vehicle tire of the vehicle assigned to a microphone crosses a lane marking.

[0007] By applying this method, it is possible to detect when a vehicle crosses a lane marking even under comparatively poor weather conditions, such as heavy rain, fog, etc., and to alert the driver, particularly to increase road safety. The method enables robust and relatively reliable detection of lane marking violations. In particular, detecting lane marking violations based on the captured sensor signals provides redundancy compared to optical detection of lane marking violations using image data from a lane departure warning system camera.The method detects when the lane is crossed, even if the camera and / or lidar of the lane keeping assistant cannot detect a crossing of a lane marking, for example due to poor visibility.

[0008] For example, the method can be used to determine the vehicle's position relative to its lane, particularly in highly automated or autonomous driving modes. Positional integrity can be improved by comparing the vehicle's position with satellite-acquired positioning data and image data obtained from a camera, especially regarding the vehicle's surroundings.

[0009] In one implementation, signals related to current vehicle speed, steering angle, and / or tire pressure are recorded as vehicle parameters and taken into account when detecting a lane marking violation. This means that, based on the determined vehicle parameters, it is possible to distinguish whether the vehicle intends to change lanes or, for example, crosses the lane due to driver inattention.

[0010] In a further version, the vehicle's environmental conditions are determined based on signals from a rain sensor and / or a pressure sensor and / or a light sensor. In particular, prevailing weather conditions, such as precipitation, wind conditions, and solar radiation, can thus be determined and taken into account, for example, since tire noise can change depending on the weather and wind noise, for instance, needs to be filtered out.

[0011] In one possible implementation, the vehicle's environmental conditions can be determined by evaluating data received from an infrastructure. For this purpose, the vehicle can be digitally connected to a central computing unit, which provides the vehicle with information about the environmental conditions, particularly the weather conditions.

[0012] In one implementation of the method, the environmental conditions include the road surface characteristics determined using map data available from the vehicle, since rolling noise changes depending on the specific road surface characteristics. This means that the road surface characteristics are taken into account when a lane marking is detected, particularly because increased road surface roughness, especially when crossing an existing lane marking, also increases the sound pressure level and frequency.

[0013] One implementation of the procedure involves using the pre-trained model to distinguish whether a lane marking is completely or partially crossed, so that it can then be determined whether the vehicle is changing lanes, for example because of an intended overtaking maneuver, or whether the vehicle is crossing a lane marking due to carelessness, whereby this usually does not occur with all four tires of the vehicle.

[0014] Furthermore, one version of the pre-trained model distinguishes between solid and broken lane markings. This allows the system to determine whether the vehicle is violating a no-overtaking rule or whether it is initiating a legal overtaking maneuver.

[0015] In a possible further implementation, the pre-trained model is further trained using lane markings stored in highly accurate map data, so that the accuracy of the pre-trained and further-trained model can be increased and optimized, for example to detect crossing a lane marking at an early stage and, if necessary, to issue a warning message in the vehicle.

[0016] In one implementation, image data captured by a vehicle's camera is used by a lane keeping assist system to verify whether a lane marking has been crossed, based on information determined by a pre-trained model. This can largely prevent false interventions by the lane keeping assist system, especially when image data (e.g., due to a dirty camera lens) indicates that a lane marking has been crossed, but sensor signals from one or more microphones detect no lane marking crossing while the vehicle is in motion, as the vehicle's steering angle is also taken into account.

[0017] Furthermore, the invention relates to a vehicle which is designed to supply detected sensor signals from a plurality of sensors designed as microphones and detected signals for determining vehicle parameters and environmental conditions of the vehicle to a pre-trained machine learning model and to determine, using the model, on the basis of the detected sensor signals and the detected signals, whether a vehicle tire of the vehicle assigned to a microphone crosses a lane marking.

[0018] Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

[0019] This shows: Fig. 1 schematically depicts a section of roadway with a vehicle driving in its lane, Fig. 2 schematically depicts a section of roadway with a vehicle crossing a lane marking with its right tires, Fig. 3 schematically depicts a section of roadway with a vehicle crossing a lane marking with its left tires and Fig. 4. Schematic block diagram for carrying out a procedure to detect when a vehicle crosses a lane marking that limits its lane.

[0020] Corresponding parts are marked with the same reference symbols in all figures.

[0021] Fig. Figure 1 shows a roadway section F with a left lane F1, a middle lane F2 and a right lane F3 and a vehicle 1 driving in the middle lane F2. The roadway section F is bounded by a broken lane marking M1, M2 on each side of the roadway, with a broken lane marking M3, M4 running between each of the three lanes F1 to F3.

[0022] For example, for safety reasons, road markings and lane markings M1 to M4 are applied to the road surface on highways with increased roughness compared to a commonly used road surface, such as asphalt. This provides increased grip for a vehicle driving on lane markings M1 to M4 in extreme driving situations, while the visibility of the lane markings M1 to M4 can be ensured by means of reflective particles.

[0023] The vehicle 1 has a microphone 2 in the area of ​​each wheel, in particular a tire, which continuously detects sensor signals, in particular acoustic signals, during vehicle operation. A sound pressure level SPL, in particular a weighted sound pressure level SPL, and a frequency f are determined based on these detected sensor signals.

[0024] According to the in Fig. In the embodiment shown in 1, the vehicle 1 travels centrally with respect to its lane F2, with an input variable for a Fig. The pre-trained model M shown in section 4 represents the sound pressure level (SPL) of all four vehicle tires, for example, between 70 dB(A) and 90 dB(A). In particular, model M was pre-trained using a neural network.

[0025] The following describes a method for detecting when a vehicle 1 crosses a lane marking M3, M4 that limits a lane F2, based on the recorded rolling noise of the vehicle tires of vehicle 1.

[0026] The procedure takes into account various vehicle parameters, environmental conditions and weather conditions in order to distinguish relatively reliably whether a given vehicle tire crosses a lane marking M1 to M4 or not.

[0027] For this purpose, sensor signals are continuously recorded by microphones 2 during the vehicle's operation 1, which are then used to create a system that... Fig. The four pre-trained machine learning models M, presented as examples and in a highly simplified manner, are used. The acquired sensor signals include the sound pressure level (SPL) and a frequency f of a recorded rolling noise from a respective tire of vehicle 1.

[0028] Furthermore, signals acquired by sensors on vehicle 1 are evaluated with regard to vehicle parameters and environmental conditions and fed to the pre-trained model M. For example, vehicle parameters such as vehicle speed v and steering angle δ are determined and fed to the pre-trained model M, whereby the tire pressure of each tire on vehicle 1 can also be fed to the model M. For example, an instantaneous vehicle speed v is determined based on the wheel rotation speeds recorded by a wheel sensor.

[0029] The environmental conditions are determined primarily based on signals from a vehicle-mounted rain sensor, a pressure sensor, and a light sensor. A rain signal S detected by the rain sensor indicates whether it is raining, specifically whether there is so-called rain spray on the road surface. A pressure signal detected by the pressure sensor indicates whether it is windy, and a light signal from the light sensor determines the brightness in the immediate vicinity of the vehicle 1.

[0030] Alternatively or additionally, the environmental conditions of vehicle 1 can also be determined by evaluating received data from an infrastructure. For example, vehicle 1 is connected via a radio link to a central computer unit, which provides this data to vehicle 1, in particular to the pre-trained model M.

[0031] The pre-trained model M contains various sound pressure levels (SPL) related to different road surface properties and different noise levels related to various precipitation scenarios. For example, the different road surface properties and their corresponding sound pressure levels (SPL) can be determined using map data, such as that available from the vehicle.

[0032] Based on the input variables mentioned above, in particular the instantaneous vehicle speed v, the steering angle δ, the rain signal S, the sound pressure level SPL, and the frequency f, the pre-trained model M and the comparison parameters stored within it, especially in the form of the different road surface properties and the different noise levels, are used to determine the contact direction R with respect to the respective lane marking M1 to M4, the overlap O between the vehicle tires and the lane marking M1 to M4, and a classification K of the crossing, specifically for each tire of the vehicle. The classification determines whether the vehicle 1 crosses a lane marking M1 to M4 with its left or right tires.

[0033] The information obtained is used for sensor fusion (SF) and / or fed into a function of a lane keeping assist system, in particular a lane departure warning system (LDW).

[0034] For example, image data captured by a camera on vehicle 1 for the lane departure warning system (LDW) is validated using information determined by the pre-trained model M regarding whether a lane marking M1 to M4 has been crossed. The information determined by the pre-trained model M is thus used as redundancy for image data captured by the camera of a lane departure warning system, in particular a lane departure warning system (LDW).

[0035] As described above, vehicle 1 drives in the Fig. In the embodiment shown in Figure 1, the vehicle is positioned centrally on the middle lane F2, i.e., with all vehicle tires on, for example, asphalt, which forms the road surface. The sound pressure level SPL of all four vehicle tires is, for example, in an interval between 70 dB(A) and 90 dB(A), with a determined frequency f of the respective rolling noise of all vehicle tires being, for example, between 100 Hz and 1000 Hz.

[0036] Based on the determined sound pressure levels SPL and the determined frequencies f of the respective rolling noise, it is determined that the vehicle 1 is in its lane F2 and that none of the vehicle tires cross a lane marking M3, M4 that limits the middle lane F2.

[0037] In a Fig. In the embodiment shown in Figure 2, the vehicle 1 travels on the middle lane F2, with the vehicle 1 using its right tires on a lane M4 which borders the middle lane M2 on the right side.

[0038] The two left vehicle tires are located in the middle lane F2 and thus, for example, on asphalt, so that a determined sound pressure level SPL of a rolling noise of the two left vehicle tires is, for example, between 70 dB(A) and 90 dB(A) and a determined frequency f of the rolling noise of these two vehicle tires is, for example, between 100 Hz and 1000 Hz.

[0039] However, the sound pressure level (SPL) of the rolling noise of the two right-hand vehicle tires crossing the lane marking M4 is, for example, between 80 dB(A) and 100 dB(A), and the frequency f of the rolling noise of the right-hand vehicle tires is, for example, in a range between 1500 Hz and 3000 Hz.

[0040] This also applies if vehicle 1 is traveling with its left tires on a lane marking M3 that borders the middle lane F2 on the left. The rolling noise changes if the respective lane marking M3, M4 is regularly interrupted, whereby the lane marking M3, M4 can be identified and classified as an interrupted lane marking M3, M4. The sound pressure level SPL and the frequency f remain largely constant if the lane marking M3, M4 is continuous, in particular if it is a so-called solid line.

[0041] Fig. Figure 3 shows an embodiment in which the vehicle 1 travels with its left vehicle tires on a left interrupted lane marking M1 on the edge of the roadway, wherein this lane marking M1 is integrated into a road surface, for example milled.

[0042] A measured sound pressure level SPL of the rolling noise of the two right vehicle tires located on the right lane F3 is between 70 dB(A) and 90 dB(A) and a frequency f of the rolling noise of the two right vehicle tires is between 100 Hz and 100 Hz.

[0043] The rolling noise of the two left tires of the vehicle, located on the left lane marking M1, has a sound pressure level SPL between 80 dB(A) and 100 dB(A), whereas the frequency f of the rolling noise of the two left tires lies between 1500 Hz and 3000 Hz. This also applies when the two right tires of vehicle 1 are traveling on a right-hand, interrupted lane marking M2 at the edge of the roadway.

[0044] The rolling noise remains largely constant if the lane marking M1 is continuous and uninterrupted. If it is determined that the sound pressure level SPL and the frequency f of the rolling noise of the two left vehicle tires remain largely the same, then the lane marking M1 is identified as continuous and classified accordingly.

[0045] In particular, the sound pressure level SPL of the vehicle wheels can be used to determine which of the vehicle wheels is crossing the respective lane marking M1 to M4, so that in conjunction with a detected steering angle δ, for example, it can be recognized that the vehicle 1 intends to change lanes, for example to carry out an overtaking maneuver.

[0046] Furthermore, it is planned that the pre-trained model M will be further trained using lane markings M1 to M4 of various types stored in a highly accurate map.

[0047] In particular, the method can be applied in highly automated or autonomous driving mode of a vehicle 1, whereby the method is used to determine the position of the vehicle 1 in relation to its lane M1 to M3. Positional integrity can be increased by comparing it with satellite-acquired positioning data and image data from a camera of the vehicle 1.

[0048] By applying the procedure, the driving safety of the vehicle 1 can be increased, especially in highly automated or autonomous driving operation, particularly when, based on image data captured by the camera of a lane keeping assistant, for example due to a low sun, fog and / or precipitation, a lane marking M1 to M4 can hardly be determined or not at all.

[0049] This method makes it possible to detect when a vehicle crosses a lane marking M1 to M4, regardless of how the lane marking M1 to M4 is oriented in relation to vehicle 1. For example, it is also possible to detect when a vehicle crosses a stop line at an intersection or when it crosses lettering applied to a road surface by means of increased roughness. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2014 115 057 A1

[0002]

Claims

[1] Method for detecting that a vehicle (1) has crossed a lane marking (M1 to M4) delimiting a lane (F1 to F3) by means of detected rolling noise from the vehicle's tires (1), characterized by , that - sensor signals acquired by a plurality of sensors designed as microphones (2) and signals acquired for determining vehicle parameters and environmental conditions of the vehicle (1) are fed to a model (M) pre-trained by machine learning and - using the model (M) to determine, based on the recorded sensor signals and the recorded signals, whether a vehicle tire of the vehicle (1) assigned to a microphone (2) crosses a lane marking (M1 to M4). [2] Method according to claim 1, characterized by , that signals recorded as vehicle parameters are determined in relation to an instantaneous driving speed (v) and / or a steering angle (δ) and / or a tire pressure. [3] Method according to claim 1 or 2, characterized by , that the vehicle's environmental conditions (1) are determined by means of signals received from a rain sensor and / or a pressure sensor and / or a light sensor. [4] Method according to any one of the preceding claims, characterized by , that the environmental conditions of the vehicle (1) are determined by evaluating data received from an infrastructure. [5] Method according to any one of the preceding claims, characterized by , that the environmental conditions include a road surface condition determined on the basis of map data available from the vehicle. [6] Method according to any one of the preceding claims, characterized by , that the pre-trained model (M) distinguishes whether a lane marking (M1 to M4) is completely or partially crossed. [7] Method according to any one of the preceding claims, characterized by, that the pre-trained model (M) distinguishes whether the crossed lane marking (M1 to M4) is a solid or a broken lane marking (M1 to M4). [8] Method according to any one of the preceding claims, characterized by , that the pre-trained model (M) is further trained using lane markings (M1 to M4) stored in highly accurate map data. [9] Method according to any one of the preceding claims, characterized by , that image data of a lane keeping assistant captured by a camera of the vehicle (1) are validated on the basis of information obtained by the pre-trained model (M) with regard to crossing a lane marking (M1 to M4). [10] Vehicle (1) which is equipped to carry out a method according to any of the preceding claims.

Images