Method and device for determining a distance to a vehicle ahead

Abstract

A method is proposed for determining a distance to a vehicle in front by means of a camera comprising rolling shutter sensors and a pulse-width modulated light source emanating from the vehicle in front. According to the invention, a video stream of the vehicle driving ahead is recorded with the camera, the video stream is evaluated with a detection of signals from pulse-width modulated taillights, one of the taillights is extracted from the video stream and cut out as an area of ​​interest. A Fourrier transform is used to determine a frequency of the taillight in the area of ​​interest and modeled as a Gaussian distribution to determine a frequency uncertainty. The distance to the vehicle ahead is determined by evaluating a reference table that maps the determined frequency uncertainty to a specific distance. The determined distance is then made available to the vehicle's assistance system.

Description

[0001] The present invention relates to a method and a device for determining a distance to a vehicle traveling ahead.

[0002] From DE 10 2011 081 384 A1, it is known to determine the distance between an ego vehicle and a preceding vehicle by transmitting a pulse-width-modulated light signal and receiving the light reflected from the preceding vehicle using a camera comprising rolling shutter sensors. By synchronizing the LED pulse width modulation of the headlight and the reading times of the rolling shutter sensors, Can depth information be obtained taking into account the light travel time?

[0003] The publication by Gouranga Mandal, Diptendu Bhattacharya, and Parthasarathi De, "Real-time vision-based overtaking assistance system for drivers at night on a two-lane single carriageway," in Computacion y Sistemas, Vol. 25, No. 2, 2021, pp. 403-416, discloses an assistance system to support a driver in making overtaking decisions under adverse night conditions. The assistance system can estimate the actual and relative speed, as well as the distance, between a slow-moving vehicle ahead and an oncoming vehicle based on their taillights and headlights, respectively.

[0004] DE 10 2020 109 542 A1 discloses a method in which the brake lights of a vehicle driving ahead of a motor vehicle are detected with a camera. A distance to the vehicle is determined based on the respective size of an image area of ​​the image data.

[0005] A publication by CHEN, Duan-Yul PENG, Yang-Jie entitled “Frequency-tuned taillight-based nighttime vehicle braking warning system”, see IEEE Sensors Journal, 2012, 12th vol. no. 1 1, pp. 3285-3292, describes a method in which a camera is used to detect the brake lights of a vehicle ahead at night by analyzing spatial features and frequency.

[0006] The object of the present invention is to provide an alternative method and an alternative device for determining the distance to a vehicle traveling ahead.

[0007] With regard to the method, this object is achieved by the measures specified in claim 1, and with regard to the device by the measures specified in claim 8.

[0008] Further advantageous embodiments of the present invention are the subject of the dependent claims.

[0009] According to one aspect of the present invention, a method for determining a distance to a preceding vehicle is provided: by means of a camera comprising rolling shutter sensors and a pulse-width modulated light source emanating from the vehicle in front, wherein the following steps are carried out according to the invention: -Recording a video stream of the vehicle ahead with the camera; -Evaluation of the video stream and detection of signals from pulse-width modulated rear lights; - Extracting one of the taillights from the video stream and cropping it as a region of interest; -Determination of a frequency of the backlight from the video stream using Fourier transformation -Modeling the frequency and determining a frequency uncertainty as a Gaussian distribution: -Determination of a distance to the vehicle in front by evaluating a reference table that represents the assignment of the determined frequency uncertainty to a distance and -Providing the determined distance to the vehicle’s assistance systems.

[0010] The invention takes advantage of the fact that the interaction of the rolling shutter sensors and the pulse-width modulated taillights creates a characteristic pattern in the form of flickering at a specific frequency in the video stream. The resulting frequency is initially constant and independent of the distance. Because this frequency is mapped onto the camera sensor in an increasingly smaller area with increasing distance, increasing frequency blur results. This blur / frequency dispersion can be related to the distance following a prior FFT. In this process, the relationship between frequency response (frequency dispersion) and depth is first extracted as part of an initial application and stored in one of the reference tables. This can be done, for example, in the vehicle by adding and comparing other sensor modalities.Next, a 2D mapping of the scene from the video stream is created during operation using a selected driver assistance camera. A further recommended step involves preprocessing, in which taillights are extracted from the scene using state-of-the-art algorithms. This has the advantage that the frequency can be determined more accurately than by evaluating the entire section of the image sensor. An FFT (Fast Fourier Transform) analysis of the extracted 2D image section of the video stream reveals the frequency response. This FFT then involves further masking, since taillights are generally positioned at a specific angle relative to the ego vehicle.

[0011] This reduces the search corridor for frequency determination to a selected range within the FFT. Here, the frequency response to the respective taillight is determined. This can be done, for example, by searching for a continuous change in frequency over several frames. Once this frequency is detected, the blur is modeled using a Gaussian function and the so-called FWHM (Full Width of Half Max) is determined. This half-width is converted into a depth value using the previously introduced reference table and made available to the assistance system. The advantage of this method is that depth can be extracted even in adverse situations. The method can be based on images, i.e. the video stream of a mono camera, which eliminates the need for complex stereo calibration and correspondence analysis.The term "video stream" here refers to a sequence of images captured by the camera sensor's exposure. The video stream is evaluated in a processing unit, and the determined distance is transmitted to assistance systems, such as adaptive cruise control or automated driving systems. The taillights of the vehicle ahead, for example, are brake lights or lights for night or daytime lighting.

[0012] According to a further embodiment of the present invention, the reference table is cyclically updated and expanded with data from additional sensor modalities. The reference table created in an initial application is expanded and validated during driving, so that it continuously improves over the vehicle's operating life.

[0013] According to a further embodiment of the present invention, the frequency uncertainty is defined by a half-width of the Gaussian distribution. The half-width is a metric for representing the frequency uncertainty that can be determined with little computational effort.

[0014] According to a further embodiment of the present invention, the determined distance is used as a substitute value in the event of failure of other sensor modalities such as radar, lidar, or ultrasound. By providing a distance determined according to the method as a substitute value, the availability of the assistance system can be increased.

[0015] According to a further embodiment of the present invention, the determined distance is used to verify the plausibility of additional sensor modalities. The distance determined using the proposed method is suitable for testing sensors. For example, if a significant deviation or significant fluctuation in the radar or ultrasonic sensors is detected compared to the distance determined using the proposed method, it can be concluded that they are malfunctioning.

[0016] According to a further embodiment of the present invention, the frequencies determined by means of the Fourier transformation are masked based on predetermined conditions. The arrangement of light sources, ie, the taillights of a vehicle, follows certain rules, e.g., the taillights are at the same height. Masking allows the relevant range for frequency determination to be limited or corresponding stray light to be excluded.

[0017] In a further embodiment of the invention, the distance is determined according to the proposed method only for predetermined vehicles. Using image recognition or wireless data exchange, the vehicle type and the associated modulation frequency of the light sources can be determined, for example, from a database. Since the method delivers the most reliable results with a known modulation frequency of the light source, the distance evaluation is preferably limited to vehicles of a known type.

[0018] According to a further aspect of the present invention, a device for determining a distance to a preceding vehicle is provided: by means of a camera comprising rolling shutter sensors and a pulse-width modulated light source emanating from the preceding vehicle, wherein the camera is configured to record a video stream of the preceding vehicle, an evaluation unit is set up to analyze the video stream and cuts out the vehicle’s rear lights as an area of ​​interest, the evaluation unit is further configured to determine the frequency of a rear light from the video stream by means of a Fourier transformation, to generate a Gaussian distribution for determining a frequency blur, to determine a distance associated with the frequency blur by evaluating a reference table and to make it available to an assistance function of the vehicle.

[0019] The device according to the invention enables the distance to a vehicle driving ahead to be determined using comparatively cost-effective means.

[0020] The present invention will now be explained in more detail using exemplary embodiments with reference to the accompanying drawings, in which the same or similar parts are designated by the same reference numerals.

[0021] It shows: Fig. 1 is a schematic side view of a vehicle and a vehicle driving in front of it; Fig. 2 a pattern of a light source of the vehicle ahead, which is imaged in the camera, and Fig. 3 a flowchart of a method sequence according to an embodiment of the present invention.

[0022] Fig. 1 shows a schematic side view of a vehicle 1 and a vehicle 3 driving ahead of the vehicle 1, shown in a first position with a first distance d1 from the vehicle 1 and in a second and third position shown in dashed lines with a second distance d2, d3. The vehicle 1 has a camera 5 with which the vehicle in front is observed. The camera is designed as a rolling shutter camera 5, i.e. images of the vehicle in front are read in line by line. The camera 5 captures an image sequence in the form of a video stream of the tail lights 7 of the vehicle 3 in front, and an evaluation unit 9 extracts the relevant area of ​​the image sequence imaged in the sensor for further processing.

[0023] As in Fig. 2, the light rays of the pulse width modulated rear light are formed with a frequency of 15 according to Fig. 1, depending on the distance d1, d2, d3 to the vehicle 3 in front, is reflected as a light-dark flickering in the area 13a, 13b, 13c on the lines of the image sensor 11 of the camera 5. The greater the distance to the vehicle in front, the fewer lines of the image sensor of the camera 5 the flickering is reflected in, ie, for example, d1 correlates with the area 13c, d2 with the area 13b, and d3 with the area 13a.

[0024] A representation 17 of the frequencies determined by fast Fourier transformation depicts the frequency 15 corresponding to the distances d1, d2, and d3. The different diameters of the determined frequency ranges 15 represent a frequency blur, i.e., the range in which the frequencies lie. This range becomes smaller the more of the lines of the image sensor 11 are illuminated by the rear light of the vehicle ahead.

[0025] This range is represented as a probability distribution in a Gaussian curve 19a, 19b, 19c. The Gaussian curves 19a, 19b, 19c reflect that with an increasing number of lines in the image sensor 11, in which the taillights of the preceding vehicle are imaged, the scatter, ie, the frequency uncertainty of the frequency determined by the Fourier transformation, decreases.

[0026] The half-width 21a, 21b, 21c is preferably used as a measure for evaluating the dispersion. The smaller the half-width, the smaller the distance to the preceding vehicle 3, and vice versa.

[0027] A reference table stores a mapping of frequency uncertainty, preferably the half-width, to distances. Using the reference table, the distance to the vehicle ahead can be determined after determining the frequency uncertainty.

[0028] In Fig. Figure 3 shows a sequence of the method according to the invention.

[0029] In step S1, a video stream of a vehicle driving ahead with pulse-width modulated taillights is recorded using a rolling shutter camera. In step S3, detection takes place, and in step S5, the area of ​​the taillights is cropped out as the relevant area for further analysis. The interaction of the pulse-width modulated taillights creates a video stream in the rolling shutter camera that flickers at a fixed frequency. This frequency is determined in step S7 using a Fourrier transformation. In the subsequent step S9, a Gaussian distribution corresponding to the number of exposed lines is determined to determine a frequency blur. In step S11, a distance to the vehicle driving ahead is determined from a reference table based on the frequency blur, and in step S13, this data is transmitted to an assistance system in the vehicle.

Claims

[1] Methods for determining the distance to a vehicle ahead: by means of a camera comprising rolling shutter sensors and a pulse-width modulated light source emanating from the vehicle in front characterized by , that The following steps must be performed: -Recording a video stream of the vehicle ahead with the camera (S1); -Evaluation of the video stream and detection of signals from pulse-width modulated rear lights (S3); - Extracting one of the taillights from the video stream and cutting it out as the area of ​​interest (S5); -Determining the frequency of the taillight from the video stream using Fourier transformation (S7) -Modeling of the frequency and determination of a frequency uncertainty as a Gaussian distribution (S9); -Determination of a distance to the vehicle in front by evaluating a reference table that assigns the determined frequency uncertainty to a distance (S11) and -Provision of the determined distance to the vehicle's assistance systems (S13). [2] Method according to claim 1, characterized by that the reference table is cyclically renewed and expanded by data from additional sensor modalities. [3] Method according to claim 1 or 2, characterized by , that the frequency uncertainty is defined via a half-width of the Gaussian distribution. [4] Method according to any one of the preceding claims, characterized by that the determined distance is used as a substitute value in case of failure of other sensor modalities such as radar, lidar or ultrasound. [5] Method according to any one of the preceding claims, characterized by , that the determined distance is used to validate sensor modalities. [6] Method according to any one of the preceding claims, characterized by , that the frequency determined by means of the Fourier transform is masked according to predefined conditions. [7] Method according to any one of the preceding claims, characterized by that the distance is only determined using the aforementioned steps for predetermined vehicles. [8] Device for determining a distance to a vehicle ahead (3): by means of a camera (5) comprising rolling shutter sensors and a pulse-width modulated light source emanating from the vehicle (3) in front, characterized by , that the camera (5) is set up to record a video stream of the vehicle ahead, an evaluation unit (9) is set up to analyze the video stream and to cut out the vehicle's taillights as an area of ​​interest, the evaluation unit (9) is further set up to determine the frequency of a rear light in the video stream by means of a Fourier transform, to generate a Gaussian distribution to determine a frequency uncertainty, to determine a distance associated with the frequency uncertainty by evaluating a reference table and to make it available to an assistance function of the vehicle.

Images