Method for determining a slipstream area behind a vehicle ahead, computer program product, driver assistance system and motor vehicle

Abstract

A method for determining a slipstream area behind a vehicle ahead is described, wherein a pattern is projected forward in the direction of travel by means of at least one headlight of the vehicle, wherein the pattern is generated from a plurality of headlight pixels, wherein projection lines are determined for the headlight pixels, wherein an environment located in front of the vehicle is measured three-dimensionally and three-dimensional measurement data is generated, wherein features of the pattern located in front of the vehicle, generated by reflection from the headlight pixels, are recorded by means of at least one camera pointing forward in the direction of travel and transferred as image points into a camera plane. From the distance of the image points from the epipolar pole of the camera plane, a distance of the reflected features of the pattern to the headlight is determined.Furthermore, a correspondence is established between reflected features of the captured pattern and the three-dimensional measurement data on the respective projection lines of the headlight pixels, whereby deviations between the distances of the features of the pattern to the headlight and distances to the headlight determined from corresponding three-dimensional measurement data are determined on the respective projection line, and the wind shadow area is determined on the basis of the deviations.

Description

[0001] A method for determining a slipstream area behind a vehicle in front, a computer program product, a driver assistance system, and a motor vehicle are described.

[0002] Methods for determining a slipstream area behind a vehicle in front, computer program products, driver assistance systems and motor vehicles of the type mentioned above are known in the prior art.

[0003] In modern vehicles, adaptive cruise control (ACC) often takes over the longitudinal control of the vehicle based on the traffic ahead. The distance to the vehicle ahead can be individually set in many systems, with predefined intervals usually chosen according to the driver's preferences. However, it would be desirable to also be able to drive in an energy-saving mode alongside these settings. This mode would regulate the distance so that the vehicle remains in the optimal slipstream, thus minimizing fuel consumption. This is particularly relevant for electric vehicles, as it can result in a noticeable difference in their range.

[0004] It is generally known that driving in a slipstream can reduce energy consumption, as the vehicle is exposed to less wind resistance and turbulence. Slipstreams can be detected, for example, by monitoring global wind patterns. It is also known to calculate slipstreams based on model assumptions. In this approach, input variables such as vehicle speed and acceleration, steering angle, etc., are measured by the vehicle's sensors, and the slipstream is then approximated using the model. A disadvantage of this method is the rough estimation resulting from the model assumptions and the sometimes insufficient availability of input data.

[0005] From DE10 2018 109 235 A1, a distance control system and a method for controlling the distance of a vehicle to a vehicle in front are known, in which the vehicle has a distance control system for setting an automatic distance control mode, with which a front object in front of the vehicle is detected by an environment detection system of the vehicle, the front object is recognized as a vehicle in front, and a distance to the vehicle in front is regulated to an ACC target distance, whereby it is determined that a safe following situation exists if at least the following criteria are met: - the vehicle in front is a moving object, - the vehicle in front is tracked for a minimum tracking period, - the same vehicle in front is tracked during the minimum tracking period, - the distance to the vehicle in front is within a predetermined distance range.- A relative speed is within a predetermined speed tolerance range, whereby a signal is issued to the driver upon confirmation of a safe following situation, and an automatic distance control platooning mode is set upon input of a confirmation signal by the driver, which has a smaller target distance than the ACC target distance of the automatic distance control mode.

[0006] German patent DE 10 2015 122 842 A1 describes a calibration plate for calibrating a 3D measuring device that emits rays and includes a camera with several markings that can be recorded by the camera and identified in the images. The calibration plate has a mirror that reflects incident rays from the 3D measuring device.

[0007] German patent DE 10 2018 101 023 B3 describes a method for determining the distance between a vehicle and a projection surface onto which a characteristic light pattern is projected by a headlight of the vehicle and captured by an image acquisition unit of the vehicle. The characteristic light pattern exhibits characteristic structures, each generated by a light-emitting unit of the headlight. The distance is determined based on a correlation between the position of the characteristic structure on the light beam generating it and the position of this characteristic structure in the image of the vehicle camera along a trajectory that corresponds to the generating light beam in the vehicle camera image.

[0008] US Patent 2023 / 0058081 A1 describes a device with a controller configured to generate a symbol pattern as a result of symbol placement based on epipolar lines; to instruct a spatial modulator for light to project the symbol pattern; and to obtain an image of a reflection of the symbol pattern. The controller is configured to determine two positions of the symbol in the image and a depth of the symbol based on these two positions and an essential matrix.

[0009] The task therefore is to further develop methods for determining a slipstream area behind a vehicle in front, computer program products, driver assistance systems and motor vehicles of the type mentioned above in such a way that the slipstream can be extracted directly from current sensor data without having to resort to model assumptions such as turbulence models etc.

[0010] The problem is solved by a method for determining a slipstream area behind a vehicle ahead according to claim 1, a computer program product according to dependent claim 7, a driver assistance system according to dependent claim 8, and a motor vehicle according to dependent claim 9. Further embodiments and developments are the subject of the dependent claims.

[0011] A method for determining a slipstream area behind a vehicle in front from a motor vehicle is described, wherein a pattern is projected forward in the direction of travel by means of at least one headlight of the motor vehicle, wherein patterns are generated from a plurality of headlight pixels, and wherein projection lines in space are determined for the headlight pixels.

[0012] Furthermore, the area in front of the vehicle is measured three-dimensionally and three-dimensional measurement data is generated.

[0013] Features of the pattern, generated by the reflection of headlight pixels from a preceding vehicle, are captured by at least one camera pointing forward in the direction of travel and transferred as image points to a plane within the camera. These features, also called projection features, lie on the epipolar lines of the camera plane. The distances of the reflected features of the pattern from the headlight on the corresponding projection line of the headlight are determined from the distances of the image points from the epipolar pole of the camera plane. The camera is preferably designed as a virtual pinhole camera.

[0014] Furthermore, a correspondence is established between the reflected features of the pattern on the projection lines of the headlight pixels and the three-dimensional measurement data on the corresponding projection lines, whereby deviations between the distances of the features of the pattern on the projection line of the headlight pixels to the headlight and corresponding distances determined from three-dimensional measurement data on the projection line of the headlight pixels to the headlight are determined, and the wing shadow area is determined on the basis of the deviations.

[0015] The projection lines are read from a memory or determined in real time. The projection lines for the headlight pixels can be calculated or determined experimentally and stored in the memory of a processing unit. This can be done, for example, in the laboratory, while stationary, or during slow-speed operation, and can be used for calibration. Accordingly, "determination" refers, for example, to reading predefined and available projection lines from a memory.

[0016] It may be provided that, based on the specific deviations, a flow pattern behind the vehicle in front is determined, from which the slipstream area is calculated.

[0017] The method takes advantage of the fact that different areas in the airflow pattern behind the vehicle ahead exhibit varying air densities. Air density, in turn, influences the refractive index of the air. Thus, deviations caused by air density fluctuations become visible between the expected, actual positions of the headlight pixel features reflected from the vehicle ahead (determined by three-dimensional measurement) and the positions determined by epipolar geometry. These deviations can be used to determine the air density distribution behind the vehicle ahead.

[0018] Projection lines provide a precise geometric basis for the analysis and processing of image data. Epipolar lines represent a concept in epipolar geometry, which is used, among other things, in stereo image processing and photogrammetry. There, they describe the geometric relationship between two cameras and the projections of a point from three-dimensional space onto the respective image planes of the cameras. This relationship reduces the complexity of finding correspondences between image pairs and plays a crucial role in depth estimation, camera calibration, and 3D reconstruction.

[0019] The previously described method uses the concept of epipolar lines to describe the spatial relationship of a projection line emitted by at least one spotlight to the camera plane. Depending on the reflection depth in the space in front of the spotlight and camera, a specific spotlight pixel is reflected at exactly one point on a projection line under ideal conditions.

[0020] By comparing the position determined by the camera on projection lines with the position on the projection lines under real conditions, i.e., by comparing with three-dimensional measurement data, a density fluctuation along the projection lines can be determined for each headlight pixel.

[0021] It is necessary to correlate the reflected headlight pixels recorded and detected by the camera with the corresponding headlight pixels of the pattern as emitted by the headlight. This is achieved, firstly, by the proximity of the reflected headlight pixels to the projection lines. Alternatively or additionally, the relationships between different headlight pixels can be known, thus enabling a reliable correlation of the reflected headlight pixels even with incomplete reflection of the pattern.

[0022] The projection lines for the headlight pixels are calculated or determined experimentally. This can be done, for example, in the laboratory, while stationary or at low speed, and can be used for calibration.

[0023] The deviations between the real reflected headlight pixels and hypothetical reflected headlight pixels caused by density fluctuations are sufficiently small compared to the distances between different headlight pixels to define a capture circle around an ideal reflection point of a specific headlight pixel, within which a reflected headlight pixel can be assigned to a headlight pixel.

[0024] Based on this, a deviation can be determined between the reflected features of the headlight pixels of the characteristic pattern and the actual features of the same extracted from survey data in the vicinity of the projection lines.

[0025] Thus, the vehicle can detect areas of different density fluctuations and, consequently, slipstream areas, which can be used to keep the vehicle in the most energy-efficient area behind a vehicle in front.

[0026] The process requires neither a network connection to a central computer nor the use of model assumptions that are often prone to errors in everyday practice.

[0027] When used in conjunction with an adaptive cruise control system, significant fuel savings or energy savings can be achieved, as the vehicle can always be driven in an aerodynamically favorable area behind the vehicle in front.

[0028] In a first further development, it is provided that at least one headlight and at least one camera are calibrated to each other.

[0029] The spotlight and camera must be precisely calibrated to each other in order to know the projection lines and accurately determine deviations along them. Advantageously, the image planes and the positions of the light source and camera in space, as well as their corresponding epipoles, are measured precisely. For faster calculations, these measurements are preferably transformed into a common coordinate system using, for example, radar.

[0030] This can initially be done at the factory and can be repeated if a sufficiently reliable assignment is no longer possible, which can happen, for example, after an accident.

[0031] In a further refinement, it is provided that the pattern is a random pattern generated by differing brightness levels of various headlight illumination points.

[0032] This reduces the likelihood of irritation to the driver that might occur if he were to perceive a regular pattern, such as a grid or the like, in the headlight beam.

[0033] Furthermore, good pattern recognition in the camera image can be ensured even in complex situations.

[0034] In a further embodiment, it may be provided that the pattern is dimmed and / or that light in the non-visible range, in particular IR, is used to generate the pattern, wherein the camera is sensitive in this frequency range and / or that the pattern is displayed in pulses, wherein the pulses can be so short that they are not perceived by the human eye.

[0035] In this way, it is possible not to confuse the driver with the pattern.

[0036] Based on the correspondence and the three-dimensional survey data, a three-dimensional depth map is created for the reflected features of the headlight pixels.

[0037] Various technologies can be used for three-dimensional surveying of the environment, including radar, LIDAR, stereoscopic or monoscopic cameras with appropriate image evaluation methods.

[0038] In a further, more advanced version, it is planned that a plausibility check will be carried out for the deviation.

[0039] If deviations are too large, they may be implausible; that is, the system might calculate air density or flow patterns that do not correspond to reality. The plausibility check can define certain plausible measurement ranges depending on the driving speed and other parameters. If a deviation lies outside this plausibility range, it can be discarded. If such deviations occur regularly, this may indicate that the headlights and camera are miscalibrated, and a recalibration can be initiated.

[0040] In a further, more advanced version, it is planned that data about the slipstream area will be made available to higher-level systems.

[0041] Such systems can be, for example, driver assistance systems such as adaptive cruise control or a (semi-)autonomous driving system.

[0042] In a further, more advanced version, it is planned that the deviations will be converted into a graphical representation and displayed.

[0043] This can help a driver of the motor vehicle to choose a more aerodynamically efficient area.

[0044] A first independent subject matter relates to a computer program product comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause that at least one computing unit to be equipped to execute the procedure of the aforementioned type.

[0045] The process can be executed on one or more computing units, so that certain process steps are executed on one computing unit and other process steps on at least one other computing unit, whereby calculated data can be transmitted between the computing units if necessary.

[0046] Another independent item relates to a driver assistance system comprising at least one camera, at least one headlight for projecting a pattern, and a control system with a computer program product of the type described above.

[0047] The headlight used to project the pattern could be, for example, a multibeam headlight or a lidar. A corresponding driver assistance system could be, for example, adaptive cruise control.

[0048] Another independent item concerns a motor vehicle with a driver assistance system of the type described above.

[0049] Further advantages, features, and details will become apparent from the following description, in which – possibly with reference to the drawing – at least one embodiment is described in detail. Identical, similar, and / or functionally equivalent parts are marked with the same reference numerals.

[0050] They show schematically: Fig. 1. A top view of a traffic situation with two motor vehicles driving one behind the other; Fig. 1a a representation of projection lines of headlight reflections; Fig. 1b Interaction of headlights, camera and survey data, as well as Fig. 2. A flowchart of the procedure.

[0051] Fig. Figure 1 shows a top view of a traffic situation with two motor vehicles driving one behind the other, a motor vehicle 2 and a motor vehicle 4 driving in front.

[0052] The motor vehicle 2 has a headlight 6, by means of which a headlight beam is emitted, the limits of which are indicated by limit lines 6.1 and 6.2.

[0053] The headlight 6 is a matrix headlight and projects a pattern 8, consisting of many headlight pixels, onto the vehicle 4 ahead using several of its matrix pixels. Two randomly selected features 8.1 and 8.2 of the pattern, representing the corresponding headlight pixels, are labeled with reference symbols. For illustrative purposes, the pattern 8 is shown at a distance from the vehicle 4; in reality, it is partially projected onto the vehicle 4.

[0054] A camera 10 is used to capture a reflection image of the pattern 8 on the vehicle 4 driving ahead.

[0055] For example, a depth measurement system, designed as radar system 12, on the front of the vehicle 2 detects an area in front of the vehicle 2 and thus also the vehicle 4 driving ahead. A depth map in front of the vehicle 2 can be generated from three-dimensional survey data created by the radar system 12.

[0056] The relevant data can be provided in raw or processed form to a driver assistance system 14, which can, among other things, control the longitudinal dynamics of the motor vehicle 2 and, in an energy-saving mode, detect and utilize a slipstream area W behind the motor vehicle 4.

[0057] The evaluation of the data can take place in a separate control unit or in a control unit that is part of the driver assistance system 14 or that implements the driver assistance system function of the driver assistance system 14.

[0058] Fig. Figure 1a shows a representation of projection lines with reflected features of a projected pattern.

[0059] For each point of pattern 8, i.e., for each light ray of a spotlight pixel, there is a projection line E1 to E5, of which in Fig. In Figure 1a, for example, only five are shown in total. Depending on where a reflection of a feature on the preceding vehicle 4 occurs due to a light beam from an associated headlight pixel, this reflection can be assigned to a very specific point on such a projection line E1 to E5. Such features of the projected pattern reflected from the preceding vehicle are subsequently called projection features. Fig. Figure 1a shows two such projection features, P1 and P5. By determining on which projection lines a projection feature P1 and P5 lies, it is possible to assign which spotlight pixel caused which projection feature P1, P5.

[0060] Using the depth map from radar system 12, a depth corresponding to each projection feature can be determined from three-dimensional survey data. This depth, i.e., the distance of the intersection of the projection lines with the vehicle ahead, can be calculated. If necessary, the design-related distance of radar system 12 to the headlight must be taken into account; for example, the geometry data of the radar system must be transformed into a coordinate system of the headlight.

[0061] In order to achieve the required precision, it is necessary that spotlight 6 and camera 10 as well as radar system 12 are very precisely aligned or calibrated to each other, since even slight changes in the relative position and orientation of spotlight 6, camera 10 and radar system 12 to each other could impair the evaluation.

[0062] Based on the Fig. 1b The functioning of the method according to the invention is explained below.

[0063] For better illustration, a headlight 6 and the camera 10 of the vehicle 2 are shown with a greater distance than in a real installation situation.

[0064] The headlight 4 projects pattern 8, which includes, for example, a feature 8.2 projected by a headlight pixel. Another feature P1 is projected onto the vehicle 2, lies on the projection line E, and is optically represented at the position shown.

[0065] From the calibration of spotlight 4 to camera 10, the spotlight projection plane L and the camera projection plane K are defined in space. Likewise, the projection pole pl of the spotlight and the projection pole pk of the camera are derived from this.

[0066] The distance of the optically appearing projection feature P1 to the spotlight on the projection line E1 is determined as follows. Camera 10 captures the optically appearing feature P1 on the projection plane K as an image point Bp, which is mapped onto the epipolar line Epl connecting the projection pole pk and the image point Bp. From the distance δ of the image point Bp of the projection feature P1 to the projection pole pk in the projection plane K, the distance δ1 of the optically appearing feature P1 from the spotlight 4 can be determined. Using the known geometric data, the ratio of distance δ to distance δ1 is determined using the intercept theorem.

[0067] The radar system shown in Figure 1 generates three-dimensional survey data VD of the surroundings. From this three-dimensional survey data, a depth value corresponding to feature P1 on the projection line E1 is determined as survey datum R1. The depth value R1 represents the actual distance from the intersection of the projection line with the vehicle 4 to the radar system, or, after transformation, the distance δ2 to the headlight 4.

[0068] The deviations D of the distance δ1 from the distance δ2 are determined for all projection points P of pattern 8 that are depicted on the preceding vehicle 4. The deviations D are then evaluated and displayed in an overview or heatmap, with the magnitude of the deviations representing the strength of turbulence. Using assistance systems, vehicle 2 is preferentially steered and operated behind vehicle 4 in areas with low deviations.

[0069] By measuring these distances d, a representation of the density fluctuations behind the preceding vehicle 4 can be created. From this, a slipstream W can be calculated (see Fig. 1) derive.

[0070] Fig. Figure 2 shows a flowchart of the procedure.

[0071] In a first process step VS1, a calibration of the system or system components takes place, in particular of the headlight 6 or the pattern 8 it generates, the camera 10, and the radar system 12 in relation to each other.

[0072] In a subsequent step VS2, the epipolar geometry is calculated, that is, projection lines are determined for each spotlight pixel that is part of the pattern projection.

[0073] In a third process step, VS3, a 3D environmental reconstruction takes place based on the data from radar system 12. Alternatively, other methods can be used, for example, LiDAR or stereo cameras. Fusion of different sensor data is also possible.

[0074] In the next process step VS4, a headlight-based pattern projection takes place, in which the pattern 8 is projected by the headlight 6 onto the area in front of the vehicle 2. The pattern 8 is generated by closely spaced linear light systems in order to produce sufficient contrast even at greater distances.

[0075] In a subsequent step VS5, pattern detection takes place, in which the characteristic pattern in the camera image is identified. Based on the laws of epipolar geometry, the distances of the projection features (P1, P5) detected by the camera, i.e., the detected features of the projected patterns, to the spotlight 6 are determined.

[0076] In a subsequent step VS6, a correspondence is established between reflected features of the captured pattern (P1, P5) and the three-dimensional measurement data (R1, R5) on the respective projection lines of the headlight pixels.

[0077] In a seventh process step VS7, the actual distances of the intersection points of the projection lines with the preceding vehicle 4 are determined from the survey data.

[0078] In an eighth process step VS8, a feature offset D is then determined between the distances of the projection features according to VS6 to the headlight 4 and the actual distances of the intersection points of the projection lines with the vehicle ahead according to VS7.

[0079] In a subsequent step VS9, a plausibility check takes place, that is, a check to see whether a feature offset originates from a miscalibrated system or whether it is due to the flow conditions and the resulting different refractive indices.

[0080] In a subsequent process step VS10, a flow calculation takes place, in which a flow profile is calculated on the basis of the previously determined data, whereby a wind shadow is determined from the flow profile using the feature offsets from process steps VS8.

[0081] In a subsequent process step VS11, this information is forwarded to higher-level systems such as the driver assistance system 14, which can use the information for vehicle longitudinal control.

[0082] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as a further explanation in the description.

Claims

[1] Method for determining a slipstream area (W) behind a vehicle (4) from a motor vehicle (2), characterized by , that a pattern (8) is projected forward in the direction of travel by means of at least one headlight (6) of the motor vehicle (2), wherein the pattern (8) is generated from a plurality of headlight pixels, wherein projection lines (E1 - E5) are determined for the headlight pixels, wherein an environment located in front of the motor vehicle (2) is measured three-dimensionally and three-dimensional measurement data (VD) are generated, wherein features of the pattern (P1, P5) located in the environment in front of the motor vehicle (2), generated by reflection of the headlight pixels, are recorded by means of at least one camera (10) pointing forward in the direction of travel and transferred as image points (Bp) into a camera plane (K), wherein the distances (δ1) of the reflected features of the pattern (P1, P5) to the headlight (6) are determined from the distances (δ) of the image points (Bp) from the epipolar pole (pk) of the camera plane, where a correspondence is established between reflected features of the captured pattern (P1, P5) and the three-dimensional measurement data (R1, R5) on respective projection lines (E1 - E5) of the headlight pixels, where deviations (D) between the distances (δ1) of the features of the pattern (P1, P5) to the headlight (6) and distances (δ2) to the headlight (6) determined from corresponding three-dimensional survey data (R1, R5) are determined on the respective projection line (E1 - E5) and the slipstream area (W) is determined on the basis of the deviations (D). [2] Method according to claim 1, characterized by , that at least one spotlight (6) and at least one camera (10) are calibrated to each other. [3] Method according to any of the preceding claims, characterized by , that the pattern is a random pattern (8) generated by varying brightness levels of different spotlight points. [4] Method according to any of the preceding claims, characterized by , that a plausibility check is performed for the deviation (D). [5] Method according to any of the preceding claims, characterized by , that data about the slipstream area (W) is made available to higher-level systems. [6] Method according to any of the preceding claims, characterized by , that the deviations (D) are converted into a graphical representation and displayed. [7] Computer program product comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause the at least one computing unit to be configured to execute the method according to any of the preceding claims. [8] Driver assistance system (14), comprising at least one camera (10), at least one headlight (6) for projecting a pattern (8), and a control system comprising a computer program product according to claim 7. [9] Motor vehicle (2) with a driver assistance system (14) according to claim 8.

Images