Method for controlling a lighting device of a motor vehicle and lighting system

Abstract

The invention relates to a method for controlling at least one lighting device (10) of a motor vehicle (12), in which a projection (14) is determined and projected onto a predetermined position (10a) in a movement area (10) of the motor vehicle (12) depending on at least one projection parameter, wherein the projection (14) is simulated in an electronic computing device (22) by means of an optical detection device (26) detecting the movement area (10) and in a resulting image array linking the respective pixels of the image array required for the projection (14) with at least one point of contact (10b) on the movement area (10), and a back reflection (28) of the projection (14) from the individual points of contact (10b) to an eye position (30) of a driver (30a) is determined, and the projection (14) is projected onto the movement area (10) depending on the simulation.Furthermore, the invention relates to a lighting system (1).

Description

[0001] The invention relates to a method for controlling a lighting device of a motor vehicle according to the preamble of patent claim 1. Furthermore, the invention relates to a lighting system.

[0002] With the introduction of high-resolution lighting systems in vehicles, based for example on "Liquid Crystal Display" (LCD), "Digital Micromirror Device" (DMD) or "Micro LED" (µLED) technologies, increasingly complex light distributions can be projected. These technologies open up the possibility of displaying ever more sophisticated light patterns, symbols and animations in the vehicle's surroundings. The placement of these elements often depends on the current vehicle position and orientation as well as the relative position of the road surface. The projected light images are typically manufactured or designed to achieve the desired display in the basic state, in which the road surface is essentially flat in the illumination area and has no inclination to the vehicle plane.

[0003] However, if this relationship changes, whether due to changes in the road surface or relative movement of the vehicle, the headlights must adjust their display accordingly. This may mean rotating, shifting, or distorting the projected light patterns to maintain the desired display.

[0004] The object of the invention is to provide a method for controlling a lighting device of a motor vehicle and a lighting system by means of which a particularly stable and constant illumination and / or projection onto a surface, in particular a roadway or road surface, is made possible, in particular taking into account the condition of the roadway.

[0005] One aspect of the invention relates to a method for controlling a lighting device of a motor vehicle, in which a symbol to be projected is determined and, depending on at least one projection parameter such as luminous intensity and / or size of the symbol, the symbol is projected onto a predetermined position within the movement range of the motor vehicle. In particular, it is provided that the projection or the symbol to be projected is directed onto a surface, in particular a roadway or road surface. However, it is also possible to implement wall projection or other projections with different orientations.

[0006] For example, many headlight systems (lighting devices) include an automatic headlight leveling system designed to compensate for pitching movements through the mechanical rotation of the light units (light sources). These headlight systems are therefore specifically designed to ensure that the alignment of the headlights remains constant even when the vehicle's inclination changes. However, other types of movement and road conditions are not taken into account. These include lateral movements, changes in vehicle height due to loading or road irregularities, and changes in road surface conditions.

[0007] Other types of rotations and translations, as well as changes in road surface conditions, are not compensated for by this state-of-the-art headlight range adjustment. These include, for example, lateral movements of the vehicle, uneven road surfaces, or changes in the vehicle's height due to loading or unevenness. These factors can affect the alignment of the headlights and lead to suboptimal illumination of the road.

[0008] In order to achieve the object of the invention and thus provide a method for controlling a lighting device of a motor vehicle, which enables particularly stable and constant illumination and / or projection onto a roadway, in particular taking the road surface conditions into account, it is provided that a simulation of the symbol projection is first carried out. In this case, the movement area is detected using an optical detection device, and in a resulting image array, the pixels of the image array required for each symbol are linked to a point of occurrence on the trajectory. Furthermore, a back reflection of the projection from the individual points of occurrence to a driver's eye position is determined. The projection is then projected onto the movement area depending on the simulation.

[0009] The proposed method for controlling the lighting device, in particular a spotlight, is thus based on three-dimensional detection of the projection area. For each projection point, the respective three-dimensional positions of the background relative to the projecting lighting device (spotlight) are required, which can be detected, for example, by various optical detection devices or camera systems using methods such as shape-from-X, stereo matching, active triangulation, or lidar systems.

[0010] To maintain consistent image quality across different road surfaces, the desired view is first defined. This is based on the direct backprojection of light rays across the road into the eyes of the driver or operator. Assuming a level road surface, the calculated vehicle position and thus the position of the lighting device (headlights), and its known projection properties, the point of incidence on the theoretical road surface is calculated for each pixel in the current image array.

[0011] The path of the reflected light from the road to the driver's eye is then determined. This can be done by either specifying a static and average eye position in the passenger compartment, or by tracking the eye position using additional driver monitoring in the interior. In other words, this calculates how the symbol might appear to the driver / vehicle occupant under given conditions, particularly ideal conditions (e.g., high visibility without weather influences). This is later used to calculate, via a rear projection -> road -> headlight image array, what the digital image must look like in order to achieve a consistently clean display despite changing projection surfaces and relative headlight positions.

[0012] This approach adjusts the projection so that the projected symbol is sharp, stable, and particularly well-visible, especially for the driver, taking the road conditions into account. The combination of three-dimensional detection, theoretical roadway calculation, and back-reflection calculations allows the projection to be adjusted for improved visibility and safety while driving.

[0013] In an advantageous embodiment of the invention, the driver's eye position is detected by means of an optical detection device in the interior of the motor vehicle. This optical detection device can be an interior camera, designed, for example, as a mirror camera or another camera already present in the vehicle. It is capable of continuously capturing images of the interior, in particular images of the driver. The captured image data are then forwarded to a control system or an electronic computing device of or for the lighting device in order to incorporate them into the method for controlling the lighting device. By using this internal optical detection device, the movements and positions of the driver's eyes can be continuously recorded without the need to install additional external devices.

[0014] In an advantageous embodiment of the invention, the driver's eye position is determined by an average eye position. Multiple recordings or images are used, allowing for statistical compilation. This means that the eye position is not determined by individual, isolated images, but rather by analyzing multiple images to determine a representative and stabilized eye position. Thus, it is not necessary to process or capture each individual image and implement controls directly from it, which, in particular, reduces computing power.

[0015] In an advantageous embodiment of the invention, the driver's eye position is determined using a predefined static eye position. Characteristics of the driver, such as height, facial structures, or characteristic points in the eye area, are recorded or captured. Based on these features, a position representative of the driver's eye position is then determined. In this way, the eye position can be determined without having to directly record the driver. Here, too, it is not necessary to process or capture each individual image and implement controls directly from it, which also leads to a reduction in computing power.

[0016] In an advantageous embodiment, the projection is performed depending on a calculated simulation quality. Thus, the simulation is evaluated based on various criteria, such as the accuracy of data acquisition and / or the accuracy of 3D reconstruction and / or the reliability of obstacle detection. Depending on this evaluation, a decision is made as to whether and / or how the symbol projection is adjusted to provide the most realistic representation possible. This makes it possible to dynamically adapt the projection to changing conditions of the movement range.

[0017] In another advantageous embodiment of the invention, it is provided that, for improved simulation quality, the position and / or projection parameters of the projection or symbol are changed by means of the lighting device. This means that the lighting device is designed to actively adjust the position of the projected symbol within the range of movement of the motor vehicle as well as the projection parameters, such as the luminous intensity or the size of the symbol. Through such dynamic adjustment, the simulation of the symbol can be improved in real time, in particular to achieve a more precise and stable projection that better corresponds to actual driving conditions.

[0018] In another advantageous embodiment of the invention, it is provided to use a point cloud in the simulation. An estimation of the three-dimensional coordinates of the impact point on the surface is based on the spanning of a plane between three extracted points from the point cloud in order to define them as the impact surface. Subsequently, an intersection point of a line of sight with this plane description is calculated. In order to account for missing detection points in the 3D reconstruction, a maximum distance of these detection points from the line of sight is defined. Depending on the availability of points within this distance, different calculation variants are used. If only two points can be detected, the determination is made using the perpendicular projection to the straight line between these two points. If only one point can be detected, the determination is made using the direct perpendicular projection to the point.If no point is detectable, a known point is used to assume an unchanged road surface at that point. Alternatively, regardless of the distance, the three detection points with the shortest distance are always used and used to create the plane. This allows for the observation and transformation of each pixel to avoid artifacts caused by partial adjustment. However, incorrect adjustment due to inaccurate road surface detection must be accepted. Since the 3D acquisition system has a limited resolution, a mathematically precise 3D point of impact cannot be assigned to the visual or projection rays – this point must / can therefore be interpolated in various ways.

[0019] In another advantageous embodiment of the invention, an estimate of the three-dimensional coordinates of an impact point on a surface, where a plane is spanned between three extracted points from the point cloud, is defined as the impact area. Furthermore, the detection points can only be considered in the 3D reconstruction if they are defined below a maximum distance of these detection points from the line of sight. Accordingly, when considering individual 3D points, points with excessive distance from the line of sight can be discarded.

[0020] A further aspect of the invention relates to a lighting system for carrying out the method according to the previous aspect, comprising a lighting device and an electronic computing device that acquires, coordinates, and processes all data and information and outputs control signals for adjusting the position and / or the projection parameters of the symbol to the lighting device. The electronic computing device is connected, for example, to sensor devices with sensors that acquire real-time data about the vehicle's surroundings, including information about the road surface, vehicle movements, and other environmental conditions. Furthermore, the computing device comprises an algorithm for dynamically adjusting the position and / or the projection parameters of the symbol, which algorithm continuously evaluates the current conditions and makes appropriate adjustments.The lighting system is configured to automatically adjust the position and / or projection parameters of the symbol to provide uninterrupted projection. Additionally, the lighting system includes an interface for integrating machine learning or artificial intelligence to detect patterns and trends in driving behavior and environmental conditions and, based on these, further improve the control of the lighting device and the projection, particularly to maintain stability.

[0021] Further advantages, features, and details of the invention will become apparent from the following description of a preferred embodiment and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0022] Showing: Fig. 1 a side view of a motor vehicle and a road area from the motor vehicle and thus a movement area of ​​the motor vehicle, on which a projection is projected by means of a lighting device (headlights) of the motor vehicle; Fig. 2 a schematic representation of a line of sight to illustrate a possible simulation of projections and resulting output signals for controlling the light sources; Fig. 3 a schematic representation of a comparison of two reconstruction variants using the example of threshold detection by means of simulation; and Fig. 4 a schematic representation of a comparison of three pixel assignments using simulation.

[0023] In the figures, identical or functionally identical elements are provided with the same reference numerals.

[0024] Fig. 1 shows a side view of a motor vehicle 12 and a road area from the motor vehicle 12, and thus a movement area 10 of the motor vehicle 12 in the direction of movement x, onto which a projection 14 is projected by means of a lighting device 16 (headlights) of the motor vehicle 12. By means of the projection 14, information can be made visible to a driver outside an interior 18 of the motor vehicle 12, such as warning symbols, markings, or indications of obstacles or traffic situations.

[0025] A lighting system 1 or headlight system comprises, for example, two lighting devices 16 (headlights) with respective adjustable (position) light sources 20, which are adjusted, for example, via a respective adjustment device or headlight range control by means of control signals transmitted by an electronic computing device 22 of the lighting system 1.

[0026] For this purpose, many such lighting systems 1 or headlight systems include, for example, an automatic headlight range adjustment for the respective lighting devices 16, which is designed to compensate for pitching movements caused by the mechanical rotation of the lighting units or light sources 20. These lighting systems 1 are therefore particularly aimed at ensuring that the orientation of the lighting devices 16 and thus headlights and thus also of the light sources 20 remains constant even when the inclination of the motor vehicle 12 changes.

[0027] However, other types of movements and road conditions remain unconsidered. These include other types of rotations and translations, as well as changes in the road surface conditions within the movement area, which are also not compensated for by this headlight range adjustment known from the prior art. These include, for example, lateral movements of the motor vehicle 12, unevenness in the road surface and thus in the movement area 10, or changes in the height of the motor vehicle 12 due to loading or unevenness. These factors can impair the alignment of the lighting device 16 and thus of the light sources 20 and lead to suboptimal illumination of the road surface and thus of the movement area 10.

[0028] To solve this problem, it is now provided to use a method for controlling the lighting device 16 of the motor vehicle 12, in which method a symbol 24 or a projection 14 to be projected is determined and, depending on at least one projection parameter such as, for example, the luminous intensity and / or size of the symbol, the projection 14 is projected onto a predetermined position 10a in the movement area 10 of the motor vehicle 12. Here, the projection 14 or the symbol 24 is simulated by detecting the movement area 10 using an optical detection device 26 and, in a resulting image array, linking respective pixels of the image array required for the projection 14 or for the symbol 24 to a point of occurrence on the movement area 10.In addition, a back reflection 28 of the projection 14 or the symbol 24 from the at least one, in particular from the individual, points of occurrence to a driver's eye position 30 is determined. Finally, the projection 14 is projected onto the movement area 10 depending on the simulation, with the respective light sources 20 being adjusted for this purpose so that they reproduce the simulation on the movement area 10.

[0029] In other words, the optical detection device 26 uses sensors or cameras to monitor the movement area 10 of the motor vehicle 12 and to capture imaging data. This data is then transmitted or transferred to the electronic computing device 22 via a suitable interface. The received data is processed and analyzed in the electronic computing device 22 to perform the simulation of the projection 14 or the symbol 24 accordingly. This includes calculating the necessary adjustments to the light sources 20 based on the captured data.

[0030] After the simulation has been performed, control signals are generated that determine the required adjustments of the light sources 20. These control signals are then transmitted to the adjustment devices of the light sources 20 in order to adjust the actual light sources 20 accordingly and adapt them to the road surface conditions. This enables the desired projection 14 or symbol 24 to be projected as stably and sharply as possible onto the movement area 10 of the motor vehicle 12, in particular to improve visibility and safety. Not only mechanical adjustments are possible here, but above all adjustments to the digital 2D image of the projection 14. Mechanical adjustments, for example by the stepper motors, are also used or taken into account / calculated out; finer adjustments can thus be made for projection 14 in the 2D image.

[0031] Fig. Figure 2 shows a schematic representation of a viewing beam 34 to illustrate a possible simulation of projections and resulting output signals for controlling the light sources 20.

[0032] In a first step, the relationship between the lighting device 16 and the movement area 10 or the headlights and the roadway is to be defined.

[0033] The lighting device 16 or the headlight can be designed as an inverse camera with the associated projection matrix MProj,Headlights be defined.

[0034] This definition can be done either via an approximation with reduced accuracy or, more precisely, via a complete intrinsic calibration of the headlight.

[0035] If, in addition, the position and orientation of the headlight relative to the ideal movement area 10 or the ideal roadway are known from the dynamic calculation, a 3D point on the movement area 10 or on the roadway can be calculated for each pixel. For this purpose, a plane description 36 of the movement area 10 or the roadway is established in the coordinate system of the lighting device 16 or the headlight, and then the respective intersection point with individual beams 16a of the lighting devices 16 or individual headlight beams is calculated.

[0036] This results in the following for the three-dimensional projection point P': P′=pT×MProj,Headlight Once this projection point is known, the next step is to calculate the effect on the driver.

[0037] In a second step, the relationship between movement area 10 and eye position 30 or roadway to eye is to be defined.

[0038] For eye position 30, or the eye, a theoretical projection matrix similar to a camera can be designed to calculate pseudo-2D points. Eye position 30, or the eye, can be modeled as a pinhole camera, with an exemplary average focal length of 2 cm: Mproj,eye=[f′0000f′000010]=[0.0200000,02000010]

[0039] The 3D point from the previous step is then simply multiplied by the projection matrix to obtain the 2D point: p″=Mproj,eye×p′

[0040] With this point, the relationship between p and p'' is now known, which means which pixel of projection 14 or headlight projection is perceived in which field of vision of the eye. This information is used in the subsequent steps to calculate the required new position of the individual image sections based on the road surface conditions.

[0041] In a third step, the calculated beam paths and the depth reconstruction of the movement area 10 or the roadway are used to calculate the deviation of a current or actual situation from an ideal or target situation. Since there is no closed form of the surface description, for each point p'' in the eye, three points A, B, and C are selected from a point cloud 32 that are closest to the line of sight 34. A straight line equation of this light ray is determined by projecting the eye pixel p'' over the projection matrix of the eye. Mproj,Eye certainly.

[0042] In the Fig. 2 shows the procedure using a simplified example.

[0043] The three-dimensional coordinates of the impact point on the surface are then estimated on this basis: A plane is spanned between the three points extracted from the point cloud 32, which is defined as the impact surface. E:(xyz)=O→A÷λA→B+μA→C⇔E:a⋅x+b⋅y+c⋅z+d=0

[0044] The intersection point P'' of the line of sight 34 with this plane description 36 is then calculated. In order to account for missing detection points in the 3D reconstruction, a maximum distance of these detection points from the line of sight 34 can be specified. If only two points A, B can be detected within this distance, P'' is determined via a perpendicular projection to the straight line between these two points A, B. If only one point can be detected, the determination is made via the direct perpendicular projection to the point. If no point can be detected, point P' is used directly, and P'' = P' is set. This assumes an unchanged movement area 10 or roadway at this point.

[0045] Alternatively, the three detection points A, B, and C with the shortest distance are always used, regardless of the distance, and are used to create the plane. The advantage of this approach is that every pixel is considered and transformed, thus avoiding artifacts caused by only partial adjustment. The disadvantage, however, is incorrect adjustment due to inaccurate road detection.

[0046] Fig. Figure 3 shows a schematic representation of a comparison of two reconstruction variants using the example of threshold detection by means of simulation.

[0047] The influence on the adjustment is illustrated using the example of thresholds 38, 40 on the movement area 10 and on the roadway, respectively. Fig. Figure 3 shows two identical thresholds 38, 40, where no points were detected in the intermediate region. Depending on the calculation variant, different reconstructions result, such as in the respective middle regions 38a, 40a, where the first threshold 38 results in either a continuous middle region 38a or the second threshold 40 results in a split middle region 40a.

[0048] Regardless of the exact design of the calculation, the point P'' now serves as the target coordinate onto which the illumination device 16 of the original pixel p with its brightness value H must project.

[0049] Fig. Figure 4 shows a schematic representation of a comparison of three pixel assignments using simulation.

[0050] In the final step, the calculated 3D point is projected back into the image array of the illumination device 16 or the headlight via the inverse camera model. This is used to calculate, based on the existing headlight calibration, which pixel p_new must now receive the brightness value H of the original pixel p. pneu=Mproj, headlight⋅P″

[0051] The allocation is made hierarchically in stages: 1) All new points p_new are determined. 2) The point p_new with the shortest distance is assigned to each pixel of the image array. Only distances of less than half a pixel are accepted. 3) Unassigned points p_new are assigned to the next available pixel, as long as the distance is less than one pixel. All distances are first determined and then assigned in ascending order of distance. If the pixel is already occupied, no assignment occurs. 4) Unassigned points p_new are additionally assigned to the next pixel. 5) The resulting brightness value H and the projection distance of the individual pixels are calculated by weighting the individual brightness values ​​of the assigned points over the respective distance. If only one point is assigned, the brightness value H and the distance are adopted directly. 6) If free pixels remain without assigned points, the brightnesses and distances are calculated here by interpolating the brightnesses of the neighboring pixels.

[0052] Thus, in Fig. 4 shows the process of pixel assignment.

[0053] Before the brightness values ​​H are finally assigned, the brightness of the pixel is also adjusted using the calculated distance difference and the resulting attenuation of the light. H(pneu)=(rneuralt)2⋅H(p)

[0054] Thus, a new brightness value H is obtained for each pixel, which creates the light image to be projected for the image array.

[0055] In summary, the invention enables a consistent display of light images in the motor vehicle 12 or vehicle, regardless of the current dynamic situation and the surface in the movement area 10. This avoids false representations and unpleasant movements / distortions.

Claims

[1] Method for controlling at least one lighting device (16) of a motor vehicle (12), in which a projection (14) is determined and projected onto a predetermined position (10a) in a movement area (10) of the motor vehicle (12) as a function of at least one projection parameter, characterized by that the projection (14) is simulated in an electronic computing device (22) by detecting the movement area (10) by means of an optical detection device (26) and, in a resulting image array, respective pixels of the image array required for the projection (14) are linked to at least one point of occurrence on the movement area (10), and a back reflection (28) of the projection (14) from the individual points of occurrence to an eye position (30) of a driver is determined, and the projection (14) is projected onto the movement area (10) as a function of the simulation. [2] Method according to claim 1, characterized bythat the eye position (30) of the driver is detected by means of a further optical detection device (19) in the interior (18) of the motor vehicle (12). [3] Method according to one of the preceding claims, characterized by that the eye position (30) of the driver (30a) is determined by an average eye position. [4] Method according to one of the preceding claims, characterized by that the driver's eye position (30) is determined by a predetermined static eye position. [5] Method according to one of the preceding claims, characterized by that the projection (14) is carried out depending on a calculated quality of the simulation. [6] Method according to claim 5, characterized by that for an improved quality of the simulation the position and / or the projection parameters of the projection (14) are changed by means of the lighting device (16). [7] Method according to one of the preceding claims, characterized by that a point cloud (32) is used for the simulation. [8] Method according to claim 7, characterized by that an estimation of three-dimensional coordinates of an impact point on a surface, in which a plane is spanned between three extracted points (A, B, C) from the point cloud (32), is defined as the impact surface. [9] Method according to claim 8, characterized by that detection points are only taken into account in the 3D reconstruction if they are defined below a maximum distance of these detection points to the line of sight. [10] Lighting system (1) for carrying out a method according to one of the preceding claims, comprising a lighting device (16) and an electronic computing device (22) by means of which data and information can be acquired, coordinated and calculated, and control signals for adjusting a position and / or for adjusting projection parameters of the symbol can be transmitted to the lighting device (16).

Images