Method for projecting an image onto a road surface

Abstract

The invention relates to a method for projecting an image (B) onto a road surface in front of a vehicle (1) by means of at least one headlight (1.1) of the vehicle (1), wherein a situation-dependent and / or function-dependent image (B) is projected onto the road surface at least for an occupant of the vehicle (1) and / or for at least one road user in the immediate vicinity of the vehicle (1). According to the invention, - a direction of view of the at least one occupant of the vehicle (1) is determined at the vehicle, - a direction of view of the at least one road user is determined at the vehicle, and - depending on a current situation, the image (B) is projected onto the road surface in a targeted manner according to the determined direction of view of the at least one occupant and of the at least one road user.

Description

[0001] Mercedes-Benz Group AG Arnold Eschbach

[0002] July 3, 2025

[0003] Method for projecting an image onto a road surface

[0004] The invention relates to a method for projecting an image onto a road surface.

[0005] German patent DE 102048 120 330 A1 discloses an image projection system for vehicles. In this system, a computer is programmed to activate a light source to project a symbol from the vehicle to the outside. The computer is also programmed to activate the light source to modify the projection based on a determination that the vehicle's trajectory is changing.

[0006] German patent DE 102009 045 169 A1 describes a display device for a vehicle and a method for displaying image information in a vehicle. The method involves generating an image and projecting it onto a combiner surface, and detecting the head, eye, pupil, and / or gaze position of a driver. Furthermore, upon detection of a driver's head, eye, pupil, and / or gaze position, the device modifies the positions of image objects in a first object category so that the image objects appear analogous to real objects in the driver's field of vision.Positions of image objects of a second object category are changed independently of real objects in the driver's field of vision in such a way that the driver gets the impression that the image objects are located at a distance in front of the combiner surface which is less or greater than the distance of an image plane of the virtual image from the combiner surface.

[0007] DE 10 2009 048 619 A1 describes a method for controlling the headlights of a vehicle with a headlight and a sensor for detecting objects. In the event of a collision risk with an object, a special function of the 2024P02093WG is activated.

[0008] 2

[0009] The headlights are activated by alternately activating and deactivating a first light source of the headlight, consisting of one or more LEDs, and controlling a second light source, also consisting of one or more LEDs, in such a way as to generate flashes of light. An optical axis of the second light source is directed towards the object.

[0010] The invention is based on the objective of providing a method for projecting an image using at least one headlight of a vehicle.

[0011] The problem is solved according to the invention by a method which has the features specified in claim 1.

[0012] Advantageous embodiments of the invention are the subject of the dependent claims.

[0013] The inventive method for projecting an image onto a road surface in front of a vehicle by means of at least one headlight of the vehicle is characterized in that

[0014] - a situation- and / or function-dependent image is projected onto the road surface for at least one occupant of the vehicle and / or for at least one road user in the immediate vicinity of the vehicle, - a direction of view of at least one occupant of the vehicle is determined from the vehicle side,

[0015] - to determine a viewpoint, the position of an occupant's head is determined based on a seating position, including the position of a headrest of a vehicle seat on which the at least one occupant is seated, and / or based on the weight of the at least one occupant, whereby the seating position is weighted more heavily for determining the position of the head of the at least one occupant,

[0016] - a direction of view of at least one road user is determined from the vehicle side,

[0017] - depending on the existing situation, the image is projected onto the road surface in a targeted manner according to the determined viewing direction of at least one occupant and at least one road user and

[0018] - An image to be projected according to the current situation is stored in a memory unit of the vehicle, so that a perspective transformation for an image matrix of the headlight is determined for the projection of the image onto the road surface. By applying the method, the projected image is projected onto the road surface in front of the vehicle in such a way that the projected image can be recognized and perceived by a person, for example a driver of the vehicle, to whom a message displayed by means of the projected image is intended, according to its meaning, in particular unambiguously.Determining the perspective transformation for the headlight's image matrix for projecting the image onto the road surface, based on the image stored in the vehicle's memory unit and to be projected according to the current situation, makes it possible to project the image onto the road surface in such a way that the projected image is hardly distorted, if at all, from the respective viewing direction of a person, and can thus be recognized and perceived according to its meaning.

[0019] In one embodiment, if the projected image is to be seen by both the at least one occupant and the at least one road user, a geometric center, in particular an angle bisector, is used to position the projected image on the road surface between the determined viewing direction of the at least one occupant and the at least one road user. This ensures that the projected image is essentially, and in particular unambiguously, recognized and perceived by the occupant, especially the driver, and the at least one road user, for example, a pedestrian on one of the vehicle's lanes.

[0020] In a further embodiment of the method, the gaze direction of at least one vehicle occupant is determined using image data captured by an interior camera of the vehicle, and the gaze direction of at least one other road user is determined using image data captured by an exterior camera of the vehicle. The respective gaze direction is thus determined based on the captured image data, thereby detecting the actual gaze direction of at least one vehicle occupant and at least one other road user. The captured gaze direction forms the basis for projecting the image, allowing it to be recognized and perceived, according to its significance, by at least one vehicle occupant and / or by at least one other road user in the immediate vicinity of the vehicle.In a training course, a viewing point for projecting the image onto the road surface is determined based on the geometric center of the eyes of at least one occupant and / or at least one other road user. This viewing point serves as a reference point for projecting the image onto the road surface, ensuring that at least one occupant and / or at least one other road user recognizes and perceives the projected image. Determining this head position can also be used, particularly for plausibility checks.

[0021] Furthermore, in one embodiment, a distance between the front of the vehicle and a position of the image to be projected onto the road surface is specified. Based on the aspect ratio, in particular the display aspect ratio, of the projected image, an angular range for the eye of at least one occupant of the vehicle is determined. Target corner points of the projected image are then determined based on this angular range, particularly to reduce distortion of the projected image. This ensures, to a large extent, that the projected image is recognized and perceived.

[0022] In a possible advanced training, the target corner points are transmitted to at least one headlight, the pixel coordinates of these target corner points are determined, and a transformation matrix is ​​calculated based on these pixel coordinates. This matrix is ​​then used to adjust the projected image accordingly. Specifically, the projected image is adjusted using the transformation matrix so that it can be seen precisely, for example, by the driver of the vehicle.

[0023] Furthermore, the system is designed to determine, based on the current situation, to whom the image projected onto the road surface is intended. The image is then positioned accordingly. For example, a construction site warning image is projected onto the road surface in such a way that it is recognizable and its meaning is understandable to the driver, as well as to other occupants of the vehicle.

[0024] A lane change indicator, on the other hand, is an additional image projected onto the road surface for other road users, for example, in the area of ​​the lane into which the vehicle intends to change lanes. The projection is designed so that the image indicating the lane change can be recognized and perceived by both the vehicle's occupants and the other road user.

[0025] In a possible further implementation, the number of vehicle occupants is determined based on signals from a seat occupancy sensor and / or the status of each seatbelt buckle. This allows the system to determine whether, in addition to the driver, other occupants need to be considered when projecting a situation-specific image onto the road surface.

[0026] Exemplary embodiments of the invention are explained in more detail below with reference to a drawing.

[0027] This shows:

[0028] Fig. 1 schematically shows a perspective view of a vehicle and an image projected onto a road surface.

[0029] The single figure shows a roadway section F with a vehicle 1 traveling in lane F1. An image B, in this embodiment a construction site warning, is projected onto the roadway surface at a predetermined distance in front of the vehicle 1 by means of at least one headlight 1.1 of the vehicle 1. In particular, the projected image B alerts the driver of the vehicle 1 that there is a construction site some distance away. Alternatively or additionally, the projected construction site warning can alert the driver of the vehicle 1 that they are in a construction zone and are currently driving through it. For this purpose, the headlights 1.1 of the vehicle 1 are high-resolution and therefore have a number of so-called LED pixels.

[0030] In general, such an auxiliary and warning projection of images B, in particular symbols, by means of high-resolution headlight systems is known, wherein the projection takes place regardless of the position of a driver and / or other occupants of a vehicle 1 and / or regardless of a road user in the immediate vicinity of a vehicle 1, so that depending on the eye and / or head position of a person, it is possible that they may not be able to recognize or perceive the image B.

[0031] The following describes a method whereby image B is projected onto the road surface in such a way that it is possible for one or more persons, to whom image B, and in particular its meaning, is intended, to recognize the projected image B essentially accurately and thus perceive its meaning. In particular, the projection of image B onto the road surface serves to assist or warn the occupants of vehicle 1 and / or road users in the immediate vicinity of vehicle 1, especially those in front of vehicle 1.

[0032] Projectable images B can be designed in various ways, but they are primarily divided into two categories. One category includes projectable images B intended for the occupants of vehicle 1, particularly the driver, while the other category includes projectable images B intended for other road users in the vicinity of vehicle 1.

[0033] Projectable images B intended for occupants, in particular a driver of vehicle 1, include, for example, a reference to a work site / construction site, as shown in Figure 1, and a warning for slippery conditions.

[0034] A projectable image B intended for road users and occupants of vehicle 1 is a lane change indication, whereby a so-called spotlight with an additional warning symbol can also be generated by means of at least one headlight 1.1 of vehicle 1 in order to mark a road user, since this is on or near the lane F1 of vehicle 1 or in a comparatively insufficiently visible section of the roadway section F.

[0035] With regard to image projections intended for the driver of vehicle 1, or for the driver and a passenger, or for all occupants of vehicle 1, the projection of each image B is adapted depending on the occupants of vehicle 1 to whom the projected image B is intended, as described below. 2024P02093WG

[0036] 7

[0037] For the projection of an image B intended for the driver of vehicle 1, the driver's eye position is determined based on signals captured by sensors installed in the vehicle. For example, the driver's gaze direction is determined using image signals captured by an interior camera, such as an eye-tracking camera, and / or time-of-flight sensors. The driver's viewing point is then defined in relation to the geometric center of both eyes, i.e., one pair of eyes.

[0038] Alternatively or in addition to determining the driver's direction of gaze based on captured image signals, the position of the driver's head is determined, whereby the viewing point is also specified based on the geometric center.

[0039] Alternatively or additionally, the position of the driver's head in vehicle 1 is estimated using two pieces of information obtainable from the vehicle. One piece of information is the seat setting, including the headrest position, of the vehicle seat in which the driver is seated, while the other piece of information is the driver's weight. Greater emphasis is placed on the seat setting, for which a weighting factor may be specified. If the weighting factor is 1, only the seat setting is considered when estimating the driver's head position.

[0040] Regarding the driver's weight, their height is determined using a Body Mass Index (BMI) table, with an average BMI of 21.75 for a person of normal weight. The height is then used to determine the correct vehicle seat shape for a specific model series, specifying the vertical, longitudinal, and lateral positions.

[0041] If information about the vehicle seat setting is available, a seat setting value can be used as the basis for a more specific determination of the driver's head position. If the vehicle does not have seat setting information, a determined weight value is used exclusively to determine the driver's head position; conversely, if no weight value is available, only the seat setting is used to determine the head position. 2024P02093WG

[0042] 8

[0043] If neither the seating position nor the driver's weight is available, the position of the driver's head for vehicle 1 is determined based on a typical average position in the direction of a vehicle vertical axis, a vehicle longitudinal axis and a vehicle transverse axis that is commonly used.

[0044] The possible projectable images for supporting and / or warning the driver, other occupants of vehicle 1, and / or other road users in the immediate vicinity of vehicle 1 are stored in a memory unit of vehicle 1, possibly with additional information such as the aspect ratio, in particular the display aspect ratio, of the projected image B. Based on this, a perspective transformation for an image matrix of the headlight system can be determined so that the driver of vehicle 1 sees the projected image 1 exactly.

[0045] For example, a position, that is, a distance between the front of a vehicle and a position of the image B to be projected onto the road surface, is specified. Based on the aspect ratio and the determined viewing direction of the driver of vehicle 1, an angular range for the driver's eye is known, so that target vertices E of the image B to be projected onto the road surface can be determined. Subsequently, the determined target vertices E are transmitted to the headlights 1.1 of vehicle 1, thus determining the pixel coordinates of the target vertices E. This allows the transformation matrix to be calculated and the projected image B to be adjusted.

[0046] If the projected image B is intended for the passenger in addition to the driver, the passenger's viewing direction and / or the position of his head, as described in relation to the driver, is determined.

[0047] The determination of the passenger's gaze direction and / or head position only occurs if, for example, based on image signals captured by the vehicle, signals from a seat occupancy sensor, and / or the status of a seatbelt buckle assigned to the passenger, it is determined that a passenger is present in vehicle 1. Subsequently, it is determined whether the projected image B can be split or duplicated. If the projected image B can be duplicated, the projection is duplicated so that one image B is projected onto the road surface in the driver's line of sight and the same image B is projected onto the road surface in the passenger's line of sight. The respective projections are adjusted according to the determined gaze direction so that the projected image can be recognized and perceived by both the driver and the passenger.The two images B are projected onto the road surface by means of a spotlight 1.1, whereby it may also be possible to split the projection of both images B using both spotlights 1.1.

[0048] In an alternative or additional implementation, image-dependent, and especially symbol-dependent, metrics are used. These evaluate distortions at individual points and sections of the projected image B and project them onto the road surface in such a way that comparatively important areas of image B are depicted as ideally as possible for viewers, especially the driver and passenger, while deviations in other areas are accepted. In such a case, different results may be obtained than with an adjustment based on the geometric center.

[0049] If there are more than one occupant besides the driver and the front passenger in the vehicle 1, an average is calculated, particularly with regard to the geometric center, of the gaze directions of the eye pairs of all occupants of the vehicle 1, so that each occupant can see the projected image B with the least possible distortion.

[0050] The occupants, particularly those in the rear of vehicle 1, are identified using captured image signals, for example, from the interior camera, signals from a seat occupancy sensor, and / or the status of the seatbelt buckle. If the projected image B cannot be duplicated, the geometric center can be determined with respect to the viewing direction of all occupants of vehicle 1. Alternatively, it may be possible to consider only the viewing direction of an occupant sitting in the center of the rear, as this occupant is normally the only one who can see the image B projected onto the road surface. The image projection can also be duplicated, as described above, in which case a potential occupant sitting in the center is assigned to either the left or right longitudinal section of the vehicle.Here too, the adjustment of the projected image B can be refined using symbol-dependent metrics.

[0051] A projection of an image B, which is also intended to be visible to a road user in the vicinity of vehicle 1 and / or whose recognizability by other road users gives vehicle 1 an advantage, especially with regard to the other road users, is described below.

[0052] One embodiment of this is Spotlight, by means of which pedestrians and two-wheelers are marked who are on or relatively close to the lane F1 of vehicle 1 and / or in an insufficiently visible section of the roadway section F.

[0053] For example, a light band is projected from vehicle 1 towards the road user, with a warning symbol projected onto this light band. For instance, a flashing exclamation mark can be integrated into the projected light band, its position adjusted according to the road user's head position. For example, from vehicle 1's perspective, the exclamation mark appears as if it is positioned on the road user's head. The projection calculation is performed as described above.

[0054] Another embodiment of the projection of an image B, which is intended for the occupants of vehicle 1 and the road users in the vicinity of vehicle 1, is the lane change indication, for example in the highly automated or autonomous ferry operation of a vehicle 1.

[0055] When a vehicle changes lanes, for example to the right, an image B, such as an animated graphic, is projected onto the road surface. Specifically, this image B is projected onto a lane marking between the vehicle lanes F1 involved in the lane change. 2024P02093WQ

[0056] 11

[0057] This projection can be adjusted, for example, so that the image to be projected, B, is projected onto the lane to which the vehicle is to change lanes, relative to the position of another vehicle behind or beside vehicle 1. Alternatively, the projection can be based on the geometric center of the determined viewing directions of the occupants of vehicle 1.

[0058] The position of the other vehicle is determined using sensor fusion. If so-called bounding boxes are present, the position of the driver's head in the other vehicle is placed within them based on their dimensions.

[0059] The bounding box provided by the sensor fusion represents a cuboid within which the vehicle is located. The direction of travel, speed, etc., are usually also known. Here, it can be defined schematically where the average head positions of the drivers lie within this "vehicle cuboid." This naturally depends on the vehicle type but can be approximated, for example, with position 0.7 corresponding to the longitudinal dimension, 0.8 to the vertical dimension, and 0.2 or 0.8 to the lateral dimension (depending on left- or right-hand traffic), with each dimension normalized to 1.

[0060] This will not be exactly correct, but the errors will be almost irrelevant for visibility / perception, as they are small enough.

Claims

2024P02093WG 12 Mercedes-Benz Group AG Arnold Eschbach July 3, 2025 Patent claims 1. Method for projecting an image (B) onto a road surface in front of a vehicle (1) by means of at least one headlight (1.1) of the vehicle (1), wherein - a situation- and / or function-dependent image (B) is projected onto the road surface for at least one occupant of the vehicle (1) and / or for at least one road user in the immediate vicinity of the vehicle (1), - the direction of view of at least one occupant of the vehicle (1) is determined from the vehicle side, - to determine a viewpoint, the position of an occupant's head is determined based on a seating position, including the position of a headrest of a vehicle seat on which the at least one occupant is seated, and / or based on the weight of the at least one occupant, whereby the seating position is weighted more heavily for determining the position of the head of the at least one occupant, - a direction of view of at least one road user is determined from the vehicle side, - depending on the existing situation, the image (B) is projected onto the road surface in a targeted manner according to the determined viewing direction of at least one occupant and at least one road user and - in a storage unit of the vehicle (1) an image (B) to be projected according to the present situation is stored, so that a perspective transformation for an image matrix of the headlight (1.1) is determined for the projection of the image (B) onto the road surface.

2. Method according to claim 1, characterized in that if the image (B) to be projected is to be seen by both the at least one occupant and the at least one road user, a geometric center is used to position the image (B) to be projected onto the road surface between the determined viewing direction of the at least one occupant and the at least one road user.

3. Method according to claim 1 or 2, characterized in that the direction of view of the at least one occupant of the vehicle (1) is determined on the basis of image data captured by an interior camera of the vehicle (1) and the direction of view of the at least one road user is determined on the basis of image data captured by an exterior camera of the vehicle (1).

4. Method according to one of the preceding claims, characterized in that a viewing point for projecting the image (B) onto the road surface is determined on the basis of a geometric center of a pair of eyes of the at least one occupant and / or the at least one road user.

5. Method according to one of the preceding claims, characterized in that a distance between a vehicle front and a position of the image (B) to be projected onto the road surface is specified, an angular range for the eye of at least one occupant of the vehicle (1) is determined on the basis of an aspect ratio of the image (B) to be projected, wherein target corner points (E) of the projected image (B) are determined on the basis of the angular range.

6. Method according to claim 5, characterized in that the target vertices (E) are transmitted to the at least one spotlight (1.1), pixel coordinates of these target vertices (E) are determined and a transformation matrix is ​​determined on the basis of the pixel coordinates and the image (B) to be projected is adjusted accordingly on the basis of the transformation matrix.

7. Method according to one of the preceding claims, characterized in that, based on the present situation, a relevance is determined on the vehicle side as to whom the image (B) projected onto the road surface applies.

8. Method according to one of the preceding claims, characterized in that a number of occupants of the vehicle (1) is determined on the basis of detected signals from a respective seat occupancy sensor and / or on the basis of a respective seat belt buckle status.

Images