Method for generating an image representation, control device, driver assistance system and vehicle

Abstract

The invention relates to a device for generating an image representation (20), wherein the image representation (20) is generated from at least one image (21) and at least one overlay (11) overlaid over the image (21), wherein the overlay (11) comprises at least one graphical element (12), which has, in a colour channel of the overlay (11), an inner region (13) and a first border region (14) at least partially surrounding the inner region (13), wherein a second border region (18) at least partially surrounding the inner region (13) is allocated to the at least one graphical element (12) in an alpha channel of the overlay (11), wherein the second border region (18) has a transparency which is greater than the transparency of the inner region (13).

Description

[0001] Description

[0002] Method for generating an image display, control device, driver assistance system and vehicle

[0003] The invention relates to a method for generating an image representation, wherein the image representation is generated from at least one image and at least one overlay superimposed on the image. Furthermore, the invention relates to a control device, a driver assistance system, and a vehicle.

[0004] Modern vehicles use image representations generated from one or more camera images, for example, to depict a portion of the vehicle's surroundings. One or more overlays can be superimposed on the camera images to supplement or augment the surroundings representation with additional information. In this way, information such as driving trajectories, target positions, or warning markings can be inserted into the surroundings view to make it easier for the driver to maneuver the vehicle based on the camera images. Various methods for generating surroundings representations for vehicles are known from the prior art.

[0005] DE 10 2016 225 066 A1 describes an all-round vision system for a vehicle. The all-round vision system comprises at least one real camera for capturing an image of the vehicle's surroundings from a non-centered position, an image processing unit, and a display unit. The display unit is configured to display a projected representation of the image captured by the real camera and at least one geometric shape, wherein the geometric shape is displayed as an overlay over the image from the real camera.

[0006] DE 10 2014 208 663 A1 discloses a camera surround view system for a vehicle with at least one vehicle camera. The vehicle camera provides camera images, which are processed by a data processing unit to generate a live video image stream that is displayed on a display unit of the vehicle.

[0007] The camera surround view system loads local cloud information data regarding an object displayed in the live video stream from a data network and displays the received local cloud information data as an overlay in the displayed live video stream.

[0008] When superimposing an image, such as a camera image, with an overlay, it is desirable for the resulting image display to have the best possible image quality. However, particularly when using the image display in a vehicle, the computing power available to generate the image display may be limited, so there is still a need for computationally efficient image display generation with the best possible image quality.

[0009] The invention is therefore based on the object of specifying an improved method for generating an image representation from an image and an overlay superimposed on the image, which method in particular has improved image quality with only low requirements on the computing resources required for generation.

[0010] To achieve this object, it is provided according to the invention in a method of the type mentioned at the outset that the overlay comprises at least one graphic element which has an inner region and a first border region at least partially enclosing the inner region in a color channel of the overlay, wherein a second border region at least partially enclosing the inner region is assigned to the at least one graphic element in an alpha channel of the overlay, wherein the second border region has a transparency which is greater than the transparency of the inner region.

[0011] The image representation can, for example, be an environmental representation that depicts at least a partial area of ​​the vehicle's surroundings. For example, the image representation can be a view of the vehicle from above or at an angle. It is also possible for it to be a representation of an area in front of, to the side, and / or behind the vehicle.

[0012] The image display can use one or more images generated by one or more cameras in the vehicle. For example, one or more images generated by a front camera, at least one side camera, and / or a rearview camera can be used. In particular, the image display can be generated continuously for a temporal sequence of images or for the images of a video stream.

[0013] The image representation generated by the method according to the invention is described, in particular, by a plurality of image points or pixels. In other words, the generated image representation can be digital image information, which can be reproduced, in particular, on a display device, such as an electronic display or the like. The image used to generate the image representation can also be in digital form and be part of a digital image sequence or a digital video stream.

[0014] The overlay superimposed on the image to generate the image display comprises at least one line and can, in particular, be part of driver assistance information displayed to a driver of a vehicle. The overlay can, in particular, be digital image information superimposed on the at least one image. The position, orientation, or perspective of the overlay can be selected or determined, in particular, depending on the image content of the at least one camera image.

[0015] The overlay comprises one or more pieces of graphic information stored in a color channel of the overlay or in a piece of digital image information representing the overlay. The overlay can be in the "YUV420 Semiplanar" color format, for example, or a comparable color format. The overlay comprises one or more graphic elements, each of which can be, for example, a line, an arc segment, a curve bent once or multiple times, or a comparable geometric shape. In particular, the overlay can comprise a plurality of graphic elements, which together can form one or more complex graphics and / or other elements, for example numbers and / or text.

[0016] In the color channel, the graphic element of the overlay comprises the inner area and the first border area, which completely or at least partially encloses the inner area. The first border area can describe the edge section of the graphic element and already be a component of the graphic element or overlay. Alternatively, the graphic element of the overlay can initially only comprise the inner area, with the first border area being assigned to the graphic element or overlay when the image representation is created.

[0017] The graphic element comprises the second border region in an alpha channel in which transparency information of the overlay is stored. The second border region encloses the inner region completely or at least partially and has a transparency that is greater than the transparency of the inner region. The second border region provides the edge region of the graphic element with transparency in the image representation at least partially. In particular, the first border region of the graphic element is displayed at least partially transparent in the image representation due to the second border region assigned to the graphic element.This means that a pixel of the first border area in the image display not only consists of the color information encoded in the color channel, but that color information from a pixel of the image located at the same position is also partially reproduced in the image display, resulting in an intermediate color between the color of the first border area and the background. In this way, an even smoother transition is created, particularly at the edge area of ​​the overlay, resulting in improved anti-aliasing and thus an improved appearance of the image display. The occurrence of step-like artifacts at the edge of the graphic element and / or other artifacts such as false colors in the edge area of ​​the graphic element or similar can thus be advantageously avoided.

[0018] Advantageously, by using the graphic element with the second border area in the alpha channel, further calculations for anti-aliasing can be omitted, so that only comparatively low computing power is required to generate the image representation. This makes the method particularly suitable for applications where only limited computing power is available, but the fastest possible provision of image information is still desired, for example, when generating an environmental representation from one or more camera images in a vehicle's driver assistance system.

[0019] In a preferred embodiment of the invention, it can be provided that the first border region has the same color as the inner region or that the first border region has a color which is different from a color of the inner region. With a first border region which has the same color as the inner region, an improved representation in the edge region is brought about only by the transparency assigned to this region by means of the second border region. Alternatively, however, the first border region can also have a color which is different from the color of the inner region. For example, the first border region can have a lighter color than the inner region. In this way, a further improvement in the edge representation of the overlay in the image representation can be achieved, at least for some types of image representations or for some types of images.

[0020] According to the invention, it can be provided that the second border region is just as wide as the first border region or that the second border region has a smaller width than the first border region. This applies in particular to the case in which the first border region is only assigned to the graphic element when the image representation is generated and / or to the case in which the first border region has a different color than the inner region. A second border region which has the same width as the first border region can in particular have a geometry corresponding to the first border region and a spatial position in the overlay corresponding to the first border region. As a result, the second border region has the effect in particular that the entire first border region of the graphic element is assigned transparency.

[0021] A second border area, which is narrower than the first border area, can in particular have transparency on an outer edge of the first border area, ie the edge of the first border area opposite the inner area of ​​the graphic element.

[0022] In a preferred embodiment of the invention, it can be provided that the image representation is described by a plurality of pixels, wherein the second border region has an average width of less than five pixels, in particular between one and two pixels inclusive. The pixels in the alpha channel correspond, in particular in size, arrangement, and number, to the pixels in the color channel.

[0023] The width refers to the extent of the second border area between an inner edge, which adjoins the inner area, and an outer edge of the overlay, which points to an image area of ​​the image display in which the image is no longer overlaid by the overlay. The width of the first border area can, for example, be between one and ten pixels inclusive. The width of the inner area can also be between one and several pixels and depends on the desired size or line width of the graphic element in the image display.

[0024] According to the invention, the graphic element can be linear, with the width of the second border region being selected as a function of an angle between the graphic element and the edges of a pixel grid used to describe the image information. In particular, with a linear graphic element, stair-stepped artifacts can arise if the linear element does not run horizontally or vertically, or not parallel to one of the edges of the pixel grid used to describe the image representation.

[0025] It is therefore possible to choose the width of the second border area depending on the angle between the line and one of the edges. In particular, the largest width for the second border area can be chosen for an angle of 45°. For angles between 0° and 45°, the width can increase with the angle, and for an angle between 45° and 90°, the width can decrease again accordingly. This advantageously allows a wider second border area and thus a smoother transition to be created, especially for those graphic elements that require a lot of anti-aliasing due to their course in the image display, than for graphic elements that have no or only very few artifacts due to their at least approximately parallel course to one of the edges of the pixel grid. This can advantageously save computing power when generating the image display.

[0026] In order to save further computing power when generating the image representation, it can be provided according to the invention that, when there are several linear graphic elements in the overlay, a second border area is assigned only to those linear graphic elements which do not run parallel to one of the edges of a pixel grid used to describe the image information.

[0027] According to the invention, the transparency of the second border region can be between complete transparency and a transparency assigned to the inner region. This results in a smooth transition of the display starting from the transparency of the inner region. It is possible for the inner region to be assigned a transparency of zero in the alpha channel, i.e., complete opacity, or for the inner region to also have a transparency greater than zero. Depending on this, the inner region of the graphic element can completely or partially overlay the image content of the image when generating the image display.

[0028] In a preferred embodiment of the invention, the second border region can have a transparency gradient, with the transparency increasing along the width of the second border region from the inner region to the outer edges of the second border region. The increase in transparency can be linear or defined by a function, e.g., exponential, logarithmic, or similar. By using a transparency gradient with increasing transparency, a particularly smooth and harmonious transition can be created.

[0029] For a control device according to the invention, it is provided that it is configured to carry out a method according to the invention. The at least one image can be transmitted to the control device, for example, from at least one camera. The control device can determine the overlay depending on an image content of the at least one image and / or depending on further information, for example, driving state information describing a driving state of a vehicle. It can be provided that the control device is configured to determine an image content of the at least one image. The control device can be configured to transmit the generated image representation to a display device.

[0030] A driver assistance system according to the invention comprises at least one camera, a display device and a control device according to the invention, wherein the at least one camera is designed to provide the at least one image and the display device is designed to reproduce the image representation.

[0031] A vehicle according to the invention is provided with a driver assistance system according to the invention. All advantages and configurations described above with reference to the method according to the invention also apply accordingly to the control device according to the invention, the driver assistance system according to the invention, the vehicle according to the invention, and vice versa. The advantages and configurations described for the control device according to the invention, the driver assistance system according to the invention, and the vehicle according to the invention are also analogously transferable to the other subject matters of the invention.

[0032] Further advantages and refinements will become apparent from the exemplary embodiments described below and from the drawings. These are schematic representations and show:

[0033] Fig. 1 shows an embodiment of a vehicle according to the invention,

[0034] Fig. 2 is a flowchart of an embodiment of a method according to the invention,

[0035] Fig. 3 shows an embodiment of a graphic element of an overlay in the color channel,

[0036] Fig. 4 a representation of the pixels of the graphic element of the overlay in the color channel,

[0037] Fig. 5 shows an example of the graphic element in the alpha channel and

[0038] Fig. 6 shows an example of an image representation generated using an embodiment of the method according to the invention.

[0039] Fig. 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck, or another type of commercial vehicle. It is also possible for the vehicle 1 to be a non-motorized vehicle such as a trailer or a combination consisting of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle, such as a tram or the like, or a robot.

[0040] The vehicle 1 comprises an exemplary embodiment of a driver assistance system 2, which comprises one or more cameras 3 and a control device 4. In the present exemplary embodiment, the vehicle 1 comprises four cameras 3, which form a camera arrangement designed as a surround-view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear-view camera of the vehicle 1, and a third camera 7 and a fourth camera 8 are each arranged as a side camera of the vehicle 1. The control device 4 can be designed, for example, as a microcontroller, as a processor, or as another type of computing device.

[0041] The driver assistance system 2 further comprises a display device 9, via which graphic information can be presented to a driver or a user of the vehicle 1. The display device 9 can, for example, be one or more screens or displays arranged in an interior of the vehicle 1.

[0042] Using the cameras 3, image data can be generated from a sub-area of ​​the surroundings of the vehicle 1. The cameras 3 are connected to the control device 4 via a communication link 10. The communication link 10 can, for example, comprise a plurality of point-to-point connections or be a bus connection such as a CAN bus or the like. Images or image data are transmitted, in particular continuously, from the cameras 3 to the control device 4 via the communication link 10.

[0043] The control device 4 is configured to execute a method for generating an image representation, wherein the image representation is generated from at least one image and an overlay superimposed on the image. For this purpose, the control device 4 uses one or more of the camera images recorded by the cameras 3 of the vehicle 1 as the at least one image.

[0044] The image display can, in particular, be an environmental display that reproduces at least part of the surroundings of the vehicle 1. The image display can, in particular, be a surround view that displays the vehicle 1 and part of the surroundings of the vehicle 1 from a virtual camera position. For this purpose, the images generated by the cameras 3 and / or the overlay can be projected, for example, onto a projection surface, from which the image display can be derived from an arbitrarily positioned virtual camera position.

[0045] Alternatively, the image display can also be a view using only one image from one of the cameras 3. The image display can, for example, be a front view based on the images from the front camera 5, a rear view based on the images from the rear view camera 6, or a side view based on the images from one of the side cameras 7, 8.

[0046] Regardless of which of the cameras 3 provides the images for the image display, the image display can be a video image based on continuously generated images from one or more of the cameras 3. The image display can thus essentially reproduce at least a portion of the vehicle's surroundings in real time.

[0047] In addition to the at least one image, the image display is also based on an overlay that is superimposed on the at least one image. The overlay can be understood as further image information that is superimposed on the image or the camera image. The overlay comprises at least one graphic element that serves to display further information in the image display. Fig. 2 shows a flowchart of a method for generating image information, which is carried out by the control device 4. In a step S1, one or more images from one or more cameras 3 are transmitted to the control device via the communication connection 10, for example an image from the rear-view camera 6.

[0048] In step S2, the control device 4, or alternatively another control device of the vehicle 1 that is communicatively connected to the control device 4, determines an overlay. The overlay can be determined based on an image content of the image and / or based on at least one driving state parameter of the vehicle 1 that describes at least one driving state of the vehicle 1.

[0049] The image content of the at least one image can be determined, for example, by the control device 4 and / or by another control device of the vehicle 1 using artificial intelligence or comparable methods. The at least one driving state parameter can be, for example, a direction of travel of the vehicle 1, a speed of the vehicle 1, a steering angle of the vehicle 1, or the like. The driving state parameter can be transmitted, for example, from at least one sensor device of the vehicle 1 to the control device 4, in particular via the communication connection 10.

[0050] Fig. 3 shows an embodiment of an overlay 11. The overlay 11 comprises at least one graphic element 12. Fig. 3 shows a color channel of the overlay 11. In this color channel, the graphic element 12 comprises an inner region 13 and a first border region 14 which at least partially surrounds the inner region 13, wherein the first border region 14 has a color which corresponds to the color of the inner region 13 or which is different from a color of the inner region 13. In the present case, the first border region 14 has, for example, a different color than the inner region 13, wherein the different colors of the inner region 13 and the first border region 14 are schematically represented in the color channel by different hatching. Alternatively, it is possible for the first border region 14 to have the same color as the inner region 13.

[0051] The first border region can be assigned to a graphic element 12 of the overlay 11 during the method for generating the image representation, for example by the control device 4. Alternatively, the first border region 14 can also already be contained in the overlay 11 used for generating the image representation or can describe a section of the graphic element 12 contained in the overlay.

[0052] The at least one graphic element 12 shows, in particular, information associated with a driver assistance system of the vehicle 1. In principle, the graphic element 12 can be a part of any desired piece of graphic or written information, which is superimposed on the at least one image as an overlay 11 and thus augments it in the image representation. For example, the graphic element 12 can be a part of an anticipated vehicle trajectory of the vehicle 1, a marking of a parking space in the vicinity of the vehicle 1, a part of a text displayed in the image representation, or the like.

[0053] In Fig. 4, the overlay 11 is shown as a pixel graphic formed by a plurality of pixels 15. In the present case, the graphic element 12 is linear and extends at an angle a to one of the edges 16 of a rectangular pixel grid 17, which is also used below to describe the image representation.

[0054] Due to the angle a between the linear graphic element 12 and the edge 16 of the pixel grid 17, step-like artifacts occur at the edge of the graphic element 12 when the graphic element 12 is displayed. The perception of this artifact can, for example, be at least partially reduced in the image representation by the first border region 14 having a different color than the inner region 13. For example, the color of the border region 14 can be lighter than the color of the inner region. Fig. 5 shows an alpha channel of the overlay 11. In the alpha channel, the control device 4 assigns transparency values ​​to the individual pixels 15 of the overlay. The graphic element 12 comprises a second border region 18 in the alpha channel, which at least partially surrounds the inner region 13 of the graphic element 12.The transparency described by the second border region 12 is schematically shown as hatching of the respective pixels 15 of the second border region 12.

[0055] The width of the second border region 18 can correspond to the width of the first border region 14 or be narrower than this. In the illustrated embodiment, the first border region 14 in the color channel of the overlay and the second border region 18 in the alpha channel of the overlay 11 each have the same width of one pixel. The second border region 14 can, in particular, have an average width of less than five pixels, in particular between one and two pixels.

[0056] The inner region 13 can also be assigned a transparency in the alpha channel, with the transparency of the second border region 18 preferably lying between complete transparency and the transparency assigned to the inner region. The pixels 15 of the second border region 18 can each have the same transparency, or different transparency values ​​can be assigned to the pixels of the second border region 18.

[0057] The second border region 18 can also have a transparency gradient, wherein the transparency increases in particular along the width of the second border region 18 from the inner region 13 to the outer edges 19 of the second border region 18. A transparency gradient can be provided in particular for those second border regions 18 which have a width of more than one pixel 15. Preferably, the transparency increases in each case starting from a center of the inner region 13 to the outer edges 19 of the second border region 18, i.e. along a positive and negative direction s, which is perpendicular to the extension direction t of the graphic element 12, as shown in Fig. 5. The transparency gradient can be described by a function and can be, for example, a linear, exponential or logarithmic gradient.

[0058] If there are several linear graphic elements 12 in the overlay 11, a second border area 18 in the alpha channel of the overlay 11 can preferably only be assigned to those linear graphic elements 12 which do not run parallel to one of the edges 16 of the pixel grid 17 used to describe the image information.

[0059] In step S3 of the exemplary embodiment of the method for generating the image representation shown in Fig. 2, an image representation is generated by overlaying the at least one image with the overlay 11. For this purpose, the overlay 11 is inserted into the at least one image at a position assigned to the overlay 11. The position and / or geometry of an overlay 11 to be inserted can, as described above, be determined depending on the interpretation of the image content and / or depending on the at least one driving state parameter.

[0060] In step S4 of the method, the image representation is displayed on the display device 9 of the vehicle 1. At least part of the first border region 14 is assigned transparency due to the second border region 18, resulting in a smoother transition between the image and the overlay 11 in the image representation. This improves the perceptibility of the overlay 11 in the image representation by a driver or a user of the vehicle 1, in particular the perceptibility in relation to the image content of the at least one image also contained in the image representation.

[0061] Fig. 6 shows an embodiment of an image display 20. This comprises an image

[0062] 21 of the rear view camera 6 of the vehicle 1, which has been superimposed with an overlay 11. In the present case, the overlay 11 comprises a plurality of linear graphic elements 12, which mark a parking space 22 located in the vicinity of the vehicle 1. In addition, the overlay 11 also comprises two curved graphic elements 23, which mark a corridor 24 through which the vehicle 1 must travel to reach the parking space 22. The geometry of the overlay 11, for example the geometry of the parking space 22, can be selected and displayed depending on an interpretation of the image content. Additionally or alternatively, the geometry of the overlay 11, in particular the geometry or the course of the curved graphic elements 23, can also be dependent on a driving state parameter, for example a set steering angle or the like.

[0063] In order to achieve anti-aliasing, when generating the image representation 20, at least some of the linear graphic elements 12 and the arcuate graphic elements 23 are each assigned a first border region 14 in the color channel of the overlay 11, the color of which border region differs from an inner region 13 of the respective graphic element 12, 23. Alternatively, the respective first border regions 14 can also have the same color as the respective inner region 13 of the graphic element 12, 23 and / or can already be provided as part of the overlay 11. At least the graphic elements 12, 23 that do not run parallel to one of the edges 16 of the pixel grid 17 used to display the image representation 19 are each assigned a second border region in an alpha channel of the overlay, as described above with reference to Figures 3 to 5.

[0064] It is possible for the image representation 20 to be generated from more than one image from one of the cameras 3 and / or for images from several of the cameras 3 to be used to image a larger portion of the surroundings of the vehicle 1. It is possible for additional and / or different overlays 11 to be used to generate the image representation. In particular, at least some of the graphic elements 12, 23 of an overlay 11 can also represent text or other written information.

Claims

Patent claims 1 . Method for generating an image representation (20), wherein the image representation (20) is generated from at least one image (21) and at least one overlay (11) superimposed on the image (21), wherein the overlay (11) comprises at least one graphic element (12) which, in a color channel of the overlay (11), has an inner region (13) and a first border region (14) at least partially enclosing the inner region (13), wherein a second border region (18) at least partially enclosing the inner region (13) is assigned to the at least one graphic element (12) in an alpha channel of the overlay (11), wherein the second border region (18) has a transparency which is greater than the transparency of the inner region (13).

2. Method according to claim 1, characterized in that the first border region (14) has the same color as the inner region (13) or that the first border region (14) has a color which is different from a color of the inner region (13).

3. Method according to claim 1 or 2, characterized in that the second border region (18) is as wide as the first border region (14) or that the second border region (18) has a smaller width than the first border region (14).

4. Method according to one of the preceding claims, characterized in that the image representation (20) is described by a plurality of pixels (15), wherein the second border region (18) has an average width of less than five pixels (15), in particular between one and two pixels (15) inclusive.

5. Method according to one of the preceding claims, characterized in that the graphic element (12) is linear, the width of the second border region (18) being selected as a function of an angle (α) between the graphic element (12) and the edges (16) of a pixel grid (17) used to describe the image information (20).

6. Method according to one of the preceding claims, characterized in that, when several linear graphic elements (12) are present in the overlay (11), a second border region (18) is assigned only to those linear graphic elements (12) which do not run parallel to one of the edges (16) of a pixel grid (17) used to describe the image information (20).

7. Method according to one of the preceding claims, characterized in that the transparency of the second border region (18) lies between a complete transparency and a transparency assigned to the inner region (13).

8. Method according to one of the preceding claims, characterized in that the second border region (18) has a transparency gradient, wherein the transparency increases along the width of the second border region (18) from the inner region (13) to the outer edges (19) of the second border region (18).

9. Control device configured to carry out a method according to one of the preceding claims.

10. Driver assistance system comprising at least one camera (3), at least one display device (9) and a control device (4) according to claim 9, wherein the camera (3) is designed to provide the at least one image (21) and the display device (9) is designed to reproduce the image representation (20).

11. Vehicle comprising a driver assistance system (2) according to claim 10.

Images