Method and device for providing an environment display on board a motor vehicle and motor vehicle

Abstract

A method (200) for providing an environment display on board a motor vehicle (105) comprises steps of producing a first image (305) of an area behind the motor vehicle (105); producing a second image (310) of an area behind the motor vehicle (105); wherein an angle of view (140) of the first image (305) is larger than an angle of view (145) of the second image (310); assembling the second image (310) into the first image (305); and providing a resulting third image (315).

Description

[0001] The present invention relates to the provision of an on-board environment display for a motor vehicle. In particular, the invention relates to a camera monitor system.

[0002] A motor vehicle includes a camera monitor system (CMS) designed to replace a traditional rearview mirror. The CMS includes a camera designed to scan an area of ​​the environment behind the motor vehicle and a monitor for displaying a captured camera image. The mounting locations of the camera and monitor are preferably coordinated to provide a driver of the motor vehicle with a visual impression similar to that of a rearview mirror.

[0003] To replace a side mirror, a camera can be arranged on the side of the motor vehicle, with the monitor preferably being attached to the same side of the vehicle. To replace an interior mirror, a camera can be provided that is arranged centrally in the transverse direction of the vehicle. An associated monitor can also be arranged centrally. In one embodiment, two CMS can be combined by using the same monitor for both cameras. This monitor is usually installed in a center console that lies between a driver and a front passenger. However, such a central monitor can be largely or completely utilized by displaying two camera images. The operation of another device, for example an air conditioning system or an entertainment system, may then no longer be possible.In another embodiment, a camera image occupies only a part of the monitor, whereby only one camera image from either one camera or the other can be displayed.

[0004] Document DE 10 2021 118 100 A1 discloses a method for providing a view for a driver of a vehicle, comprising steps of scanning a first area laterally behind the driver; scanning a second area behind the vehicle; combining scans of the first and second areas to form a panorama of an area laterally and behind the vehicle; and displaying the panorama to a person on board the vehicle.

[0005] Document US 2023 / 0 114 340 A1 discloses a camera that captures an image behind a vehicle to display an image behind the vehicle on an electronic rearview mirror and a monitor installed on the vehicle, and is equipped with an optical system configured to form a high-resolution region having a high imaging magnification and a peripheral region formed around the high-resolution region and having a lower imaging magnification than the high-resolution region, and to form an optical image on an image sensor;and an image pickup means configured to output an image based on an optical image formed on the image sensor, wherein the camera is arranged on the vehicle such that a ground surface located behind and near the vehicle and the position is 80 cm away from the ground surface lies in a second field area in which an image is formed on the image sensor via the second area of ​​the optical system;

[0006] Different views of the surroundings may be required in different driving conditions of the motor vehicle. One object underlying the present invention is to provide an improved technology for displaying the surroundings of a motor vehicle using a camera-mirror system. The invention solves this problem by means of the subject matter of the independent claims. Subordinate claims specify preferred embodiments.

[0007] According to a first aspect of the present invention, a method for providing an environment display on board a motor vehicle comprises steps of creating a first image of an area behind the motor vehicle; creating a second image of an area behind the motor vehicle; wherein an image angle of the first image is greater than an image angle of the second image; assembling the second image into the first image; and providing a resulting third image.

[0008] It has been recognized that different cameras on a motor vehicle have different angles of view, with the camera with the smaller angle of view providing an image that represents part of an image from the other camera with the larger angle of view. Thus, the second image can be superimposed onto the first to produce the third image. A transition between the first and second images can be designed such that no or very slight geometric distortions, missing image content, or jumps in the image are perceptible. A technique known as stitching can be used to superimpose the second image onto the first image.

[0009] The third image is enlarged by a predetermined magnification factor and cropped to a predetermined size. The predetermined size can, in particular, correspond to a display area available on a monitor for the third image.

[0010] The magnification factor can now be freely selected, allowing a section of the third image to be shown on the predetermined display area, which is selected depending on the magnification factor. If the magnification factor is changed, the display can give the impression of a tracking shot or a change in the focal length of an underlying camera. This effect is known as zooming in or zooming out. The larger the magnification factor selected, the more strongly the third image can be influenced or represented by the second image. If the magnification factor is selected to be small, the emphasis on the second image can be reduced.

[0011] It is also possible to animate a change in the magnification factor. A change in the magnification factor is determined, and the magnification of the third image is successively adjusted to the magnification factor. For this purpose, a series of third images with different magnification factors can be provided. The magnification of the third image is preferably adjusted in such a way that a viewer perceives a noticeable zoom-in or zoom-out effect. In one embodiment, sufficient third images are displayed to provide a seamless animation for the viewer.

[0012] The third image can be provided to a driver of the vehicle. The third image can be displayed on a single monitor that simultaneously displays content from both cameras.

[0013] Preferably, geometries of the first and second images are adapted to each other for the third image. For this purpose, the selection of an image section, a size, or a distortion can be adjusted. In various embodiments, the geometry of the first image can be adapted to the geometry of the second, or vice versa. Geometries of both images can also be adapted to a higher-level geometry. In a further embodiment, the brightness or color ranges of the two images can also be adjusted to each other.

[0014] It is preferred that the parameters of the adaptation are fixed and predetermined. A parameter can be predetermined based on properties of a camera that provides the first or second image. The mounting location of a camera on the motor vehicle, an orientation, an angle of view, or a focal length can be assumed to be known. These properties of a camera are usually unchangeable. The adaptation of the images to one another based on predetermined geometric information can be referred to as pre-warp. In this case, an analysis of image content may not be necessary for the geometric adaptation. The geometric adaptation can be carried out quickly and precisely. The processing resources required can be modest.

[0015] The magnification can be adjusted at a predetermined speed. The predetermined speed can be constant or follow a predetermined curve. In a further embodiment, the magnification follows a predetermined speed profile. The predetermined profile can represent a slow change, particularly at the beginning and / or end of the animation, and a faster change in between. Switching between different views characterized by different magnification factors can be intuitive for the viewer. A view of the surroundings can be continuously provided to the viewer, and the view can be adjusted depending on the prevailing circumstances by selecting the appropriate magnification factor.

[0016] In a particularly preferred embodiment, a first and a second magnification factor are predetermined. The speed curve can be applied between the magnification factors. The first magnification factor can be assigned to a first representation, and the second magnification factor to a second representation.

[0017] One of the predetermined magnification factors can be selected depending on a signal indicating whether the motor vehicle should drive forward or backward. The signal can be provided, in particular, by a transmission that can switch between forward and reverse. The signal can also be provided by a control element for such a transmission. With an automatic transmission, forward drive can be expressed by the drive position D, and reverse drive by the drive position R. With a manual transmission, several forward gears and one or more reverse gears can be engaged manually.

[0018] If the vehicle is to be reversed, a smaller magnification factor can be selected so that the angle of the displayed third image is large and the viewer has a good overview of the area behind the vehicle. If the vehicle is to be driven forward, the magnification factor can be higher so that the viewer is presented with the third image with a smaller angle of view. This allows adapted views for forward and reverse travel to be shown on a single monitor. Transitions between the images can be animated so that the vehicle can be controlled more effectively depending on the display of the third image.

[0019] According to a further aspect of the present invention, a device for providing an environment display on board a motor vehicle comprises a first camera for providing a first image of an area behind the motor vehicle; a second camera for providing a second image of an area behind the motor vehicle; wherein an image angle of the first image is greater than an image angle of the second image; and a processing device configured to assemble the second image into the first image and to display a resulting third image.

[0020] The processing device is preferably configured to partially or completely execute a method described herein. For this purpose, the processing device can be implemented electronically and, for example, comprise an integrated circuit, a programmable logic module, or a programmable microcomputer. The method can be implemented in the form of a configuration or as a computer program product with program code means for the processing device. The configuration or the computer program product can be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the device, or vice versa.

[0021] The device may comprise a display device for presenting the third image to a driver of the motor vehicle. The display device is preferably mounted in front of the driver so that the driver can view an area of ​​the surroundings of the motor vehicle located behind him on the display device.

[0022] The display device can be characterized by being significantly wider than it is tall. In one embodiment, the ratio of a width to a height of the third image on the display device is greater than four. The ratio can also be greater than five or, more preferably, greater than six. In another embodiment, the ratio can be twelve or more.

[0023] For example, the display device can comprise a head-up display. The display device can be presented on board the motor vehicle using projection technology. It is possible for the display area of ​​the display device to occupy virtually the entire width of the vehicle's interior. The height of the display area can therefore be significantly smaller. For example, a projection can be made at the lower edge of a windshield. Such a display is known as a panoramic head-up display.

[0024] According to yet another aspect of the present invention, a motor vehicle comprises a device described herein. The motor vehicle preferably comprises a passenger car. In further embodiments, the motor vehicle can also comprise, for example, a truck or a bus.

[0025] The invention will now be described in more detail with reference to the accompanying drawings, in which: Fig. 1 a system; Fig. 2 a flow diagram of a process; Fig. 3 an exemplary montage of a second image into a first image; and Fig. 4 illustrates an exemplary view of a third image on a display device on board a motor vehicle.

[0026] Fig. 1 shows a system 100 with a motor vehicle 105, which comprises a device 110. The device 110 is configured to scan a portion of an environment 115 of the motor vehicle 105 and to display it in the motor vehicle 105. The display is directed in particular to a person 120, who may be a driver of the motor vehicle 105. Purely by way of example, the person 120 is shown in Fig. 1 shown on the left side of the vehicle.

[0027] The device 110 comprises a processing device 125, a first camera 130, and a second camera 135. The first camera 130 can, in particular, comprise a rear-view camera, which is mounted more preferably centrally on the motor vehicle 105 in the transverse direction. The second camera 135 can comprise a side camera, which can be mounted on a right or left side of the motor vehicle 105. In the longitudinal direction, the second camera 135 can be mounted approximately at the height of a first row of seats, as shown. In another embodiment, the second camera 135 can also be displaced in the longitudinal direction, in particular to the rear. In the illustration of Fig. 1, two second cameras 135 are shown on opposite sides.

[0028] The first camera 130 operates with a first angle of view 140, and the second camera 135 with a second angle of view 145. The first angle of view 140 is larger than the second angle of view 145. The first angle of view 140 can, for example, be in the range of approximately 180°. The second angle of view 145 is typically approximately 60° or less. The scanning ranges of the cameras 130, 135 overlap, particularly in an area behind the motor vehicle 105.

[0029] The device 110 can further comprise a display device 150, which is embodied, for example, as a projection surface of a panoramic head-up display (pHUD). An optional interface 155 is configured to detect a signal indicating whether the motor vehicle 105 should drive forward or backward. The interface 155 can be connected, for example, via a data bus to a transmission, an operating unit, or a control device for a transmission. The signal is preferably bivalent, with one value being assigned to forward travel and another to reverse travel. A corresponding signal can be used to control a reversing light of the motor vehicle 105.

[0030] The processing device 125 is configured to assemble an image provided by a second camera 135 into an image provided by the first camera 130. Depending on a magnification factor, which can be selected based on the signal determined at the interface 155, the resulting image can be scaled and cropped to a predetermined size. The result of the processing can be displayed on the display device 150 or a portion thereof.

[0031] Fig. 2 shows a flowchart of a method 200 for displaying an environment 115 on board a motor vehicle 105. In a step 205, a first image is provided at a first image angle 140. In particular, the first camera 130 can be used for this purpose. In a step 210, a second image is provided at a second image angle 145, which is smaller than the first image angle 140. The second image is preferably captured by a second camera 135.

[0032] In a step 215, the second image is merged into the first image. Due to the narrower viewing angle, the second image shows only a section of the first image. The second image is merged into the first image in such a way that transitions between the first and second images are as unobtrusive as possible. For this purpose, the first image and / or the second image can be appropriately distorted or rectified. Furthermore, color or brightness adjustment can be performed. The merge results in a third image.

[0033] In a step 220, a direction of travel in which the motor vehicle 105 is to travel can be determined. The direction of travel can be, in particular, forward or reverse. Based on the direction of travel, a magnification factor can be determined in a step 225. A magnification factor can be assigned to a direction of travel. The magnification factor assigned to reverse travel is preferably smaller than the magnification factor assigned to forward travel.

[0034] In a step 230, the third image can be scaled using the determined magnification factor. If the magnification factor changes, an animation can be created in which the third image is scaled using a series of monotonically decreasing or monotonically increasing magnification factors. A change in the magnification factor over time can be determined after a predetermined transition. The duration of the transition is preferably predetermined and can range from a few seconds to less than approximately one second. The scaled third image can be provided in a step 235. In particular, the third image can be output on the display device 150 to a person 120 on board the motor vehicle 105.

[0035] Fig. Figure 3 shows an exemplary montage of images. A lower section shows a first image 305, which was captured by a rearview camera 130 of a motor vehicle 105. A top section shows a second image 310, which may have been captured by a side camera 135 on the same motor vehicle 105.

[0036] The second image 310 is inserted in scaled form at the appropriate location into the first image 305, resulting in a third image 315. The third image 315 can then be subjected to a predetermined magnification factor in order to display a section in an enlarged manner. A vanishing point of the magnification is preferably predetermined and preferably lies in the region of the second image 310 within the third image 315. The magnification factor can be increased until the display shows exclusively the second image 310. Optionally, the display can be further enlarged until only a section of the second image 310 is visible.

[0037] Fig.4 shows an exemplary view 400 of a third image 315 on a display device 150 on board a motor vehicle 105. The illustration is based on a perspective that, for example, a passenger in the first row of seats of the motor vehicle 105 could take. In a lower area, a dashboard of the motor vehicle 105 is shown, which at its upper end adjoins a windshield of the motor vehicle 105. In a left-hand area, the dashboard is partially obscured by a steering wheel with which the motor vehicle 105 can be steered.

[0038] The display device 150 is designed as a PHUD, with a wide, but only a few centimeters high, strip at the lower end of the windshield serving as a projection surface. A third image 315 can be seen, for example, in a central area of ​​the projection surface. Above the projection surface, a view through the windshield of the motor vehicle 105 is possible. Reference symbol 100 systems 105 Motor vehicle 110 Device 115 Environment 120 people 125 processing facility 130 first camera, rear view camera 135 second camera, side camera 140 first angle of view 145 second angle of view 150 display device 155 Interface 200 procedures 205 provide first image 210 provide second image 215 Stitching 220 Determine direction of travel 225 Determine magnification factor 230 Animate transition 235 Print image 305 first picture 310 second picture 315 third picture 400 views

Claims

[1] Method (200) for providing an environment display on board a motor vehicle (105), the method (200) comprising the following steps: - creating (205) a first image (305) of an area behind the motor vehicle (105); - creating (210) a second image (310) of an area behind the motor vehicle (105); - wherein an image angle (140) of the first image (305) is greater than an image angle (145) of the second image (310); - Mounting (215) the second image (310) into the first image (305) and providing (235) a resulting third image (315); wherein the third image (315) is enlarged by a predetermined magnification factor and cropped to a predetermined extent; wherein a change in the magnification factor is determined; and the magnification of the third image (315) is successively adjusted to the magnification factor. [2] The method (200) of claim 1, wherein geometries of the first and second images (310) are adapted to each other for the third image (315). [3] Method (200) according to claim 2, wherein parameters of the adaptation are fixedly predetermined. [4] Method (200) according to one of the preceding claims, wherein the magnification is adjusted (230) at a predetermined speed. [5] Method (200) according to one of the preceding claims, wherein the magnification is tracked (230) with a predetermined speed profile. [6] Method (200) according to one of the preceding claims, wherein a first and a second magnification factor are predetermined. [7] Method (200) according to claim 6, wherein one of the predetermined magnification factors is selected (225) in dependence on a signal indicating whether the motor vehicle (105) should travel forward or backward. [8] Device (110) for providing an environmental display on board a motor vehicle (105), the device comprising the following elements: - a first camera (130) for providing a first image (305) of an area behind the motor vehicle (105); - a second camera (135) for providing a second image (310) of an area behind the motor vehicle (105); - wherein an image angle (140) of the first image (305) is greater than an image angle (145) of the second image (310); and - a processing device (125) configured to assemble the second image (310) into the first image (305) and to provide a resulting third image (315); wherein the third image (315) is enlarged by a predetermined magnification factor and cropped to a predetermined extent; wherein a change in the magnification factor is determined; and the magnification of the third image (315) is successively adjusted to the magnification factor. [9] The device (110) of claim 8, further comprising a display device for displaying the third image (315) to a driver of the motor vehicle (105). [10] The device (110) of claim 9, wherein a ratio of a width to a height of the third image (315) on the display device exceeds four. [11] Device (110) according to claim 9 or 10, wherein the display device comprises a head-up display. [12] Motor vehicle (105) comprising a device (110) according to one of claims 9 to 11.

Images