Camera monitoring system for vehicle and method for controlling such camera monitoring system

By using optical components in the camera monitoring system to insert the image of the display into the camera scanning area and adjust the brightness and contrast, the problem of image unclearity caused by changes in lighting conditions is solved, and high-quality image display under different lighting conditions is achieved, simplifying the System design and reduce costs.

CN118973865APending Publication Date: 2024-11-15BAYERISCHE MOTOREN WERKE AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380024489.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-06-07
Filing Date
2023-05-25
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing camera surveillance systems have difficulty maintaining image clarity when lighting conditions suddenly change, especially when the driver enters or exits the tunnel, which can lead to unclear images and adding another camera to solve this problem can increase costs and introduce scanning error.

Method used

The image of the display is inserted into the scanning area of ​​the camera by using optical elements, and the brightness and contrast of the image is adjusted according to the lighting conditions by the processing device. The deflection and deformation distortion are used with reflective or refractive optical elements to compensate for viewing angle deformation, avoiding the use of additional Camera.

Benefits of technology

The display system construction is simplified, the system components are reduced, the visibility and resolution of images are improved, the image clarity is ensured under different lighting conditions, and the driver's visual interference during driving is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118973865A_ABST
    Figure CN118973865A_ABST
Patent Text Reader

Abstract

A display system (100) for a vehicle (105) comprises: a camera (125) having a predetermined scanning area (160) for scanning a first image (405) from the surroundings of the vehicle (105); processing means for providing a second image (410) based on the first image (405); a graphical display (140) for rendering the second image (410); and an optical element (135) for inserting the display (140) into the scanning area (160) of the camera (125) such that a section of the first image (405) comprises the second image (410). In this case, the processing device is arranged to provide a second image (410) as a function of the insertion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a camera monitoring system for a vehicle. In particular, the present invention relates to controlling a camera monitoring system. Background Art

[0002] Motor vehicles include exterior rear-view mirrors in order to provide the driver with an impression of the traffic situation behind him. The exterior rear-view mirrors can be replaced by a camera monitoring system (CMS) comprising a camera mounted on the motor vehicle and a graphic display in the interior. The camera is usually mounted on the vehicle approximately where the exterior rear-view mirrors would otherwise be and scans the area laterally behind the driver. The image provided by the camera can be displayed on the display so that the driver recognizes on the display an image that is substantially the same as the image of the exterior rear-view mirror.

[0003] DE 10 2021 004 637 A1 discloses a driver assistance system for supporting the reversing of a vehicle with a trailer and a correspondingly equipped vehicle having a reversing camera and a projection device arranged on the reversing camera.

[0004] One advantage of CMS is the ability to adapt the display to the existing lighting conditions. For example, in nighttime surroundings, the contrast of the displayed image can be increased and the brightness can be reduced. If sunlight is directly shining on the camera, the contrast and brightness can be adaptively reduced. In very bright surroundings, the contrast and brightness can be increased. This allows the driver to better recognize the area behind him.

[0005] However, the change in the appearance can also have the effect that the driver can no longer clearly discern details in the image provided to him. In particular, if the ambient lighting conditions change suddenly, for example when entering or exiting a tunnel, it may be difficult to control the display.

[0006] It has been proposed to scan the image provided by means of a display by means of a further camera and to control the provision of the displayed image based on the scanning. However, this requires a further camera which must be installed in the interior of the motor vehicle. This can be associated with considerable costs. Furthermore, due to scanning errors of the further camera, new errors can be introduced into the image provided to the driver. Summary of the invention

[0007] The object of the present invention is to provide an improved technology for controlling a camera monitoring system for a vehicle. The present invention achieves this object by means of the technical solution of the independent claim. The dependent claims present preferred embodiments.

[0008] According to a first aspect, a display system for a vehicle comprises: a camera with a predetermined scanning area for scanning a first image from the surroundings of the vehicle; a processing device for providing a second image based on the first image; and a graphic display for displaying the second image. In addition, an optical element is provided for inserting (einblenden: fading) the display into the scanning area of ​​the camera so that a section of the first image includes the second image. The processing device is configured to provide the second image based on the insertion.

[0009] It has been found that the second image provided to the vehicle driver can be inserted into the scanning area of ​​the camera in a purely optical manner by means of an optical element, so that the camera additionally scans the second image on the display. In this case, the insertion preferably only occupies a small portion of the image surface of the second image, so that the visible surface or the resolution of the second image is not reduced or is only slightly reduced.

[0010] An additional camera for scanning the display or the second image can be omitted, so that the display system can be constructed more simply or improved without using components external to the system (such as internal cameras). The optical element is preferably reflective and can in particular include a mirror, a prism or the like. Optionally, a plurality of optical elements can also be used, which can optionally be mounted on one another or connected to one another.

[0011] The processing device is preferably arranged to provide the second image such that the insertion in the first image has a predetermined brightness and / or a predetermined contrast. The predetermined brightness or the predetermined contrast can be determined according to the lighting conditions existing in the vehicle surroundings. The lighting conditions can be determined by means of corresponding sensors or based on the first image. Under different predetermined lighting conditions, the brightness and / or the contrast can be increased to a value above normal or reduced to a value below normal. The normal value can be predetermined by a user of the CMS.

[0012] The scanning area is preferably directed backward from the camera; the optical element on the vehicle is located behind the camera and behind the second image. Unless otherwise specified, the direction descriptions used here are based on the common driving direction of the vehicle along its longitudinal axis forward.

[0013] The optical element can deflect the light path between the display and the camera, for which purpose the optical element preferably comprises at least one reflective element, such as a mirror or a prism. The deflection can generally occur at the interface between media of different optical density, the shape and orientation of the interface determining the type of deflection. The optical element can also have a refracting effect, for example in order to reduce or enlarge the second image inserted into the camera. In another embodiment, the optical element can also undergo a predetermined deformation distortion, such as a trapezoidal or pincushion deformation distortion. For this purpose, the interface can be appropriately curved. This can partially or completely compensate for the deformation distortion that has already occurred due to the viewing angle of the optical element on the display.

[0014] The vehicle may have transparent glass and the optical element may be mounted on the glass. In one embodiment, the existing glass may be supplemented with an optical element mounted on the inside or outside. The optical element may be glued, bonded or manufactured integrally with the glass. In another embodiment, a section of the glass may be provided with a material having a different optical density. Thus, the thickness of the glass may remain the same, but the desired reflection effect may occur at the location of the optical material. The optical element may be designed to be small and inconspicuous.

[0015] Glass can delimit the interior of a vehicle (where the vehicle occupants are located) from the exterior and can include in particular windshields or side windows. Cameras are usually mounted in the area of ​​the front wheels, front fenders, A-pillars or front doors of a vehicle, where the vehicle can include in particular a passenger car or a truck. If there is a possible line of sight between the camera and the side window, an optical element can be used therein. Of course, another glass delimiting the interior of the vehicle from the exterior can also be used to mount the optical element.

[0016] If the windshield is visible from the camera, then the windshield is also suitable for mounting the optical element. For example, if the windshield is used to display the image provided by the display or to reflect it toward the head area, the optical element can couple out a second image in a limited area in the windshield and direct it toward the camera. The optical element is preferably also small and can be inconspicuous to the people on the vehicle. In this case, it usually has a significantly smaller visible surface than a direct graphic display. Usually, the optical element has a surface that is comparable to the light-inlet surface of the camera.

[0017] The vehicle may comprise a driver position such that the driver's head in the driver position is located in a predetermined head area. The display may be arranged to provide the second image in the area of ​​a line between the head area and the camera. Display errors between the perceived second image and the actual origin of the second image may thereby be minimized. In particular in dangerous or precise maneuvers, such as when driving on a motorway or when parking in the area of ​​an obstacle, such an arrangement may help to provide the driver with a realistic impression of the area laterally behind him.

[0018] The camera is located in the exterior space, while the display is preferably mounted in the interior space. In some embodiments, the display is configured to provide the second image as a virtual image so that the driver can perceive the second image at a location different from where the display is located. For example, the display may include a head-up display that can project the second image toward the driver by means of tilted glass. From the driver's perspective, the second image may appear to be located outside the vehicle, even though the display is preferably located in the interior space.

[0019] A further optical element can be mounted on the camera for inserting the first optical element into the scanning area. The further optical element can be arranged in the area of ​​the opening of the camera for the light to enter. The optical element can also be designed to be small here so that it is not exposed in any particular way to environmental influences in the area outside the vehicle, such as wind, rain or dust. The optical element can be mounted on or behind a protective glass that is mounted on the light-inlet side of the camera in order to protect the optics of the camera from mechanical environmental influences, such as water or dust.

[0020] In a preferred embodiment, the optical element is located behind the line between the camera and the driver's head area. This can be advantageous in particular when the optical element is mounted on the driver's side, for example on the side window of the vehicle. The optical element can be located behind the head area so that the driver's direct line of sight through the side window is not disturbed by the optical element. If the optical element is located in the windshield, a further optical element can be provided to project the second image towards the optical element. The second image can be reflected at two or more optical elements before it falls into the camera.

[0021] The display can be configured to project the second image. The driver can recognize the second image as a virtual image. A further optical element for coupling out the second image can be installed in the light path of the display. It is also possible to combine a plurality of optical elements with one another in order to insert the second image into the scanning area of ​​the camera. The optical elements are preferably configured to deflect the light path and can optionally also perform other optical functions, such as focusing, deforming, reducing, enlarging, polarizing, filtering or coloring the light passing through.

[0022] According to another aspect, a vehicle includes the display system described herein. The vehicle preferably includes a motor vehicle and may also include, for example, a motor vehicle, a passenger car or a bus. In the case of a motor vehicle, there may be no interior space, in which case the camera, optical elements and display may be installed in the exterior space.

[0023] According to yet another aspect, a method for controlling a display system described herein comprises the following steps: providing a first image by means of a camera; determining a control segment representing a second image in the first image; and providing the second image as a function of the control segment.

[0024] The method can preferably be performed with the aid of a processing device as described herein. For this purpose, the processing device is preferably implemented electronically and can include a programmable microcomputer or microcontroller or an integrated circuit. The method can be in the form of a computer program product including program code means. The computer program product can also be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the device and vice versa.

[0025] It is particularly preferred that the second image is divided non-overlappingly into a control segment and a display segment; wherein the second image is further preferably provided based only on the display segment. The control segment is preferably not the basis for the second image. Optionally, the first image can still include one or more additional parts that are not further evaluated or forwarded. For example, the position of the display segment can be adjusted in the first image so that a predetermined scanning area is detected. In addition to the control segment, the surrounding segments can be ignored.

[0026] In other words, the second image is preferably provided based on only a portion of the first image. The provision is based on the section of the first image that does not enter the second image. This prevents parts from being recognizable in the second image that show content that differs from the content derived from the camera scanning area. In this way, the driver is not confused or distracted and can control the vehicle without problems using the second image shown to him.

[0027] The camera, the display and the processing device can together form a closed control loop. Preferably, when controlling the provision of the second image, a time delay between a change in the second image and a change in the control section is taken into account. Here, a first delay to be provided between the change in the control and the output of the changed second image can be taken into account. Likewise, a second delay between providing the changed surroundings and providing the correspondingly changed first image can be taken into account. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will now be described in more detail with reference to the accompanying drawings, in which:

[0029] Figure 1 A display system in a first embodiment is shown;

[0030] Figure 2 A display system in a second embodiment is shown;

[0031] Figure 3 A display system in a third embodiment is shown;

[0032] Figure 4 showing a presentation of the first and second images; and

[0033] Figure 5 A flow chart of a method is shown. DETAILED DESCRIPTION

[0034] Figure 1 The display system 100 is shown in an exemplary first embodiment on a vehicle 105. The vehicle 105 comprises a driver position 110, which is shown in the driving direction in an exemplary manner on the left side of the vehicle 105. The driver 115 is located in the driver position 110, with his head substantially located in a predetermined head area 120.

[0035] The display system 100 comprises a camera 125, a processing device 130, an optical element 135 and a graphic display 140. The camera 125 is mounted on the outside of the vehicle 105, preferably substantially at the location where the exterior rearview mirrors are usually mounted. However, there does not necessarily have to be a line of sight between the head area 120 and the camera 125. In the longitudinal direction, the camera is usually located behind the front wheel axle and in front of the head area 120. In the vertical direction, the camera 125 is preferably located below the upper edge and optionally also below the lower edge of the windshield 145 or the side glass 150, which can respectively separate the interior of the vehicle 105 from the exterior.

[0036] exist Figure 1In the illustration of FIG. 1 , the camera 125 is shown larger and laterally further away from the vehicle 105 than it would be on a real vehicle 105 in order to better explain the described technology. Purely by way of example, the camera 125 is located on the left side of the vehicle 105. It should be noted that the described technology does not presuppose a specific position of the driver's position 110 in the vehicle 105, nor does it proceed from a predetermined side on which the camera 125 is mounted on the vehicle 105. In particular, the described technology can be mounted on both sides of the vehicle 105 in order to enable a rear view on both sides for the driver 115.

[0037] Processing device 130 is usually designed as a control unit or a control device and usually includes a programmable microcomputer. Here, processing device 130 is configured to control display 140 based on the image provided by camera 125 to provide an image to driver 115.

[0038] Figure 1 The display 140 in the embodiment is embodied as a screen, on the surface of which an image recognizable to the driver 115 can be displayed. As will be explained in more detail here, for the geometric relationships considered, the position of the display 140 is not as important as the position of the image that it provides and is recognizable to the driver 115. For a display 140 embodied as a screen, its position coincides with the position of the image displayed by it.

[0039] Display 140 is preferably located on a line 155 between camera 125 and head region 120. Ideally, line 155 extends through the center point of the image plane of camera 125, the center point of the display surface of display 140 or the center point of the eye or the perception of driver 115. However, a certain deviation of the position of display 140 or the image displayed thereby from line 155 is tolerable.

[0040] It is recommended that the optical element 135 be positioned so that it projects at least a portion of the image provided by the display 140 for the driver 115 into a portion of the detection area 160 of the camera 125 so that it is detected by the camera 125. If the display 140 is located in the interior of the vehicle 105 and the camera 115 is located outside the interior, the light path must pass through the boundary of the interior for this purpose. In the embodiment shown, this occurs on the side window 150. The optical element 135 can be arranged in the interior or outside, but is preferably installed in the area of ​​the passage.

[0041] It is particularly preferred that the optical element 135 is implemented to be integrated with the transparent element (here, the side glass 150) in the boundary area. For this purpose, the optical element 135 can be embedded in the side glass 150. Optionally, the optical element 135 can also partially extend beyond the boundary of the side glass 150 on the inside and / or outside. In another embodiment, the optical element 135 is placed on the side glass 150 on the inside or outside. The optical element 135 preferably includes a mirror or a prism, in particular an inverted prism, such as an Amisi prism, a Porro prism or a dove prism. It can also include a spherical element, such as an optical lens or a curved reflective surface. The optical element 135 can also include multiple reflective surfaces or refractive surfaces.

[0042] exist Figure 1 , the optical element 135 is not located in the scanning area 160, so that light from the optical element 135 can reach the camera 125 but cannot be detected by it. In such a configuration, the scanning area 160 can be pivoted toward the vehicle 105 until the optical element 135 is located in the scanning area 160. At the same time, the focal length of the camera 125 can be shortened, wherein the aperture angle of the scanning area is increased.

[0043] Alternatively, a further optical element 165 can be used to couple the light of the first optical element 135 into the camera 125. The further optical element 165 is preferably mounted on the camera 125 and can also include a reflective element or a refractive element. In one embodiment, the camera 125 includes an optical lens or an objective lens, which includes the further optical element 165.

[0044] Figure 2 The display system 100 in the exemplary second embodiment is shown. Figure 1 , at least a portion of the image provided to the driver 115 by means of the display 140 is at least partially deflected to the camera 125 by means of the optical element 135. However, here the camera 125 is located at a higher position on the vehicle 105 so that the light of the image can be guided through the windshield 145.

[0045] When using a screen-type display 140, the mirror-type optical element 135 can be positioned, for example, in the area of ​​the driver's seat 110. Figure 2In an embodiment, the display 140 projects an image for the driver 115 onto the glass 205, which turns the projected image toward the head area 120, so that the driver 115 can perceive the image at a predetermined distance on an axis extending from his head through the glass 205. The glass 205 can be implemented separately or coincident with the windshield 145. The light of the display 140 can be coupled out by means of the optical element 135 before or after reflection on the glass 205 and turned to the camera 125. The optical element 135 can be implemented in combination with the glass 205 or the windshield 145, wherein the options mentioned above for the side glass 150 can be adopted.

[0046] Figure 3 The display system 100 is shown in an exemplary third embodiment. Here, the image of the display 140 is reflected on the mirror 305 onto the projection surface 310, where the driver 115 can perceive the image. In this case, the projection surface 310 is only slightly offset relative to the connecting line 155, so that the image perceptible by the driver 115 can have practically the same viewing angle as the image provided by the camera 125, which is symbolically represented on the camera 125.

[0047] It should be noted that, refer to Figures 1 to 3 The features and embodiments of the described elements of vehicle 105 or display system 100 may be used interchangeably.

[0048] Figure 4 An exemplary representation of a first image 405 and a second image 410 is shown. First image 405 is scanned by camera 125 from the surroundings of vehicle 105 and shows the contents of scan area 160. A representation section 415 and a control section 420 are defined in first image 405.

[0049] Here, the segments 415 and 420 are preferably non-overlapping or non-intersecting. The first image 405 has no segments that are located both in the display segment 415 and in the control segment 420. The control segment 420 is usually smaller than the display segment 415 and can include only a few image elements (pixels). For example, the control segment 420 can include an array of approximately 3x3 pixels or 4x4 pixels. Figure 4 In the diagram of FIG. 4 , the control section 420 is shown enlarged for convenience of explanation.

[0050] The display section 415 is the basis for the second image 410. In particular, the second image 410 can display the content of the display section 415, but the second image 410 can be adapted, for example, in terms of brightness, contrast and / or its color.

[0051] The control section 420 includes that portion of the first image 405 in which the second image 410 provided by means of the display 140 is deflected into the scanning area 160 by means of the optical element 135. The content of the control section 420 can be analyzed to determine the display parameters, such as the brightness and / or the contrast of the second image 410 that can be perceived by the driver 115. Based on this determination, the processing device 130 can adapt the display of the second image 410 on the display 140. For example, if it is determined that the second image 410 is too dark in the control section 420, the display 140 can be controlled to output the second image 410 brighter.

[0052] If the visualization segment 415 and the control segment 420 do not intersect in the first image 405, then an "endless tunnel" may appear in the second image 410 at the location where the control segment 420 is located, which tunnel is generated as a cascade of interlaced, increasingly smaller images of the second image 410. In this case, the visualization parameters associated with the control segment 420 can be determined without regard to the segment in which the tunnel is located.

[0053] Figure 5 A flow chart of a method 500 for controlling a display system 100 is shown. In step 505, the surroundings of the vehicle 105 are scanned in a scanning area 160 by means of the camera 125. In step 510, a first image 405 is provided, in which a display segment 410 is determined in step 515. As long as the orientation and the focal length of the camera 125 remain unchanged, the display segment 410 usually has a predetermined position and size in the first image 405. In step 520, the content to be displayed can be determined in the display segment 410.

[0054] In parallel, in step 525 control section 420 can be determined in first image 405. In step 530 the content of control section 420 can be analyzed, for example to determine whether brightness, color or contrast respectively corresponds to a predetermined value. In step 530 one or more visualization parameters can be determined.

[0055] Based on the at least one visualization parameter and the content to be visualized, the display 140 can be actuated in step 540 to provide the second image 410. In step 545, the visualized second image 410 can be fed back into the first image 405 by coupling the second image 410 into the scanning region 160 of the camera 125 by means of the optical element 135. In step 545, a first delay between actuating the display 140 and providing the corresponding second image 410 and a second delay between detecting the output second image 410 as part of the first image 404 and providing the new first image 405 can be taken into account or simulated. Reference numerals

[0056] 100 systems

[0057] 105 vehicles

[0058] 110 Driver's Position

[0059] 115 Driver

[0060] 120 Head area

[0061] 125 cameras

[0062] 130 Processing device

[0063] 135 Optical Components

[0064] 140 Graphic Display

[0065] 145 Windshield

[0066] 150 side glass

[0067] 155 Connections

[0068] 160 scanning area

[0069] 165 Additional optical components

[0070] 205 Glass

[0071] 305 mirror

[0072] 310 projection surface

[0073] 405 First Image

[0074] 410 Second Image

[0075] 415 manifestation section

[0076] 420 control section

[0077] 500 Methods

[0078] 505 Scan the surrounding environment

[0079] 510 Provide the first image

[0080] 515 Determine the display section

[0081] 520 Determine the content to be displayed

[0082] 525 Determine the control section

[0083] 530 Analysis and Display

[0084] 535 Determine display parameters

[0085] 540 control display

[0086] 545 Feedback through reflection

Claims

1. A display system (100) for a vehicle (105), in, The display system (100) comprises the following: A camera (125) having a predetermined scanning area (160) for scanning a first image (405) from the surroundings of the vehicle (105); A processing device (130) for providing a second image (410) based on the first image (405); A graphics display (140) for displaying a second image (410); and An optical element (135) for inserting the display (140) into a scanning area (160) of the camera (125) such that a portion of the first image (405) includes the second image (410); Therein, the processing device is configured to provide a second image (410) based on the inserting.

2. The display system (100) according to claim 1, in, The processing device is configured to provide the second image (410) such that the insertion in the first image (405) has a predetermined brightness and / or a predetermined contrast.

3. The display system (100) according to claim 1 or 2, in, The scanning area (160) is directed backward from the camera (125); the optical element (135) on the vehicle (105) is located behind the camera (125) and behind the second image (410).

4. The display system (100) according to one of the preceding claims, in, The vehicle (105) includes transparent glass (205), and the optical element (135) is mounted on the glass (205).

5. The display system (100) according to one of the preceding claims, in, The vehicle (105) comprises a driver position (110), such that the head of a driver (115) at the driver position (110) is located in a predetermined head area (120); the display (140) is configured to provide a second image (410) in the area of ​​a line (155) between the head area and the camera (125).

6. The display system (100) according to one of the preceding claims, in, A further optical element (165) is mounted on the camera (125) for inserting the first optical element (135) into the scanning area (160).

7. The display system (100) according to one of the preceding claims, in, The display (140) is configured to project a second image (410); a further optical element (135) is installed in the optical path of the display (140) for coupling out the second image (410).

8. Vehicle (105) comprising a display system (100) according to one of the preceding claims.

9. A method (500) for controlling a display system (100) according to any one of claims 1 to 7, in, The method comprises the following steps: Providing (510) a first image (405) by means of a camera (125); determining (525) a control segment (420) in the first image (405) that represents the second image (410); and A second image (410) is provided according to a control section (420).

10. The method (500) according to claim 9, in, The first image (405) is divided non-overlappingly into a control section (420) and a display section (415); wherein the second image (410) is provided only based on the display section (415).

11. The method (500) according to any one of the preceding claims 9 and 10, in, Consider the time delay between the second image (410) changing and the control section (420) changing.