CAMERA-MONITOR SYSTEM AS WELL AS METHOD AND DEVICE FOR OPERATING A CAMERA-MONITOR SYSTEM FOR A MOTOR VEHICLE

Description

[0001] The invention relates to a camera monitor system for a motor vehicle, in particular for a motor vehicle with a trailer. The invention further relates to a method for operating a camera monitor system for a motor vehicle and a device designed to carry out the method.

[0002] Motor vehicles, such as trucks and cars, traditionally have exterior mirrors. Mirror replacement systems are also known in which the viewing areas of the standard exterior mirrors are reproduced using a camera and a monitor. Such mirror replacement systems are also referred to as camera-monitor systems.

[0003] A method and apparatus for operating a camera monitor system is known, for example, from WO 2016 / 026870 A1. WO 2016 / 026870 A1 discloses a display system for generating a composite view of an area behind a vehicle towing a trailer. The display system comprises a first camera for outputting first image data corresponding to a first image, wherein the first camera is configured to be mounted on the vehicle in a rearward-facing orientation. The display system further comprises a second camera for outputting second image data corresponding to a second image, wherein the second camera is configured to be mounted on the trailer in a rearward-facing orientation.

[0004] US 2008 / 044061 A1 discloses an apparatus and method for displaying an image representing a field of view of the surroundings of a connected vehicle consisting of a tractor and a trailer on a display device inside the tractor.

[0005] US 2017 / 341583 A1 relates to systems and methods for a towing vehicle and a trailer with surround view imaging devices for determining the dynamics of the towing vehicle and trailer, generating a view of the rear of a towing vehicle with a transparent trailer and generating a 360-degree bird's eye view of the towing vehicle and / or the trailer.

[0006] US 2016 / 366336 A1 discloses a system and method that facilitate the generation of a surround view image for an articulated commercial vehicle with interchangeable vehicle segments.

[0007] DE 10 2015 214611 A1 discloses a method and a device for displaying an ambient scene of a vehicle combination. Furthermore, a surround-view system for a vehicle combination is described.

[0008] It is desirable to provide a method for operating a camera monitor system for a motor vehicle that enables reliable operation, particularly when coupled to a trailer. It is further desirable to provide a device for operating a camera monitor system that enables reliable operation. It is further desirable to provide a camera monitor system that enables reliable operation, particularly when coupled to a trailer.

[0009] The invention is characterized by a method according to claim 1 for operating a camera monitor system for a motor vehicle with a trailer and by a corresponding device according to claim 7, which is designed to carry out the method.

[0010] According to the invention, the camera monitor system has a camera assigned to a rear side of the trailer. The camera is designed to be mounted on the rear side of the trailer in order to record an environment at the rear of the trailer. The camera monitor system has a further camera assigned to a side of the motor vehicle. The side is in particular a longitudinal side that extends in the normal direction of travel between the front windshield of the motor vehicle and a rear end of the trailer. The further camera is in particular designed to provide an image of the environment of the motor vehicle in the region of the side of the motor vehicle.

[0011] The camera monitor system also includes a monitor for displaying images from the cameras.

[0012] An image from the camera is provided. Another image from the other camera is provided. An overall image is created from this image and the other image based on a determined trajectory of the vehicle. The overall image is displayed on the monitor.

[0013] The method thus enables a combination of the camera, which can also be referred to as a rear-view camera, and the additional camera, which is, for example, part of a mirror replacement system for replacing a conventional exterior mirror. No additional monitor is required to display the image from the camera in the vehicle. The image from the camera is displayed together with the additional image from the additional camera on the monitor of the mirror replacement system. The monitor is arranged, for example, in a side area of ​​the motor vehicle. This means that the user can intuitively perceive the displayed surroundings even in the motor vehicle with a trailer. In particular, areas that cannot be displayed with a conventional mirror replacement system or conventional exterior mirrors can be displayed on the monitor.

[0014] The area behind the trailer becomes visible. Legal requirements for the position of the monitor can be met as a replacement for a conventional exterior mirror.

[0015] The driver does not have to look at another separate monitor, which is located in the middle of the vehicle, for example, in order to see the area behind the trailer. This makes use more convenient for the driver. The two monitors, which replace the left and right conventional exterior mirrors, show, for example, the inside and outside of a curve as well as the area to the rear when reversing. The rear area can therefore be viewed comfortably and intuitively. In particular, no separate monitor is required to display the rear area. An additional monitor is optionally available, for example in the middle of the vehicle. The driver can thus comfortably view the rear area without having to look at another monitor.

[0016] The motor vehicle is, for example, a truck with a trailer, such as a tractor with a semi-trailer. The motor vehicle can also be a truck with a drawbar trailer and / or a bus with a trailer. According to further embodiments, the motor vehicle is alternatively or additionally a passenger car with a trailer.

[0017] Taking the vehicle's trajectory into account allows the image and the rest of the image to be adapted to the respective driving situation. For example, the overall image displayed is adapted depending on the articulation angle between the vehicle's towing vehicle and the trailer.

[0018] According to at least one further embodiment, determining the trajectory comprises determining a speed of the motor vehicle. Alternatively or additionally, determining the trajectory comprises determining a steering angle.

[0019] Alternatively or additionally, determining the trajectory includes determining the yaw rate of the towing vehicle. This makes it possible, depending on the vehicle's driving condition, to combine the image and the additional image into a complete image so that a desired section of the surroundings is displayed on the monitor.

[0020] According to at least one embodiment, the image and the additional image are combined by stitching. Thus, the image and / or the additional image are distorted such that a coherent overall image is displayed on the monitor.

[0021] According to at least one embodiment, the image and the further image are combined in such a way that a field of view is displayed in the overall image, which is composed of a field of view of the camera and a field of view of the further camera. In particular, the field of view of the further camera is supplemented by the field of view of the camera. The field of view of the further camera essentially corresponds, for example, to the field of view of the conventional exterior mirrors on one side. This field of view is supplemented by the field of view of the camera assigned to the rear of the trailer. This makes it possible, for example, to display the trailer partially transparently in the overall image on the monitor. In addition to displaying the surroundings to the side of the motor vehicle.

[0022] According to at least one further embodiment, the image and the further image are combined in such a way that a field of view is displayed in the overall image, which is composed of a field of view of the camera and a partial field of view of the further camera. In particular when cornering, the field of view of the further camera is covered by the trailer itself. This part of the field of view is not displayed in the overall image, for example. The partial field of view of the further camera is displayed, which continues to depict the surroundings next to the trailer. This partial field is supplemented by the field of view of the camera, so that overall the conventional field of view of the further camera can be simulated at an articulation angle of 0°.

[0023] The camera is designed as a removable camera. The camera can be mounted and removed from the trailer without causing any damage. This makes it possible to use the method and the camera-monitor system even in vehicles that are not equipped with a camera on the rear of the trailer. For calibration, a predetermined image content is determined in the image and in the additional image. The position of the camera relative to the additional camera is determined based on the determined image content. This makes it possible to compensate for tolerances during installation of the removable camera.

[0024] According to at least one embodiment, determining the image content includes determining side edges of the trailer in the image and in the additional image. Alternatively or additionally, side lights of the trailer are determined in the image and the additional image. Alternatively or additionally, clearly recognizable, prominent points of the surroundings depicted in the image and the additional image are used. It is also possible to calibrate the camera and the additional camera using calibration points in a workshop environment.

[0025] The camera-monitor system comprises the camera, the further camera, the monitor and the device which is designed to carry out a method according to at least one embodiment.

[0026] Advantages, features and developments described for the method also apply to the device and the system and vice versa.

[0027] Further advantages, features, and refinements will become apparent from the following examples, explained in conjunction with the figures. Identical, similar, and functionally identical elements may be provided with the same reference numerals throughout the figures.

[0028] They show: Figure 1 a schematic representation of a motor vehicle with a camera monitor system according to an embodiment, Figure 2 a schematic representation of the motor vehicle, and Figure 3 a schematic representation of the motor vehicle.

[0029] Figure 1 shows a schematic representation of a motor vehicle 101 according to one exemplary embodiment. The motor vehicle 101 is, for example, a truck with a towing vehicle 201 and a trailer 202. According to further exemplary embodiments, the motor vehicle 101 is a bus and / or passenger car with a trailer 202.

[0030] The motor vehicle 101 has a camera monitor system 100 as a mirror replacement system. The camera monitor system 100 has a camera 103 on each longitudinal side 106 of the motor vehicle 101. The cameras 103 serve to record a respective lateral environment 111 of the respective associated side 106.

[0031] The camera monitor system 100 has two monitors 104, each assigned to one of the sides 106. During operation, a section of the surroundings 111 is displayed on the respective assigned monitor 104. In particular, a field of view S1 of the camera 103 is displayed in whole or in part on the assigned monitor 104 in order to display the legally prescribed field of view classes, for example, field of view classes 2, 4, and 5. The image displayed on the monitor 104 is configured, in particular, to display a mirror image of a conventional exterior mirror and, in particular, to display further sections of the surroundings 111 that cannot be displayed with conventional exterior mirrors.

[0032] A device 110 is provided, which is signal-coupled to the cameras 103 and the monitors 104. The device 110 is configured to control both the cameras 103 and the monitors 104. For example, the device 110 is a control unit or part of a control unit of the motor vehicle 101.

[0033] An additional camera 102 is arranged on a rear side 105 of the trailer 202. During normal operation, the rear side 105 of the trailer 202 faces away from the towing vehicle 201. The camera 102 is signal-coupled to the device 110. The device 110 is configured to control the camera 102. For example, the camera 102 is wirelessly connected to the device 110. A cable connection is also possible.

[0034] According to embodiments, the device 110 is a separate component from the cameras 102, 103 and the monitors 104. According to further embodiments, the device 110 is a component of at least one of the cameras 102, 103 and / or at least one of the monitors 104.

[0035] The camera 102, which can also be referred to as a rearview camera, is, for example, a mobile camera. The mobile camera 102 can be mounted on the trailer 202 and removed again without causing damage and, for example, mounted on another trailer 202. According to further embodiments, the camera 102 is permanently mounted on the trailer 202.

[0036] The camera 102 is configured to capture an area of ​​the surroundings 111 behind the trailer 202. A field of view S2 of the camera 102 captures the surroundings 111 behind the trailer 202. The field of view S2 is, in particular, an area facing away from the towing vehicle 101.

[0037] The camera image from camera 102 is transmitted wired or wirelessly to device 110 in towing vehicle 201. An image from camera 102 is displayed on monitor 104 of the mirror replacement system. For example, monitor 104 is an existing monitor of a mirror replacement system.

[0038] The additional field of view S2 compared to the conventional mirror replacement system is adaptively combined with the field of view S1, for example by means of stitching. For this purpose, the device 110 determines in particular the trajectory of the motor vehicle 101. In this case, the speed of the motor vehicle 101 is taken into account. For example, the speed is determined from CAN information (CAN: controller area network). Alternatively or additionally, a steering angle of the motor vehicle 101 is taken into account. Alternatively or additionally, a bend angle 107 ( Figure 3) and / or a yaw rate are taken into account. The yaw rate is determined, for example, using a gyroscope sensor or another control unit.

[0039] As from Figure 2 As can be seen, the viewing areas S1 and S2 of the cameras 102 and 103 are at least partially combined to form an overall viewing area S3 and displayed on the associated monitor 104.

[0040] Depending on the vehicle trajectory, known parameters of the motor vehicle 101 such as length, height, and / or width, as well as the known viewing areas S1 and S2 of the cameras 102 and 103, it is determined how the viewing areas S1 and S2 are combined to form the overall viewing area S3 in order to also display the area at the rear 105 on the monitor 104 to the driver. The composition of the viewing areas S1 and S2 is changed and adapted, in particular, depending on the determined trajectory, for example, depending on the articulation angle between the towing vehicle 201 and the trailer 202.

[0041] The data connection between camera 102 and device 110 is established, for example, via a wired connection using a so-called BroadR-Reach network or wirelessly via a wireless local area network, for example, according to an IEEE 802.11 standard. When using a wireless network, the antennas can be located in the camera arm of camera 103 of the mirror replacement system.

[0042] The images from the different cameras 102 and 103 are combined, in particular, in the device 110 by means of stitching. Different views can be displayed on the monitor 104. For example, the image from the rear view camera is displayed in addition to the conventional representations of the legally prescribed field of view classes. For example, the field of view class 4 is displayed on the monitor 104, and the image from camera 102 is displayed alongside field of view class 2. For example, the image from camera 102 is displayed alongside field of view class 5 in addition to field of view class 4. A partially transparent display of the trailer 202 is also possible, as in Figure 2Here, the field of view S1 of camera 103 is supplemented with a partial field S2' of the field of view S2 of camera 102, resulting in an expanded overall field of view S3 compared to conventional mirror replacement systems. This enables a display on monitor 104 that simulates a view through trailer 202.

[0043] Various combinations of the fields of view S1 and S2 to form the overall field S3 are possible, particularly depending on a driving situation or the determined trajectory of the motor vehicle 101. For example, the field of view S1 of the camera 103 is designed such that it is aligned along the side 106 of the motor vehicle 101 and in particular of the trailer 202, or minimally overlaps or intersects it. The field of view S1 of the additional camera 103 is composed, for example, of the fields of view for the statutory field of view classes 2, 4, and 5. Multiple cameras can also be used for this purpose.

[0044] As in Figure 2As shown, by expanding the field of view S1 by the partial area S2, it is possible to display the conventional area S1 of the mirror replacement system, and at the same time, by combining it with the partial area S2', the area behind the trailer 202 can also be displayed. This would not be possible to display without the camera 102. The two fields of view S1 and S2' are combined to form the overall field of view S3 as if the overall field of view S3 were a single field of view with a larger aperture angle.

[0045] The images from the two cameras 102 and 103 are distorted to achieve the best possible image stitching. For this purpose, the position of the respective cameras 102 and 103 relative to each other and relative to the motor vehicle 101 is determined. During operation, for example, when cornering, the respective changed relative orientation of the cameras 102, 103 is taken into account, for example, according to the articulation angle 107, and the stitching of the images from cameras 102 and 103 is adjusted accordingly.

[0046] The figures schematically depict the fields of vision for the driver's side. The method and system can also be implemented for the passenger side.

[0047] Figure 3 shows an alternative composition of the viewing areas S1 and S2 of the cameras 102 and 103 or of sub-areas of the viewing areas.

[0048] For example, when the motor vehicle 101 is aligned straight, i.e., at a bend angle 107 of 180° ± 5°, only the field of view S1 is displayed on the monitor 104. When cornering, the bend angle 107 decreases. Depending on the bend angle 107, only a partial area S1' of the field of view S1 is displayed. The partial area S1' is the area of ​​the field of view S1 that is directed outward from a side edge 108 of the trailer 202. The remaining partial area S1" of the field of view S1, which only images the side 106 of the trailer 202, is not used to display the overall field of view S3 on the monitor 104. The area S1" is replaced by the partial area S2' of the camera 102, which images the surroundings 111 at the rear 105. Thus, the total field of view S3 always remains the same, regardless of the bend angle 107.

[0049] Instead of masking the trailer 202 from the overall field of view S3 using the partial field of view S2', it is also possible to still display the trailer 202, but almost transparently, whereby the transparency can be adjusted, for example. It is also possible to overlay the position of the trailer 202 in the displayed overall field of view S3 with color and transparency. Alternatively or additionally, it is also possible to display the edges of the trailer 202 on the monitor 104 with lines, for example as a box. Alternatively or additionally, a wireframe view is also possible. It is also possible to overlay an image filtered using conventional edge detection.

[0050] It is also possible to display the partial area S1" on the monitor 104 in addition to the total field of view S3 without the partial area S2'. For example, the two areas S1' and S1" as well as S1' and S2' are displayed side by side on the monitor 104. This allows the driver to consider both views.

[0051] It is also possible to display the viewing area S2 or S2' in one of the two lower Class 5 areas on the monitor 104 in addition to the conventional display of the mirror replacement system.

[0052] Alternatively or additionally, the image from camera 102 is displayed on a dedicated area of ​​monitor 104 or on an additional monitor. For example, the images from cameras 103 of the mirror replacement system are used as a supplement, and a combined view is displayed, for example, on the monitor of a navigation system.

[0053] To determine the position of camera 102, if camera 102 is designed as a removable camera, image contents from cameras 102 and 103 are used, for example. If the camera lens of camera 102 covers an angle greater than 180°, for example, the two side edges 108 of trailer 202 are detected in the image of camera 102. The edges of one side of the sides 106 of motor vehicle 101 are also detected, for example, in the image of the further camera 103. Alternatively or additionally, for example, side lights 109 are detected in the respective images of cameras 102, 103. The side lights 109 are also called rear marker lights. According to exemplary embodiments, the side lights 109 are controlled with a predetermined pattern during the determination of the position of cameras 102, 103 in order to be able to clearly identify the side lights 109 in the image and the further image. For example, a predefined blinking pattern is executed.

[0054] Other image content can also be used for calibration, for example road markings and / or calibration points and / or prominent image content that is easily recognizable in the environment.

[0055] The camera monitor system (100) according to the invention has at least two cameras assigned to the rear, wherein the cameras are designed as removable cameras. The method of the invention comprises determining a predetermined image content in the respective image of the at least two cameras, determining a position of the at least two cameras relative to one another and relative to the additional camera depending on the determined image content, wherein the two cameras establish a radio connection to one another, and then one of the cameras assumes the master role, so that a position determination of the two cameras relative to one another is possible.

[0056] The camera monitor system 100 according to the described embodiments enables the driver of the motor vehicle 101 to conveniently and intuitively display on the monitor 104 the area of ​​the surroundings 111 behind the trailer 202, which would conventionally not be visible without the camera 102.

Claims

1. Method for operating a camera monitor system (100) for a motor vehicle (101) with a trailer (202), in which the camera monitor system (100) has a camera (102), which is assigned to a rear side (105) of the trailer (202), and a further camera (103), which is assigned to a longitudinal side (106) of the motor vehicle (101), and a monitor (104) for presenting images of the cameras (102, 103), comprising: - providing an image of the camera (102), - providing a further image of the further camera (103), - ascertaining a specified image content in the image and in the further image, - ascertaining an overall image comprising the image and the further image in dependence on an ascertained trajectory of the motor vehicle (101), - presenting the overall image on the monitor (104), characterized in that the camera monitor system (100) has at least two cameras (102) assigned to the rear side (105), wherein the cameras (102) are embodied as removable cameras (102), comprising: - ascertaining a specified image content in the respective image of the at least two cameras (102), - ascertaining a position of the at least two cameras (102) relative to one another and relative to the further camera (103) in dependence on the ascertained image content, - wherein the two cameras (102) establish a radio connection to one another and then one of the cameras (102) assumes the master role, so that a position determination of the two cameras (102) relative to one another is possible.

2. Method according to Claim 1, in which ascertaining the trajectory comprises at least one of: - ascertaining a speed of the motor vehicle (101), - ascertaining a steering angle, - ascertaining an articulation angle (107) between a towing vehicle (201) of the motor vehicle (101) and the trailer (202), and - ascertaining a yaw rate of the trailer (202).

3. Method according to Claim 1 or 2, in which ascertaining the overall image comprises: - combining the image and the further image by way of stitching.

4. Method according to one of Claims 1 to 3, in which ascertaining the overall image comprises: - combining the image and the further image such that a total field of view (S3) that is composed of a field of view (S2, S2') of the camera (102) and a field of view (S1, S2', S1") of the further camera (103) is presented in the overall image.

5. Method according to one of Claims 1 to 4, in which ascertaining the overall image comprises: - combining the image and the further image such that a total field of view (S3) that is composed of a field of view (S2, S2') of the camera (102) and a partial region (S1') of the field of view (S1) of the further camera (103) is presented in the overall image.

6. Method according to one of Claims 1 to 5, in which ascertaining the image content comprises at least one of: - ascertaining side edges (108) of the trailer (202) in the image and the further image, and - ascertaining side lamps (109) of the trailer (202) in the image and the further image.

7. Device for a motor vehicle (101) with a trailer (202), the device being designed to carry out a method according to one of Claims 1 to 6.

8. Camera monitor system (100) for a motor vehicle (101) with a trailer (202), having: - a device (110) according to Claim 7, - two cameras (102) assigned to a rear side (105) of the trailer (202), wherein the cameras (102) are embodied as removable cameras (102), - a further camera (103), which is assigned to a longitudinal side (106) of the motor vehicle (101), - a monitor (104) for presenting images of the cameras (102, 103).