Method for aligning a camera of a vehicle camera monitoring system with a vehicle

Abstract

The present disclosure relates to a method for aligning a camera of a vehicle camera monitoring system with the vehicle. The present disclosure relates to a method for aligning a camera of a vehicle camera monitoring system (CMS), wherein a camera (1) is mounted on a side of a vehicle (20) and has a camera opening angle (10), the camera (1) comprises at least one image sensor having an entire area (11) of a full sensor field of view (FoV), including at least a first FoV in the form of a class II FoV in a first area (15) which is legally prescribed for a rear view device of a vehicle, the full sensor FoV is defined by the camera opening angle (10) and the alignment of the camera (1) relative to the vehicle (10), the class II FoV extends away from the vehicle from an edge of said side of the vehicle (20), and image data recorded by said at least one image sensor is forwarded to a monitor (2) of the CMS to display an image to a driver of the vehicle which includes the first area (15).

Description

[0001] This application relates to vehicle camera monitoring systems. In particular, this application relates to a method for aligning a camera of a vehicle camera monitoring system with a vehicle having the camera monitoring system.

[0002] An improved method is provided for a camera for a vehicle-mounted camera monitoring system, wherein the camera is mounted to one side of the vehicle and has a camera opening angle, the camera including at least one image sensor having the entire area of ​​the full sensor field of view (FoV), including at least a first FoV in the form of a Class II FoV legally required for vehicle rearview devices in a first area, the full sensor FoV being defined by the camera opening angle and the camera's alignment relative to the vehicle, a Class II FoV extending away from the edge of said side of the vehicle, and image data recorded by the at least one image sensor being forwarded to a monitor of the CMS to display an image including the first area to the driver of the vehicle.

[0003] In current general-purpose camera surveillance (CMS) systems, at least one camera is oriented such that the camera's field of view (FoV) has a Category II FoV that conforms to ECE Guideline R 46 and is substantially centered on the camera's optical axis, as shown in Figure 4 for a truck 20 traveling along a straight road 30. The truck 20 has a cab 22 and a trailer 24, with the CMS camera 1 mounted on the driver's side of the left side of the cab 22, and a monitor (not shown) for the CMS provided as an alternative to a rearview mirror, as a head-up display, etc. The known camera orientation has the advantage of recording a Category II FoV that defines a first area 15 to be displayed on the monitor with optimal clarity when traveling straight ahead. However, when the truck 20 is turning and has a Category IV FoV that defines a second area 17 to be displayed on the monitor, the clarity decreases because the large area 14 of the recorded segment cannot be used.

[0004] Figure 4 also illustrates the camera opening angle 10, which defines the entire sensor FoV region 11 of camera 1. This region 11 covers two edge regions 12 and 13 that cannot meet the sharpness requirements due to lensing effects. Between these two edge regions 12 and 13, there is a so-called wasted region 14, which is not used to maximize sharpness in the Class II FoV region 15, which partially overlaps with the Class IV FoV region 17. Regions 15 and 17 both begin along the side of truck 20, rather than along the side of trailer 24 of truck 20, but the first region 15 begins with an offset relative to the second region 17, which complies with the legal requirements of ECE Guideline R 46, for which more details on the different FoV categories can be found.

[0005] EP 2 765 031 B1 relates to a vehicle equipped with an observation system, particularly a commercial vehicle, wherein the observation system includes an image capture unit attached to the vehicle, a computing unit connected to the image capture unit, and a reproduction unit connected to the computing unit. The image capture unit includes a lens having an optical axis and a digital image sensing unit. The image capture unit is attached to the vehicle such that, in use, it senses an observation area on the side of the vehicle using at least a portion of a first legally defined field of view and at least a portion of a second legally defined field of view, wherein the first legally defined field of view is configured such that the driver can see a flat, horizontal portion of the road at least 5 meters wide, defined by a plane parallel to the median longitudinal vertical plane and passing through the outermost points of the vehicle on the driver's and passenger's sides respectively, and extending 30 meters behind the driver's eyes to the horizon, and allows the driver to see a 1-meter width of the road, defined by a plane parallel to the median longitudinal vertical plane and passing through the outermost points of the vehicle starting 4 meters behind the vertical plane passing through the driver's eyes. A lens is arranged relative to a digital image sensing unit such that the optical axis extends through a portion of a first legally defined field of view reproduced on the digital image sensing unit, wherein the optical axis is moved toward the vehicle such that the optical axis extends through a portion of the first legally defined field of view.

[0006] Another vision system for a vehicle according to WO 2013 / 019707 A1 includes at least one imaging sensor disposed on the vehicle and having an external field of view. The imaging sensor is operable to capture image data. The imaging sensor includes or is associated with a tilt sensor. At least one other tilt sensor is disposed on the vehicle. A processing system is operable to process the output of the tilt sensor to determine whether the at least one imaging sensor on the vehicle is aligned or misaligned.

[0007] A method for calibrating and / or aligning a camera mounted in a vehicle is known from EP 2 490 175 A1, wherein the camera includes an image sensor and a computing device that can calculate the values ​​of parameters in a region of interest in an image.

[0008] WO 2017 / 080753 A1 describes a system and method for creating a unified output image based on image data from multiple cameras with overlapping fields of view, by using a lookup table to transform raw images from each camera into corrected output images. Camera misalignment is mitigated by generating an updated lookup table based on feature point detection and matching in the overlapping fields of view.

[0009] DE 10 2014 006 153 A1 relates to a method for displaying an indirect field of view in a vehicle environment, particularly for displaying the rear view and field of view of a driver and / or passenger of a vehicle, especially a commercial vehicle, by means of a camera monitoring system. At least one camera of the camera monitoring system, particularly by means of a digital camera, records the overall field of view and displays it on a monitor, such that the overall field of view displayed on the monitor is divided into several adjacent local fields of view, each displayed with a different associated magnification and / or distortion.

[0010] US 10,284,818 B2 provides a calibration process that uses multiple parallel lines with markings or scales for multi-camera image stitching calibration. The calibration system can calibrate the cameras and systems while the vehicle is moving along a vehicle assembly line. Image capture is triggered by specific targets while the vehicle is moving. Optionally, the calibration system can utilize user-actuable input to provide a manual calibration process, which can be performed while the user views a display image derived from image data captured from the vehicle cameras. The captured image data includes areas where the camera fields of view overlap, with one or more targets or markings placed in these overlapping areas to facilitate the calibration of one or more cameras.

[0011] For Category II and Category IV, simple adjustments to the FoV of individual cameras are required while ensuring high resolution and FoV utilization.

[0012] The purpose of this disclosure is to further develop known methods to overcome the shortcomings of the prior art.

[0013] The objective of this disclosure is achieved by a method comprising aligning a camera and opening the camera at an angle such that a first region is positioned at the horizontal outermost position of an image captured by the camera's image sensor and / or at the vertical outermost position of an image displayed on a monitor.

[0014] According to one embodiment, the entire area of ​​the entire sensor FoV includes two edge regions that do not meet the image sharpness requirements on the monitor of the rear-view device, and a first region is arranged adjacent to the first edge region.

[0015] According to the invention, it is also proposed that the entire region spans an angle of at least 90°, preferably about 100°, and / or the first edge region and the second edge region each span an angle of up to 5°.

[0016] The method disclosed may also include, depending on the turning conditions of the vehicle, particularly when the vehicle is a truck having a cab and a trailer, translating a first area to a first area of ​​translation, wherein a camera is mounted in the cab and a Class II FoV extends from the edge of the trailer on said side of the vehicle, and when passing through a curve, the translation occurs after the trailer has bent at least 90° relative to the cab.

[0017] Furthermore, it is proposed to place the first region in the upper right region of the monitor of the simulated rearview mirror, and to place the translated first region in the upper left region of the monitor of the simulated rearview mirror.

[0018] Embodiments of this disclosure may further include: aligning a Class IV FoV extending away from the edge of said side of the vehicle, and aligning a Class II FoV such that a second area of ​​the Class IV FoV is displayed on a monitor.

[0019] It is also proposed that the method of this disclosure includes providing Class V FoV in a third region extending away from the edge of the vehicle from the vehicle between the second region and the second edge region.

[0020] According to this disclosure, a camera is associated with at least one lens, which preferably has uniform deformation or is a total variable lens.

[0021] This disclosure also proposes to arrange at least a first region in an image region having distortion reduced by the lens of the CMS, preferably arranging the first region, a first region translated by translation, a second region, and / or a third region.

[0022] Additionally, a vehicle, particularly a truck-type vehicle having a cab and a trailer, is provided, which has a CMS having a single camera mounted on one side of the vehicle and aligned according to the invention.

[0023] It was also proposed that a single camera be placed on the driver's side and / or that the vehicle have a single camera on each of its sides.

[0024] Therefore, this application provides a solution for a CMS having a single camera capable of recording at least Category II FoV and a lens with uniform distortion to display an image of a legally required area on the monitor of the CMS. The recorded image can be processed, for example, as described in US 2018 / 0224108, to simulate a flat and / or curved rearview mirror.

[0025] In short, according to this application, the camera orientation is selected such that the camera opening angle is aligned in such a way that the legally required image portion (i.e., Class II FoV) is located at the horizontal outermost position of the image captured by the camera's image sensor and / or the vertical outermost position of the image displayed on the monitor.

[0026] This choice is limited only when the edge areas of the camera's image sensor cannot meet the sharpness requirements due to lens effects. Therefore, the distance from the image to the image sensor can vary depending on the lens. Another limitation may be the physical edges of the image sensor when it has curvature.

[0027] The foregoing summary of the invention and the following detailed description will be better understood when read in conjunction with the accompanying drawings. For illustrative purposes, certain examples of this specification are shown in the drawings. However, it should be understood that the invention is not limited to the precise arrangements and apparatus shown. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of systems consistent with this specification and, together with the specification, serve to explain the advantages and principles consistent with the invention.

[0028] Other features, details, and advantages of the invention are explained in the appended claims, drawings, and the following description of preferred embodiments according to the head portion of the invention.

[0029] Figure 1a , 1b 1c is a schematic plan view of the truck of this disclosure traveling on the street, illustrating the full sensor FoV area of ​​the cameras of the vehicle camera monitoring system having straight trailer, curved trailer and Class V FoV respectively.

[0030] Figure 2a , 2b 2c is as shown on the monitor regarding Figure 1a , 1b A schematic diagram of the region defined by 1c;

[0031] Figure 3 This is a monitor view illustrating the effect of a total variable lens; and

[0032] Figure 4 is a schematic plan view of a truck driving on the street, illustrating the full sensor FoV area of ​​a camera in a prior art vehicle camera monitoring system.

[0033] Figure 1aA situation similar to that already discussed with respect to Figure 4 is shown, where truck 20 is traveling on road 30, with its trailer 24 positioned directly behind its cab 22. However, according to an embodiment of this disclosure, a camera 1 with a uniformly deformable lens (not shown) of the CMS of truck 20 is oriented such that the camera opening angle 10 is aligned such that the first region 15 of the legally required image portion (i.e., Category II FoV) of a rearview device simulating a rearview mirror of truck 20 is positioned at the horizontal outermost position of the image sensor of camera 1, immediately adjacent to the first edge region 12 of the FoV, which does not meet the sharpness requirement, primarily due to lens effects. This orientation also provides a sharp image not only in the required Category II region 15 but also in the Category II region 17.

[0034] Figure 2a The CMS monitor 2 is depicted, replacing the external rearview mirrors attached to the side of the truck 20. The displayed image of area 11, including all sensor FoV values ​​of first area 15 and second area 17, as a stitched image, provides an improved overall impression, allowing the driver of the truck 20 to see more detail. First area 15 is shown in the upper right region of monitor 2.

[0035] Compared to Figure 1a and 2a The described camera orientation allows for the maximum possible translation of the first region 15, such as... Figure 1b The diagram shows a curved trailer 24 that depicts a first translational region 16 for Class II FoV. For example, even with a camera opening angle of 100° and assuming that a 5° angled region of edge region 12 is unusable due to blurring or inaccuracy in the image portion, the end of trailer 24 can still be captured with a curvature greater than 90° because the first region 15 is translated to the first translational region 16. This translation results in a curve along the upper corner region of monitor 2. Figure 2b The arrow A in the diagram is shifted. Therefore, it is similar to... Figure 1b Compared to the case shown with the original first region 15 associated with the straight trailer 24, the camera orientation implemented by the method of this disclosure allows for a very high translational margin when turning forward and backward, so that the end of the trailer 24 remains within the translated first region 16.

[0036] Therefore, the alignment described above allows for correction of Type II FoV by translation along direction A during cornering, thereby achieving, as Figure 2b All the necessary information shown on monitor 2.

[0037] This alignment, in addition to enabling translation of Type II FoV, also allows for observation of... Figure 1c As shown and for Figure 2cThe monitor image further illustrates the third region 18 of the Class V FoV. This is because the FoV region indicated by region 14 for prior art camera alignment, as shown in Figure 4, is not wasted. The region obtained is between the second region 17 and the second edge region 13 for Class IV FoV, with the second edge region 13 freeing up enough area to allow recording of the third region 18.

[0038] Although a uniform deformable lens has been used to achieve [the desired effect] against [the specific problem] Figures 1a to 2c The alignment described herein can also be applied to panoramic lenses, where the camera is aligned, and furthermore, as... Figure 3 As shown, the first region 15 of the Class II FoV is arranged within the low-deformation region 19 of the lens. In this case, the translation range without loss of sharpness is smaller compared to the case of a lens with uniform deformation.

[0039] Those skilled in the art will recognize that changes can be made to the above embodiments without departing from the broad inventive concept of the invention. Therefore, it should be understood that the invention disclosed herein is not limited to the specific embodiments disclosed, but is intended to cover modifications within the spirit and scope of the invention.

[0040] Reference Mark

[0041] 1CMS camera

[0042] 2 monitors

[0043] 10 camera opening angles

[0044] 11 All sensor FoV areas

[0045] 12FoV edge region

[0046] 13FoV edge region

[0047] 14FoV unused area

[0048] 15 Category II FOV area

[0049] Translation region of Class II FoV

[0050] 17. Class IV FoV region

[0051] 18V FoV region

[0052] 19 Low deformation areas

[0053] 20 trucks

[0054] 22 driver's cab

[0055] 24 trailers

[0056] 30th Street

[0057] A. Translation direction

Claims

1. A method for aligning a camera in a vehicle camera monitoring system (CMS), wherein... The camera (1) is mounted on one side of the vehicle (20), and the camera (1) has a camera opening angle (10). The camera (1) includes at least one image sensor having a full area (11) that covers the entire sensor field of view (FoV), including at least a first FoV in the form of a Class II FoV for vehicle rearview devices as required by law in a first area (15). All sensor FoV is defined by the camera opening angle (10) and the alignment of the camera (1) relative to the vehicle (20). Category II FoV extends from the edge of the side of the vehicle (20) away from the vehicle, and Image data recorded by the at least one image sensor is forwarded to the monitor (2) of the CMS to display an image including the first region (15) to the driver of the vehicle. Its features are, Align the camera (1) and the camera opening angle (10) of the camera, such that the first region (15) is positioned at the horizontal outermost position of the image captured by the image sensor of the camera (1) and at the vertical outermost position of the image displayed on the monitor (2), wherein, The entire area (11) of the entire sensor FoV includes two edge regions (12, 13) that do not meet the image sharpness requirements on the monitor (2) of the rear-view device; and a first region (15) is arranged to be directly adjacent to the first edge region (12). Align the Class IV FoV extending away from the edge of the side of the vehicle (20) away from the vehicle, and align the Class II FoV, such that the second area (17) of the Class IV FoV is displayed on the monitor (2), and Depending on the turning behavior of the vehicle (20), the first region (15) is translated to the translated first region (16), where The first area (15) is positioned in the upper right area of ​​the monitor (2) simulating a rearview mirror, and The first translation area (16) is positioned in the upper left area of ​​the monitor (2) of the simulated rearview mirror.

2. The method as described in claim 1, wherein, The entire area (11) spans an angle of at least 90°, and / or The first edge region (12) and the second edge region (13) each span an angle of up to 5°.

3. The method of claim 2, wherein the entire region (11) spans an angle of approximately 100°.

4. The method as described in any one of claims 1-3, wherein when the vehicle (20) is a truck having a cab (22) and a trailer (24), the camera (1) is mounted to the cab (22), and the Class II FoV extends from the edge of the trailer (24) on said side of the vehicle (20), and when passing through a curve, the translation occurs after the trailer (24) has been bent at least 90° relative to the cab (22).

5. The method as described in any one of claims 1-3, further comprising: Class V FoV is provided in a third region (18) extending away from the edge of the vehicle (20) from the side of the vehicle (20) between the second region (17) and the second edge region (13).

6. The method as described in any one of claims 1-3, wherein The camera (1) is associated with at least one lens.

7. The method of claim 6, wherein the lens has uniform deformation.

8. The method of claim 6, further comprising: At least a first region (15) is arranged in the image region with distortion reduced by the lens of CMS.

9. The method of claim 8, further comprising arranging a first region (15) and a translated first region (16), a second region (17) and / or a third region (18) in an image region having distortion reduced by the lens of the CMS.

10. A vehicle (20), a truck-type vehicle having a cab (22) and a trailer (24), having a CMS having a single camera (1) mounted on one side of the vehicle (20) and aligned according to the method of any one of claims 1-9.

11. The vehicle (20) as claimed in claim 10, having a single camera (1) on the driver's side and / or a single camera on each side of the vehicle.

Images