A method, system, device and medium for adjusting color temperature consistency of electronic rearview mirror

By calibrating and fusing different camera images in the electronic rearview mirror, the color temperature difference is eliminated, the problem of inconsistent image color temperature in the prior art is solved, the image quality and the driver's field of view are improved, and the ambient light changes are achieved. Real-time response.

CN118644430BActive Publication Date: 2024-11-15AUTOMOBILE TECH (WUHAN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411112724.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-15
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

In the existing electronic rearview mirrors, the images captured by the camera may have obvious color temperature differences, causing the driver to feel unnatural color jumps when watching the display, affecting the driving experience and judgment, and in extreme weather. Stable image quality cannot be maintained under conditions.

Method used

By obtaining image data from different types of cameras, performing image calibration and fusion, determining the overlapping area of ​​the field of view and performing image column fusion, calculating and adjusting the color temperature value of the fused image, real-time response to ambient light changes, and eliminating the color temperature between different cameras difference.

Benefits of technology

Improve image quality and accuracy, expand the driver's effective field of view, provide clear visual information, eliminate color temperature differences between different cameras, and ensure image consistency and nature.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644430B_ABST
    Figure CN118644430B_ABST
Patent Text Reader

Abstract

The present invention proposes a method, system, device and medium for adjusting the color temperature consistency of an electronic rearview mirror, which belongs to the field of color temperature adjustment, and includes: S1, obtaining original image data at different viewing angles from different types of cameras of the electronic rearview mirror; S2, respectively performing image calibration on the original image data at different viewing angles to obtain a first image and a second image; S3, determining the overlapping area of ​​the field of view according to the first image and the second image, and performing image column fusion on the overlapping area of ​​the field of view to obtain a fused image; S4, calculating the color temperature value of the fused image, and adjusting the color temperature of the electronic rearview mirror according to the color temperature value of the fused image. The present application realizes real-time response to changes in ambient light and eliminates color temperature differences between different cameras by integrating image data of different types of cameras, calibrating and fusing them, and calculating and adjusting the color temperature based on the fused image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of color temperature adjustment, and in particular to a method, system, device and medium for adjusting the color temperature consistency of an electronic rearview mirror. Background Art

[0002] In recent years, with the sustained and rapid development of China's economy and society, the number of motor vehicles in the country has shown a trend of sustained and rapid growth. Against this background, the disadvantages of traditional physical rearview mirrors in certain scenarios have become increasingly prominent. For example, when driving at night, encountering headlights of rear vehicles, and in rainy days, physical rearview mirrors can easily cause the driver's field of vision to be blocked or limited, thus affecting driving safety.

[0003] At the same time, with the continuous advancement of technology and the promulgation of relevant regulations, the installation rate of electronic rearview mirrors (Camera Monitor System, CMS for short) in new cars has continued to increase. The CMS system has gradually covered the I, II, III, IV, V, and VI mirrors on the car, becoming an important means to improve driving safety. CMS is mainly composed of one or more cameras arranged outside the car and a display screen arranged in the cab. The corresponding field of view is presented in real time through the display screen, providing the driver with a wider and clearer information about the vehicle's surrounding environment. This system needs to adapt to various complex road conditions and weather conditions, including but not limited to various light scenes such as day, night, cloudy, rainy, foggy, evening, and dusk.

[0004] However, in the existing technology, since Class II and IV mirrors and Class V and VI mirrors cover different visual fields of the truck respectively, the images captured by these cameras may show obvious color temperature differences under different lighting conditions, causing the driver to experience unnatural color jumps when viewing the display screen, affecting the driving experience and judgment. In addition, under extreme weather conditions (such as heavy rain, dense fog, etc.), stable image quality cannot be maintained. In overlapping areas, the color, brightness and details of the image may be incoherent.

[0005] Therefore, finding a method that can adjust the color temperature difference of images captured by different cameras while ensuring image quality is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0006] In view of this, the present invention proposes a method, system, device and medium for adjusting the color temperature consistency of an electronic rearview mirror, which can integrate image data from different types of cameras, calibrate and fuse them, and calculate and adjust the color temperature based on the fused image, so as to achieve real-time response to changes in ambient light and eliminate color temperature differences between different cameras.

[0007] The technical solution of the present invention is achieved in this way:

[0008] In a first aspect, the present invention provides a method for adjusting the color temperature consistency of an electronic rearview mirror, comprising the following steps:

[0009] S1, obtaining raw image data at different viewing angles from different types of cameras of the electronic rearview mirror;

[0010] S2, performing image calibration on the original image data at different viewing angles to obtain a first image and a second image;

[0011] S3, determining a visual field overlap region according to the first image and the second image, and performing image column fusion on the visual field overlap region to obtain a fused image;

[0012] S4. Calculate the color temperature value of the fused image, and adjust the color temperature of the electronic rearview mirror according to the color temperature value of the fused image.

[0013] On the basis of the above technical solution, preferably, step S3 specifically includes:

[0014] Acquire a first set of image size information and a second set of image size information according to the first image and the second image respectively;

[0015] determining an overlapping area according to the first set of image size information and the second set of image size information;

[0016] Cut out the image parts of the overlapping area in the first image and the second image respectively to obtain a first cropped image and a second cropped image;

[0017] The first cropped image is scaled and aligned with the second cropped image to obtain a third cropped image and a fourth cropped image;

[0018] The image columns are fused according to the third cropped image and the fourth cropped image to obtain a fused image.

[0019] On the basis of the above technical solution, preferably, the image sequence fusion specifically includes:

[0020] Set the priority factor;

[0021] Calculate the fusion coefficient of each column based on the priority factor and the height of the overlapping area;

[0022] The same columns in the third cropped image and the fourth cropped image are fused according to the fusion coefficient of each column to obtain a fused image.

[0023] On the basis of the above technical solution, preferably, the calculation formula of the fusion is:

[0024]

[0025] in, represents the fused image, represents the third cropped image, represents the fourth cropped image, represents the i-th column pixel of the image, Represents the fusion coefficient of each column, i represents the number of columns, Indicates the image height after scaling and alignment.

[0026] On the basis of the above technical solution, preferably, the calculation formula of the fusion coefficient is:

[0027]

[0028] in represents the priority factor, It indicates the image height after scaling and alignment, and i indicates the number of columns.

[0029] More preferably, step S2 specifically includes:

[0030] S21, calibrating the original image data at different viewing angles respectively to obtain a first perspective transformation matrix and a second perspective transformation matrix;

[0031] S22. Perform coordinate transformation on the first perspective transformation matrix and the second perspective transformation matrix respectively to obtain a first image and a second image.

[0032] On the basis of the above technical solution, preferably, the coordinate transformation specifically includes:

[0033] Set the calibration coordinate system;

[0034] The first perspective transformation matrix and the second perspective transformation matrix are transformed according to the calibration coordinate system to obtain a first image and a second image; wherein the coordinate transformation formula is:

[0035]

[0036]

[0037] in, , and Represents the pixel coordinates of the original image data at the first viewing angle, represents the pixel coordinates of the first perspective transformation matrix, , and represents the pixel coordinates of the first image, , and represents the pixel coordinates of the original image data at the second viewing angle, represents the pixel coordinates of the second perspective transformation matrix, , and Represents the pixel coordinates of the second image.

[0038] In a second aspect, the present invention provides an electronic rearview mirror color temperature consistency adjustment system, which adopts the electronic rearview mirror color temperature consistency adjustment method as described above, comprising:

[0039] An image acquisition module configured to acquire raw image data at different viewing angles from different types of cameras of the electronic rearview mirror;

[0040] An image calibration module, configured to perform image calibration on original image data at different viewing angles to obtain a first image and a second image;

[0041] An image fusion module is configured to determine a visual field overlap region according to the first image and the second image, and fuse image sequences in the visual field overlap region to obtain a fused image;

[0042] The color temperature adjustment module is configured to calculate the color temperature values ​​of the fused images respectively, and adjust the color temperature of the electronic rearview mirror according to the color temperature values ​​of the fused images.

[0043] In a third aspect, the present invention provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein:

[0044] The processor, memory, and communication interface communicate with each other via the bus;

[0045] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method for adjusting the color temperature consistency of the electronic rearview mirror as described above.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the method for adjusting the color temperature consistency of an electronic rearview mirror as described above.

[0047] The electronic rearview mirror color temperature consistency adjustment method of the present invention has the following beneficial effects compared with the prior art:

[0048] (1) By integrating image data from different types of cameras and performing calibration and fusion, problems such as lens distortion and color deviation are corrected, image quality and accuracy are improved, and the driver's effective field of view is expanded. Based on the color temperature calculation and adjustment of the fused image, real-time response to changes in ambient light is achieved, eliminating the color temperature differences between different cameras and providing clear visual information;

[0049] (2) By obtaining and comparing the size information of the first image and the second image, the overlapping area between images of different perspectives is accurately determined, and the overlapping area is cropped and aligned to reduce redundant information, ensure that the fused images are completely matched, improve the fusion quality and naturalness, and effectively eliminate the differences between different camera perspectives, thereby improving the consistency of the overall image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 The present invention is a flow chart of a method for adjusting color temperature consistency of an electronic rearview mirror. DETAILED DESCRIPTION

[0052] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, the present invention provides a method for adjusting the color temperature consistency of an electronic rearview mirror, comprising the following steps:

[0054] S1. Acquire raw image data at different viewing angles from different types of cameras of the electronic rearview mirror.

[0055] It can be understood that the different types of cameras include II and IV mirror cameras and V and VI mirror cameras in the electronic rearview mirror. The image data under different viewing angles include image data under a first viewing angle, i.e., image data obtained by II and IV mirror cameras in the electronic rearview mirror, and image data under a second viewing angle, i.e., image data obtained by V and VI mirror cameras in the electronic rearview mirror.

[0056] S2. Perform image calibration on original image data at different viewing angles to obtain a first image and a second image.

[0057] Specifically, step S2 specifically includes:

[0058] S21, calibrating the original image data at different viewing angles respectively to obtain a first perspective transformation matrix and a second perspective transformation matrix;

[0059] S22. Perform coordinate transformation on the first perspective transformation matrix and the second perspective transformation matrix respectively to obtain a first image and a second image.

[0060] By calibrating and converting the original image data at different viewing angles, image distortion caused by factors such as camera installation position and angle is eliminated, and image data from different cameras (II, IV type mirrors and V, VI type mirrors) are mapped to a unified coordinate system to compensate for the inherent differences between different cameras, such as lens distortion and field of view differences, thereby improving the consistency between images from different viewing angles. By using data calibration and coordinate conversion, the image displayed to the driver more accurately reflects the actual scene.

[0061] Furthermore, the coordinate transformation specifically includes:

[0062] Set the calibration coordinate system;

[0063] The first perspective transformation matrix and the second perspective transformation matrix are transformed according to the calibration coordinate system to obtain a first image and a second image; wherein the coordinate transformation formula is:

[0064]

[0065]

[0066] in, , and Represents the pixel coordinates of the original image data at the first viewing angle, represents the pixel coordinates of the first perspective transformation matrix, , and represents the pixel coordinates of the first image, , and represents the pixel coordinates of the original image data at the second viewing angle, represents the pixel coordinates of the second perspective transformation matrix, , and Represents the pixel coordinates of the second image.

[0067] Specifically,

[0068]

[0069]

[0070] in, Indicates the horizontal scaling of the original image data at the first viewing angle, Indicates the horizontal shearing of the original image data from the first perspective, Indicates the horizontal translation of the original image data from the first perspective, Indicates the vertical shearing of the original image data from the first perspective, Indicates the vertical scaling of the original image data at the first viewing angle, Represents the vertical translation of the original image data from the first perspective, Indicates the horizontal perspective effect of the original image data from the first perspective, Indicates the vertical perspective effect of the original image data from the first perspective, Represents the overall scaling factor of the original image data at the first viewing angle; represents the horizontal scaling of the original image data at the second viewing angle, represents the horizontal shearing of the original image data at the second viewing angle, represents the horizontal translation of the original image data at the second viewing angle, represents the vertical shearing of the original image data at the second viewing angle, represents the vertical scaling of the original image data at the second viewing angle, represents the vertical translation of the original image data at the second viewing angle, Indicates the horizontal perspective effect of the original image data under the second viewing angle, Indicates the vertical perspective effect of the original image data under the second viewing angle, Represents the overall scaling factor of the original image data at the second viewing angle.

[0071] By converting images from different types of cameras into the same calibration coordinate system, the differences between images from different perspectives are reduced, the overall visual consistency is improved, and images from different perspectives can be compared and adjusted under the same benchmark, achieving high-quality correction and unification of images from different types of cameras.

[0072] S3. Determine a visual field overlap region according to the first image and the second image, and fuse the image sequences of the visual field overlap region to obtain a fused image.

[0073] Specifically, step S3 specifically includes:

[0074] Acquire a first set of image size information and a second set of image size information according to the first image and the second image respectively;

[0075] determining an overlapping area according to the first set of image size information and the second set of image size information;

[0076] Cut out the image parts of the overlapping area in the first image and the second image respectively to obtain a first cropped image and a second cropped image;

[0077] The first cropped image is scaled and aligned with the second cropped image to obtain a third cropped image and a fourth cropped image;

[0078] The image columns are fused according to the third cropped image and the fourth cropped image to obtain a fused image.

[0079] By acquiring and comparing the size information of the first image and the second image, the overlapping area between images of different perspectives is accurately determined, and the overlapping area is cropped and aligned to reduce redundant information, ensure that the fused images are completely matched, improve the fusion quality and naturalness, and effectively eliminate the differences between different camera perspectives to improve the consistency of the overall image.

[0080] In one embodiment of the present application, the calculation formula for the first cropped image and the second cropped image is:

[0081] ;

[0082] ;

[0083] in, represents the first cropped image, represents the first image, , , , Respectively represent the row start point, row end point, column start point, and column end point of the first cropped image; represents the second cropped image, represents the second image, , , , Respectively represent the row start point, row end point, column start point, and column end point of the second cropped image;

[0084] In one embodiment of the present application, the formula for scaling alignment is:

[0085] ;

[0086] ;

[0087] in, represents the third cropped image, represents the image scaling function, Indicates the image height after scaling and alignment. Indicates the image width after scaling and alignment; Represents the fourth cropped image.

[0088] Furthermore, the image sequence fusion specifically includes:

[0089] Set the priority factor;

[0090] Calculate the fusion coefficient of each column based on the priority factor and the height of the overlapping area;

[0091] The same columns in the third cropped image and the fourth cropped image are fused according to the fusion coefficient of each column to obtain a fused image.

[0092] By setting the priority factor and calculating the fusion coefficient of each column according to the priority factor, we can achieve refined control over the image fusion process, thereby obtaining a more natural and smooth fusion effect and avoiding the abruptness of the fusion boundary. Image column fusion can better retain the detail information of the original image during the fusion process, improve the clarity and information integrity of the fused image, eliminate the differences in color, brightness, etc. between different cameras, and improve the consistency of the overall image.

[0093] It will be appreciated that the priority factors are calculated based on the design parameters of different types of cameras.

[0094] In one embodiment of the present application, the weight multiples of II and IV are V and VI. times, the priority factor is

[0095]

[0096] In one embodiment of the present application, =1.25, indicating that the design parameters of the II and IV mirrors are 1.25 times the design parameters of the V and VI mirrors. is 0.2.

[0097] Understandably, when When it is less than 1, it means that the weight of Class V and VI mirrors is higher.

[0098] Furthermore, the calculation formula of the fusion is:

[0099]

[0100] in, represents the fused image, represents the third cropped image, represents the fourth cropped image, represents the i-th column pixel of the image, Represents the fusion coefficient of each column, i represents the number of columns, Indicates the image height after scaling and alignment.

[0101] Furthermore, the calculation formula of the fusion coefficient is:

[0102]

[0103] in represents the priority factor, It indicates the image height after scaling and alignment, and i indicates the number of columns.

[0104] S4. Calculate the color temperature value of the fused image, and adjust the color temperature of the electronic rearview mirror according to the color temperature value of the fused image.

[0105] Specifically, the calculation formula for the color temperature value is:

[0106]

[0107] in, represents the smoothing factor, Indicates the color temperature value of the previous fused image. Represents the estimated color temperature of the fused image.

[0108] By integrating image data from different types of cameras and performing calibration and fusion, problems such as lens distortion and color deviation are corrected, image quality and accuracy are improved, and the driver's effective field of view is expanded. Based on the color temperature calculation and adjustment of the fused image, real-time response to changes in ambient light is achieved, eliminating color temperature differences between different cameras and providing clear visual information.

[0109] The present invention also provides an electronic rearview mirror color temperature consistency adjustment system, which adopts the electronic rearview mirror color temperature consistency adjustment method as described above, and comprises:

[0110] An image acquisition module configured to acquire raw image data at different viewing angles from different types of cameras of the electronic rearview mirror;

[0111] An image calibration module, configured to perform image calibration on original image data at different viewing angles to obtain a first image and a second image;

[0112] An image fusion module is configured to determine a visual field overlap region according to the first image and the second image, and fuse image sequences in the visual field overlap region to obtain a fused image;

[0113] The color temperature adjustment module is configured to calculate the color temperature values ​​of the fused images respectively, and adjust the color temperature of the electronic rearview mirror according to the color temperature values ​​of the fused images.

[0114] The present invention also provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein,

[0115] The processor, memory, and communication interface communicate with each other via the bus;

[0116] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method for adjusting the color temperature consistency of the electronic rearview mirror as described above.

[0117] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the method for adjusting the color temperature consistency of the electronic rearview mirror as described above.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for adjusting the color temperature consistency of an electronic rearview mirror, characterized in that: The following steps are involved: S1, obtaining raw image data at different viewing angles from different types of cameras of the electronic rearview mirror; S2, performing image calibration on the original image data at different viewing angles to obtain a first image and a second image; S3, determining a visual field overlap region according to the first image and the second image, and performing image column fusion on the visual field overlap region to obtain a fused image; S4, calculating the color temperature value of the fused image, and adjusting the color temperature of the electronic rearview mirror according to the color temperature value of the fused image; Step S3 specifically includes: Acquire a first set of image size information and a second set of image size information according to the first image and the second image respectively; determining an overlapping area according to the first set of image size information and the second set of image size information; Cut out the image parts of the overlapping area in the first image and the second image respectively to obtain a first cropped image and a second cropped image; The first cropped image is scaled and aligned with the second cropped image to obtain a third cropped image and a fourth cropped image; Performing image column fusion according to the third cropped image and the fourth cropped image to obtain a fused image; The image column fusion specifically includes: Set the priority factor; Calculate the fusion coefficient of each column based on the priority factor and the height of the overlapping area; fusing the same columns in the third cropped image and the fourth cropped image according to the fusion coefficient of each column to obtain a fused image; The calculation formula of the fusion is: ; in, represents the fused image, represents the third cropped image, represents the fourth cropped image, represents the i-th column pixel of the image, Represents the fusion coefficient of each column, i represents the number of columns, Indicates the image height after scaling and alignment; The calculation formula of the fusion coefficient is: ; in represents the priority factor, It indicates the image height after scaling and alignment, and i indicates the number of columns.

2. The method for adjusting color temperature consistency of an electronic rearview mirror according to claim 1, characterized in that: Step S2 specifically includes: S21, calibrating the original image data at different viewing angles respectively to obtain a first perspective transformation matrix and a second perspective transformation matrix; S22. Perform coordinate transformation on the first perspective transformation matrix and the second perspective transformation matrix respectively to obtain a first image and a second image.

3. A method for adjusting color temperature consistency of an electronic rearview mirror as claimed in claim 2, characterized in that: The coordinate transformation specifically includes: Set the calibration coordinate system; The first perspective transformation matrix and the second perspective transformation matrix are transformed according to the calibration coordinate system to obtain a first image and a second image; wherein the coordinate transformation formula is: ; ; in, , and Represents the pixel coordinates of the original image data at the first viewing angle, represents the pixel coordinates of the first perspective transformation matrix, , and represents the pixel coordinates of the first image, , and represents the pixel coordinates of the original image data at the second viewing angle, represents the pixel coordinates of the second perspective transformation matrix, , and Represents the pixel coordinates of the second image.

4. An electronic rearview mirror color temperature consistency adjustment system, characterized in that: The method for adjusting color temperature consistency of an electronic rearview mirror according to any one of claims 1 to 3 comprises: An image acquisition module configured to acquire raw image data at different viewing angles from different types of cameras of the electronic rearview mirror; An image calibration module, configured to perform image calibration on original image data at different viewing angles to obtain a first image and a second image; An image fusion module is configured to determine a visual field overlap region according to the first image and the second image, and fuse image sequences in the visual field overlap region to obtain a fused image; The color temperature adjustment module is configured to calculate the color temperature values ​​of the fused images respectively, and adjust the color temperature of the electronic rearview mirror according to the color temperature values ​​of the fused images.

5. An electronic device, characterized in that: include: At least one processor, at least one memory, a communication interface and a bus; wherein, The processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method for adjusting the color temperature consistency of the electronic rearview mirror as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the method for adjusting the color temperature consistency of the electronic rearview mirror as described in any one of claims 1 to 3.