Multi-camera color consistency correction method and device

Abstract

The present application provides a multi-camera color consistency correction method and apparatus, the method comprising: acquiring a first image captured by a first camera and a second image captured by a second camera; determining at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, wherein the image information includes at least one of the color information of the first image and the ambient light source of the first image, the color mapping parameter indicating the color conversion relationship between the image captured by the first camera and the image captured by the second camera, the N color mapping parameters corresponding one-to-one to N standard light sources, where N is a positive integer; and performing color consistency correction on the second image based on the at least one color mapping parameter to obtain a corrected image. The multi-camera color consistency correction method of the embodiments of the present application can achieve real-time color consistency correction of the second image under different lighting conditions.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and device for multi-camera color consistency correction. Background Art

[0002] To meet user demands for high-quality imaging, panoramas, image stitching, and recognition, multiple cameras are often used to capture the same scene. In an increasing number of products and systems, multiple cameras are often installed to capture images. For example, equipping mobile devices with multiple cameras of varying focal lengths and characteristics can provide users with high-quality images. However, due to differences in the device itself, its field of view, and debugging style, the colors of the images actually captured by each camera may vary. The degree of color difference varies with changes in lighting and the shooting scene. Therefore, in practical applications, the colors of images captured by multiple cameras must be adjusted in real time based on changes in lighting and the shooting scene to ensure consistent color across the images. Summary of the Invention

[0003] The present application provides a multi-camera color consistency correction method and device, which can achieve color consistency adjustment of images acquired by multiple cameras under different lighting conditions.

[0004] In a first aspect, a multi-camera color consistency correction method is provided, the method comprising: acquiring a first image captured by a first camera and a second image captured by a second camera; determining at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, the image information comprising color information of the first image and at least one of an ambient light source of the first image, the color mapping parameter indicating a color conversion relationship between the image captured by the first camera and the image captured by the second camera, the N color mapping parameters corresponding one-to-one to N standard light sources, where N is a positive integer; performing color consistency correction on the second image based on the at least one color mapping parameter to obtain a corrected image.

[0005] By calibrating color mapping parameters under different standard light sources, we can obtain color mapping parameters corresponding to each standard light source. During the color consistency correction process, combining at least one color mapping parameter with the second image for color consistency correction can achieve real-time color correction under different lighting conditions, improving the accuracy of color consistency correction.

[0006] In one possible implementation, before determining at least one color mapping parameter from N color mapping parameters, the method further includes: determining a first calibration image and a second calibration image under each standard light source, where the first calibration image and the second calibration image are color card images generated based on spectral response curves of the first camera and the second camera, respectively; and determining the color mapping parameters corresponding to each standard light source based on the first calibration image and the second calibration image.

[0007] Based on the spectral response of the camera, calibration images under different light source conditions can be simulated and generated, which reduces the time cost of capturing calibration data, reduces the unstable factors introduced by capturing, and improves the stability and accuracy of color mapping parameter calibration.

[0008] In another possible implementation, performing color consistency correction on the second image based on at least one color mapping parameter includes: determining a common image area of ​​the first image and the second image; determining a color compensation parameter based on the common image area and at least one color mapping parameter; determining a white balance compensation parameter based on the common image area; and performing color consistency correction on the second image based on the white balance parameter and the color compensation parameter.

[0009] The gray area of ​​the second image can be corrected according to the white balance compensation parameters, and the color part of the second image can be corrected according to the color compensation parameters. By combining white balance compensation and color compensation, the gray area and the color area of ​​the second image can both be corrected for color consistency, thereby improving the color correction effect.

[0010] In another possible implementation, determining the common image area of ​​the first image and the second image includes: determining a search area according to the relative positions and fields of view of the first camera and the second camera; and determining the common image area according to the search area.

[0011] When the calibration information such as the position information of multiple cameras is known, the image matching range can be determined in combination with the camera calibration information to improve the accuracy of image matching and search efficiency.

[0012] In another possible implementation, determining a color compensation parameter based on a common image area and at least one color mapping parameter includes: applying N color mapping parameters to the common image area in the second image, respectively, to obtain N third images; calculating the color difference between the common image area in the first image and each third image, respectively; determining at least one color mapping parameter based on the color difference, the at least one color mapping parameter being a color mapping parameter corresponding to at least one third image with the smallest color difference; determining a target color mapping parameter based on the at least one color mapping parameter, the target color mapping parameter being a weighted value of the at least one color mapping parameter; and determining a color compensation parameter based on the target color mapping parameter.

[0013] In another possible implementation, determining a color compensation parameter based on a common image area and at least one color mapping parameter includes: determining an ambient light source based on a white balance gain of the common image area in a first image; determining at least one color mapping parameter corresponding to at least one standard light source based on the ambient light source, wherein the difference between the at least one standard light source and the ambient light source is minimized; determining a target color mapping parameter based on the at least one color mapping parameter, wherein the target color mapping parameter is a weighted value of the at least one color mapping parameter; and determining a color compensation parameter based on the target color mapping parameter.

[0014] The color compensation parameters can be determined in a variety of ways, and a plurality of related color mapping parameters can be fused according to actual lighting conditions. The target color mapping parameters determined in this way are more accurate.

[0015] In another possible implementation, the color compensation parameter is a target color mapping parameter, or the color compensation parameter is a product of the target color mapping parameter, the white balance gain of the first image, and the color restoration parameter.

[0016] In another possible implementation, determining the white balance compensation parameters based on the common image area includes: separately determining the weighted average values ​​or weighted color histograms of the pixels of the common image area in the first image in the three color channels; separately determining the weighted average values ​​or weighted color histograms of the pixels of the common image area in the second image in the three color channels; and determining the white balance compensation parameters based on the weighted average values ​​or weighted color histograms of the three color channels.

[0017] In another possible implementation, before determining the white balance compensation parameters based on the common image area, the method further includes: dividing the common image area into M blocks based on the spatial position, color similarity, and edge information of the common image area, where M is a positive integer; determining the white balance compensation parameters based on the common image area includes: respectively determining the weighted average values ​​or weighted color histograms of the image blocks in the common image area in the first image in three color channels; respectively determining the weighted average values ​​or weighted color histograms of the image blocks in the common image area in the second image in three color channels; and determining the white balance compensation parameters based on the weighted average values ​​or weighted color histograms of the three color channels.

[0018] By dividing the image into blocks, the calculation can be simplified and the efficiency of color consistency correction can be improved.

[0019] In a second aspect, a multi-camera color consistency correction device is provided, which includes: an acquisition module for acquiring a first image taken by a first camera and a second image taken by a second camera; a determination module for determining at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, the image information including the color information of the first image and at least one of the ambient light sources of the first image, the color mapping parameters indicating a color conversion relationship between the image taken by the first camera and the image taken by the second camera, the N color mapping parameters corresponding one-to-one to N standard light sources, and N being a positive integer; a correction module for performing color consistency correction on the second image based on at least one color mapping parameter to obtain a corrected image.

[0020] By calibrating color mapping parameters under different standard light sources, we can obtain color mapping parameters corresponding to each standard light source. During the color consistency correction process, combining at least one color mapping parameter with the second image for color consistency correction can achieve real-time color correction under different lighting conditions, improving the accuracy of color consistency correction.

[0021] In one possible implementation, before determining at least one color mapping parameter from N color mapping parameters, the determination module is specifically configured to: determine a first calibration image and a second calibration image under each standard light source, where the first calibration image and the second calibration image are color card images generated based on spectral response curves of the first camera and the second camera, respectively; and determine the color mapping parameters corresponding to each standard light source based on the first calibration image and the second calibration image.

[0022] Based on the spectral response of the camera, calibration images under different light source conditions can be simulated and generated, which reduces the time cost of capturing calibration data, reduces the unstable factors introduced by capturing, and improves the stability and accuracy of color mapping parameter calibration.

[0023] In another possible implementation, the determination module is specifically used to: determine the common image area of ​​the first image and the second image; determine the color compensation parameters based on the common image area and at least one color mapping parameter; determine the white balance compensation parameters based on the common image area; and the correction module is specifically used to: perform color consistency correction on the second image based on the white balance parameters and the color compensation parameters.

[0024] White balance compensation can correct the gray area of ​​the second image, and color compensation can correct the color part of the second image. By combining white balance compensation and color compensation, both the gray area and the color area of ​​the second image can be corrected for color consistency, thereby improving the color correction effect.

[0025] In another possible implementation, the determination module is specifically configured to: determine a search area according to the relative positions and fields of view of the first camera and the second camera; and determine the common image area according to the search area.

[0026] When the calibration information such as the position information of multiple cameras is known, the image matching range can be determined in combination with the camera calibration information to improve the accuracy of image matching and search efficiency.

[0027] In another possible implementation, the determination module is specifically used to: apply N color mapping parameters to the common image area in the second image respectively to obtain N third images; calculate the color difference between the common image area in the first image and each third image respectively; determine at least one color mapping parameter based on the color difference, and the at least one color mapping parameter is the color mapping parameter corresponding to at least one third image with the smallest color difference; determine the target color mapping parameter based on the at least one color mapping parameter, and the target color mapping parameter is a weighted value of the at least one color mapping parameter; determine the color compensation parameter based on the target color mapping parameter.

[0028] In another possible implementation, the determination module is specifically used to: determine the ambient light source based on the white balance result gain of the common image area in the first image; determine at least one color mapping parameter corresponding to at least one standard light source based on the ambient light source, and the difference between the at least one standard light source and the ambient light source is minimized; determine the target color mapping parameter based on the at least one color mapping parameter, and the target color mapping parameter is a weighted value of the at least one color mapping parameter; and determine the color compensation parameter based on the target color mapping parameter.

[0029] Color compensation parameters can be determined in a variety of ways, and related color mapping parameters can be fused according to actual lighting conditions. The target color mapping parameters determined in this way are more accurate.

[0030] In another possible implementation, the color compensation parameter is a target color mapping parameter, or the color compensation parameter is a product of the target color mapping parameter, the white balance gain of the first image, and the color restoration parameter.

[0031] In another possible implementation, the determination module is specifically used to: respectively determine the weighted average values ​​or weighted color histograms of the pixels in the common image area of ​​the first image in the three color channels; respectively determine the weighted average values ​​or weighted color histograms of the pixels in the common image area of ​​the second image in the three color channels; and determine the white balance compensation parameters based on the weighted values ​​or weighted color histograms of the three color channels.

[0032] In another possible implementation, before determining the white balance compensation parameters based on the common image area, the determination module is further used to: divide the common image area into M blocks based on the spatial position, color similarity and edge information of the common image area, where M is a positive integer; determine the weighted average values ​​or weighted color histograms of the image blocks in the common image area in the first image in the three color channels; determine the weighted average values ​​or weighted color histograms of the image blocks in the common image area in the second image in the three color channels; and determine the white balance compensation parameters based on the weighted values ​​or weighted color histograms of the three color channels.

[0033] By dividing the image into blocks, the calculation can be simplified and the efficiency of color consistency correction can be improved.

[0034] In a third aspect, a computer-readable medium is provided, which stores a program code for execution by a device, wherein the program code includes instructions for executing the color consistency correction method in the first aspect or any implementation of the first aspect.

[0035] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the color consistency correction method in the first aspect or any one of the implementations of the first aspect.

[0036] In a fifth aspect, a chip is provided, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the color consistency correction method in the above-mentioned first aspect or any one of the implementations of the first aspect.

[0037] Optionally, as an implementation method, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the color consistency correction method in the first aspect or any one of the implementation methods of the first aspect.

[0038] In a sixth aspect, a device is provided, comprising: a processor and a memory, the memory being used to store the computer program code, and when the computer program code runs on the processor, the device executes the color consistency correction method in the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is an application scenario of the embodiment of the present application;

[0040] Figure 2 1 is a flow chart of a multi-camera color consistency correction method according to an embodiment of the present application;

[0041] Figure 3 1 is a flow chart of a method for determining color mapping parameters according to an embodiment of the present application;

[0042] Figure 4 is a flow chart of another method for determining color mapping parameters according to an embodiment of the present application;

[0043] Figure 5 is a flow chart of a second image color correction method according to an embodiment of the present application;

[0044] Figure 6 is a schematic diagram of image matching in an embodiment of the present application;

[0045] Figure 7 is a flowchart of another second image color correction method according to an embodiment of the present application;

[0046] Figure 8 This is a schematic structural diagram of a multi-camera color consistency correction device according to an embodiment of the present application;

[0047] Figure 9 Schematic diagram of the structure of another multi-camera color consistency correction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0049] To meet user demands for high-quality imaging, panoramic views, image stitching, and recognition, it's often necessary to process images captured by multiple cameras. For example, mobile phones, cars, computers, and surveillance systems often utilize multiple cameras (or multiple cameras) to meet these requirements. Due to differences in device characteristics, field of view, and debugging styles, the actual image color captured by each camera may vary. The degree of color variation varies with lighting and scene conditions, necessitating real-time adjustments to these changes in real-time.

[0050] In existing multi-camera color correction technology, the image acquired by one of the multiple cameras is usually used as the main image (for example, the first image), and the other images are used as auxiliary images (for example, the second image). The parameters of the three color channels of the first image and the parameters of the three color channels of the second image are mapped to each other, and the parameters of the three color channels of the second image are adjusted separately. However, this adjustment processes the parameters of each color channel separately. For example, the compensation component of the red (red, R) channel is calculated, and the compensation component of the R channel is applied to the R channel of the second image. When performing color compensation on the R channel of the second image, the influence of the green (green, G) and blue (blue, B) channels is often not considered. When the spectrums of different cameras are significantly different, the adjustment of a single channel has a poor effect on image color consistency.

[0051] Furthermore, under varying lighting conditions, the color mapping parameters between different cameras often differ. The color mapping parameters indicate the color conversion relationship between an image acquired by one camera (e.g., the first camera) and an image acquired by another camera (e.g., the second camera). Existing multi-camera color correction techniques, while taking varying lighting conditions into account when calibrating the color mapping parameters, only consider a single parameter when adjusting image color consistency. This makes it impossible to achieve color consistency across multiple images when lighting changes.

[0052] The embodiments of the present application provide a multi-camera color consistency correction method and apparatus, which can achieve real-time correction of the color consistency of images acquired by multiple cameras under different lighting environments.

[0053] The embodiments of the present application can be applied to multi-camera (or multi-camera) scenarios, wherein the multiple cameras can be located on the same device, for example, a terminal device such as a mobile phone equipped with multiple cameras, a vehicle equipped with multiple cameras, etc.; the multiple cameras can also be located on different devices, for example, using cameras on multiple different devices to shoot the same scene, etc., and the embodiments of the present application are not limited to this.

[0054] Below Figure 1 Take this as an example to introduce the application scenario of the embodiment of this application. Figure 1As shown, one of the multiple cameras is selected as the primary camera (e.g., referred to as the "first camera"), and the other cameras are selected as auxiliary cameras (e.g., referred to as the "second camera"). Light reflected from the scene is projected onto two image sensors through the camera lenses to form digital image signals. The two image signals are then passed through their respective pre-processing modules, where operations such as bad pixel correction, black level compensation, shadow correction, white balance gain calculation, and color restoration parameter calculation can be performed. The two images and related parameters then enter the color correction module. The image from the first camera is used as the target image effect. In the color correction module, the color compensation parameters and white balance compensation parameters of the second image acquired by the second camera are calculated by combining the various parameters of the first image and the calibrated color mapping parameters. Subsequently, the second image is adjusted based on the calculated color compensation parameters and white balance compensation parameters to obtain a corrected image with consistent color with the first image. In some embodiments, the first image and the color-corrected second image can be subjected to post-processing such as gamma transformation and dynamic adjustment in the post-processing module to obtain a first image and a corrected image with better display effects.

[0055] Figure 2 FIG. 1 is a flow chart of a multi-camera color consistency correction method according to an embodiment of the present application. Figure 2 As shown, the multi-camera color consistency correction method of the embodiment of the present application includes steps S210 to S220.

[0056] S210: Determine N color mapping parameters corresponding to N standard light sources.

[0057] Among them, N standard light sources correspond one to one with N color mapping parameters.

[0058] The color mapping parameters can represent the conversion relationship between the colors of two images (for example, image 1 and image 2). The description of the color mapping parameters may include linear transformation matrices, high-order polynomials, neural networks, etc. For example, the image of the embodiment of the present application adopts the RGB color mode, and each pixel in the image is described by the numerical values ​​of the three color channels R, G, and B. In this case, the color mapping parameters can be a 3×3 linear transformation matrix. The color mapping parameters in matrix form are shown in formula (1):

[0059]

[0060] Among them, (R 1i G 1i B 1i ) represents the values ​​of the R, G, and B color channels of the i-th pixel in the image;

[0061] (R 2i G 2i B2i ) represents the values ​​of the R, G, and B color channels of the i-th pixel in image 2;

[0062] The 3×3 matrix T is the color mapping parameter, which represents the color mapping relationship between image 1 and image 2.

[0063] When the color mapping parameters are in the matrix form of formula (1), for N standard light source conditions, the embodiment of the present application determines N matrices for each of the N light sources as color mapping parameters under different light source conditions. Specifically, the standard light sources may include American window spotlights (A), simulated sunlight (D50), simulated blue sky daylight (D65), simulated northern average sunlight (D75), simulated horizontal daylight (H), American warm white store light sources (U30), European, Japanese, and Chinese store light sources (TL84), American cool white fluorescent (CWF), and other light sources.

[0064] In the embodiment of the present application, since color mapping parameters are calibrated for different standard light sources, when performing color consistency correction, appropriate color mapping parameters can be selected according to different lighting conditions to perform color consistency correction on the second image, and the dynamic adaptability is good.

[0065] The following combination Figure 3 and Figure 4 This section describes in detail the color mapping parameter calibration method. In the embodiments of this application, there are two methods for calibrating color mapping parameters. Method 1: Calibrate the color mapping parameters based on the first and second calibration images after white balancing to determine the first color mapping parameters. Method 2: Calibrate the color mapping parameters based on the first and second calibration images before white balancing to determine the second color mapping parameters.

[0066] Figure 3 This is a flowchart of method 1 for determining the first color mapping parameters.

[0067] S310: Measure the spectral response curves of the two cameras. The spectral response curves can be used to describe the spectral sensitivity of the cameras, usually with wavelength as the horizontal axis and the corresponding spectral response as the vertical axis.

[0068] S320: Acquire calibration images. Calibration images are color chart images used to calibrate color mapping parameters. Specifically, the calibration images can be color chart images or other reference images generated by simulating two cameras based on an imaging model, a spectral response curve, a light source spectrum, and a 24-color chart reflectance spectrum. For example, a first calibration image and a second calibration image are generated based on the spectral response curves of the first camera and the second camera, respectively.

[0069] It should be understood that in the above step S320, the calibration method of the first color mapping parameter is introduced by taking the 24-color card as an example. In other embodiments, other color cards or other objects containing multiple colors that can be used for reference images can also be used to calibrate the color mapping parameters.

[0070] At step S330, white balance processing is performed on the first calibration image and the second calibration image, respectively. For example, in step S320, the first calibration image and the second calibration image are generated based on the spectral response curves of the first camera and the second camera, respectively. White balance gains are calculated for the two images based on the color values ​​of the gray blocks in the images, and white balance processing is performed on the first calibration image and the second calibration image, respectively. In other embodiments, information about the current light source may be used to calculate a light source reference value, and this light source reference value may be applied to the two calibration images for white balance processing.

[0071] S340, calculate the first color mapping parameter. From the two calibration images after white balance, obtain the values ​​of the three RGB color channels corresponding to each pixel in each image. For example, (R 1i 1 G 1i 1 B 1i 1 ) represents the RGB value of the i-th pixel of the first calibration image after white balance, (R 2i 1 G 2i 1 B 2i 1 ) represents the RGB value of the i-th pixel of the second calibration image after white balance.

[0072] In some embodiments, according to (R 1i 1 G 1i 1 B 1i 1 ) and (R 2i 1 G 2i 1 B 2i 1 ), using the least squares method, calculate the first color mapping parameter T corresponding to the light source 1 In other embodiments, according to (R 1i 1 G 1i 1 B 1i 1 ) and (R 2i 1 G 2i 1 B2i 1 ), other regression methods can also be used to calculate the first color mapping parameters corresponding to the light source.

[0073] It should be understood that in some embodiments, when there is no spectral response measuring instrument, the above steps S310 and S320 may not be performed, and the image data captured by the camera may be directly used as the first calibration image and the second calibration image to calibrate the first color mapping parameters of the above steps S330 and S340.

[0074] In step S210 of the present application, for each of the N standard light sources, the color mapping parameters are calibrated according to the methods of steps S310 to S340 above under each standard light source, and N first color mapping parameters corresponding to the N standard light sources can be determined.

[0075] Figure 4 FIG2 is a flow chart of determining the second color mapping parameters in Method 2. In Method 2, the first calibration image and the second calibration image are directly obtained according to the spectral response, and there is no need to perform white balance processing on the calibration images.

[0076] S410, measuring the spectral response curves of the two cameras.

[0077] S420: Acquire a calibration image.

[0078] The specific implementation method of step S410 is similar to that of the above-mentioned step S310, and the specific implementation method of step S420 is similar to that of the above-mentioned step S320, which will not be repeated here.

[0079] S430, calculate the second color mapping parameters. Specifically, for the first calibration image and the second calibration image acquired by the two cameras, respectively obtain the values ​​of the three RGB color channels corresponding to each pixel in each image. For example, (R 1i 2 G 1i 2 B 1i 2 ) represents the RGB value of the i-th pixel of the first calibration image, (R 2i 2 G 2i 2 B 2i 2 ) represents the RGB value of the i-th pixel of the second calibration image.

[0080] In some embodiments, according to (R 1i 2 G 1i 2 B 1i2 ) and (R 2i 2 G 2i 2 B 2i 2 ), using the least squares method or other regression methods, obtain the second color mapping parameter T corresponding to the light source 2 .

[0081] It should be understood that in some embodiments, when there is no spectral response measuring instrument, similar to the above method 300, the above steps S410 and S420 may not be performed, and the images captured by the camera may be directly used as the first calibration image and the second calibration image to perform calibration of the second color mapping parameters in step S430.

[0082] It should also be understood that in the above method 300, the first camera and the second camera determine the first color mapping parameters based on the calibration image after white balancing, and therefore the first color mapping parameters do not include a white balance adjustment component. In the above method 400, the first camera and the second camera determine the second color mapping parameters based on the calibration image before white balancing, and therefore the second color mapping parameters include a white balance component.

[0083] During the color mapping parameter calibration process, color charts or other benchmark data can be used to simulate calibration images under different lighting conditions based on the spectra of different light sources and the camera's spectral response parameters. This reduces the time and cost of data capture, while also reducing the instability introduced by capture and improving the stability and accuracy of color mapping parameter calibration.

[0084] It should be understood that the process of calibrating the N color mapping parameters introduced in the above step S210 can be performed offline. After the calibration of the N color mapping parameters is completed, the correspondence between the N standard light sources and the N color mapping parameters can be preset in the image processing device. When the image processing device performs color consistency correction on images obtained by multiple cameras, the calibrated color mapping parameters can be directly used without repeated calibration.

[0085] S220: Correct the second image according to at least one color mapping parameter.

[0086] Specifically, since the first image is used as the target image, at least one color mapping parameter can be selected according to the image information indicated by the first image to perform color consistency correction on the second image so that the corrected image obtained after correction has the same color as the first image.

[0087] In the embodiment of the present application, correcting the second image based on at least one of the N color mapping parameters primarily includes image matching and performing color compensation and white balance compensation on the second image. Since there are two methods for calibrating the color mapping parameters in step S210, there are correspondingly two methods for performing color compensation and white balance compensation on the second image.

[0088] Figure 5 3 is a schematic diagram of the second image color correction process corresponding to the color mapping parameter calibration method 1.

[0089] S510, image matching. The purpose of image matching is to determine the common image area between the first image and the second image. Based on the common image area in the two images, the color difference between the first image and the second image can be determined, and color compensation parameters and white balance compensation parameters are calculated based on the color difference to perform color correction on the second image. A variety of methods can be used to determine the common image area between the two images, such as methods based on feature point matching, methods based on three-dimensional space projection, methods based on template matching, methods based on machine learning, etc., which are not limited in the embodiments of the present application.

[0090] Figure 6 Schematic diagram of a method for image matching between a first image and a second image. In some embodiments, when the relative positions of the two cameras are fixed, stable and fast image matching can be achieved by combining camera calibration. Figure 6 As shown in FIG, during the image matching process, the scaling factor of the two images, the offset of the common image area on the two images, and other parameters are obtained based on the calibration of multiple cameras (such as the position and focal length of the cameras). The two images are scaled according to the above parameters, and the approximate search range of the common image area is determined based on the offset. For example, Figure 6 As shown in (a), a search starting point and a search end point can be determined on the first image, and image matching is performed within the range of the search starting point and the search end point to finally obtain a common image area of ​​the first image and the second image.

[0091] In other embodiments, considering the problem of incomplete synchronization between the two images, in order to obtain more accurate image matching, methods such as feature point-based, brightness-based, and edge template matching can be used to achieve more accurate matching near the search range.

[0092] When multiple cameras are relatively fixed in position, combining camera calibration information can narrow the search area for image matching, enabling rapid image matching. Furthermore, template matching technology can also achieve good image matching results in small image scenes, meeting the real-time requirements of color consistency correction.

[0093] S520: Calculate white balance compensation parameters for the second image relative to the first image. The white balance compensation parameters can be calculated based on the color information of the common image area of ​​the two images. This step aims to calculate the color difference in the common image area of ​​the two images and calculate the white balance compensation parameters required for the second image relative to the first image based on the color difference in the common image area of ​​the two images. Calculating the white balance compensation parameters includes the following steps:

[0094] Step 1: Apply white balance gain to the two images to obtain two images after white balance processing. Specifically, Figure 1 As shown, in some embodiments, after the first image captured by the first camera and the second image captured by the second camera pass through the image sensor, they enter the preprocessing module for preprocessing. The preprocessing module can calculate a white balance gain for each image channel, and the color correction module performs white balance processing on the image channel based on the white balance gain. Subsequent steps 2 through 5 calculate white balance compensation parameters based on the two white-balanced images obtained in step 1.

[0095] Step 2: Divide the common image area of ​​the first image and the second image after white balance processing into blocks.

[0096] In some embodiments, in order to reduce the amount of calculation in subsequent calculations and improve the efficiency of image processing, the common image area obtained by image matching in step S510 can be divided into blocks. For example, the common image area image can be divided into M blocks according to the spatial position, color similarity, edge information, semantic information, etc. of the image, where M is a positive integer, and subsequent calculations are performed in units of blocks. The embodiment of the present application does not limit the way of image segmentation. The color value of each image block after segmentation can be represented by the average value of the three-channel colors of all pixels of the image block. For example, the image block includes 10 pixels, and the R, G, and B three-channel color values ​​of the image block can be represented by the R average value, G average value, and B average value of the 10 pixels, respectively. The color value of the image block can also be represented by the mode, median, etc. of the three-channel color values ​​of all pixels of the image block, which is not limited in the embodiment of the present application.

[0097] In some other embodiments, the public image area image may not be divided into blocks. In this case, M=1, and subsequent calculations are performed in pixels. In this case, step 2 may not be performed.

[0098] Step 3: Assign confidence (or “weight”) to the common image area images.

[0099] In some embodiments, after the common image area image is divided into blocks in step 2, confidence levels (or weights) can be assigned to the common image area image blocks based on information such as brightness, saturation, and color characteristics of each image block. For example, overexposed or oversaturated image blocks can be discarded; for example, overexposed or oversaturated image blocks can be assigned a smaller confidence level (or weight), while image blocks close to gray can be assigned a larger confidence level (or weight).

[0100] In other embodiments, the image of the common image area is not segmented in step 2 (or step 2 is not performed). In this case, confidence levels (or weights) can be assigned to the pixels of the common image area based on information such as the brightness, saturation, and color characteristics of each pixel in the common image area. For example, overexposed or oversaturated pixels can be discarded; for example, overexposed or oversaturated pixels can be assigned a smaller confidence level (or weight), while pixels close to gray can be assigned a larger confidence level (or weight).

[0101] In other embodiments, all image blocks in a common image area or all pixels in a common image area may be assigned the same weight.

[0102] The embodiment of the present application does not limit the way of assigning confidence (or weight) or the size of the weight.

[0103] Step 4: Calculate the color features of the common image area of ​​the two images.

[0104] For example, the weighted average values ​​of the three color channels of the image blocks in the common image area of ​​the first image and the second image can be calculated respectively. Specifically, in step 3, the three-channel color values ​​of each image block are weighted. Therefore, the R channel color feature of the common image area of ​​the first image is the weighted average value of the R value of each image block, the G channel color feature of the common image area of ​​the first image is the weighted average value of the G value of each image block, and the B channel color feature of the common image area of ​​the first image is the weighted average value of the B value of each image block. The color features of the common image area of ​​the second image can also be calculated in a similar manner.

[0105] For example, the weighted color histograms of the image blocks in the common image area of ​​the first image and the second image can be calculated respectively; the color histogram can represent the frequency of occurrence of a certain color value in the image. Specifically, in step 3, weights are assigned to the three-channel color values ​​of each image block. Taking the common image area of ​​the first image as an example, in the R channel color histogram, the frequency corresponding to each R value is the weighted sum of the number of occurrences of the corresponding R value of the image block. For example, the common image area of ​​the first image includes 2 image blocks, the weight of image block 1 is 1, and the R value is 255; the weight of image block 2 is 3, and the R value is also 255. Then, in the weighted color histogram of the image R channel, the frequency corresponding to the value 255 is 1×1+1×3=4.

[0106] For example, the weighted average values ​​of the three color channels of the pixels in the common image area of ​​the first image and the second image may be calculated respectively; the calculation method is similar to the calculation method after the above-mentioned block division and will not be described in detail.

[0107] For example, weighted color histograms of pixels in the common image area of ​​the first image and the second image may be calculated respectively. The calculation method is similar to the calculation method after the above-mentioned block division, and will not be described in detail.

[0108] Step 5: Calculate the white balance compensation parameters of the second image relative to the first image. Specifically, the color difference can be calculated based on the color features of the common image area of ​​the two images extracted in step 4, thereby obtaining the white balance compensation parameters of the second image relative to the first image.

[0109] For example, the average weighted values ​​of the three color channels of the common image area images of the two images obtained in step 4 may be compared to calculate the white balance compensation parameter of the second image relative to the first image;

[0110] For example, the white balance compensation parameter of the second image relative to the first image may be calculated based on the weighted color histogram features of the common image area of ​​the two images obtained in step 4;

[0111] For example, the white balance compensation parameters of the second image relative to the first image may be calculated using histogram matching based on the weighted color histogram features of the common image area of ​​the two images obtained in step 4.

[0112] S530, based on the first color mapping parameters, calculate the color compensation parameters of the second image relative to the first image. The color compensation parameters can be calculated based on the color information of the common image area of ​​the two images and the color mapping parameters. Since the first color mapping parameters determined in method 1 in step S210 do not include the component of white balance compensation, during the color correction of the second image in step S220, it is necessary to first calculate the white balance compensation parameters of the second image with respect to the first image (step S520), and then calculate the color compensation parameters of the second image with respect to the first image. Specifically, in this step, based on at least one first color mapping parameter, the color mapping parameters applicable to the scene are determined so that the color difference between the two images after color compensation is minimized.

[0113] In some embodiments, a global search may be used to find a suitable first color mapping parameter among the N first color mapping parameters as the first target color mapping parameter, so that the difference between the third image generated after the common image area of ​​the second image is transformed by the first color mapping parameter and the common image area of ​​the first image is minimized. For example, the search may be performed according to the method shown in formula (2):

[0114]

[0115] Where N represents the number of standard light sources;

[0116] T i 1 represents the first color mapping parameter corresponding to the i-th standard light source among N standard light sources;

[0117] M represents the number of image blocks or pixels in the common image area of ​​the first image and the second image;

[0118] and Three-channel values ​​of the m-th block, or three-channel values ​​of the m-th pixel, respectively representing the common image area of ​​the first image and the common image area of ​​the second image;

[0119] Indicates that the color mapping parameter T i 1 The three-channel values ​​of the m-th block, or the three-channel values ​​of the m-th pixel, of the third image generated after being applied to the common image area of ​​the second image;

[0120] The Dis() function is used to calculate the difference of images. For example, the absolute value distance, Euclidean distance or other color difference measurement methods can be used; for example, the image color can also be converted from the RGB space to other color spaces to measure the difference of the images.

[0121] Through the above formula (2), a first color mapping parameter can be selected from N first color mapping parameters as the first target color mapping parameter, and the first target color mapping parameter can be used as the color compensation parameter of the second image, so that the difference between the second image and the first image after the color compensation parameter is applied is minimized.

[0122] In other embodiments, multiple first color mapping parameters may be selected, where the multiple first color mapping parameters are those that minimize the difference between the common image area of ​​the third image and the first image, and the multiple color mapping parameters are fused according to a preset rule to obtain first target color mapping parameters, which are then used as color compensation parameters for the second image. For example, a weighted value of the multiple first color mapping parameters may be used as the first target color mapping parameter, and the first target color mapping parameter may be used as the color compensation parameter for the second image.

[0123] In other embodiments, the ambient light source information of the first image can be determined based on the white balance gain of the first image in step S520, and then a light source closest to the ambient light source indicated by the first image can be selected from N standard light sources based on the estimated light source information, and the first color mapping parameters corresponding to the standard light source are used as the first target color mapping parameters, which are the color compensation parameters of the second image. Alternatively, multiple light sources closest to the ambient light source indicated by the first image can be selected from N standard light sources based on the estimated light source information, and the multiple first color mapping parameters corresponding to these standard light sources are fused (e.g., a weighted average of the multiple first color mapping parameters) to serve as the first target color mapping parameters. The first target color mapping parameters can be used as the color compensation parameters of the second image.

[0124] It should be understood that, since the first color mapping parameters are determined according to the first image and the second image after white balance in the first color mapping parameter determination method 1, the first target color mapping parameters are Does not include white balance.

[0125] S540: Color correct the second image. Specifically, in this step, the white balance compensation parameters calculated in step S520 and the color compensation parameters calculated in step S530 are applied to the second image to obtain a color-corrected image. The color-corrected image of the second image can be consistent with the color of the first image.

[0126] Figure 7 2 is a schematic diagram of the second image color correction process corresponding to the color mapping parameter calibration method 2.

[0127] S710, image matching. The purpose of image matching is to determine the common image area between the two images for subsequent calculation of color compensation parameters and white balance compensation parameters. The image matching process in this step is similar to the image matching process in step S510 above and will not be repeated here.

[0128] S720: Calculate color compensation parameters of the second image relative to the first image. The process of calculating color compensation parameters includes two steps:

[0129] Step 1: Calculate the second target color mapping parameters; Step 2: Calculate the color compensation parameters.

[0130] When calculating the second target color mapping parameter in step 1, one of the N second color mapping parameters may be selected as the second target color mapping parameter according to the scheme described in step S530, or multiple second color mapping parameters may be selected and fused to form the second target color mapping parameter. For the sake of brevity, this will not be described in detail.

[0131] Step 2: Calculate color compensation parameters. The color compensation parameters of the second image can be calculated according to formula (3):

[0132]

[0133] in, represents the second target color mapping parameters obtained in step 1;

[0134] and CC mat Respectively represent the white balance gain and color restoration parameters of the first image, which can be obtained from Figure 1 obtained in the preprocessing steps shown;

[0135] G represents the color compensation parameter, which includes the white balance gain component and the color restoration parameter component.

[0136] S730: Calculate white balance compensation parameters of the second image relative to the first image. Perform white balance compensation calculation on the color-compensated second image to cope with complex and changing scenes.

[0137] Step 1: Apply white balance gain to the common image area of ​​the first image and color reproduction parameters CC mat Among them, the white balance gain and color restoration parameters of the first image can be obtained from Figure 1 Calculated in the preprocessing steps shown.

[0138] Step 2: Apply the color compensation parameter G obtained in step S720 to the common image area of ​​the second image.

[0139] Step 3: Calculate white balance compensation parameters based on the common area of ​​the processed first image obtained in step 1 and the common area of ​​the processed second image obtained in step 2. The calculation method of white balance compensation is similar to the method described in step S520 and will not be described in detail to avoid repetition.

[0140] In other embodiments, step S730 may also be implemented by the following steps:

[0141] Step 1: The common image area of ​​the second image is applied

[0142] Step 2: Calculate white balance compensation parameters based on the common image area of ​​the first image before white balancing and the common image area of ​​the processed second image obtained in Step 1. By comparing the overall color difference between the common area of ​​the first image before white balancing and the common area of ​​the second image after color mapping, the white balance compensation amount in the scene can be dynamically adjusted. The calculation method of the white balance compensation parameters is similar to the method described in Step S520 and will not be detailed here to avoid repetition.

[0143] S740: Color correct the second image. Specifically, in this step, the color compensation parameters calculated in step S720 and the white balance compensation parameters calculated in step S730 are applied to the second image to obtain a color-corrected image. The color-corrected image of the second image can be consistent with the color of the first image.

[0144] In the embodiments of the present application, color consistency correction of the second image includes color compensation and white balance compensation. White balance compensation and color compensation can be performed based on white balance compensation parameters and color compensation parameters, respectively. White balance compensation corrects the consistency of gray areas in the image, while color compensation corrects the consistency of color areas in the image. Combining these two corrections achieves a good correction effect for both gray and color areas of the image.

[0145] Figure 8 Schematic diagram of a multi-camera color consistency correction device according to an embodiment of the present application. Figure 8 As shown, the multi-camera color consistency correction device of the embodiment of the present application includes an acquisition module 810, a determination module 820 and a correction module 830.

[0146] The acquisition module 810 is configured to acquire a first image captured by a first camera and a second image captured by a second camera.

[0147] Determination module 820 is used to determine at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, where the image information includes color information of the first image and at least one of the ambient light sources of the first image. The color mapping parameter indicates a color conversion relationship between the image captured by the first camera and the image captured by the second camera, and the N color mapping parameters correspond one-to-one to N standard light sources.

[0148] The correction module 830 is configured to perform color consistency correction on the second image according to at least one color mapping parameter to obtain a corrected image, where the color of the corrected image is consistent with that of the first image.

[0149] In some embodiments, the determination module 820 and the correction module 830 can implement the function of determining the color mapping parameters in step S220 of the above method 200 and performing color consistency correction on the second image according to the color mapping parameters. Figures 5 to 7 The color consistency correction method 500 and the method 700 are shown. The specific functions and beneficial effects of the determination module 820 and the correction module 830 can be found in the description of the above methods, and are not repeated here for the sake of brevity.

[0150] In some embodiments, Figure 8 The color consistency correction device shown can achieve Figure 1 The functionality of the correction module is shown.

[0151] In some embodiments, the determination module 820 may also be used to determine Figure 1 The color mapping parameters in the application scenario shown can realize the function of determining the color mapping parameters in step S210 of method 200. Specifically, the determination module 810 can be used to realize Figure 3 The steps in the color mapping parameter calibration method 300 shown in FIG. Figure 4 The various steps in the color mapping parameter calibration method 400 are shown. In this case, the specific functions and beneficial effects of the determination module 820 can be found in the description of the above method, and will not be described in detail for the sake of brevity.

[0152] It should be understood that Figure 8 The color consistency correction device 800 shown in the figure only includes an acquisition module 810, a determination module 820 and a correction module 830. In other embodiments, the color consistency correction device may also include other modules or components, such as Figure 1 The pre-processing module, post-processing module, etc. shown in the figure are not limited in the embodiments of the present application.

[0153] Figure 9 Schematic diagram of another multi-camera color consistency correction device according to an embodiment of the present application. Figure 9The color consistency correction device 900 shown includes a memory 910, a processor 920, a communication interface 930, and a bus 940. The memory 910, the processor 920, and the communication interface 930 are connected to each other via the bus 940.

[0154] The memory 910 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 910 may store a program. When the program stored in the memory 910 is executed by the processor 920, the processor 920 is used to execute the various steps of the multi-camera color consistency correction method of the embodiment of the present application, for example, executing Figures 2 to 7 The steps shown.

[0155] It should be understood that the color consistency correction device shown in the embodiment of the present application can be a server, for example, a server in the cloud, or a chip configured in a server in the cloud; or, the color consistency correction device shown in the embodiment of the present application can be a smart terminal, or a chip configured in a smart terminal.

[0156] The color consistency correction method disclosed in the above embodiments of the present application can be applied to or implemented by processor 920. Processor 920 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the color consistency correction method can be completed by hardware integrated logic circuits in processor 920 or by software instructions.

[0157] The processor 920 described above may be a central processing unit (CPU), an image signal processor (ISP), a graphics processing unit (GPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art, such as a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 910, and the processor 920 reads the instructions in the memory 910 and combines the hardware to complete the implementation of the present application. Figures 2 to 7 The various steps of the color consistency correction method are shown.

[0158] The communication interface 930 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the apparatus 900 and other devices or a communication network.

[0159] The bus 940 may include a path for transmitting information between the various components of the color consistency correction apparatus 900 (eg, the memory 910 , the processor 920 , and the communication interface 930 ).

[0160] It should be noted that although the above color consistency correction device 900 only shows a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the color consistency correction device 900 may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above color consistency correction device 900 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the above color consistency correction device 900 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 9 All devices shown in .

[0161] An embodiment of the present application also provides a computer-readable medium, which stores a computer program (also referred to as code, or instructions) that, when executed on a computer, enables the computer to execute the color consistency correction method in any of the above method embodiments.

[0162] An embodiment of the present application also provides a chip system, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device or apparatus equipped with the chip system executes a method in any of the above method embodiments.

[0163] Among them, the chip system may include an input circuit or interface for sending information or data, and an output circuit or interface for receiving information or data.

[0164] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0165] It should be understood that the “some embodiments” or “one embodiment” mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, “in some embodiments” or “in one embodiment” appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0166] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0167] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided in this application, it is understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not performed.

[0170] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0171] In addition, the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0172] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A multi-camera color consistency correction method, characterized in that: The method comprises: Acquire a first image captured by a first camera and a second image captured by a second camera; determining at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, the image information including at least one of color information of the first image and an ambient light source of the first image, the color mapping parameter indicating a color conversion relationship between an image captured by the first camera and an image captured by the second camera, the N color mapping parameters corresponding one-to-one to N standard light sources, where N is a positive integer; performing color consistency correction on the second image according to the at least one color mapping parameter to obtain a corrected image; The performing color consistency correction on the second image according to at least one of the color mapping parameters comprises: determining a common image area of ​​the first image and the second image; determining a color compensation parameter for correcting a color area based on the common image area and the at least one color mapping parameter; determining a white balance compensation parameter for correcting a gray area according to the common image area; Color consistency correction is performed on the second image according to the white balance compensation parameter and the color compensation parameter.

2. The method according to claim 1, characterized in that Before determining at least one color mapping parameter from the N color mapping parameters, the method further includes: Determine a first calibration image and a second calibration image under each of the standard light sources, wherein the first calibration image and the second calibration image are color card images generated according to spectral response curves of the first camera and the second camera, respectively; The color mapping parameter corresponding to each of the standard light sources is determined according to the first calibration image and the second calibration image.

3. The method according to claim 1, characterized in that Determining the common image area of ​​the first image and the second image includes: Determining a search area based on the relative positions and fields of view of the first camera and the second camera; The common image area is determined according to the search area.

4. The method according to claim 1, wherein Determining the color compensation parameter for correcting the color area according to the common image area and the at least one color mapping parameter comprises: applying the N color mapping parameters to the common image areas in the second image respectively to obtain N third images; respectively calculating a color difference between the common image area in the first image and each of the third images; determining at least one color mapping parameter according to the color difference, wherein the at least one color mapping parameter is a color mapping parameter corresponding to at least one third image having the smallest color difference; determining a target color mapping parameter based on the at least one color mapping parameter, the target color mapping parameter being a weighted value of the at least one color mapping parameter; The color compensation parameters are determined according to the target color mapping parameters.

5. The method according to claim 1, wherein Determining the color compensation parameter for correcting the color area according to the common image area and the at least one color mapping parameter comprises: determining an ambient light source according to a white balance gain of the common image area in the first image; determining, based on the ambient light source, at least one color mapping parameter corresponding to at least one standard light source, wherein the difference between the at least one standard light source and the ambient light source is minimal; determining a target color mapping parameter according to the at least one color mapping parameter, the target color mapping parameter being a weighted value of the at least one color mapping parameter; The color compensation parameters are determined according to the target color mapping parameters.

6. The method according to claim 4 or 5, characterized in that The color compensation parameter is the target color mapping parameter, or the color compensation parameter is the product of the target color mapping parameter and the white balance gain and color restoration parameter of the first image.

7. The method according to any one of claims 1 to 5, characterized in that Determining the white balance compensation parameter according to the common image area includes: respectively determining weighted average values ​​or weighted color histograms of pixels in the common image area in the first image in three color channels; respectively determining weighted average values ​​or weighted color histograms of pixels in the common image area in the second image in three color channels; The white balance compensation parameter is determined according to a weighted average value of the three color channels or the weighted color histogram.

8. The method according to any one of claims 1 to 5, characterized in that Before determining the white balance compensation parameter, the method further includes: Dividing the common image area into M blocks according to the spatial position, color similarity and edge information of the common image area, where M is a positive integer; Determining the white balance compensation parameter according to the common image area includes: respectively determining weighted average values ​​or weighted color histograms of the image blocks of the common image area in the first image in three color channels; respectively determining weighted average values ​​or weighted color histograms of the image blocks of the common image area in the second image in three color channels; The white balance compensation parameter is determined according to a weighted average value of the three color channels or the weighted color histogram.

9. A multi-camera color consistency correction device, characterized in that: The device comprises: An acquisition module, configured to acquire a first image captured by the first camera and a second image captured by the second camera; a determination module, configured to determine at least one color mapping parameter from N color mapping parameters based on image information indicated by the first image, the image information including at least one of color information of the first image and an ambient light source of the first image, the color mapping parameter indicating a color conversion relationship between an image captured by the first camera and an image captured by the second camera, the N color mapping parameters corresponding one-to-one to N standard light sources, where N is a positive integer; a correction module, configured to perform color consistency correction on the second image according to the at least one color mapping parameter to obtain a corrected image; The determining module is specifically configured to: determining a common image area of ​​the first image and the second image; determining a color compensation parameter for correcting a color area based on the common image area and the at least one color mapping parameter; determining a white balance compensation parameter for correcting a gray area according to the common image area; The correction module is specifically used for: Color consistency correction is performed on the second image according to the white balance compensation parameter and the color compensation parameter.

10. The device according to claim 9, characterized in that Before determining at least one color mapping parameter from the N color mapping parameters, the determining module is specifically configured to: Determine a first calibration image and a second calibration image under each of the standard light sources, wherein the first calibration image and the second calibration image are color card images generated according to spectral response curves of the first camera and the second camera, respectively; The color mapping parameter corresponding to each of the standard light sources is determined according to the first calibration image and the second calibration image.

11. The device according to claim 9, characterized in that The determining module is specifically configured to: Determining a search area based on the relative positions and fields of view of the first camera and the second camera; The common image area is determined according to the search area.

12. The device according to claim 9, characterized in that The determining module is specifically configured to: respectively applying the N color mapping parameters to the common image areas in the second image to obtain N third images; respectively calculating a color difference between a common image area in the first image and each of the third images; determining at least one color mapping parameter according to the color difference, wherein the at least one color mapping parameter is a color mapping parameter corresponding to at least one third image having the smallest color difference; determining a target color mapping parameter based on the at least one color mapping parameter, the target color mapping parameter being a weighted value of the at least one color mapping parameter; The color compensation parameters are determined according to the target color mapping parameters.

13. The device according to claim 9, characterized in that The determining module is specifically configured to: determining an ambient light source according to a white balance gain of the common image area in the first image; determining, based on the ambient light source, at least one color mapping parameter corresponding to at least one standard light source, wherein the difference between the at least one standard light source and the ambient light source is minimal; determining a target color mapping parameter according to the at least one color mapping parameter, the target color mapping parameter being a weighted value of the at least one color mapping parameter; The color compensation parameters are determined according to the target color mapping parameters.

14. The device according to claim 12 or 13, characterized in that The color compensation parameter is the target color mapping parameter, or the color compensation parameter is the product of the target color mapping parameter and the white balance gain and color restoration parameter of the first image.

15. The device according to any one of claims 9 to 13, characterized in that The determining module is specifically configured to: respectively determining weighted average values ​​or weighted color histograms of pixels in the common image area in the first image in three color channels; respectively determining weighted average values ​​or weighted color histograms of pixels in the common image area in the second image in three color channels; The white balance compensation parameter is determined according to a weighted average value of the three color channels or the weighted color histogram.

16. The device according to any one of claims 9 to 13, characterized in that Before determining the white balance compensation parameter, the determining module is further configured to: Dividing the common image area into M blocks according to the spatial position, color similarity and edge information of the common image area, where M is a positive integer; respectively determining weighted average values ​​or weighted color histograms of the image blocks of the common image area in the first image in three color channels; respectively determining weighted average values ​​or weighted color histograms of the image blocks of the common image area in the second image in three color channels; The white balance compensation parameter is determined according to a weighted average value of the three color channels or the weighted color histogram.

17. A multi-camera color consistency correction device, characterized in that: include: A processor and a memory, the memory being used to store a program, and the processor being used to call and run the program from the memory to execute the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that The invention comprises a computer program which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 8.

Images