Image processing method, device, equipment, storage medium and system

Abstract

The invention discloses an image processing method, device and system, equipment and a storage medium. According to the method, a mapping relation between white balance parameters and color mapping parameters in different shooting scenes is calibrated in advance, and the color mapping parameters of a first image are determined according to a target white balance parameter index mapping relation corresponding to the first image. And further, converting the first image in the color space of the first camera device into a second image in the color space of the second camera device by using the color mapping parameters. It can be understood that the converted second image is consistent with the image actually acquired by the second camera device in color representation.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, device, storage medium, and system. Background Art

[0002] Color response difference refers to the difference in color representation of images obtained when capturing the same shooting scene using different imaging devices. This difference may stem from various factors, including different sensor types, differences in color filter designs, differences in image processing algorithms, and aging of the imaging device.

[0003] When shooting the same shooting scene using imaging devices with color response differences, obvious differences may occur in the colors of the obtained images, thus affecting the consistency of the images. Summary of the Invention

[0004] Embodiments of the present application are expected to provide an image processing method, apparatus, device, storage medium, and system.

[0005] The technical solution of the present application is implemented as follows:

[0006] In a first aspect, an image processing method is provided, including:

[0007] Obtain a first image collected by a first imaging device and its corresponding target white balance parameter;

[0008] Based on the target white balance parameter and the mapping relationship, determine the color mapping parameter of the first image, where the mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0009] Perform color mapping on the first image based on the color mapping parameter of the first image to obtain a second image, where the first image is an image in the color space of the first imaging device, and the second image is an image in the color space of the second imaging device.

[0010] In a second aspect, an image processing apparatus is provided, including:

[0011] An obtaining unit, configured to obtain a first image collected by a first imaging device and its corresponding target white balance parameter;

[0012] A determining unit, configured to determine the color mapping parameter of the first image based on the target white balance parameter and the mapping relationship, where the mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0013] A processing unit for performing color mapping on a first image based on color mapping parameters of the first image to obtain a second image, where the first image is an image in the color space of a first imaging device, and the second image is an image in the color space of a second imaging device.

[0014] In a third aspect, an electronic device is provided, including: a processor and a memory configured to store a computer program that can run on the processor,

[0015] wherein, when the processor is configured to run the computer program, it executes the steps of the foregoing method.

[0016] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the steps of the foregoing method.

[0017] In a fifth aspect, a computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the foregoing method.

[0018] In a sixth aspect, an image acquisition system includes a first imaging device, a second imaging device, and a shooting object, and the positions of the first imaging device and the second imaging device are relatively fixed;

[0019] The first imaging device is configured to capture a first reference image of the shooting object in response to a first shooting instruction;

[0020] The second imaging device is configured to capture a second reference image of the shooting object in response to a second shooting instruction;

[0021] The first imaging device is further configured to determine white balance parameters corresponding to the first reference image based on the first reference image; or, the second imaging device is further configured to determine white balance parameters corresponding to the first reference image based on the second reference image.

[0022] In an embodiment of the present application, an image processing method, apparatus, device, storage medium, and system are provided. The method pre-calibrates the mapping relationship between white balance parameters and color mapping parameters in different shooting scenarios, and determines the color mapping parameters of the first image according to the target white balance parameters of the first image indexing the mapping relationship. Furthermore, using these color mapping parameters, the first image in the color space of the first imaging device is converted into a second image in the color space of the second imaging device. It can be understood that the converted second image is consistent with the image actually captured by the second imaging device in terms of color performance. Description of the Drawings

[0023] Figure 1 It is the first flowchart of the image processing method in an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of the mapping relationship of the reference area in the embodiment of the present application;

[0025] Figure 3 It is a second process schematic diagram of the image processing method in the embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of the composition framework of an image acquisition system provided by the embodiment of the present application;

[0027] Figure 5 It is a third process schematic diagram of the image processing method in the embodiment of the present application;

[0028] Figure 6 It is a schematic diagram of the composition structure of the image processing device in the embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the composition structure of the electronic device in the embodiment of the present application;

[0030] Figure 8 It is a schematic diagram of the composition framework of another image acquisition system provided by the embodiment of the present application. Detailed implementation manners

[0031] In order to be able to understand the features and technical content of the embodiments of the present application in more detail, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and illustration purposes and are not used to limit the embodiments of the present application.

[0032] The following problems may be brought about by the color response differences existing in different imaging devices in the embodiments of the present application:

[0033] In the same shooting scene, if imaging devices with color response differences are used for shooting, the colors of the obtained images may be different. For example, in some application scenarios that require multiple imaging devices to work together, the switching between different imaging devices may cause the display screen to jump. In addition, in application scenarios that rely on image color information for recognition and analysis, such as machine vision, autonomous driving and other fields, the color response differences may affect the accuracy and reliability of the system.

[0034] The Auto White Balance (AWB) prediction technology based on artificial intelligence (AI) uses artificial intelligence technologies such as machine learning algorithms to predict and correct color deviations in images, aiming to reproduce natural colors. The training of this AI-based AWB model relies on a large image dataset that contains image data of various scenes (indoor, outdoor, daytime, night, etc.) shot under different lighting conditions. These data need to be preprocessed for model training, so that the model can learn to extract features from the image and predict the appropriate white balance parameters.

[0035] However, due to the differences in color response between different cameras, an AWB model trained for a certain camera cannot usually be directly applied to a new camera. It would be extremely costly to collect tens of thousands of images for each new camera to train the model.

[0036] In view of this, an embodiment of the present application proposes an image processing method. The method pre-calibrates the mapping relationship between white balance parameters and color mapping parameters under different shooting scenes, and determines the color mapping parameters of the first image according to the target white balance parameter index mapping relationship corresponding to the first image. Then, using these color mapping parameters, the first image in the color space of the first camera device is converted into the second image in the color space of the second camera device. It can be understood that the converted second image is consistent with the image actually captured by the second camera device in terms of color expression.

[0037] For example, in a scenario where the color consistency of different cameras needs to be ensured, the image captured by the first camera (non-reference camera) can be converted into the color space of the second camera (reference camera), so that the images captured by all non-reference cameras are consistent with the images of the reference camera in color expression, thereby achieving color consistency between different cameras. In this way, when switching between different cameras, the color consistency of the displayed image can be ensured; in addition, when identifying and analyzing images captured by different cameras, the accuracy and reliability of the system can be improved.

[0038] For another example, in a scenario where a large number of data sets are required, the image processing method provided by the embodiment of the present application does not need to collect images for each camera device. It is only necessary to perform color space conversion on the data set of one camera device to obtain a data set suitable for other camera devices, thereby adapting to different camera devices with different color responses, avoiding the collection of tens of thousands of data sets for each camera device, improving the quality of the data set, and reducing the cost of data set production.

[0039] Figure 1This is the first flowchart diagram of the image processing method in the embodiments of this application. As Figure 1 shown, the method may specifically include:

[0040] Step 101: Obtain the first image collected by the first imaging device and its corresponding target white balance parameter;

[0041] The target white balance parameter is the true value or reference value of the white balance parameter of the first image, and serves as a benchmark for measuring the color accuracy of the first image. In one example, the target white balance parameter may be the white balance parameter determined based on the first image. In another example, when the first imaging device and the second imaging device shoot the same shooting scene, the target white balance parameter may be the white balance parameter determined based on the third image, and the third image is the image collected by the second imaging device.

[0042] The white balance parameter is a key setting for adjusting the color balance of an image or video, aiming to ensure that white and neutral colors can be accurately restored under different light source conditions, thereby avoiding color deviation. It is an important part of color management in photography, videography, and image processing. In the embodiments of this application, the target white balance parameter is used to index the mapping relationship to determine the color mapping parameter adapted when the first image undergoes color space conversion.

[0043] In one example, the white balance parameter is the white balance gain or the color temperature value.

[0044] The white balance gain refers to that the imaging device adjusts the pixel values of the red (R), green (G), and blue (B) channels of the image through the internal automatic white balance algorithm to compensate for color deviation under different light sources and ensure the accurate restoration of neutral colors (such as white and gray).

[0045] The color temperature is an index for measuring the color of a light source, with the unit of Kelvin (K). Different light sources have different color temperature values. For example, daylight is about 5500K, incandescent lamps are about 2700K, and fluorescent lamps have various color temperature values. During shooting, the color temperature value indicates the color temperature of the light source in the current shooting scene, thereby automatically adjusting the color balance of the image to make the image color consistent with the real environment.

[0046] In one example, the first image may be the original image output by the image sensor, such as a Raw image. In another example, the first image may also be other common image formats, such as RGB images, YUV images, etc.

[0047] Step 102: Determine the color mapping parameter of the first image based on the target white balance parameter and the mapping relationship. The mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0048] The color mapping parameter is used to implement the conversion between different color spaces, specifically for adjusting the color channels of the first image, so as to correct the color deviation of the image captured by the first imaging device compared with the image captured by the second imaging device. The converted second image is consistent with the image actually captured by the second imaging device in terms of color performance.

[0049] In the embodiments of the present application, a mapping relationship including a plurality of white balance parameters and color mapping parameters can be preset based on the prior knowledge obtained in a variety of shooting scenarios.

[0050] Exemplarily, the method further includes: in a variety of shooting scenarios, acquiring a first reference image captured by the first imaging device for a shooting object, a second reference image captured by the second imaging device for the shooting object, and the corresponding white balance parameter for each shooting scenario; determining the color mapping parameter between the first reference image and the second reference image; and constructing a mapping relationship by using the corresponding white balance parameter and color mapping parameter for each shooting scenario.

[0051] In one example, the corresponding white balance parameter for each shooting scenario may be the white balance parameter determined based on the first reference image. In another example, when the first imaging device and the second imaging device shoot the same shooting scenario, the corresponding white balance parameter for each shooting scenario may be the white balance parameter determined based on the second reference image captured by the second imaging device.

[0052] In one example, the positions of the first imaging device and the second imaging device are relatively fixed. The first imaging device and the second imaging device respond to the shooting instructions issued at the same moment and respectively acquire the first reference image and the second reference image. In this way, the reference images in various open environments and under various light sources can be quickly acquired, so that the color difference mapping parameter between the two imaging devices can be quickly calibrated according to the reference images of different imaging devices, which is beneficial to improving the image acquisition and processing efficiency.

[0053] In one example, the first imaging device and the second imaging device may be imaging devices on different electronic devices. In another example, the first imaging device and the second imaging device may be different imaging devices on the same electronic device, and different imaging devices correspond to different focal lengths. For example, the electronic device includes but is not limited to: ultra-wide-angle lens, wide-angle lens, telephoto lens, etc.

[0054] In some embodiments, the shooting object is a color card including a plurality of color blocks with different colors. By comparing the color response differences of different color blocks by different imaging devices, it is beneficial to estimate the color mapping parameters between the two. For example, the shooting object may be a 24-color card, and the 24-color card includes color blocks of 24 common colors that have been precisely proportioned and standardized, and these colors can achieve color consistency under different light conditions.

[0055] In some embodiments, determining the color mapping parameters between a first reference image and a second reference image includes: detecting and matching feature points of the first reference image and the second reference image to determine a plurality of pairs of matching points in the first reference image and the second reference image; determining the pixel values of a plurality of pairs of reference regions in the first reference image and the second reference image based on the plurality of pairs of matching points; and determining the color mapping parameters between the first reference image and the second reference image based on the pixel values of the plurality of pairs of reference regions.

[0056] In some embodiments, determining the pixel values of a plurality of pairs of reference regions in the first reference image and the second reference image based on the plurality of pairs of matching points includes: determining the coordinate mapping relationship between the first reference image and the second reference image based on the plurality of pairs of matching points; aligning the first reference image and the second reference image based on the coordinate mapping relationship; and obtaining the pixel values of the plurality of pairs of reference regions from the aligned first reference image and the second reference image respectively.

[0057] It should be noted that the precise matching of two images can be achieved through operations such as feature point detection and matching, coordinate mapping relationship estimation, and image alignment. Obtaining the pixel values of similar or identical pairs of reference regions from the aligned two images is beneficial to improving the accuracy of subsequent processing.

[0058] The coordinate mapping relationship can be a certain geometric transformation between two images (such as translation, rotation, scaling, or affine transformation, etc.). The goal of image alignment is to minimize the distance or error between corresponding key points in the two images.

[0059] For example, the coordinate mapping relationship is a homography transformation matrix. The homography transformation matrix is an invertible 3x3 matrix used to map the homogeneous coordinates of a point on one image to the homogeneous coordinates of a point on another image. The mapping formula is as follows:

[0060]

[0061] where the homogeneous coordinates of a pixel point in the first reference image are represented as [x 1 , y 1 , 1], x 1 and x 1 are the coordinates of the pixel point in the first reference image, and the third component is a fixed 1, used to indicate that this is a two-dimensional point. The homogeneous coordinates of a pixel point in the second reference image are represented as [x 2 , y 2 , 1], x 2 and x 2 are the coordinates of the pixel point in the second reference image. Homogeneous coordinates enable coordinate transformation to be represented using a matrix.

[0062] The homography transformation matrix H can be expressed as:

[0063]

[0064] Solving the homography transformation matrix includes: Usually, it is necessary to solve the homography transformation matrix H by the coordinates of multiple known matching point pairs (at least 4 matching point pairs). For example, it is implemented by the direct linear transformation algorithm. The direct linear transformation algorithm uses at least 4 matching point pairs to construct a linear equation system and solves the equation system to obtain an estimated value of H.

[0065] In some embodiments, determining the pixel values of multiple reference region pairs in the first reference image and the second reference image based on multiple matching point pairs includes: determining the position coordinates of multiple reference region pairs in the first reference image and the second reference image based on multiple matching point pairs; and respectively obtaining the pixel values of multiple reference region pairs from the first reference image and the second reference image based on the position coordinates of multiple reference region pairs.

[0066] It should be noted that only through feature point detection and matching, some corresponding feature points between two reference images can be found, and these corresponding feature points can, to a certain extent, indicate the position coordinates of similar or identical reference region pairs in the two images; thus, according to the position coordinates of the reference region pairs, the pixel values of multiple reference region pairs are obtained, which is beneficial to improving the processing efficiency.

[0067] In some embodiments, the pixel values of the reference region pair include the average pixel value of the first reference region and the average pixel value of the second reference region. The first reference region is the reference region in the first reference image, and the second reference region is the region in the second reference image that is similar or identical to the first reference region. As Figure 2 shown, the color blocks 1 in the two images form a reference region pair, the color blocks 2 form a reference region pair, the color blocks 3 form a reference region pair, the color blocks 4 form a reference region pair, and so on.

[0068] In some embodiments, determining the color mapping parameter of the first image based on the target white balance parameter and the mapping relationship includes: determining at least one white balance parameter close to the target white balance parameter from the mapping relationship based on the target white balance parameter; and determining the color mapping parameter of the first image based on the color mapping parameters corresponding to at least one white balance parameter in the mapping relationship.

[0069] Exemplarily, the white balance parameter is a one-dimensional parameter, such as the color temperature value. By comparing the difference between the target white balance parameter and the white balance parameters in the mapping relationship, the first N white balance parameters with the smallest difference from the target white balance parameter are determined, where N is an integer greater than or equal to 1.

[0070] Exemplarily, the white balance parameter is a two-dimensional parameter, such as white balance gain. By comparing the similarity between the target white balance parameter and the white balance parameters in the mapping relationship, the top N white balance parameters with the smallest difference from the target white balance parameter are determined, where N is an integer greater than or equal to 1.

[0071] Exemplarily, one white balance parameter closest to the target white balance parameter is determined from the mapping relationship as the color mapping parameter of the first image.

[0072] Exemplarily, interpolation operations are performed on the color mapping parameters corresponding to at least two white balance parameters in the mapping relationship to obtain the color mapping parameter of the first image. Through interpolation operations, accurate prediction can be made based on at least two color mapping parameters, thereby obtaining the color mapping parameter finally used for the first image.

[0073] Exemplarily, the color mapping parameter includes but is not limited to at least one of the following: linear transformation coefficient, high-order polynomial coefficient, neural network parameter, etc.

[0074] The linear transformation coefficient can be a color correction matrix (CCM). The CCM is mainly used to perform color correction on the raw image output by the image sensor to obtain a more accurate and natural color reproduction effect. It linearly transforms the image through a 3x3 matrix or a 3x4 matrix (the latter additionally includes an overall offset of the RGB component) to precisely adjust the color channels of the image and ensure that the color of the image is consistent with the real environment. This correction is crucial for ensuring excellent color effects of the image under different lighting conditions.

[0075] The expression of the high-order polynomial coefficient can be:

[0076] P(x) = a n x n + a n-1 x n-1 +... + a 1 x 1 + a 0

[0077] where x is the input variable, specifically including the pixel value of the pixel point in the first image or the first reference pixel, and a n , a n-1 ,..., a 1 , a 0 are polynomial coefficients, n is a non-negative integer and represents the highest power of the polynomial. P(x) is the output variable, that is, the pixel value of the corresponding pixel point in the second image or the second reference image.

[0078] Neural network parameters refer to the internal variables learned during the training process of a neural network model. These variables play a decisive role in the prediction ability of the neural network model, that is, they determine how to map the input data to the output. In the embodiments of this application, the neural network model is used to perform color mapping on the input first image to obtain a second image. For such a neural network model, its parameters include but are not limited to weights and biases.

[0079] Step 103: Perform color mapping on the first image based on the color mapping parameters of the first image to obtain a second image. The first image is an image in the color space of the first imaging device, and the second image is an image in the color space of the second imaging device.

[0080] The color mapping process can be flexibly configured according to different parameter types to meet different color adjustment requirements and application scenarios.

[0081] For example, when the color mapping parameter is a linear transformation coefficient, the color values of the first image are linearly transformed by the linear transformation coefficient to obtain the color values of the second image. This linear transformation can maintain the relative relationship and distribution of colors and achieve simple color adjustment.

[0082] Another example is that when the color mapping parameter is a high-order polynomial coefficient, a high-order polynomial is configured by the high-order polynomial coefficient, and the color values of the first image are used as the input variables of the high-order polynomial to output the color values of the second image. High-order polynomial mapping can capture more complex color transformation relationships and achieve a more delicate color adjustment effect.

[0083] Another example is that when the color mapping parameter is a neural network parameter, a neural network model is configured by the neural network parameter, and the first image is input into the neural network model to output the second image. Neural network mapping has strong learning ability and non-linear transformation ability, can capture and simulate complex color transformation patterns, and thus achieve highly flexible and accurate color adjustment.

[0084] It should be noted that the first image and the second image may include a first image component, a second image component, and a third image component. Among them, these three image components are respectively a luminance component, a blue chrominance component, and a red chrominance component, and these three image components are also respectively a red (R) channel component, a green (G) channel component, and a blue (B) channel component.

[0085] Assume that the first image includes a first image component, and the first image component is a luminance component. Then, perform color mapping on the first image component included in the first image based on the color mapping parameters of the luminance component to obtain the first image component included in the second image; or, assume that the first image includes a second image component, and the second image component is a chrominance component. Then, perform color mapping on the second image component included in the first image based on the color mapping parameters of the chrominance component to obtain the second image component included in the second image.

[0086] In one example, the color mapping parameters of the three image components are the same. In another example, the color mapping parameters of the three image components are different. In yet another example, the color mapping parameters of some of the image components are the same, and the color mapping parameters of some other image components are different. For example, the luminance component uses separate color mapping parameters, and the two chrominance components use the same color mapping parameters.

[0087] It should be noted that according to the target white balance parameters corresponding to the first image, appropriate color mapping parameters are configured for the first image, thereby improving the accuracy of color mapping of the first image.

[0088] In some embodiments, when the target white balance parameters are the white balance parameters determined based on the first image, obtaining the first image captured by the first imaging device and its corresponding target white balance parameters includes: obtaining the first image captured by the first imaging device and its corresponding target white balance parameters from the first data set, where the first data set is used to train the automatic white balance model of the first imaging device. The first data set includes a large number of first images and the true values of the white balance parameters corresponding to each first image.

[0089] Correspondingly, the method may further include: constructing a second data set based on the second image; training the automatic white balance model of the second imaging device based on the second data set.

[0090] It should be noted that the automatic white balance model is a machine learning model, and the training of the model depends on a large image data set, which contains image data of various scenes (indoor, outdoor, day, night, etc.) captured under different lighting conditions. The first data set is a large-scale data set captured by the first imaging device. If there is a color response difference between the first imaging device and the second imaging device, the automatic white balance model trained for the first imaging device generally cannot be directly applied to the second imaging device, and the first data set cannot be used for the training of the automatic white balance model of the second imaging device either.

[0091] In view of this, through the image processing method provided in the embodiments of the present application, the first data set is converted into a second data set adapted to the second imaging device, improving the quality of the data set.

[0092] In some embodiments, when the target white balance parameter is the white balance parameter determined based on the third image, the method further includes: obtaining a target image based on the second image; and displaying the target image.

[0093] It should be noted that in the same shooting scene, if shooting is performed using imaging devices with different color response characteristics, the colors of the obtained images may be different. To ensure the color consistency of images captured by different imaging devices, the images captured by the first imaging device (non-reference imaging device) can be converted into the color space of the second imaging device (reference imaging device), so that the colors of the images captured by all non-reference imaging devices are consistent with those of the reference imaging device, thereby achieving color consistency among different imaging devices. In this way, when switching between different imaging devices, the color consistency of the displayed screen can be ensured; in addition, when identifying and analyzing images captured by different imaging devices, the accuracy and reliability of the system can be improved.

[0094] To better illustrate the purpose of the present application, further examples are provided based on the above embodiments of the present application. Figure 3 This is the second flowchart of the image processing method in the embodiments of the present application, as Figure 3 shown, the method may specifically include:

[0095] Step 301: Obtain the first image captured by the first imaging device and its corresponding target white balance parameter from the first data set, where the first data set is used to train the automatic white balance model of the first imaging device;

[0096] Exemplarily, the target white balance parameter is the true value of the white balance parameter corresponding to the first image, and the white balance parameter (such as white balance gain) for each image in the first data set is known. During model training, it is used to guide the model and help the model gradually approach the optimal solution. During color space conversion, it is used as an index of the mapping relationship to determine the color mapping parameter adapted to the first image.

[0097] Step 302: Determine the color mapping parameter of the first image based on the target white balance parameter and the mapping relationship, where the mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0098] In some embodiments, the method further includes: obtaining, by the image acquisition system, the first reference image captured by the first imaging device for the shooting object and the second reference image captured by the second imaging device for the shooting object in multiple shooting scenes; determining the color mapping parameter between the first reference image and the second reference image; and constructing the mapping relationship using the white balance parameter and the color mapping parameter corresponding to each shooting scene.

[0099] The white balance parameter corresponding to each shooting scenario can be the white balance parameter determined based on the first reference image. Exemplarily, the white balance of the first reference image is predicted through the automatic white balance model of the first imaging device to obtain the white balance parameter of the first reference image.

[0100] An embodiment of the present application provides a portable image acquisition system for quickly acquiring data required for color space conversion.

[0101] Figure 4 It is a schematic diagram of the composition framework of an image acquisition system provided by an embodiment of the present application, as Figure 4 shown, the image acquisition system 400 includes: a first imaging device 401, a second imaging device 402, a control device 403, a small-size 24-color card 404, and a fixing device 405.

[0102] Among them, the first imaging device 401, the second imaging device 402, and the small-size 24-color card 404 are fixed on the fixing device 405. The control device 403 can also be fixed on the fixing device 405 or can be independently arranged. The control device 403 communicates with the first imaging device 401 and the second imaging device 402 by wire or wirelessly. The control device 403 simultaneously sends shooting instructions to the two imaging devices to obtain reference images of the two imaging devices in the same shooting scenario.

[0103] The small-size 24-color card is located within the common field of view of the first reference image and the second reference image, so that the two imaging devices can acquire complete 24-color card images.

[0104] Through the image acquisition system, the color difference mapping parameters of different imaging devices in various shooting environments can be quickly acquired and calibrated, so as to convert the existing large dataset to the color space of the target imaging device, avoid collecting tens of thousands of datasets for each type of imaging device, improve the quality of the dataset, and reduce the cost of dataset production.

[0105] It should be noted that compared with the massive images of the large dataset, the number of reference images obtained through the image acquisition system is significantly smaller, reducing the cost of dataset production.

[0106] Step 303: Perform color mapping on the first image based on the color mapping parameter of the first image to obtain a second image;

[0107] Exemplarily, taking the color mapping parameter as the color correction matrix as an example, the color mapping formula is as follows:

[0108]

[0109] Among them, R 1 、G 1 、B 1Represents the color value (components of the three primary colors red, green, and blue) of a certain pixel point in the first image, R 2 、G 2 、B 2 Represents the color value of a certain pixel point in the converted second image. C is the color correction matrix. When constructing the mapping relationship, the least squares method is applied to solve the color correction matrix.

[0110] Step 304: Based on the second image, construct a second data set;

[0111] Exemplarily, the method further includes: using the true value of the white balance parameter of the first image as the true value of the white balance parameter of the second image; or, performing white balance prediction on the second image based on a high-precision automatic white balance algorithm to obtain the true value of the white balance parameter of the second image; constructing a second data set based on the second image and its true white balance parameter value.

[0112] Step 305: Train the automatic white balance model of the second imaging device based on the second data set.

[0113] The training process of the automatic white balance model of the second imaging device includes: inputting the second image in the second data set into the automatic white balance model of the second imaging device, and outputting the predicted value of the white balance parameter of the second image; calculating the loss function based on the true value and the predicted value of the white balance parameter; continuously adjusting the internal parameters of the automatic white balance model through optimization techniques such as the backpropagation algorithm to minimize the value of the loss function; when the value of the loss function fluctuates within a certain range and no longer decreases significantly, we can consider that the model has converged, that is, the training process is completed.

[0114] Through the image processing method provided by the embodiments of the present application, there is no need to collect images for each imaging device. Only by performing color space conversion on the data set of one type of imaging device can a data set applicable to other imaging devices be obtained, so as to adapt to different imaging devices with color response differences, avoid collecting tens of thousands of data sets for each type of imaging device, improve the quality of the data set, and reduce the cost of data set production.

[0115] In order to better reflect the purpose of the present application, based on the above embodiments of the present application, further examples are given. Figure 5 This is the third process schematic diagram of the image processing method in the embodiments of the present application. As Figure 5 shown, the method may specifically include:

[0116] Step 501: Obtain the first image collected by the first imaging device and its corresponding target white balance parameter;

[0117] Exemplarily, the target white balance parameter is based on the white balance parameter determined from the third image, which is the image captured by the second imaging device. For example, the white balance of the third image is predicted through the automatic white balance model or algorithm of the second imaging device to obtain the target white balance parameter.

[0118] Exemplarily, the method further includes: starting the second imaging device, and the second imaging device captures the third image.

[0119] It can be understood that the second imaging device serves as the reference imaging device, and the first imaging device serves as the non-reference imaging device. When the first imaging device is the main camera, the second imaging device is the secondary camera, and the white balance parameter of the secondary camera image is used as the target white balance parameter of the main camera image.

[0120] Step 502: Based on the target white balance parameter and the mapping relationship, determine the color mapping parameter of the first image, where the mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0121] In some embodiments, the method further includes: through the image acquisition system, in multiple shooting scenarios, obtaining the first reference image captured by the first imaging device for the shooting object, the second reference image captured by the second imaging device for the shooting object, and the corresponding white balance parameter for each shooting scenario; determining the color mapping parameter between the first reference image and the second reference image; using the corresponding white balance parameter and color mapping parameter for each shooting scenario to construct the mapping relationship.

[0122] The corresponding white balance parameter for each shooting scenario may be the white balance parameter determined based on the second reference image. Exemplarily, the white balance of the second reference image is predicted through the automatic white balance model of the second imaging device to obtain the white balance parameter of the first reference image.

[0123] In one example, the first imaging device and the second imaging device may be imaging devices on different electronic devices. The image acquisition system may be as Figure 4 shown.

[0124] In another example, the first imaging device and the second imaging device may be different imaging devices on the same electronic device, and different imaging devices correspond to different focal lengths. In this way, when switching between different imaging devices, the color consistency of the display screen can be ensured, and the display screen jump can be avoided. The image acquisition system includes: an electronic device, a small-sized 24-color card, and a bracket, where the electronic device includes at least two imaging devices.

[0125] Step 503: Perform color mapping on the first image based on the color mapping parameter of the first image to obtain the second image;

[0126] Step 504: Obtain a target image based on the second image;

[0127] Step 505: Display the target image.

[0128] To ensure the color consistency of images captured by different imaging devices, the images captured by the first imaging device (non-reference imaging device) can be converted to the color space of the second imaging device (reference imaging device), so that the images captured by all non-reference imaging devices are consistent with the images of the reference imaging device in terms of color representation, thereby achieving color consistency among different imaging devices.

[0129] To implement the method of the embodiments of the present application, based on the same inventive concept, the embodiments of the present application also provide an image processing device, as Figure 6 shown. The image processing device 600 includes:

[0130] An acquisition unit 601, configured to acquire a first image captured by a first imaging device and its corresponding target white balance parameter;

[0131] A determination unit 602, configured to determine the color mapping parameter of the first image based on the target white balance parameter and the mapping relationship, where the mapping relationship includes the mapping relationships between multiple white balance parameters and color mapping parameters;

[0132] A processing unit 603, configured to perform color mapping on the first image based on the color mapping parameter of the first image to obtain a second image, where the first image is an image in the color space of the first imaging device, and the second image is an image in the color space of the second imaging device.

[0133] In some embodiments, the determination unit 602 is further configured to, in multiple shooting scenarios, acquire a first reference image captured by the first imaging device for shooting an object, a second reference image captured by the second imaging device for shooting the object, and the corresponding white balance parameter for each shooting scenario; determine the color mapping parameter between the first reference image and the second reference image; and construct a mapping relationship by using the corresponding white balance parameter and color mapping parameter for each shooting scenario.

[0134] In an example, the white balance parameter corresponding to each shooting scenario may be the white balance parameter determined based on the first reference image. In another example, when the first imaging device and the second imaging device shoot the same shooting scenario, the white balance parameter corresponding to each shooting scenario may be the white balance parameter determined based on the second reference image captured by the second imaging device.

[0135] In some embodiments, the positions of the first imaging device and the second imaging device are relatively fixed, and the first imaging device and the second imaging device respond to a shooting instruction issued at the same moment to respectively acquire a first reference image and a second reference image.

[0136] In some embodiments, the object to be photographed is a color card including a plurality of color patches, and different color patches have different colors.

[0137] In some embodiments, the determining unit 602 is configured to perform feature point detection and matching on the first reference image and the second reference image to determine a plurality of pairs of matching points in the first reference image and the second reference image; based on the plurality of pairs of matching points, determine pixel values of a plurality of pairs of reference regions in the first reference image and the second reference image; and based on the pixel values of the plurality of pairs of reference regions, determine color mapping parameters between the first reference image and the second reference image.

[0138] In some embodiments, the determining unit 602 is configured to determine a coordinate mapping relationship between the first reference image and the second reference image based on the plurality of pairs of matching points; align the first reference image and the second reference image based on the coordinate mapping relationship; and respectively obtain pixel values of a plurality of pairs of reference regions from the aligned first reference image and the second reference image.

[0139] In some embodiments, the determining unit 602 is configured to determine at least one white balance parameter similar to the target white balance parameter from the mapping relationship based on the target white balance parameter; and determine color mapping parameters of the first image based on the color mapping parameters corresponding to at least one white balance parameter in the mapping relationship.

[0140] In some embodiments, the determining unit 602 is configured to perform interpolation operations on color mapping parameters corresponding to at least two white balance parameters in the mapping relationship to obtain color mapping parameters of the first image.

[0141] In some embodiments, the target white balance parameter is a white balance parameter determined based on the first image.

[0142] In some embodiments, the obtaining unit 601 is configured to obtain a first image captured by a first imaging device and its corresponding target white balance parameter from a first data set, where the first data set is used to train an automatic white balance model of the first imaging device;

[0143] In some embodiments, the processing unit 603 is further configured to construct a second data set based on the second image; and train an automatic white balance model of a second imaging device based on the second data set.

[0144] In some embodiments, the obtaining unit 601 is further configured to obtain a third image captured by the second imaging device; determine a white balance parameter of the third image; and use the white balance parameter of the third image as the target white balance parameter. That is, the target white balance parameter is a white balance parameter determined based on the third image, and the third image is an image captured by the second imaging device.

[0145] In some embodiments, the processing unit 603 is further configured to obtain a target image based on the second image; the image processing apparatus 600 further includes a display unit configured to display the target image.

[0146] In practical applications, the above-mentioned image processing apparatus may be an electronic device or a chip applied to an electronic device. In this application, the apparatus may implement the functions of multiple units through software, hardware, or a combination of software and hardware, enabling the apparatus to execute the image processing method provided in any of the foregoing embodiments. The technical effects of the various technical solutions of the apparatus may refer to the technical effects of the corresponding technical solutions in the image processing method, and details thereof are not described herein again.

[0147] Based on the hardware implementation of each unit in the above-mentioned image processing apparatus, an embodiment of the present application further provides an electronic device, as Figure 7 shown. The electronic device 700 includes a processor 701 and a memory 702 configured to store a computer program that can run on the processor.

[0148] Wherein, when the processor 701 is configured to run the computer program, it executes the method steps in the foregoing embodiments.

[0149] Of course, in practical applications, as Figure 7 shown, the various components in the electronic device 700 are coupled together through a bus system 703. It can be understood that the bus system 703 is used to implement the connection and communication between these components. In addition to the data bus, the bus system 703 further includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all kinds of buses are labeled as the bus system 703 in the figure.

[0150] In practical applications, the above-mentioned processor may be at least one of an application specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic devices for implementing the functions of the above-mentioned processor may be others, and the embodiments of the present application do not make specific limitations.

[0151] The above-mentioned memory may be a volatile memory, such as a random-access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or a combination of the above types of memories, and provides instructions and data to the processor.

[0152] The electronic devices described in this application may include, such as mobile phones, tablet computers, laptop computers, palmtop computers, personal digital assistants (PDAs), portable media players (PMPs), cameras, wearable devices, cameras, etc.

[0153] In an exemplary embodiment, the embodiment of this application also provides a computer-readable storage medium, such as a memory including a computer program, and the computer program can be executed by a processor of an electronic device to complete the steps of the foregoing method.

[0154] The embodiment of this application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in any one of the embodiments of this application.

[0155] Optionally, the computer program product can be applied to the electronic device in the embodiment of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the electronic device in each method of the embodiment of this application. For the sake of brevity, it will not be elaborated here.

[0156] The embodiment of this application also provides a computer program.

[0157] Optionally, the computer program can be applied to the electronic device in the embodiment of this application. When the computer program runs on the computer, it causes the computer to execute the corresponding processes implemented by the electronic device in each method of the embodiment of this application. For the sake of brevity, it will not be elaborated here.

[0158] The embodiment of this application also provides an image acquisition system, as Figure 8 shown, the system includes: The image acquisition system 800 includes a first imaging device 801, a second imaging device 802, and a shooting object 803, and the positions of the first imaging device 801 and the second imaging device 802 are relatively fixed;

[0159] The first imaging device 801 is configured to capture a first reference image of a subject in response to a first shooting instruction.

[0160] The second imaging device 802 is configured to capture a second reference image of a subject in response to a second shooting instruction.

[0161] The first imaging device 801 is further configured to determine white balance parameters corresponding to the first reference image based on the first reference image; alternatively, the second imaging device 802 is further configured to determine white balance parameters corresponding to the first reference image based on the second reference image.

[0162] In some embodiments, the image acquisition system 800 further includes a control device configured to issue the first shooting instruction and the second shooting instruction at the same time.

[0163] In some embodiments, the image acquisition system 800 further includes a fixing device for assembling the first imaging device, the second imaging device and the subject together so that their positions are relatively fixed.

[0164] In some embodiments, the subject is a color card including multiple color patches with different colors.

[0165] In some embodiments, the first imaging device and the second imaging device are integrated in the same electronic device, for example, a mobile phone integrating at least two imaging devices.

[0166] In some embodiments, the first imaging device and the second imaging device are respectively assembled on different electronic devices.

[0167] With this image acquisition system, reference images in various open environments and under various light sources can be quickly captured, and thus, according to the reference images of different imaging devices, the chromatic aberration mapping parameters between the two imaging devices can be quickly calibrated, which is beneficial to improving the efficiency of image acquisition and processing.

[0168] It should be understood that in the embodiments of the present application, when it comes to data related to user information, etc., when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0169] It should be understood that the terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. Expressions such as "has", "may have", "includes", "contains", "may include", and "may contain" in this application can be used to indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations, or components), but do not exclude the presence of additional features.

[0170] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and do not necessarily describe a specific order or sequence. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0171] Among the technical solutions described in the embodiments of this application, they can be arbitrarily combined without conflict.

[0172] In several embodiments provided in this application, it should be understood that the disclosed methods, devices, and equipment can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0173] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in the embodiments of this application can be all integrated in one processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0175] As described above, it is only the specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image captured by a first camera device and a corresponding target white balance parameter; Determining a color mapping parameter of the first image based on the target white balance parameter and a mapping relationship, wherein the mapping relationship includes mapping relationships between a plurality of white balance parameters and color mapping parameters; Color mapping is performed on the first image based on the color mapping parameters of the first image to obtain a second image, wherein the first image is an image in the color space of the first camera device, and the second image is an image in the color space of the second camera device.

2. The method according to claim 1, characterized in that The method further comprises: In various shooting scenes, obtaining a first reference image captured by the first camera device on a shooting object, a second reference image captured by the second camera device on the shooting object, and corresponding white balance parameters in each shooting scene; determining color mapping parameters between the first reference image and the second reference image; The mapping relationship is constructed using the white balance parameters corresponding to each shooting scene and the color mapping parameters.

3. The method according to claim 2, characterized in that The positions of the first camera device and the second camera device are relatively fixed. The first camera device and the second camera device respectively capture the first reference image and the second reference image in response to a shooting instruction issued at the same time.

4. The method according to claim 2, characterized in that: The photographed object is a color card including a plurality of color blocks, and different color blocks have different colors.

5. The method according to claim 2, characterized in that: The determining of a color mapping parameter between the first reference image and the second reference image comprises: Performing feature point detection and matching on the first reference image and the second reference image to determine a plurality of matching point pairs in the first reference image and the second reference image; Determining pixel values ​​of a plurality of reference region pairs in the first reference image and the second reference image based on the plurality of matching point pairs; The color mapping parameters between the first reference image and the second reference image are determined based on the pixel values ​​of the plurality of reference region pairs.

6. The method according to claim 5, characterized in that The step of determining pixel values ​​of a plurality of reference area pairs in the first reference image and the second reference image based on the plurality of matching point pairs comprises: Determining a coordinate mapping relationship between the first reference image and the second reference image based on the multiple matching point pairs; Based on the coordinate mapping relationship, aligning the first reference image and the second reference image; Pixel values ​​of the plurality of reference region pairs are acquired respectively from the aligned first reference image and the second reference image.

7. The method according to claim 1, characterized in that The determining, based on the target white balance parameter and the mapping relationship, the color mapping parameter of the first image includes: Based on the target white balance parameter, determining at least one white balance parameter close to the target white balance parameter from the mapping relationship; Based on the color mapping parameters corresponding to the at least one white balance parameter in the mapping relationship, the color mapping parameters of the first image are determined.

8. The method according to claim 7, characterized in that The determining the color mapping parameter of the first image based on the color mapping parameter corresponding to the at least one white balance parameter in the mapping relationship includes: An interpolation operation is performed on color mapping parameters corresponding to at least two white balance parameters in the mapping relationship to obtain color mapping parameters of the first image.

9. The method according to any one of claims 1 to 8, characterized in that: The target white balance parameters are white balance parameters determined based on the first image.

10. The method according to claim 9, characterized in that The step of acquiring a first image captured by a first camera device and a corresponding target white balance parameter thereof includes: Acquire a first image captured by a first camera device and a corresponding target white balance parameter thereof from a first data set, wherein the first data set is used to train an automatic white balance model of the first camera device; The method further comprises: Based on the second image, construct a second data set; An automatic white balance model of the second camera device is trained based on the second data set.

11. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquire a third image captured by the second camera device; determining a white balance parameter of the third image; The white balance parameters of the third image are used as the target white balance parameters.

12. The method according to claim 9, characterized in that The method further comprises: Based on the second image, obtaining a target image; The target image is displayed.

13. An image processing device, characterized in that: The device comprises: An acquisition unit, used to acquire a first image captured by a first camera device and a corresponding target white balance parameter; a determining unit, configured to determine a color mapping parameter of the first image based on the target white balance parameter and a mapping relationship, wherein the mapping relationship includes a mapping relationship between a plurality of white balance parameters and color mapping parameters; A processing unit is used to perform color mapping on the first image based on the color mapping parameters of the first image to obtain a second image, wherein the first image is an image in the color space of the first camera device, and the second image is an image in the color space of the second camera device.

14. An electronic device, characterized in that: The electronic device comprises: a processor and a memory configured to store a computer program capable of running on the processor, Wherein, the processor is configured to execute the steps of the method according to any one of claims 1 to 12 when running the computer program.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. An image acquisition system, characterized in that: The image acquisition system includes a first camera device, a second camera device and a photographed object, and the positions of the first camera device and the second camera device are relatively fixed; The first camera device is configured to capture a first reference image of the object in response to a first shooting instruction; The second camera device is configured to capture a second reference image of the object in response to a second shooting instruction; The first camera device is further configured to determine a white balance parameter corresponding to the first reference image based on the first reference image; or the second camera device is further configured to determine a white balance parameter corresponding to the first reference image based on the second reference image.

17. The system according to claim 16, characterized in that The image acquisition system further comprises a control device, The control device is configured to issue the first shooting instruction and the second shooting instruction at the same time.

Images