A color correction matrix optimization method and device, electronic equipment and medium

Abstract

Embodiments of the present application disclose a color correction matrix optimization method and device, electronic equipment and medium. The method comprises: based on a color correction matrix, performing correction processing on a color source matrix of a color card image to obtain a color output matrix; determining a global hue error and a saturation constraint regularization term according to Lab coordinates of the color output matrix and Lab coordinates of a color target matrix of a target color card image, and constructing a loss function; updating the color correction matrix according to the loss function value until the loss function value converges, to obtain an optimal color correction matrix, so as to perform color correction on a to-be-processed image based on the optimal color correction matrix. In the above technical solution, when the color correction matrix is optimized, the loss function is determined by considering the perception of the human eye to the hue difference and the saturation difference, so that the image collected by the image collector can be closer to the color perceived by the human eye after color correction based on the optimized color correction matrix.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a color correction matrix optimization method, apparatus, electronic device, and medium. Background Technology

[0002] Images captured by digital cameras often exhibit color deviations from the actual images perceived by the human eye. Factors contributing to this color deviation include the color temperature of the light source, lens transmission characteristics, and the photosensitive characteristics of the image sensor. In digital color correction matrix optimization, the Automatic White Balance (AWB) algorithm is used to correct color casts caused by color temperature variations, simulating the constancy of human eye perception of gray. Even after AWB correction, color images still exhibit significant color casts. The primary reason for this is the substantial difference between the image sensor's response and the human eye's response to spectral density and brightness. Therefore, after AWB restores the color of gray objects, a Color Correction Matrix (CCM) is needed to correct the color cast of colored objects in the images captured by the camera.

[0003] However, current color correction schemes simply optimize the color correction matrix using the Euclidean distance between the coordinates of the image to be corrected and the image after correction. But Euclidean distance cannot fully represent the difference between colors perceived by the human eye and colors captured by the camera. Therefore, the colors in the color-corrected image do not closely match human perception. Furthermore, current color correction matrix optimization schemes use the least squares method to obtain an initial color correction matrix, which can easily lead to local minima traps in subsequent optimization processes, preventing the attainment of a globally optimal solution. Summary of the Invention

[0004] This application provides a color correction matrix optimization method, apparatus, electronic device, and medium, which optimizes the color correction matrix by determining a loss function based on the actual perception of hue and saturation by the human eye, thereby improving the fit between the color of the color-corrected image and the color perceived by the human eye.

[0005] In one embodiment, this application provides a color correction matrix optimization method, including:

[0006] Based on the color correction matrix, the color source matrix of the acquired color card image is corrected to obtain the color output matrix;

[0007] Based on the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color chart image, global hue error and saturation constraint regularization terms are determined, and a loss function is constructed based on the global hue error and the saturation constraint regularization terms; where Lab coordinates are coordinates in the Lab color space;

[0008] The color correction matrix is ​​updated based on the loss function value until the loss function value converges to obtain the optimal color correction matrix, which is then used to perform color correction on the image to be processed acquired by the image acquisition device.

[0009] In one embodiment, this application provides a color correction matrix optimization device, comprising:

[0010] The correction processing module is used to correct the color source matrix of the acquired color card image based on the color correction matrix to obtain the color output matrix;

[0011] The loss function construction module is used to determine the global hue error and saturation constraint regularization term based on the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, and to construct the loss function based on the global hue error and the saturation constraint regularization term; wherein, the Lab coordinates are the coordinates in the Lab color space;

[0012] An optimization module is used to update the color correction matrix based on the loss function value until the loss function value converges to obtain the optimal color correction matrix, so as to perform color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix.

[0013] In one embodiment, this application provides an electronic device, including:

[0014] One or more processors;

[0015] Storage device for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the color correction matrix optimization method as described in any of the above embodiments.

[0017] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the color correction matrix optimization method as described in any of the above embodiments.

[0018] This application provides a method, apparatus, electronic device, and medium for optimizing a color correction matrix. The implementation involves: First, correcting the color source matrix of a captured color chart image based on the color correction matrix to obtain a color output matrix. Then, determining global hue error and saturation constraint regularization terms based on the Lab coordinates of the color output matrix and the Lab coordinates of the target color matrix of the target color chart image, and constructing a loss function based on the global hue error and the saturation constraint regularization terms. Here, Lab coordinates are coordinates in the Lab color space. The color correction matrix is ​​updated based on the loss function value until the loss function value converges, resulting in an optimal color correction matrix. This optimal color correction matrix is ​​then used to perform color correction on the image to be processed acquired by the image acquisition device. In the above technical solution, when optimizing the color correction matrix, the loss function is constructed considering the human eye's perception of hue and saturation deviations, so that the image acquired by the image acquisition device, after color correction based on the optimized color correction matrix, can more closely approximate the colors perceived by the human eye. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart of a color correction matrix optimization method provided in one embodiment of this application;

[0021] Figure 2 This is a schematic diagram of color block pixels provided in one embodiment of this application;

[0022] Figure 3 A flowchart of a color correction matrix optimization method provided in another embodiment of this application;

[0023] Figure 4 A flowchart illustrating a color correction matrix optimization method provided in yet another embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an iterative optimization process provided in another embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of a color correction matrix optimization device provided in one embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0027] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, the embodiments and features described herein can be combined with each other unless otherwise specified. It should also be noted that, for ease of description, only the parts relevant to the present application are shown in the accompanying drawings, not the entire structure.

[0028] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0029] To better understand the embodiments of this application, the relevant technologies are described below.

[0030] Figure 1 This is a flowchart illustrating a color correction matrix optimization method provided in an embodiment of this application. This embodiment is applicable to situations requiring optimization of the color correction matrix. Specifically, the color correction matrix optimization method can be executed by a color correction matrix optimization device, which can be implemented through software and / or hardware and integrated into an electronic device.

[0031] like Figure 1 As shown, the method specifically includes the following steps:

[0032] S110. Based on the color correction matrix, the color source matrix of the acquired color card image is corrected to obtain the color output matrix.

[0033] The acquired color chart image is obtained by an image acquisition device from a 24-color standard color chart. Before color correction, preprocessing such as black level correction, Bayer interpolation, and white balance correction can be performed on the acquired color chart image. A color source matrix for the acquired color chart image is constructed based on the grayscale values ​​of the R, G, and B channels of each color block. Specifically, pixels closest to the center pixel are selected from the entire pixel region of each color block. The average grayscale values ​​of the R, G, and B channels of these extracted pixels are calculated and used as the grayscale values ​​of the R, G, and B channels for that color block, forming one column of the matrix. For 24 color blocks, this results in 24 columns of grayscale values ​​for the R, G, and B channels, constructing the color source matrix. The color correction matrix is ​​then compared with the corresponding matrix of the color source matrix to perform color correction on the color source matrix.

[0034] For example, such as Figure 2 As shown, assuming Figure 2 To capture one color patch from a color chart image, select the pixel region closest to the center pixel. Figure 2 The shading represents 25 pixels. Calculate the average grayscale values ​​of the R channel, G channel, and B channel of these 25 pixels to obtain the corresponding color block. and After performing the above processing on all 24 color patches, the color source matrix of the acquired color card image can be represented as follows: The first column represents the grayscale values ​​of the R, G, and B channels of the first color patch; the second column represents the grayscale values ​​of the R, G, and B channels of the second color patch; and the last column represents the grayscale values ​​of the R, G, and B channels of the 24th color patch. In this embodiment, Figure 2 The number of pixels in the entire pixel area and the number of pixels in a partial pixel area of ​​the color block are just examples. The specific number is determined based on the acquired color chart image and is not limited here.

[0035] The color correction matrix can be used to The parameters in CCM can be adjusted during subsequent optimization. The color source matrix is ​​corrected by undergoing a linear transformation using CCM. The specific expression is C... o =CCM·C s , where C o This is the color output matrix.

[0036] S120. Based on the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, determine the global hue error and saturation constraint regularization terms, and construct a loss function based on the global hue error and the saturation constraint regularization terms; wherein, the Lab coordinates are the coordinates in the Lab color space.

[0037] The target color chart image is a pre-determined color chart image that reflects human visual perception of color. The color target matrix is ​​a matrix obtained by sampling and averaging the three-channel grayscale values ​​of each color block pixel in the target color chart image. The process of determining the color target matrix of the target color chart image is the same as the process of determining the color source matrix of the acquired color chart image, as detailed in S110. The obtained color target matrix is... In actual color correction processes, after the image acquired by the image acquisition device is color-corrected using a color correction matrix, it still needs to undergo nonlinear operations using gamma correction to obtain the final corrected image. Therefore, in this embodiment of the application, in order to compare the color output matrix after color correction using the color correction matrix with the color target matrix, the color target matrix is ​​subjected to an inverse gamma transform, thereby enabling the color output matrix and the color target matrix to be compared and analyzed under the same processing dimension.

[0038] The Lab color space is a color space where luminance and chromaticity are relatively separated, and it is a uniform color space recommended by the International Commission on Illumination (ICI) in China. This space is a three-dimensional Cartesian coordinate system and is currently the most widely used color measurement system. Here, L represents the luminance component, and the (a,b) components together form the chromaticity plane, where a and b represent the color's opposing dimensions. To evaluate the color difference between the color output matrix and the color target matrix, the data from other color spaces needs to be converted to Lab color space for comparison. In the Lab color space, the coordinates of the color output matrix represent the hue and saturation information, and the coordinates of the color target matrix represent the hue and saturation information, allowing for the determination of global hue error and saturation constraint regularization terms for both the color output and color target matrices. The global hue error represents the difference in chromaticity. If the loss function is determined solely based on the global hue error, the saturation in the chromaticity space can be arbitrarily scaled, failing to achieve the desired color correction effect. In this embodiment of the application, to address the aforementioned issues, a saturation constraint regularization term is constructed based on saturation differences. This restricts arbitrary scaling of saturation during the loss function optimization process, thereby improving the color correction effect. The saturation constraint regularization term can be determined based on saturation differences. The specific determination method can be determined according to the actual situation. For example, the saturation can be represented by the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix to determine the saturation differences, and then the saturation constraint regularization term can be constructed based on these differences.

[0039] Specifically, a loss function can be constructed by calculating the global hue error and saturation constraint regularization terms. The specific values ​​of the elements in the color output matrix and the color target matrix are then substituted into the loss function to obtain its value. During the calculation process, coefficients can be introduced to adjust the weights of the global hue error and saturation constraint regularization terms in the loss function, depending on the actual situation.

[0040] The solution in this application takes into account the global hue and saturation differences that are sensitive to human vision. Then, based on the global hue error and saturation constraint regularization term, the loss function value is determined to update the color correction matrix so that the color correction of the acquired image based on the optimized color correction matrix can be closer to the color perceived by human vision.

[0041] S130. Update the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix, and perform color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix.

[0042] For example, the color correction matrix is ​​iteratively optimized until the loss function converges, obtaining the minimum value of the loss function. The color correction matrix corresponding to the minimum value of the loss function is then taken as the optimal color correction matrix. In subsequent applications, the relevant parameters of the optimal color correction matrix can be input into the image acquisition device, enabling the image acquisition device to generate the optimal color correction matrix based on the relevant parameters, perform color correction on the acquired image to be processed, and then perform a gamma transform on the color-corrected image to obtain the final output image. Because the output image has undergone color correction and gamma transformation, its colors are closer to the colors perceived by the human eye, improving the user's viewing experience.

[0043] It should be noted that the electronic device implementing the above scheme in this application embodiment can be the same as or different from the device that subsequently uses the optimal color correction matrix to perform color correction on the image to be processed. Generally, since the processing data volume of the scheme in this application embodiment is large, it can be executed by an electronic device with high processing performance. After obtaining the optimal color correction matrix, the relevant parameters of the optimal color correction matrix are then input into the image acquisition device so that the image acquisition device can apply the relevant parameters to generate the optimal color correction matrix to process the acquired image.

[0044] This application provides a color correction matrix optimization method, apparatus, electronic device, and medium. The implementation scheme involves: correcting the color source matrix of a captured color chart image based on the color correction matrix to obtain a color output matrix; determining a loss function value based on the global hue error and saturation constraint regularization term of the color output matrix and the target color matrix of the target color chart image in the Lab color space; updating the color correction matrix based on the loss function value until the loss function value converges to obtain the optimal color correction matrix; and then performing color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix. In the above technical solution, when optimizing the color correction matrix, the loss function is determined considering the human eye's perception of hue and saturation deviations, so that the image acquired by the image acquisition device, after color correction based on the optimized color correction matrix, can more closely approximate the colors perceived by the human eye.

[0045] Figure 3This is a flowchart illustrating a color correction matrix optimization method according to another embodiment of this application. This embodiment is an optimization based on the above embodiments. It should be noted that technical details not described in detail in this embodiment can be found in any of the above embodiments.

[0046] Specifically, such as Figure 3 As shown, the method specifically includes the following steps:

[0047] S210. Based on the color correction matrix, the color source matrix of the acquired color card image is corrected to obtain the color output matrix.

[0048] S220. For each color patch represented by each column element in the color output matrix and the color target matrix, determine the chromaticity coordinates corresponding to each color patch; wherein, the chromaticity coordinates are the coordinates of the color output matrix and the color target matrix in the chromaticity plane after being converted to the Lab color space.

[0049] For example, each column element in the color output matrix and color target matrix represents a color patch. Converting the color output matrix and color target matrix to the Lab color space yields their respective elements in the Lab color space, allowing us to determine the chromaticity coordinates of each color patch in the Lab color space. For example, in the RGB color space, the i-th color patch in the color output matrix is ​​represented by the i-th column element. and This means that, when converted to the Lab color space, the i-th color patch in the color output matrix is ​​represented by... and express, This represents the coordinates of the i-th color patch in the Lab color space. The matrix in the Lab color space composed of the coordinates of the 24 color patches is as follows: Similarly, the color target matrix in the Lab color space is:

[0050] In this embodiment, chromaticity coordinates are coordinates on the chromaticity plane within the Lab color space, representing the a and b coordinates of the color. Since the evaluation criteria for color error directly affect the visual effect of color correction, the global hue error and saturation constraint regularization term of the image are analyzed based on the chromaticity coordinates.

[0051] S230. Determine the global hue error and saturation constraint regularization terms based on the chromaticity coordinates of each color block.

[0052] Since the Lab color space is a color space where luminance and chrominance are relatively separated, the (a,b) components together form the chrominance plane, where a and b represent opposing dimensions of color. Therefore, the hue and saturation differences between the color output matrix and the color target matrix in the Lab color space can be characterized using chrominance coordinates.

[0053] In this embodiment of the application, the global hue error is determined based on the chromaticity coordinates corresponding to each color block, including: for color block pairs represented by elements in the same column of the color output matrix and the color target matrix, the global hue error of the color block pair is determined based on the angle between the chromaticity coordinate vectors of the color block pair; wherein, the chromaticity coordinate vector is a vector with the origin as the starting point and the chromaticity coordinate as the ending point; the global hue error is determined based on the weighted average of the global hue errors of each color block pair.

[0054] In this context, a color patch pair consists of two color patches represented by elements in the same column of the color output matrix and the color target matrix. For example, the color patch represented by the element in the 5th column of the color output matrix and the color patch represented by the element in the 5th column of the color target matrix form a color patch pair; similarly, the color patch represented by the element in the 24th column of the color output matrix and the color patch represented by the element in the 24th column of the color target matrix form a color patch pair. Exemplarily, in the chromaticity plane, the chromaticity coordinates of a color patch are represented by a point in the chromaticity plane. Taking the origin of the chromaticity plane as the starting point and the chromaticity coordinates as the ending point, the vector pointing from the starting point to the ending point is taken as the chromaticity coordinate vector of that color patch. Since the chromaticity coordinates of the same color patch pair are different, their chromaticity coordinate vectors are different, and the chromaticity coordinate vectors of the color patch pair have a certain angle. This angle represents the global hue error of the color patch pair. The global hue error of the color output matrix and the color target matrix is ​​the weighted average of the global hue errors of the 24 color patch pairs.

[0055] Specifically, assume that in a color patch pair, the patch belonging to the color output matrix is ​​patch 1, and the patch belonging to the color target matrix is ​​patch 2. The chromaticity coordinates of patch 1 are (a1, b1), and the chromaticity coordinates of patch 2 are (a2, b2). The global hue error of this color patch pair is... For color patch pair i, the chromaticity coordinates of the color patch belonging to the color output matrix are: The chromaticity coordinates of the color patches belonging to the color target matrix are The global hue error of the color block for i is The global hue error of the color output matrix and the color target matrix can be represented by θ1, θ2, ..., θ 24 To represent, for example:

[0056]

[0057] Where ΔHue is the global hue error between the color output matrix and the color target matrix, w iThis is the weight value, which can be determined based on the actual situation. If w i If all are equal, then ΔHue is θ1, θ2, ..., θ 24 The arithmetic mean. In this embodiment, the human eye perceives little difference between black, white, and gray. Therefore, the weight of the global hue error of the corresponding black, white, and gray color blocks can be set to 0 to avoid the influence of the corresponding black, white, and gray color blocks on the color evaluation of the colored color blocks, thereby improving the accuracy of the global hue error.

[0058] In this embodiment, determining a saturation constraint regularization term based on the chromaticity coordinates corresponding to each color block includes: determining the average value of a first distance between the chromaticity coordinates of each color block represented by each column element in the color output matrix and the origin, and determining the average value of a second distance between the chromaticity coordinates of each color block represented by each column element in the color target matrix and the origin; determining the ratio of the average value of the first distance to the average value of the second distance, and determining the saturation constraint regularization term based on the difference between the ratio and one. For example, in the chromaticity plane, the saturation of a color block can be characterized by the distance between its chromaticity coordinates and the origin. For each color block represented by each column element in the color output matrix, the first distance between the chromaticity coordinates of the color block and the origin is calculated, and the average value of the first distances corresponding to each color block is calculated to characterize the saturation of the color output matrix. For each color block in the color target matrix, the second distance between the chromaticity coordinates of the color block and the origin is calculated, and the average value of the second distances corresponding to each color block is calculated to characterize the saturation of the color target matrix. The saturation constraint regularization term is determined by calculating the ratio of the average of the first distance to the average of the second distance, and then calculating the difference between this ratio and the first distance. Here, the first distance refers to the ratio of two second distances. Specifically, for color patch i in the color output matrix, the first distance is used... To characterize its saturation, since the human eye perceives little difference between black, white, and gray, the saturation of the corresponding black, white, and gray color patches can be disregarded; only the saturation of the remaining 18 color patches needs to be considered, thus using the average value of the first distance. The saturation of the color output matrix is ​​represented by the second distance. For color patch i in the target color matrix, the saturation is represented by the second distance. To characterize its saturation, since the human eye perceives little difference between black, white, and gray, the saturation of the corresponding black, white, and gray color patches can be disregarded, and only the saturation of the remaining 18 color patches can be considered, thus using the average value of the second distance. Characterizes the saturation of the color target matrix. The ratio of the average value of the first distance to the average value of the second distance is... The difference between the ratio and one is As a saturation constraint regularization term, it is used to characterize the saturation difference between the color output matrix and the color target matrix. Since the saturation constraint regularization term is of interest in practical applications and is a positive value, its absolute value can be taken, i.e. As a saturation constraint regularization term, constraint parameters can also be introduced in this embodiment to adjust the weight of ΔChroma. These constraint parameters can be parameters determined according to actual conditions, such as values ​​like 0.1, which can be determined based on the tolerance for relative saturation differences.

[0059] S240. Construct a loss function based on the sum of the saturation constraint regularization term and the global hue error.

[0060] Specifically, the loss function can be expressed as E = ΔHue + λ·ΔChroma, where E is the loss function, ΔHue is the global hue error, ΔChroma is the saturation constraint regularization term, and λ is the constraint parameter. The loss function value is obtained by substituting the coordinates of the color output matrix and the color target matrix in the Lab color space.

[0061] In this embodiment, considering the different perceptions of global hue error and saturation constraint regularization terms by the human eye, with a much higher sensitivity to global hue error than to saturation constraint regularization terms, a loss function is introduced to calculate the loss function value based on global hue error, thus correcting the image from the perspective of human eye perception. Saturation constraint regularization restricts arbitrary scaling of saturation in the color space without affecting the optimization result of global hue error, thereby making the colors of the corrected image closer to human visual perception, and the color correction effect more closely approximating human perception.

[0062] S250. Update the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix, and perform color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix.

[0063] This application provides a color correction matrix optimization method, which optimizes the above embodiments. For each color patch represented by each column element in the color output matrix and the color target matrix, the chromaticity coordinates corresponding to each color patch are determined. The chromaticity coordinates are the coordinates of the color output matrix and the color target matrix in the chromaticity plane after conversion to the Lab color space. Based on the chromaticity coordinates corresponding to each color patch, the global hue error and saturation constraint regularization terms of the color output matrix and the color target matrix in the Lab color space are determined. A loss function is constructed based on the sum of the saturation constraint regularization term and the global hue error. Taking into full account the high sensitivity of the human eye to hue error, the color correction matrix is ​​optimized based on the global hue error. During the optimization process, the saturation constraint regularization term is used to constrain and prevent arbitrary scaling of saturation, thereby enabling the optimized image color to closely resemble the color perceived by the human eye, improving the user experience.

[0064] Figure 4 This is a flowchart illustrating a color correction matrix optimization method according to another embodiment of this application. This embodiment is an optimization based on the above embodiments. It should be noted that technical details not described in detail in this embodiment can be found in any of the above embodiments.

[0065] Specifically, such as Figure 4 As shown, the method specifically includes the following steps:

[0066] S310. For the color correction matrix composed of the hue rotation matrix and the saturation scaling matrix, multiply it on the left by the mapping matrix from the YUV color space to the RGB color space, and multiply it on the right by the mapping matrix from the RGB color space to the YUV color space to obtain the third-order color correction matrix.

[0067] YUV is a color encoding method; YIQ, Y'UV, YUV, YCbCr, and YPbPr can all be referred to as YUV. "Y" represents luminance, or grayscale value, while "U" and "V" represent chrominance, describing the color and saturation of an image and specifying the color of a pixel. We will explain this in detail using the YIQ color space as an example. The YIQ color space is a color space where luminance and chrominance are relatively independent. The YIQ color space and the RGB color space have a linear relationship; therefore, a linear transformation in the YIQ color space can be linearly mapped to a linear transformation in the RGB space. In the YIQ color space, we design a hue rotation parameter α and construct a hue rotation matrix. Design the saturation scaling parameter s, and construct the saturation scaling matrix. The linear transformation of the YIQ color gamut is ultimately expressed as T = M S ·M R The initial color correction matrix BaseCCM = M is determined based on T. YIQ2RGB ·T·MRGB2YIQ , of which M YIQ2RGB M is the mapping matrix from the YIQ color space to the RGB color space. RGB2YIQ The mapping matrix from the RGB color space to the YIQ color space yields the third-order color correction matrix.

[0068] S320. Based on white balance constraints, one element in each row of the third-order color correction matrix is ​​represented by the other two elements to obtain a color correction matrix containing six variables.

[0069] For example, a third-order matrix consisting of nine parameters can be Parameter set S1 = {c 11 ,c 12 ,c 13 ,c 21 ,c 22 ,c 23 ,c 31 ,c 32 ,c 33 The matrix contains nine variables. If this matrix is ​​used as a color correction matrix, the parameters must adhere to white balance constraints, meaning the sum of the parameters in each row of the matrix must equal 1. r1 +c r2 +c r3 =1, therefore, based on the white balance constraint, one element in each row of the third-order color correction matrix can be represented by the other two elements, resulting in a variable set such as S2 = {c 11 ,c 13 ,c 22 ,c 23 ,c 31 ,c 32}, c 12 =1-c 11 -c 13 c 21 =1-c 22 -c 22 c 33 =1-c 31 -c 32 Alternatively, you can select off-diagonal elements to design the variable set S2 = {c 12 ,c 13 ,c 21 ,c 23 ,c 31 ,c 32 The 3D matrix with nine variables can be expressed as a 3D matrix with six variables. If we represent the diagonal elements using off-diagonal elements, we obtain a 3D matrix with six variables as follows:

[0070] S330. Based on the color correction matrix, the color source matrix of the acquired color card image is corrected to obtain the color output matrix.

[0071] S340. Based on the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, determine the global hue error and saturation constraint regularization terms, and construct a loss function based on the global hue error and the saturation constraint regularization terms; wherein, the Lab coordinates are the coordinates in the Lab color space.

[0072] S350. The six variables are updated according to the loss function value to update the color correction matrix, thereby obtaining the optimal color correction matrix, and the image to be processed acquired by the image acquisition device is color corrected based on the optimal color correction matrix.

[0073] For example, the specific implementation process of the embodiments of this application is as follows: Figure 5 As shown, the color source matrix is ​​color-corrected based on the color correction matrix (CCM) to obtain the color output matrix. A loss function value is calculated based on the color output matrix and the color target matrix. The color correction matrix is ​​then updated iteratively based on the loss function value until it converges, yielding the minimum value of the loss function and the corresponding optimal color correction matrix.

[0074] In this embodiment, the update process of the color correction matrix is ​​actually an optimization of six variables. This optimization process is an unconstrained optimization process, and the optimization algorithm is not limited; for example, it can be gradient descent, Newton's method, quasi-Newton's method, conjugate gradient method, etc. The process of optimizing the color correction matrix containing nine variables based on white balance constraints is a constrained optimization process. However, the scheme in this embodiment simplifies this to an unconstrained optimization process, thus accelerating the convergence speed.

[0075] In this embodiment, the initialization process of the color correction matrix includes: determining the hue rotation factor and the saturation scaling factor based on the value range of the hue rotation factor in the hue rotation matrix and the value range of the saturation scaling factor in the saturation scaling matrix; determining at least two color correction matrices based on the hue rotation factor and the saturation scaling factor; or, randomly generating at least two sets of hue rotation factors and saturation scaling factors based on the probability distribution model of the hue rotation factor and the saturation scaling factor; and determining at least two color correction matrices based on the at least two sets of hue rotation factors and saturation scaling factors. This embodiment provides two exemplary schemes for batch determination of color correction matrices. In practical applications, the scheme for batch generation of color correction matrices can be selected according to the actual situation and is not limited; any scheme involving batch generation is within the scope of protection of this application. Specifically, the hue rotation factor is the hue rotation matrix M.R In this context, α represents the saturation scaling factor, which is the saturation scaling matrix M. S In the s, the range of values ​​for the hue rotation factor and saturation scaling factor can be determined in advance based on the actual situation, α∈[α... min ,α max ], s∈[s min ,s max The process involves pre-setting a certain step size and selecting hue rotation factors and saturation scaling factors from a range of values. For example, hue rotation factors are selected as α1 and α2, and saturation scaling factors as s1 and s2. These hue rotation factors and saturation scaling factors are combined to obtain four combinations: (α1, s1), (α2, s1), (α1, s2), and (α1, s2). These four combinations are then substituted into T, and the color space is converted from YUV to RGB to obtain four color correction matrices. The specific number of hue rotation factors and saturation scaling factors selected is not limited and can be determined based on the number of initial color matrices required.

[0076] The solution in this application embodiment can also be to establish a probability distribution model for α and s, such as a uniform distribution model, randomly select at least two sets (α, s) from the probability distribution model, substitute them into T, and convert them from the YUV color space to the RGB color space to obtain at least two initial color matrices.

[0077] In this embodiment, at least two (α, s) can also be determined through other means, such as random generation, to obtain at least two color correction matrices. This enables batch generation of initial color correction matrices, thus solving the problem of easily getting trapped in local minima when optimizing a single initial color correction matrix. This allows for optimization of at least two initial color correction matrices to find the global minimum, improving optimization accuracy. In this embodiment, the number of hue rotation factors and saturation scaling factors can be at least on the order of hundreds to batch generate a sufficient number of initial color correction matrices, more accurately finding the optimal solution and resolving the problem of getting trapped in local minima.

[0078] After determining at least two color correction matrices, extract the values ​​of the corresponding elements in the color correction matrices, initialize the variable set S2, and iteratively solve for the optimal solution of S2.

[0079] In this embodiment, correcting a color source matrix based on a color correction matrix to obtain a color output matrix includes: correcting the color source matrix based on at least two color correction matrices respectively to obtain at least two color output matrices; updating the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix, including: optimizing the at least two color correction matrices according to the loss function value until the loss function value converges to a minimum value to obtain at least two minimum values; determining the minimum value among the at least two minimum values, and using the updated color correction matrix corresponding to the minimum value as the optimal color correction matrix. For example, correcting the color source matrix based on at least two color correction matrices respectively can obtain at least two color output matrices. The color correction processes can be executed in parallel without affecting each other. Assume at least two color correction matrices CCM. j The number of elements is N. Traverse the color correction matrix, correct the color source matrix to obtain N color output matrices. Based on the global hue error and saturation constraint regularization terms of the color output matrix and the color target matrix in the Lab color space, determine the loss function value, and update and optimize the N color correction matrices until N minimum values ​​(MinE) are obtained. j With the corresponding optimization matrix MinCCM j By selecting the minimum value among the minimum values, opt = argmin j＝1,2,…,N (MinE j ), to obtain the global minimum value OptE = MinE opt And the corresponding global optimal solution OptCCM = MinCCM opt As the optimal color correction matrix. In this embodiment, by determining at least two color correction matrices, color correction matrices are generated in batches, which maximizes the dispersion of the optimization starting point, thereby increasing the probability of finding the global minimum and avoiding the problem of getting trapped in local minima when optimizing a single color correction matrix, thus failing to converge to the global minimum.

[0080] Figure 6 This is a schematic diagram of a color correction matrix optimization device provided in one embodiment of this application. The color correction matrix optimization device provided in this embodiment includes:

[0081] The correction processing module 410 is used to perform correction processing on the color source matrix of the acquired color card image based on the color correction matrix to obtain the color output matrix;

[0082] The loss function construction module 420 is used to determine the global hue error and saturation constraint regularization term based on the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, and to construct a loss function based on the global hue error and the saturation constraint regularization term; wherein, the Lab coordinates are coordinates in the Lab color space;

[0083] The optimization module 430 is used to update the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix, so as to perform color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix.

[0084] In this embodiment of the application, the loss function construction module 420 includes:

[0085] The coordinate determination unit is used to determine the chromaticity coordinates corresponding to each color patch represented by each column element in the color output matrix and the color target matrix; wherein, the chromaticity coordinates are the coordinates of the color output matrix and the color target matrix in the chromaticity plane after being converted to the Lab color space;

[0086] The difference data determination unit is used to determine the global hue error and saturation constraint regularization terms based on the chromaticity coordinates of each color block.

[0087] The numerical determination unit is used to construct a loss function based on the sum of the saturation constraint regularization term and the global hue error.

[0088] In this embodiment of the application, the difference data determination unit includes:

[0089] The global hue error determination subunit is used to determine the global hue error of a pair of color blocks represented by elements in the same column of the color output matrix and the color target matrix, based on the angle between the chromaticity coordinate vectors of the color block pairs; wherein, the chromaticity coordinate vector is a vector that starts from the origin and ends at the chromaticity coordinates.

[0090] The weighted average calculation unit is used to determine the global hue error based on the weighted average of the global hue error of each color block pair.

[0091] In this embodiment of the application, the difference data determination unit includes:

[0092] The distance data determination subunit is used to determine the average value of the first distance between the chromaticity coordinates of each color patch represented by each column element in the color output matrix and the origin, and to determine the average value of the second distance between the chromaticity coordinates of each color patch represented by each column element in the color target matrix and the origin.

[0093] The relative saturation difference determination subunit is used to determine the ratio of the average value of the first distance to the average value of the second distance, and to determine the saturation constraint regularization term based on the difference between the ratio and a.

[0094] In this embodiment of the application, the device further includes:

[0095] The matrix construction module is used to multiply the color correction matrix, which is composed of the hue rotation matrix and the saturation scaling matrix, by the left multiplication of the mapping matrix from the YUV color space to the RGB color space, and by the right multiplication of the mapping matrix from the RGB color space to the YUV color space, to obtain the third-order color correction matrix.

[0096] The transformation module is used to represent one element in each row of the third-order color correction matrix with the other two elements based on the white balance constraint, so as to obtain a color correction matrix containing six variables.

[0097] Optimization module 430 is specifically used for:

[0098] The six variables are updated based on the loss function value in order to update the color correction matrix.

[0099] In this embodiment of the application, the method further includes:

[0100] The first factor selection unit is used to determine the hue rotation factor and the saturation scaling factor based on the value range of the hue rotation factor in the hue rotation matrix and the value range of the saturation scaling factor in the saturation scaling matrix.

[0101] The first matrix construction unit is used to determine at least two color correction matrices based on the hue rotation factor and the saturation scaling factor. Alternatively,

[0102] The second factor unit is used to randomly generate at least two sets of hue rotation factors and saturation scaling factors according to the probability distribution model of the hue rotation factor and the saturation scaling factor.

[0103] The second matrix construction unit is used to determine at least two color correction matrices based on at least two sets of hue rotation factors and saturation scaling factors.

[0104] In this embodiment of the application, the correction processing module 410 is specifically used for:

[0105] The color source matrix is ​​corrected based on at least two color correction matrices to obtain at least two color output matrices.

[0106] Optimization module 430 includes:

[0107] The minimum value determination unit is used to optimize at least two color correction matrices based on the loss function value until the loss function value converges to a minimum value, thereby obtaining at least two minimum values.

[0108] The optimal color correction matrix determination unit is used to determine the minimum value among at least two minimum values, and the updated color correction matrix corresponding to the minimum value is used as the optimal color correction matrix.

[0109] The color correction matrix optimization device provided in this application embodiment can be used to execute the color correction matrix optimization method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0110] Figure 7 This is a schematic diagram of an electronic device provided according to one embodiment of this application. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, user equipment, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0111] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0112] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks and wireless networks.

[0113] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the color correction matrix optimization method.

[0114] In some embodiments, the color correction matrix optimization method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the color correction matrix optimization method by any other suitable means (e.g., by means of firmware).

[0115] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 10, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 10. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0119] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0120] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0121] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0122] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for optimizing a color correction matrix, characterized in that, The method includes: Based on the color correction matrix, the color source matrix of the acquired color card image is corrected to obtain the color output matrix; For each color patch represented by each column element in the color output matrix and the color target matrix of the target color chart image, the chromaticity coordinates corresponding to each color patch are determined. Based on the chromaticity coordinates corresponding to each color patch, the global hue error and saturation constraint regularization term are determined. Based on the sum of the saturation constraint regularization term and the global hue error, a loss function is constructed. Wherein, the chromaticity coordinates are the coordinates in the chromaticity plane of the color output matrix and the color target matrix after being transformed into the Lab color space. The color correction matrix is ​​updated based on the loss function value until the loss function value converges to obtain the optimal color correction matrix, and the color correction of the image to be processed acquired by the image acquisition device is performed based on the optimal color correction matrix. The step of determining the global hue error based on the chromaticity coordinates of each color block includes: for color block pairs represented by elements in the same column of the color output matrix and the color target matrix, determining the global hue error of the color block pair based on the angle between the chromaticity coordinate vectors of the color block pair; wherein the chromaticity coordinate vector is a vector with the origin as the starting point and the chromaticity coordinate as the ending point; and determining the global hue error based on the weighted average of the global hue errors of each color block pair. The step of determining the saturation constraint regularization term based on the chromaticity coordinates corresponding to each color block includes: determining the average value of the first distance between the chromaticity coordinates of each color block represented by each column element in the color output matrix and the origin, and determining the average value of the second distance between the chromaticity coordinates of each color block represented by each column element in the color target matrix and the origin; determining the ratio of the average value of the first distance to the average value of the second distance, and determining the saturation constraint regularization term based on the difference between the ratio and one.

2. The method according to claim 1, characterized in that, Before updating the color correction matrix based on the loss function value, the method further includes: For the color correction matrix composed of the hue rotation matrix and the saturation scaling matrix, multiply it on the left by the mapping matrix from the YUV color space to the RGB color space, and multiply it on the right by the mapping matrix from the RGB color space to the YUV color space to obtain the third-order color correction matrix. Based on white balance constraints, one element in each row of the third-order color correction matrix is ​​represented by the other two elements, resulting in a color correction matrix containing six variables. The color correction matrix is ​​updated based on the loss function value, including: The six variables are updated based on the loss function value in order to update the color correction matrix.

3. The method according to claim 1 or 2, characterized in that, The initialization process of the color correction matrix includes: The hue rotation factor and the saturation scaling factor are determined based on the range of values ​​for the hue rotation factor in the hue rotation matrix and the range of values ​​for the saturation scaling factor in the saturation scaling matrix. Determine at least two color correction matrices based on the hue rotation factor and the saturation scaling factor; or, Based on the probability distribution model of the hue rotation factor and the saturation scaling factor, at least two sets of hue rotation factors and saturation scaling factors are randomly generated; Determine at least two color correction matrices based on at least two sets of hue rotation factors and saturation scaling factors.

4. The method according to claim 3, characterized in that, The color source matrix is ​​corrected based on the color correction matrix to obtain the color output matrix, including: The color source matrix is ​​corrected based on at least two color correction matrices to obtain at least two color output matrices. The color correction matrix is ​​updated based on the loss function value until the loss function value converges, resulting in the optimal color correction matrix, including: Based on the loss function value, at least two color correction matrices are optimized until the loss function value converges to a minimum value, resulting in at least two minimum values. Determine the minimum value among at least two local minima, and use the updated color correction matrix corresponding to the minimum value as the optimal color correction matrix.

5. A color correction matrix optimization device, characterized in that, The device includes: The correction processing module is used to correct the color source matrix of the acquired color card image based on the color correction matrix to obtain the color output matrix; The loss function construction module is used to determine the chromaticity coordinates of each color patch represented by each column element in the color target matrix of the color output matrix and the target color chart image, determine the global hue error and saturation constraint regularization term based on the chromaticity coordinates of each color patch, and construct the loss function based on the sum of the saturation constraint regularization term and the global hue error; wherein, the chromaticity coordinates are the coordinates of the color output matrix and the color target matrix in the chromaticity plane after being transformed into the Lab color space; An optimization module is used to update the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix, so as to perform color correction on the image to be processed acquired by the image acquisition device based on the optimal color correction matrix. The loss function construction module includes: The global hue error determination subunit is used to determine the global hue error of a pair of color blocks represented by elements in the same column of the color output matrix and the color target matrix, based on the angle between the chromaticity coordinate vectors of the color block pairs; wherein, the chromaticity coordinate vector is a vector that starts from the origin and ends at the chromaticity coordinates. The weighted average calculation unit is used to determine the global hue error based on the weighted average of the global hue error of each color block pair; The distance data determination subunit is used to determine the average value of the first distance between the chromaticity coordinates of each color patch represented by each column element in the color output matrix and the origin, and to determine the average value of the second distance between the chromaticity coordinates of each color patch represented by each column element in the color target matrix and the origin. The relative saturation difference determination subunit is used to determine the ratio of the average value of the first distance to the average value of the second distance, and to determine the saturation constraint regularization term based on the difference between the ratio and a.

6. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the color correction matrix optimization method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the color correction matrix optimization method as described in any one of claims 1-4.

Images