Image color difference correction method and device, electronic equipment and storage medium

Abstract

The application discloses an image color difference correction method and device, electronic equipment and a storage medium. The image color difference correction method comprises the following steps: acquiring first color difference gray values and second color difference gray values corresponding to each pixel point in an input image, and first color difference index values and second color difference index values of the pixel points; for any pixel point in the input image, based on the first color difference index values and the second color difference index values of the pixel point, and first mapping relationship and second mapping relationship, first color difference correction weight values and second color difference correction weight values of the pixel point are obtained; based on the color difference correction weight values, the input image is subjected to color difference correction to generate a corrected output image; wherein the first mapping relationship and the second mapping relationship are obtained by training a neural network model based on a plurality of sample images and a color difference perception result data set of the plurality of sample images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image color difference correction method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, the importance of image quality in the fields of monitoring equipment, machine recognition, and automatic driving is self-evident, and in particular, whether the restoration and display of colors meet the subjective perception of the human eye is particularly important. Under the existing color space representation, there will be certain color differences that do not meet the human visual sense. Under the color space representation based on Riemann geometry, the color difference on the three-dimensional Riemann space is greater than the color difference perceived by the human eye, so it is an important requirement to optimize the color space to meet the subjective perception of the human eye.

[0003] The human eye has three types of cone cells that can perceive the color of visible light, which have different responses to light of different wavelengths, which leads to the human perception of various colors. This response can be linked to color by establishing a model, but there is often an error in color. SUMMARY

[0004] The present application provides an image color difference correction method, device, electronic equipment and storage medium to solve the problem that the current image color difference correction does not meet the human visual sense.

[0005] In a first aspect, the present application provides an image color difference correction method, comprising:

[0006] Obtaining a first color difference gray value and a second color difference gray value corresponding to each pixel point in an input image; each first color difference gray value is the difference between a first primary color gray value and a second primary color gray value of the corresponding pixel point, and each second color difference gray value is the difference between the second primary color gray value and a third primary color gray value of the corresponding pixel point;

[0007] Obtaining a first color difference index value corresponding to the first color difference gray value of the pixel point, and a second color difference index value corresponding to the second color difference gray value of the pixel point;

[0008] For any pixel point in the input image, based on the first color difference index value of the pixel point and the first mapping relationship between the first color difference index value and the first color difference correction weight value, obtaining the first color difference correction weight value of the pixel point; based on the second color difference index value of the pixel point and the second mapping relationship between the second color difference index value and the second color difference correction weight value, obtaining the second color difference correction weight value of the pixel point;

[0009] perform color difference correction on the input image based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, and generate a corrected output image;

[0010] The first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result dataset of the plurality of sample images by a neural network model.

[0011] In some embodiments, the first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result dataset of the plurality of sample images by a neural network model.

[0012] The neural network model is trained based on the plurality of sample images and the color difference perception result dataset of the plurality of sample images to obtain a first effective weight distribution of a first color difference gray value and a second effective weight distribution of a second color difference gray value.

[0013] The first mapping relationship between a plurality of first color difference index value threshold intervals and the first effective weight distribution, and the second mapping relationship between a plurality of second color difference index value threshold intervals and the second effective weight distribution are established.

[0014] In some embodiments:

[0015]

[0016]

[0017] The first primary color is red, the second primary color is green, the third primary color is blue, and the output image is a YUV image.

[0018] In some embodiments:

[0019] The plurality of first color difference index value threshold intervals include a first color difference threshold interval in which a first color difference index value is less than a first color difference threshold, and a plurality of first color difference threshold intervals in which a first color difference index value is greater than or equal to a first color difference threshold. The first color difference threshold interval in which the first color difference index value is less than the first color difference threshold corresponds to a first effective weight of 1. In the plurality of first color difference threshold intervals in which the first color difference index value is greater than or equal to the first color difference threshold, each first color difference threshold interval corresponds to a first effective weight obtained by training the neural network model.

[0020] The plurality of second color difference index value threshold intervals include a second color difference threshold interval in which a second color difference index value is less than a second color difference threshold value, and a plurality of second color difference threshold intervals in which a second color difference index value is greater than or equal to a second color difference threshold value, the second effective weight corresponding to the second color difference threshold interval in which the second color difference index value is less than the second color difference threshold value is 1, and each of the plurality of second color difference threshold intervals in which the second color difference index value is greater than or equal to the second color difference threshold value corresponds to a second effective weight obtained by training the neural network model.

[0021] In some embodiments, the neural network model is a U-Net convolutional neural network model.

[0022] In some embodiments, the training of the neural network model based on the plurality of sample images and the color difference perception result dataset of the plurality of sample images by the human eye obtains a first effective weight distribution of the first color difference grayscale value and a second effective weight distribution of the second color difference grayscale value, including:

[0023] Based on the relative response values of the three types of cone cells of the human eye to light of different wavelengths, modeling is performed according to the plurality of sample images to obtain the color difference perception result dataset of the plurality of sample images;

[0024] The root mean square test error is used as the loss function of the U-Net convolutional neural network model, and the first effective weight distribution and the second effective weight distribution are obtained by the U-Net convolutional neural network model based on the color difference perception result dataset of the plurality of sample images.

[0025] In a second aspect, the present application provides an image color difference correction device, comprising:

[0026] A first acquisition module is configured to acquire first color difference grayscale values and second color difference grayscale values corresponding to each pixel point in an input image, each first color difference grayscale value is the difference between a first primary color grayscale value and a second primary color grayscale value of the corresponding pixel point, and each second color difference grayscale value is the difference between the second primary color grayscale value and a third primary color grayscale value of the corresponding pixel point.

[0027] A second acquisition module is configured to acquire first color difference index values corresponding to the first color difference grayscale values of the pixel points, and second color difference index values corresponding to the second color difference grayscale values of the pixel points.

[0028] determining, for any pixel point in the input image, a first color difference correction weight value of the pixel point based on a first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value, and a second color difference correction weight value of the pixel point based on a second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value;

[0029] correcting, based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, the input image to generate a corrected output image;

[0030] The first mapping relationship and the second mapping relationship are obtained based on training a neural network model based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye.

[0031] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to implement the method of any of the above.

[0032] In a fourth aspect, the present application provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method of any of the above.

[0033] In a fifth aspect, the present application provides a computer program product comprising a computer program, and the computer program is executed by a processor to implement the method of any of the above.

[0034] The image color difference correction method, device, electronic device and storage medium provided by the present application train a neural network model based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye to obtain a first mapping relationship and a second mapping relationship; correct the first color difference gray value and the second color difference gray value corresponding to each pixel point according to the color difference correction weight values corresponding to the pixel points in the first mapping relationship and the second mapping relationship, implement the color difference correction of the image, improve the processing precision of the image color difference correction under different color gamuts, and make the image color difference more consistent with the human sense. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0036] Figure 1 FIG. 1 is a flowchart of an image color difference correction method according to an embodiment of the present application;

[0037] Figure 2 FIG. 2 is another flowchart of an image color difference correction method according to an embodiment of the present application;

[0038] Figure 3 FIG. 3 is a flowchart of establishing a mapping matrix between a color difference index value and a color difference correction weight value according to an embodiment of the present application;

[0039] Figure 4 FIG. 4 is another flowchart of establishing a mapping matrix between a color difference index value and a color difference correction weight value according to an embodiment of the present application;

[0040] Figure 5 FIG. 5 is a structural diagram of an image color difference correction device according to an embodiment of the present application;

[0041] Figure 6 FIG. 6 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0043] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the front and rear associated objects are in an "or" relationship.

[0044] The color space used by cameras to take pictures is currently represented based on Riemann geometry, which often amplifies the difference between colors and often does not conform to the color difference perceived by the human eye. Although many companies have developed standards such as standard red green blue (sRGB), wide color gamut (Adobe red green blue, Adobe RGB), and professional photo color gamut (prophoto red green blue, prophoto RGB), there is still a large difference between image display and human perception. The distribution of red green blue (RGB) components in the spatial position and the difference between colors will affect the perception of the human eye from an objective and subjective point of view. Therefore, a more accurate way of representing the color space is needed to compensate for the difference in color perception by the human eye.

[0045] The difference between colors in the Riemann geometric space can be represented by the shortest path length between two points. The path length in this three-dimensional geometric space is a curve, which can be regarded as the sum of an infinite number of microelement distances. However, the difference in color perception by the human eye is often less than the sum of the microelements in the three-dimensional geometric space. Therefore, only the representation of the color space based on Riemann geometry may not conform to the human eye perception in the case of large color difference, so some correction is needed to make the displayed color conform to the human sense of sight.

[0046] To this end, the present application provides an image color difference correction method, device, electronic equipment and storage medium. The color difference channel information R-G and B-G is calculated through the information of the RGB three channels of the multi-sample image, the information is trained with the end-to-end convolutional neural network of the human eye color difference perception result of the multi-sample image, the weight matrix relationship of the two is finally obtained, and then the weight value and the color difference channel information are mapped to establish a mapping matrix. After the mapping matrix is established, the color difference of the color space can be processed by the mapping matrix, thereby realizing the correction of the Riemann color space and compensating for the difference in human eye perception. The present application can improve the processing accuracy under different color gamuts and make the color difference more consistent with the human sense of sight.

[0047] Figure 1 One of the flowcharts of the image color difference correction method provided by the embodiments of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, an image color difference correction method is provided, which includes the following steps: step 110, step 120, step 130, and step 140. The method flow steps are only used as one possible implementation of the present application.

[0048] Step 110, obtaining a first color difference gray value and a second color difference gray value corresponding to each pixel point in an input image; each first color difference gray value is the difference between the first primary color gray value and the second primary color gray value of the corresponding pixel point, and the second color difference gray value is the difference between the second primary color gray value and the third primary color gray value of the corresponding pixel point.

[0049] Step 120, obtaining a first color difference index value of each pixel point corresponding to the first color difference gray value, and a second color difference index value of each pixel point corresponding to the second color difference gray value.

[0050] Step 130, for any pixel point in the input image, obtaining a first color difference correction weight value of the pixel point based on the first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value; obtaining a second color difference correction weight value of the pixel point based on the second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value.

[0051] Step 140, performing color difference correction on the input image based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image to generate a corrected output image; wherein the first mapping relationship and the second mapping relationship are obtained by training a neural network model based on a multi-sample image and a color difference perception result data set of the multi-sample image by a human eye.

[0052] Specifically, when the device (hereinafter referred to as the device) for implementing the image color difference correction method receives an input image, the first primary color gray value, the second primary color gray value and the third primary color gray value corresponding to each pixel point in the input image can be obtained.

[0053] In some embodiments, the first primary color can be red (R), the second primary color can be green (G), and the third primary color can be blue (B).

[0054] The first color difference gray value of each pixel can be the difference between the first primary color gray value and the second primary color gray value of the corresponding pixel, which can be obtained by subtracting the first primary color gray value from the second primary color gray value. The second color difference gray value can be the difference between the second primary color gray value and the third primary color gray value of the corresponding pixel, which can be obtained by subtracting the second primary color gray value from the third primary color gray value.

[0055] For example, for any pixel, a method for obtaining the first color difference gray value and the second color difference gray value from the first primary color gray value, the second primary color gray value and the third primary color gray value can be:

[0056]

[0057] Wherein, R is the first primary color gray value, G is the second primary color gray value, B is the third primary color gray value, R-G is the first color difference gray value, and B-G is the second color difference gray value.

[0058] The first color difference index value of each pixel point corresponding to the first color difference gray value can be the first color difference gray value corresponding to the pixel point, or can be other index values that can be used to represent the first color difference of the pixel point. The second color difference index value of each pixel point corresponding to the second color difference gray value can be the second color difference gray value corresponding to the pixel point, or can be other index values that can be used to represent the second color difference of the pixel point. The specific calculation method is not limited here.

[0059] In the first mapping relationship, each first color difference index value can correspond to a first color difference correction weight value. In different color gamut spaces, the correspondence between the first color difference index value and the first color difference correction weight value in the first mapping relationship can be different.

[0060] In the second mapping relationship, each second color difference index value can correspond to a second color difference correction weight value. In different color gamut spaces, the correspondence between the second color difference index value and the second color difference correction weight value in the second mapping relationship can be different.

[0061] After obtaining the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, the first color difference gray value corresponding to the pixel point can be corrected according to the first color difference correction weight value of the pixel point, and the second color difference gray value corresponding to the pixel point can be corrected according to the second color difference correction weight value of the pixel point.

[0062] After completing the correction of the first color difference gray value and the second color difference gray value corresponding to each pixel point in the input image, respectively, a corrected output image can be generated.

[0063] In some embodiments, the output image can be a luminance (Y) chrominance (U) chroma (V) image.

[0064] For example, a method for correcting the first color difference gray value and the second color difference gray value can be:

[0065]

[0066] R is the first primary color gray value, G is the second primary color gray value, B is the third primary color gray value, R-G is the first color difference gray value, B-G is the second color difference gray value, α1 is the first color difference correction weight value, and α2 is the second color difference correction weight value.

[0067] The first mapping relationship and the second mapping relationship can be obtained by training a neural network model based on a multi-sample image and a result dataset of color difference perception of the multi-sample image by a human eye.

[0068] Based on the multi-sample image, the first primary color gray value, the second primary color gray value and the third primary color gray value corresponding to each pixel point in the multi-sample image can be obtained. According to the first primary color gray value, the second primary color gray value and the third primary color gray value corresponding to each pixel point in the multi-sample image, the first color difference gray value and the second color difference gray value corresponding to each pixel point in the multi-sample image can be obtained.

[0069] The first color difference correction weight value and the second color difference correction weight value of each pixel point can be obtained by training an end-to-end convolutional neural network with the first color difference gray value and the second color difference gray value corresponding to each pixel point in the multi-sample image and the result of color difference perception of the multi-sample image by a human eye as input.

[0070] The result of color difference perception of the multi-sample image by a human eye can be obtained by modeling the relative response values of three types of cone cells of the human eye to light of different wavelengths and obtaining a subjective perception result dataset of different color differences based on the response.

[0071] After obtaining the color difference correction weight matrix corresponding to the first color difference gray value and the second color difference gray value, the root mean square test error can be used as a loss function, and the effective value is below the test error. After completing the error test, the color difference correction weight matrix obtained finally is output by the convolutional neural network.

[0072] After obtaining the final color difference correction weight matrix, the color difference correction weight value corresponding to each color difference index value can be determined. Each color difference index value can correspond to one color difference correction weight value, or the threshold space of each color difference index value can correspond to one color difference correction weight value, which is not limited here.

[0073] According to the corresponding relationship between the first color difference index value and the first color difference correction weight value, the first mapping relationship can be obtained. According to the corresponding relationship between the second color difference index value and the second color difference correction weight value, the second mapping relationship can be obtained.

[0074] In the embodiments of the present application, the first mapping relationship and the second mapping relationship are obtained by training a neural network model based on a multi-sample image and a result dataset of color difference perception of the multi-sample image by a human eye. The first color difference gray value and the second color difference gray value corresponding to each pixel point are corrected according to the color difference correction weight value corresponding to each pixel point in the first mapping relationship and the second mapping relationship, the color difference correction of the image is realized, the processing precision of the color difference correction of the image under different color gamuts is improved, and the color difference of the image is more consistent with the human sense.

[0075] It should be noted that each embodiment of the present application can be freely combined, exchanged order or executed alone, and does not need to rely on or depend on a fixed execution order.

[0076] In some embodiments, the first mapping relationship and the second mapping relationship are obtained after the neural network model is trained based on the multi-sample images and the color difference perception result data set of the multi-sample images by the human eyes.

[0077] The neural network model is trained based on the multi-sample images and the color difference perception result data set of the multi-sample images by the human eyes, to obtain a first effective weight distribution of the first color difference gray value and a second effective weight distribution of the second color difference gray value.

[0078] A first mapping relationship between a plurality of first color difference index value threshold intervals and the first effective weight distribution is established, and a second mapping relationship between a plurality of second color difference index value threshold intervals and the second effective weight distribution is established.

[0079] Specifically, after the multi-sample images and the color difference perception result data set of the multi-sample images by the human eyes are used to train the neural network model, a first effective weight corresponding to each first color difference gray value and a second effective weight corresponding to each second color difference gray value can be obtained, and then a first effective weight distribution of the first color difference gray value and a second effective weight distribution of the second color difference gray value can be obtained.

[0080] The first color difference gray values corresponding to the same first effective weight can be selected, and the first color difference index values corresponding to these first color difference gray values can be combined to obtain a first color difference index value threshold interval, and the first color difference index value threshold interval corresponds to a first effective weight. According to this step, a plurality of first color difference index value threshold intervals and a first effective weight corresponding to each first color difference index value threshold interval can be obtained. The first mapping relationship can be used to indicate the corresponding relationship between the plurality of first color difference index value threshold intervals and the first effective weight distribution.

[0081] Similarly, the second mapping relationship can be used to indicate the corresponding relationship between the plurality of second color difference index value threshold intervals and the second effective weight distribution.

[0082] In some embodiments, the neural network model can be a U-Net convolutional neural network model.

[0083] The U-Net convolutional neural network model is a convolutional neural network model that can be applied to semantic segmentation, and is named as such because the visualized result of its network structure looks like a letter U. The entire U-Net convolutional neural network mainly consists of two parts: a contraction path and an expansion path. The contraction path is mainly used to capture the context information in the picture, and the expansion path, which is symmetrical to the contraction path, is used to accurately locate the part that needs to be segmented in the picture. In order to accurately locate, the local pixel features extracted in the contraction path are combined with new feature maps in the upsampling process, that is, the local information and global information are combined, the accuracy of the pixel point category prediction is improved through information integration, and the important feature information in the previous downsampling process is retained to the greatest extent. In order to make the network structure run more efficiently, there is no fully connected layer in the structure, so the number of parameters that need to be trained can be greatly reduced, and all the information in the picture can be well preserved due to the special U-shaped structure.

[0084] In some embodiments, the plurality of first color difference index value threshold intervals includes a first color difference threshold interval in which a first color difference index value is less than a first color difference threshold, and a plurality of first color difference threshold intervals in which a first color difference index value is greater than or equal to the first color difference threshold, the first color difference threshold interval in which the first color difference index value is less than the first color difference threshold corresponds to a first effective weight of 1, and each of the plurality of first color difference threshold intervals in which the first color difference index value is greater than or equal to the first color difference threshold corresponds to a first effective weight obtained by training a neural network model;

[0085] The plurality of second color difference index value threshold intervals includes a second color difference threshold interval in which a second color difference index value is less than a second color difference threshold, and a plurality of second color difference threshold intervals in which a second color difference index value is greater than or equal to the second color difference threshold, the second color difference threshold interval in which the second color difference index value is less than the second color difference threshold corresponds to a second effective weight of 1, and each of the plurality of second color difference threshold intervals in which the second color difference index value is greater than or equal to the second color difference threshold corresponds to a second effective weight obtained by training a neural network model.

[0086] For example, in a color gamut space, the first color difference threshold and the second color difference threshold are both

[0087] The first color difference index value threshold interval corresponds to a first effective weight of 1; the first color difference index value threshold interval corresponds to a first effective weight weight1; the first color difference index value threshold interval corresponds to a first effective weight weight2; … the first color difference index value threshold interval corresponds to a first effective weight weightN.

[0088] Second color difference index value threshold interval The corresponding second effective weight is 1, and the second color difference index value threshold interval The corresponding second effective weight is weight1, and the second color difference index value threshold interval The corresponding second effective weight is weight2, and the second color difference index value threshold interval The corresponding second effective weight is weightN.

[0089] In some embodiments:

[0090]

[0091]

[0092] For example, the first color difference index value can be calculated in the following manner:

[0093]

[0094] where Δ1 is the first color difference index value, R is the first primary color gray value, and G is the second primary color gray value.

[0095] The second color difference index value can be calculated in the following manner:

[0096]

[0097] where Δ2 is the second color difference index value, G is the second primary color gray value, and B is the third primary color gray value.

[0098] In some embodiments, the neural network model is trained based on a multi-sample image and a color difference perception result dataset of the multi-sample image by a human eye, to obtain a first effective weight distribution of a first color difference gray value and a second effective weight distribution of a second color difference gray value, including:

[0099] Based on the relative response values of the three types of cone cells of the human eye to light of different wavelengths, a multi-sample image is modeled to obtain a color difference perception result dataset of the multi-sample image;

[0100] The root mean square test error is used as the loss function of the U-Net convolutional neural network model, and the first effective weight distribution and the second effective weight distribution are obtained by the U-Net convolutional neural network model based on the color difference perception result dataset of the multi-sample image.

[0101] Specifically, based on the multi-sample image, first primary color gray values, second primary color gray values and third primary color gray values corresponding to each pixel point in the multi-sample image can be obtained. According to the first primary color gray values, the second primary color gray values and the third primary color gray values corresponding to each pixel point in the multi-sample image, first color difference gray values and second color difference gray values corresponding to each pixel point in the multi-sample image can be obtained.

[0102] The first color difference gray values and the second color difference gray values corresponding to each pixel point in the multi-sample image and the results of the color difference perception of the multi-sample image by the human eye are input into an end-to-end convolutional neural network for training, and the first effective weight distribution of the first color difference gray values and the second effective weight distribution of the second color difference gray values can be obtained by taking the root mean square test error as a loss function.

[0103] The results of the color difference perception of the multi-sample image by the human eye can be obtained by modeling the relative response values of the three types of cone cells of the human eye to light of different wavelengths, and obtaining a subjective perception result dataset of different color differences from the test population on the basis of the response.

[0104] The image color difference correction method provided by the embodiments of the present application is described below, so that the technical solutions described in the embodiments of the present application can be more clearly understood.

[0105] Figure 2 A flowchart of the image color difference correction method provided by the embodiments of the present application is shown in FIG. 2, and the image color difference correction process of the method includes: Figure 2

[0106] Step 210, obtaining an input image.

[0107] Step 220, obtaining RGB channel gray values corresponding to each pixel point in the input image.

[0108] Step 230, converting the RGB channel gray values corresponding to each pixel point in the input image into gray values on a color difference channel. One calculation method of converting the RGB channel gray values into gray values on the color difference channel is as follows:

[0109]

[0110] Step 240, establishing a mapping matrix between the color difference correction weight matrix output by the convolutional neural network and the color difference index value. A specific processing method in a color gamut space is as follows:

[0111] Figure 3 A flowchart of establishing a mapping matrix between the color difference index value and the color difference correction weight value provided by the embodiments of the present application is shown in FIG. 4, and the process of establishing the mapping matrix includes: Figure 3

[0112] First, the color difference index value is calculated​​ Then set different threshold to judge, when the index value is less than the threshold , the weight matrix value weight is 1, namely discard not to join the mapping matrix (the color difference index value is not modified); when the index value is greater than or equal to the threshold And less than or equal to the threshold , the corresponding weight matrix value weight1 is established (the color difference index value is modified by weight1); when the index value is greater than the threshold And less than or equal to the threshold , the corresponding weight matrix value weight2 is established (the color difference index value is modified by weight2); then further judge, in the different threshold range, the corresponding weight matrix value is established multiple to one relationship, finally all the index values and the weight matrix values are collected to form the final mapping matrix. The mapping matrix is equivalent to the first mapping relationship.

[0113] Figure 4 The flow chart two of the process of establishing the mapping matrix between the color difference index value and the color difference correction weight value provided by the embodiment of the application is shown as follows: Figure 4 The process of establishing the mapping matrix includes:

[0114] Firstly, the color difference index value is calculated Then set different threshold to judge, when the index value is less than the threshold , the weight matrix value weight is 1, namely discard not to join the mapping matrix (the color difference index value is not modified); when the index value is greater than or equal to the threshold And less than or equal to the threshold , the corresponding weight matrix value weight1 is established (the color difference index value is modified by weight1); when the index value is greater than the threshold And less than or equal to the threshold , the corresponding weight matrix value weight2 is established (the color difference index value is modified by weight2); then further judge, in the different threshold range, the corresponding weight matrix value is established multiple to one relationship, finally all the index values and the weight matrix values are collected to form the final mapping matrix. The mapping matrix is equivalent to the second mapping relationship.

[0115] Different color gamut spaces have different mapping matrices, which together constitute adaptive color adjustment.

[0116] Step 250, the weight matrix value in the final output mapping matrix is corrected and adjusted to the actual color space, and the modified image is output, and a calculation formula for correcting and adjusting the gray value on the color difference channel is as follows:

[0117]

[0118] a1 and a2 are corresponding color difference correction weight values. The first color difference index value and the second color difference index value corresponding to the color difference correction weight values a1 and a2 that adapt to human eye perception are found through a matrix mapping relationship, a threshold value β is set, when the first color difference index value and the second color difference index value are greater than the threshold value, the weight values (weight1 to weight N different weights) are obtained through the mapping matrix according to the difference value, and are applied to the color difference channel to obtain the color difference value conforming to the human eye, and finally YUV format output is obtained.

[0119] It should be noted that under different color gamut spaces, the matrix values corresponding to a1 and a2 can be very different, and the neural network will be processed accordingly, and the corresponding traditional threshold retrieval will also be adjusted accordingly.

[0120] The embodiment of the application mainly establishes the difference relationship between the spatial color difference and the human eye perception through the end-to-end convolutional neural network, obtains the appropriate weight value, and applies it to the spatial color difference. Different spatial distance color differences have different weight values, so as to establish a matrix mapping relationship. When the color difference channel is output, the weight value needs to be multiplied, and finally the YUV format output image is converted. In this way, the problem that the color difference representation in the Riemann color space does not conform to the human sense can be corrected.

[0121] After obtaining the camera input multi-sample image, it is converted into RGB format, and is converted into R+G+B, R-G, B-G three channels. The two color difference channel values are combined with the subjective feeling result value of the human eye to the multi-sample color difference one by one, and are input into the end-to-end convolutional neural network U-NET, to obtain the direct relationship between the two, and to establish a weight relationship formula. Then the weight relationship and the two color difference channel values are set to mapping matrices, that is, the first color difference correction weight value and the second color difference correction weight value. Then the mapping matrix is set in the RGB to YUV process in the software processing process, and the color difference display result conforming to the human sense is obtained.

[0122] The image color difference correction device provided by the embodiment of the application is described below. The image color difference correction device described below can be correspondingly referred to the image color difference correction method described above.

[0123] Figure 5 The structure diagram of the image color difference correction device provided by the application is shown in Figure 5 The device comprises:

[0124] The first acquisition module 510 is configured to acquire first color difference gray scale values and second color difference gray scale values corresponding to respective pixel points in an input image. Each first color difference gray scale value is a difference between a first primary color gray scale value and a second primary color gray scale value of a corresponding pixel point, and each second color difference gray scale value is a difference between the second primary color gray scale value and a third primary color gray scale value of the corresponding pixel point.

[0125] The second acquisition module 520 is configured to acquire first color difference index values of respective pixel points corresponding to the first color difference gray scale values, and second color difference index values of respective pixel points corresponding to the second color difference gray scale values.

[0126] The determination module 530 is configured to, for any pixel point in the input image, obtain a first color difference correction weight value of the pixel point based on a first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value, and obtain a second color difference correction weight value of the pixel point based on a second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value.

[0127] The correction module 540 is configured to perform color difference correction on the input image based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, and generate a corrected output image.

[0128] The first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye.

[0129] In some embodiments, the first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye by training a neural network model, including:

[0130] The neural network model is trained based on the plurality of sample images and the color difference perception result data set of the plurality of sample images by the human eye to obtain a first effective weight distribution of the first color difference gray scale values and a second effective weight distribution of the second color difference gray scale values.

[0131] A first mapping relationship between a plurality of first color difference index value threshold intervals and the first effective weight distribution is established, and a second mapping relationship between a plurality of second color difference index value threshold intervals and the second effective weight distribution is established.

[0132] In some embodiments:

[0133]

[0134]

[0135] The first primary color is red, the second primary color is green, the third primary color is blue, and the output image is a YUV image.

[0136] In some embodiments,

[0137] The plurality of first color difference index value threshold intervals include a first color difference threshold interval in which a first color difference index value is less than a first color difference threshold, and a plurality of first color difference threshold intervals in which a first color difference index value is greater than or equal to the first color difference threshold, the first effective weight corresponding to the first color difference threshold interval in which the first color difference index value is less than the first color difference threshold is 1, and the first effective weight corresponding to each of the plurality of first color difference threshold intervals in which the first color difference index value is greater than or equal to the first color difference threshold is obtained by training a neural network model;

[0138] The plurality of second color difference index value threshold intervals include a second color difference threshold interval in which a second color difference index value is less than a second color difference threshold, and a plurality of second color difference threshold intervals in which a second color difference index value is greater than or equal to the second color difference threshold, the second effective weight corresponding to the second color difference threshold interval in which the second color difference index value is less than the second color difference threshold is 1, and the second effective weight corresponding to each of the plurality of second color difference threshold intervals in which the second color difference index value is greater than or equal to the second color difference threshold is obtained by training a neural network model.

[0139] In some embodiments, the neural network model is a U-Net convolutional neural network model.

[0140] In some embodiments, the neural network model is trained based on a plurality of sample images and a color difference perception result dataset of the plurality of sample images, to obtain a first effective weight distribution of the first color difference gray value and a second effective weight distribution of the second color difference gray value, including:

[0141] Based on the relative response values of the three types of cone cells of the human eye to light of different wavelengths, a color difference perception result dataset of the plurality of sample images is obtained by modeling the plurality of sample images;

[0142] The root mean square test error is used as the loss function of the U-Net convolutional neural network model, and the first effective weight distribution and the second effective weight distribution are obtained from the color difference perception result dataset of the U-Net convolutional neural network model.

[0143] It should be noted that the image color difference correction device provided by the embodiments of the present application can realize all the method steps realized by the image color difference correction method embodiments, and can achieve the same technical effects. Therefore, the same parts and beneficial effects of the method embodiments in the embodiments will not be described in detail.

[0144] Figure 6 An example of an electronic device is shown in the schematic diagram of the physical structure of the electronic device, as shown in Figure 6As shown, the electronic device can include a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can invoke a logical instruction in the memory 630 to execute an image color difference correction method, which includes: obtaining a first color difference gray value and a second color difference gray value corresponding to each pixel point in an input image; each first color difference gray value is the difference between a first primary color gray value and a second primary color gray value of the corresponding pixel point, and each second color difference gray value is the difference between a second primary color gray value and a third primary color gray value of the corresponding pixel point;

[0145] obtaining a first color difference index value of each pixel point corresponding to the first color difference gray value, and a second color difference index value of each pixel point corresponding to the second color difference gray value;

[0146] For any pixel point in the input image, based on the first color difference index value of the pixel point, and a first mapping relationship between the first color difference index value and a first color difference correction weight value, the first color difference correction weight value of the pixel point is obtained; based on the second color difference index value of the pixel point, and a second mapping relationship between the second color difference index value and a second color difference correction weight value, the second color difference correction weight value of the pixel point is obtained;

[0147] Based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, the input image is color difference corrected to generate a corrected output image;

[0148] The first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a neural network model.

[0149] On the other hand, the present application also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, when the computer program is executed by a processor, the computer can execute the method provided by each method embodiment, the method includes: obtaining a first color difference gray value and a second color difference gray value corresponding to each pixel point in an input image; each first color difference gray value is the difference between a first primary color gray value and a second primary color gray value of the corresponding pixel point, and each second color difference gray value is the difference between a second primary color gray value and a third primary color gray value of the corresponding pixel point;

[0150] obtaining a first color difference index value of each pixel point corresponding to the first color difference gray value, and a second color difference index value of each pixel point corresponding to the second color difference gray value;

[0151] For any pixel point in the input image, a first color difference correction weight value of the pixel point is obtained based on a first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value; a second color difference correction weight value of the pixel point is obtained based on a second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value.

[0152] Based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, color difference correction is performed on the input image to generate a corrected output image.

[0153] The first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye.

[0154] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement an image color difference correction method provided by each of the above method embodiments, the method comprising: obtaining first color difference gray values and second color difference gray values corresponding to each pixel point in an input image; each first color difference gray value being a difference between a first original color gray value and a second original color gray value of a corresponding pixel point, and each second color difference gray value being a difference between a second original color gray value and a third original color gray value of the corresponding pixel point.

[0155] Obtaining first color difference index values of each pixel point corresponding to the first color difference gray values, and second color difference index values of each pixel point corresponding to the second color difference gray values.

[0156] For any pixel point in the input image, a first color difference correction weight value of the pixel point is obtained based on a first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value; a second color difference correction weight value of the pixel point is obtained based on a second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value.

[0157] Based on the first color difference correction weight value and the second color difference correction weight value of each pixel point in the input image, color difference correction is performed on the input image to generate a corrected output image.

[0158] The first mapping relationship and the second mapping relationship are obtained based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by a human eye.

[0159] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0160] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0161] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image color difference correction method characterized by, The method comprises: obtaining first color difference gray values and second color difference gray values corresponding to each pixel point in an input image; each first color difference gray value is the difference between a first primary color gray value and a second primary color gray value of the corresponding pixel point, and each second color difference gray value is the difference between the second primary color gray value and a third primary color gray value of the corresponding pixel point; obtaining first color difference index values of the pixel points corresponding to the first color difference gray values and second color difference index values of the pixel points corresponding to the second color difference gray values; for any pixel point in the input image, obtaining a first color difference correction weight value of the pixel point based on the first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value; obtaining a second color difference correction weight value of the pixel point based on the second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value; correcting the first color difference gray values corresponding to the pixel points based on the first color difference correction weight values of the pixel points in the input image and correcting the second color difference gray values corresponding to the pixel points based on the second color difference correction weight values of the pixel points, to generate a corrected output image; wherein the first mapping relationship and the second mapping relationship are obtained by training a neural network model based on a plurality of sample images and a color difference perception result data set of the plurality of sample images by human eyes; wherein the neural network model is trained based on the plurality of sample images and the color difference perception result data set of the plurality of sample images by human eyes to obtain a first effective weight distribution of the first color difference gray values and a second effective weight distribution of the second color difference gray values; establishing a first mapping relationship between a plurality of first color difference index value threshold intervals and the first effective weight distribution and a second mapping relationship between a plurality of second color difference index value threshold intervals and the second effective weight distribution; the first primary color is red, the second primary color is green, the third primary color is blue, and the output image is a YUV image.

2. The image color difference correction method according to claim 1, wherein: the plurality of first color difference index value threshold intervals include a first color difference threshold interval in which a first color difference index value is less than a first color difference threshold and a plurality of first color difference threshold intervals in which first color difference index values are greater than or equal to the first color difference threshold, the first effective weight corresponding to the first color difference threshold interval in which the first color difference index value is less than the first color difference threshold is 1, and each first color difference threshold interval of the plurality of first color difference threshold intervals in which the first color difference index values are greater than or equal to the first color difference threshold corresponds to a first effective weight obtained by training the neural network model. The plurality of second color difference index value threshold intervals include a second color difference threshold interval in which a second color difference index value is less than a second color difference threshold value, and a plurality of second color difference threshold intervals in which a second color difference index value is greater than or equal to the second color difference threshold value. The second color difference threshold interval in which the second color difference index value is less than the second color difference threshold value corresponds to a first effective weight of 1. Each of the plurality of second color difference threshold intervals in which the second color difference index value is greater than or equal to the second color difference threshold value corresponds to a second effective weight obtained by training the neural network model.

3. The image color difference correction method according to any one of claims 1 to 2, characterized in that, The neural network model is a U-Net convolutional neural network model.

4. The image color difference correction method according to claim 3, characterized by, The training of the neural network model based on the plurality of sample images and a color difference perception result dataset of the plurality of sample images by a human eye includes: modeling the plurality of sample images based on relative response values of three types of cone cells of the human eye to light of different wavelengths to obtain the color difference perception result dataset of the plurality of sample images; using a root mean square test error as a loss function of the U-Net convolutional neural network model, and obtaining the first effective weight distribution and the second effective weight distribution based on the color difference perception result dataset of the plurality of sample images by the U-Net convolutional neural network model.

5. An image color difference correction device characterized by comprising: including: a first acquisition module configured to acquire first color difference gray values and second color difference gray values corresponding to respective pixel points in an input image. Each of the first color difference gray values is a difference between a first primary color gray value and a second primary color gray value of a corresponding pixel point. Each of the second color difference gray values is a difference between the second primary color gray value and a third primary color gray value of the corresponding pixel point; a second acquisition module configured to acquire first color difference index values of the respective pixel points corresponding to the first color difference gray values, and second color difference index values of the respective pixel points corresponding to the second color difference gray values; a determination module configured to, for any pixel point in the input image, obtain a first color difference correction weight value of the pixel point based on a first color difference index value of the pixel point and a first mapping relationship between the first color difference index value and the first color difference correction weight value; obtain a second color difference correction weight value of the pixel point based on a second color difference index value of the pixel point and a second mapping relationship between the second color difference index value and the second color difference correction weight value; a correction module configured to correct the first color difference gray values corresponding to the respective pixel points in the input image based on the first color difference correction weight values of the respective pixel points, and correct the second color difference gray values corresponding to the respective pixel points in the input image based on the second color difference correction weight values of the respective pixel points, to generate a corrected output image; wherein the first mapping relationship and the second mapping relationship are obtained by training a neural network model based on a plurality of sample images and a color difference perception result dataset of the plurality of sample images by a human eye. The neural network model is trained based on the multi-sample image and a color difference perception result dataset of the multi-sample image, to obtain a first effective weight distribution of a first color difference gray value and a second effective weight distribution of a second color difference gray value. The first mapping relationship between a plurality of first color difference index value threshold intervals and the first effective weight distribution is established, and the second mapping relationship between a plurality of second color difference index value threshold intervals and the second effective weight distribution is established. The first primary color is red, the second primary color is green, the third primary color is blue, and the output image is a YUV image.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the image color difference correction method of any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the image color difference correction method of any one of claims 1 to 4.

8. A computer program product comprising a computer program which, when executed by a processor, implements the image color difference correction method of any one of claims 1 to 4.

Images