White balance correction method, apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose a white balance correction method, device, equipment and storage medium. The white balance correction method comprises: inputting a to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image; determining an illumination color parameter of the to-be-corrected image according to the achromatic point weight map; and performing white balance correction on the to-be-corrected image according to the illumination color parameter to obtain a corrected image. Embodiments of the present application determine an achromatic point weight map of a to-be-corrected image based on a pre-trained achromatic point model, obtain an illumination color of the to-be-corrected image according to the achromatic point weight map, and finally perform white balance correction according to the illumination color, thereby realizing white balance correction of the to-be-corrected image. The white balance correction method of embodiments of the present application does not limit the scene of the to-be-corrected image, can improve the adaptability of white balance correction, and improve the accuracy of the image correction result.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of Internet, and in particular, to a white balance correction method, device, equipment and storage medium. BACKGROUND

[0002] In nature, the color of an object changes with the change of the ambient light source. For humans, even if the light source changes, the eyes are not greatly affected in color perception. This is because the brain has certain prior knowledge of the color of the object, and can perceive the inherent color of the object under some changing light sources. The camera itself does not have this special function of the human eye, and when it captures images under different light sources, it will be affected by the color of the light source, so it is necessary to perform white balance correction processing on the images obtained by the camera.

[0003] At present, the commonly used white balance correction algorithm is a statistical method. That is, under certain assumptions, the ambient light source is estimated by statistically analyzing the color characteristics of the scene image. For example, the commonly used gray world method, perfect reflection method, etc. This kind of method has simple principle and is easy to implement, but due to the strong assumption-based, the use of the scene is limited. The statistical method has the shortcomings of strong assumption and poor adaptability. SUMMARY

[0004] Embodiments of the present application provide a white balance correction method, device, equipment and storage medium to improve the adaptability of white balance correction, and further improve the accuracy of the correction result.

[0005] In a first aspect, the embodiments of the present application provide a white balance correction method, comprising:

[0006] inputting a to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image;

[0007] determining a light color parameter of the to-be-corrected image according to the achromatic point weight map;

[0008] performing white balance correction on the to-be-corrected image according to the light color parameter to obtain a corrected image.

[0009] In a second aspect, the embodiments of the present application also provide a white balance correction device, comprising:

[0010] a weight map determination module configured to input a to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image;

[0011] a light color determination module configured to determine a light color parameter of the to-be-corrected image according to the achromatic point weight map;

[0012] a white balance correction module configured to perform white balance correction on the image to be corrected according to the illumination color parameter, to obtain a corrected image.

[0013] In a third aspect, an apparatus is provided, comprising:

[0014] one or more processors;

[0015] a storage configured to store one or more programs,

[0016] when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the white balance correction method according to any of the embodiments of the present application.

[0017] In a fourth aspect, a computer readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, implements the white balance correction method according to any of the embodiments of the present application.

[0018] The embodiment of the present application determines the achromatic point weight map of the image to be corrected based on the pre-trained achromatic point model, obtains the illumination color of the image to be corrected according to the achromatic point weight map, and finally performs white balance correction according to the illumination color, thereby realizing white balance correction of the image to be corrected. The white balance correction method of the embodiment of the present application does not limit the scene of the image to be corrected, and can improve the adaptability of white balance correction and the accuracy of the image correction result. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the white balance correction method in the embodiment one of the present application;

[0020] Figure 2A is a flowchart of the achromatic point model training process in the embodiment two of the present application;

[0021] Figure 2B is a schematic diagram of the processing flow of the input image by the pre-constructed network architecture in the embodiment two of the present application;

[0022] Figure 3 is a flowchart of the achromatic point model training process in the embodiment three of the present application;

[0023] Figure 4 is a structural schematic diagram of the white balance correction apparatus in the embodiment four of the present application;

[0024] Figure 5 is a structural schematic diagram of the apparatus in the embodiment five of the present application. DETAILED DESCRIPTION

[0025] The application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the application, but not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description, rather than all the structures.

[0026] Embodiment one

[0027] Figure 1 is a flowchart of the white balance correction method in the embodiment one of the application, and the embodiment can be applied to the case of white balance correction processing of the image acquired by the camera. The method can be performed by a white balance correction device, which can be realized in software and / or hardware, and can be configured in a device, for example, the device can be a background server or other device with communication and computing capabilities. As shown in Figure 1 , the method specifically includes:

[0028] Step 101, input the image to be corrected into a pre-trained achromatic point model to obtain an achromatic point weight map of the image to be corrected.

[0029] The image to be corrected refers to the change of the color of the object shown in the image shot by the camera due to the influence of different environmental light sources. To overcome this change, white balance correction is needed for the image to be corrected. White balance correction is to estimate the influence of the environmental light source factor on the color of the object in the image to be corrected, so as to convert the image under the influence of the environmental light source to a certain classic light source, restore the color of the object in the image to be corrected to the inherent color of the object under the target light source, that is, make the color of the object in the image to be corrected after white balance correction the same or similar to the color perceived by the human eye.

[0030] The achromatic point refers to the gray point of the image under the standard white light source. The achromatic point can be used to determine the environmental light source factor in the image to be corrected. The achromatic point model is a pre-trained network model that can extract the features of the achromatic point in the image to be corrected, and is used to detect the achromatic point in the image. The input of the model is the image to be corrected, and the output is the feature representation of the achromatic point in the image to be corrected. Exemplarily, the output can be an achromatic point weight map of the image to be corrected. The achromatic point weight map refers to a probability representation map of each pixel point being an achromatic point in the image to be corrected. Exemplarily, the achromatic point weight map has the same size as the image to be corrected, and the value of each pixel point in the achromatic point weight map is the probability of the corresponding point in the image to be corrected being an achromatic point. For example, the value of each point in the achromatic weight map ranges from 0 to 255, or the value is normalized to be in the range of 0 to 1, which can be set according to specific conditions, which is not limited herein.

[0031] Specifically, a non-achromatic point model capable of extracting non-achromatic point features in an image is pre-trained, and the image to be corrected is taken as an input of the non-achromatic point model, and non-achromatic point features in the image to be corrected are recognized by the model to obtain an output of the model as a non-achromatic point weight map corresponding to the image to be corrected. Illustratively, a probability value of each pixel point in the image to be corrected being a non-achromatic point is obtained by the non-achromatic point model, and a non-achromatic point weight map is formed according to the serial number of the pixel points, so that each point in the weight map corresponds to the image to be corrected.

[0032] Step 102, determining the illumination color parameter of the image to be corrected according to the non-achromatic point weight map.

[0033] The illumination color parameter is used to determine the influence of the ambient light source on the color of the object when the image to be corrected is taken. For example, the inherent color of an object is white, that is, the color of the object obtained by taking under a standard white light source is white. If the color obtained by taking the object under a certain light source is red, it indicates that the illumination color of the image to be corrected is red, and the illumination color parameter can be determined according to the specific information of the red color.

[0034] Since the non-achromatic point is a gray point under the standard white light source, the illumination color can be determined by the non-achromatic point in the image to be corrected. Illustratively, each non-achromatic point in the image to be corrected can be determined by the non-achromatic point weight map, and then the illumination color parameter when the image to be corrected is taken is determined. For example, the illumination color parameter represents the color of the ambient light source from one aspect, and therefore the illumination color can be represented by the tristimulus values of the color respectively.

[0035] In an optional embodiment, the illumination color parameter e = [r, g, b] is determined by the following formula:

[0036]

[0037]

[0038]

[0039] wherein, ω(x, y) is a non-achromatic point weight value of the (x, y) pixel point determined according to the non-achromatic point weight map, and Σω(x, y) is the weight sum of the non-achromatic point weight map, f R (x, y) is an R channel value of the (x, y) pixel point in the image to be corrected, f G (x, y) is a G channel value of the (x, y) pixel point in the image to be corrected, f B(x,y) is the B channel value of the pixel point (x,y) in the image to be corrected.

[0040] Specifically, taking the RGB format image f(x) with 8bit precision as input, the achromatic point weight map ω of f(x) is obtained, and the illumination color e of the image to be corrected f(x) is set, and the three stimulus values [r,g,b] of the color are used to represent e.

[0041] Wherein, r in the illumination color parameter represents the influence of the achromatic point of the image to be corrected on the R channel; g in the illumination color parameter represents the influence of the achromatic point of the image to be corrected on the G channel; b in the illumination color parameter represents the influence of the achromatic point of the image to be corrected on the B channel.

[0042] The three channel values f R (x,y), f G (x,y) and f B (x,y) are determined, the pixel value of each pixel point on each channel is multiplied by the achromatic point weight of the corresponding point in the achromatic point weight map and summed to obtain the achromatic point weight sum of each channel of the image to be corrected, and the ratio of the achromatic point weight sum of each channel to the weight sum in the achromatic point weight map is taken as the influence degree of the achromatic point on the image, and finally the illumination color parameter is determined according to the proportion of the influence degree of the achromatic point of each channel in the three channels.

[0043] The illumination color parameter is determined by the achromatic point weight map and the image information of the image to be corrected, which uses the original image information to determine, and is suitable for scenes under different illumination environments, thus improving the accuracy of illumination color estimation and avoiding the phenomenon of deviation of light source color estimation in some scenes.

[0044] Step 103, white balance correction is performed on the image to be corrected according to the illumination color parameter, and a corrected image is obtained.

[0045] After the illumination color parameter is determined, the influence of the illumination color on the inherent color of the object can be determined, so that the influence can be eliminated by the determined illumination color parameter, and white balance correction is realized.

[0046] For example, the gain values of the illumination on each channel, such as red gain, blue gain and green gain, are determined according to the illumination color parameter,

[0047]

[0048] Wherein, RGain refers to red gain, GGain refers to blue gain, and BGain refers to green gain. The influence of the light color on the three channels is determined, and then the influence elimination operation is realized.

[0049] In an optional embodiment, the corrected image is determined by the following formula:

[0050]

[0051] Wherein, white R (x,y) is the R channel value of the (x,y) pixel point in the corrected image, white G (x,y) is the G channel value of the (x,y) pixel point in the corrected image, white B (x,y) is the B channel value of the (x,y) pixel point in the corrected image.

[0052] On the basis of the above examples, after determining the gain of each channel, the color error caused by the channel gain to each channel pixel value is eliminated by the formula , and then the color error elimination is realized. For example, the quotient of the pixel value of each channel of the to-be-corrected image and the light color parameter value on the corresponding channel is the pixel value on the channel after white balance correction, so as to realize the white balance correction of the to-be-corrected image.

[0053] The embodiment of the present application determines the achromatic point weight map of the to-be-corrected image based on the pre-trained achromatic point model, obtains the light color of the to-be-corrected image according to the achromatic point weight map, and finally performs white balance correction according to the light color, thereby realizing the white balance correction of the to-be-corrected image. The white balance correction method of the embodiment of the present application does not limit the scene of the to-be-corrected image, can improve the adaptability of the white balance correction, and improve the accuracy of the image correction result.

[0054] Embodiment two

[0055] Figure 2A is a flowchart of the achromatic point model training process in the embodiment two of the present application. The embodiment two further describes the pre-training process of the achromatic point model in the embodiment one. As shown in the figure, Figure 2A the training process includes:

[0056] Step 201, acquiring a training sample set color image.

[0057] Since only color images have the color information required for determining the light color, color images are used for the training samples in the embodiment of the present application, and the color images in the training sample set need to cover more scenes and light source types as much as possible, so as to ensure the accuracy and universality of the achromatic point feature extraction.

[0058] Since the color images of various formats can be converted through color space conversion, the format of the color images is not limited, for example, the format of the training sample set and the color image to be corrected in the embodiment of the present application is RGB format, when the input color image is not in RGB format, such as YUV or other formats, the RGB format image is obtained through linear or nonlinear color space conversion. In addition, the color images in the training sample set are not limited to the color images with labeled illumination color parameters, and can also be color images without labeled illumination color parameters.

[0059] Step 202, determining the sample achromatic point weight map of the color image in the training sample set through the pre-constructed network.

[0060] The pre-constructed network has the ability to extract the features of the achromatic points in the image.

[0061] In an optional embodiment, the pre-constructed network at least includes a convolutional layer, an activation function layer, a pooling layer and a deconvolutional layer, so that the input training sample set color image and the output sample achromatic point weight map have the same resolution.

[0062] Specifically, the feature data of the image is extracted through the convolutional layer, the activation function layer and the pooling layer, and in order to make the input training sample set color image and the output sample achromatic point weight map have the same resolution, the feature data is processed through the deconvolutional layer.

[0063] For example, an RGB format image with an 8bit precision size of 227*227 is taken as the input of the network, a full convolutional neural network structure is built to predict the achromatic weight map of the input image. The input layer of the network is a 227*227*3 image, as shown in the following formula (1), the pre-constructed network architecture processes the input image as follows: Figure 2B The processing flow diagram of the input image processed by the pre-constructed network architecture is shown in the following figure:

[0064] (1) After the first convolutional layer and the activation function relu are processed, the size of the input image data remains unchanged; after the maximum pooling layer is processed, the size of the pooled feature data is 27*27*96;

[0065] (2) After the second convolutional layer and the activation function relu are processed, the size of the image data remains unchanged; after the maximum pooling layer is processed, the size of the pooled feature data is 13*13*256;

[0066] (3) After the third convolutional layer and the activation function relu are processed, the size of the output feature data is 13*13*384;

[0067] (4) After the fourth convolutional layer and the activation function relu are processed, the data size is unchanged; after the maximum pooling layer is processed, the feature data size after the pooling is 6*6*256;

[0068] (5) After the fifth convolutional layer and the activation function relu are processed, the output feature data size is 6*6*4096;

[0069] (6) After the sixth convolutional layer and the activation function relu are processed, the output feature data size is 6*6*21;

[0070] (7) After the first deconvolutional layer and the activation function relu are processed, the output feature data size is 13*13*21;

[0071] (8) After the second deconvolutional layer and the activation function relu are processed, the output feature data size is 227*227*21; the maximum value of the 21-dimensional data corresponding position is taken, and finally the achromatic point weight map of 227*227*1 is obtained.

[0072] The network structure described above is only one embodiment of the present application, and the present application does not limit the specific structure of the network. For the pre-constructed network, as long as it has the following characteristics: it can process the input color image; the features extracted by the convolutional layer are the features of the achromatic points in the image; the network output is the achromatic point weight map; and the number of convolutional layers and deconvolutional layers is set to ensure that the input image and the output image are the same size; and the image size is not limited for the input of the pre-constructed network in the embodiment of the present application.

[0073] Step 203, determining the sample illumination color parameter of the training sample set color image according to the sample achromatic point weight map.

[0074] Specifically, the sample illumination color parameter is determined by the method of step 102 in the above embodiment, which is not repeated here.

[0075] Step 204, determining the sample correction image of the training sample set color image according to the sample illumination color parameter.

[0076] Specifically, the sample correction image is determined by the method of step 103 in the above embodiment, which is not repeated here

[0077] Step 205, performing back propagation on the achromatic point model according to the image information of the sample correction image and the sample achromatic point weight map, to optimize the parameters in the network.

[0078] The error of the achromatic point in the sample correction image is determined according to the image information of the sample correction image and the sample achromatic point weight graph, the correction error of the sample correction image is calculated through the error of the achromatic point, and then the achromatic point model is back propagated, the parameters in the network are optimized, the extraction of the achromatic point feature by the network is more accurate, and then the accuracy of the achromatic point weight graph determination is improved.

[0079] Optionally, for the color image with a labeled illumination color parameter in the training sample set, a label correction image is determined through the labeled illumination color parameter. The labeled illumination color parameter refers to the real illumination color parameter of the sample image, and can be labeled manually or determined through prior knowledge. Then, the error of the achromatic point in the label correction image and the sample correction image is calculated through the loss function of supervised training, and then the error is continuously reduced through back propagation, the function of optimizing the parameters in the network is realized, and then the feature extraction capability of the network is improved.

[0080] The embodiment of the application realizes the extraction of the achromatic point feature of the sample image through the pre-constructed network, determines the sample correction image through the sample achromatic point weight graph obtained through the network, and then realizes the optimization of the network parameters through the back propagation of the sample correction image, improves the capability of the network to extract the achromatic point feature, and then improves the accuracy of the determination of the achromatic point weight graph. The achromatic point model obtained through the method can extract the features of the to-be-corrected image in various scenes, and improves the adaptability and accuracy of the white balance correction.

[0081] Embodiment three

[0082] Figure 3 It is the flowchart of the achromatic point model training process in the embodiment three of the application, and the embodiment three is further optimized on the basis of the above-mentioned embodiment, and the embodiment of the application is suitable for unsupervised achromatic point model training. As shown in Figure 3 , the training process comprises:

[0083] Step 301, acquiring the color image of the training sample set; wherein the color image of the training sample set does not include a labeled illumination color parameter.

[0084] The labeled illumination color parameter refers to the real illumination color parameter of the color image in the training sample set, and can be pre-labeled manually or determined through prior knowledge. In the embodiment of the application, the color image of the training sample set does not need to be pre-labeled or acquired with the labeled illumination color parameter, the acquisition efficiency of the training sample set is improved, and the training process is simplified.

[0085] Step 302, determining the sample achromatic point weight graph of the color image of the training sample set through the pre-constructed network.

[0086] Step 303, determining sample illumination color parameters of the color images of the training sample set according to the sample achromatic point weight map.

[0087] Step 304, determining sample correction images of the color images of the training sample set according to the sample illumination color parameters.

[0088] Step 305, constructing a loss function of unsupervised training.

[0089] Since the color images in the training sample set do not have labeled illumination color parameters, it is necessary to construct a loss function of unsupervised training from the features of the color images themselves.

[0090] According to the nature of the color image, the color image can be represented as:

[0091] f i (x,y)=E i (x,y)·R i (x,y),i∈{R,G,B}

[0092] Wherein, f i (x,y) represents the pixel value of the i channel of the color image at the pixel point (x, y); E i (x,y) represents the illumination color parameter of the i channel of the color image at the pixel point (x, y); R i (x,y) represents the surface reflection value of the i channel of the color image at the pixel point (x, y), which represents the intrinsic value at the point, which is not affected by the environment light source and belongs to the intrinsic characteristics of the object.

[0093] Further, the logarithmic domain representation of the above formula is obtained:

[0094] log(f i (x,y))=log(E i (x,y))+log(R i (x,y))

[0095] The color image is divided into several local regions, and the light source in the local region of the color image can be regarded as uniform, so that the E i (x,y) of each channel of the pixel point in the local region can be regarded as equal, and according to the characteristics of the achromatic point, the R i (x,y) of each channel of the achromatic point is the same, so that the achromatic point in the local region meets the following rules: log(f R (x,y))=log(f G (x,y))=log(f B(x, y))≠0. Therefore, for the corrected image, when the channel values of the pixel points where each achromatic point in the corrected image is equal or approximately equal, the color in the corrected image is close to the real color of the object.

[0096] Therefore, according to the principle that the channel values of the pixel points where each achromatic point in the corrected image are equal or approximately equal, a loss function of unsupervised training is constructed. The error between the channel values of each achromatic point in the corrected image can be determined through the constructed loss function, and then the error of the corrected image is determined through the loss function, so as to judge the image correction result.

[0097] In an optional embodiment, the loss function of unsupervised training is constructed by the following formula:

[0098]

[0099]

[0100]

[0101] wherein I represents a sample corrected image to be calculated error value; i represents the serial number of a local image block pre-divided in the sample corrected image; n represents that the sample corrected image is divided into n local image blocks; p i represents the local contrast of the i-th local image block; L i represents the brightness value of the i-th local image block; t represents the serial number of three channels of the image; SD t represents the local variance of the local image block in the t channel; represents the local mean square error of the local image block in three channels.

[0102] Specifically, the sample corrected image is divided into n local image blocks, the local contrast of the image block is determined according to the pixel value of each achromatic point in the local image block, and the average brightness value in each local image block is determined. The average brightness value can be determined according to the average value of the pixel value of each pixel point, for example, according to the formula L(x, y) = (f R (x, y) + f G (x, y) + f B(x, y)) / 3, and taking the average of the average luminance values of all the pixels in the local image block as the average luminance value of the local image block; and taking the ratio of the local contrast and the average luminance value of all the local image blocks as the loss function. Since the channel values of each achromatic point in the image after accurate correction are equal or approximately equal, the local contrast of the pixel value of each achromatic point in the sample correction image should tend to zero, and thus the value of the loss function also tends to zero, that is, the accuracy of the image correction result can be determined through the loss function.

[0103] The local contrast of the local image block is determined by the variance of the pixel values of each channel of the local image, and the variance of the pixel values of each channel of the local image is determined by a variance calculation formula, and the variance values of the red channel, the green channel and the blue channel are obtained respectively, and the mean square error of the local image block is obtained according to the variances of each channel. The local contrast is determined by the relationship between the variance of each channel and the mean square error, which accurately reflects the contrast of the achromatic pixel points in the local image block and improves the accuracy of the loss function construction.

[0104] Step 306, based on the loss function, determining the error value of the sample correction image according to the image information of the sample correction image and the sample achromatic point weight map.

[0105] According to the image information of the sample correction image and the sample achromatic point weight map, the achromatic points in the sample correction image are determined, and then the errors of the channel values of the achromatic points are determined based on the pre-constructed loss function, and the overall situation of the errors of the channel values of all the achromatic points is taken as the error value of the sample correction image. When the obtained achromatic point weight map is accurate, the error value of the sample correction image corresponding to the obtained achromatic point weight map should tend to zero, so that the accuracy of the network output result can be determined by the error value, and then the accuracy of the white balance correction by the model can be determined.

[0106] In an optional embodiment, step 306 comprises:

[0107] According to the image information of the sample correction image and the sample achromatic point weight map, the correction weight image is determined by the formula

[0108] Based on the loss function and the correction weight image, the error value of the sample correction image is determined;

[0109] wherein, is the t-channel value of the (x, y) pixel point in the correction weight image, ω(x, y) is the achromatic point weight value of the (x, y) pixel point determined according to the sample achromatic point weight map, white i ​(x, y) is the t channel value of the sample correction image at the (x, y) pixel point, and Σω(x, y) is the weight of the sample achromatic point weight map.

[0110] Specifically, according to the formula The determined correction weight image can reflect the achromatic point in the sample correction image, and the influence degree of the achromatic point in the sample correction image can be further reflected through the sample achromatic point weight map, and thus the accuracy of the error value determined according to the correction weight image is improved.

[0111] Since the error value calculated by the loss function needs to reflect the channel error value of the achromatic point in the image, the correction weight image is calculated by the above formula, so as to reflect the achromatic point, and the value of the loss function is calculated according to the image information in the correction weight image as the error value of the sample correction image. Exemplarily, on the basis of the above example, the correction weight image is divided into n local image blocks, and the local contrast and average brightness value of each local image block are calculated to obtain the value of the loss function.

[0112] Step 307, the error value is used to perform back propagation on the achromatic point model to optimize the parameters in the network.

[0113] According to the error value, the error back propagation method is used for back propagation, the parameters in the network are optimized, and then an updated sample achromatic point weight map is obtained. Exemplarily, the back propagation of the error value can assign a higher weight value to the pixel that may be an achromatic point in the image, and an updated sample achromatic point weight map is obtained. According to the updated sample achromatic point weight map, a new round of sample correction image error calculation is performed, and the error value is minimized through continuous iteration, and then the iteration is stopped. At this time, the network parameters are optimal, which can improve the ability of the network to extract the achromatic point feature and ensure the accuracy of the achromatic point weight map.

[0114] The embodiment of the application is based on the construction of the loss function of unsupervised training, realizes the unsupervised model training process, avoids the need for pre-labeling work of the training sample set, and improves the training efficiency. And through the loss function of unsupervised training, the correction accuracy of the sample correction image can be truly reflected, and thus the ability of the network to extract the achromatic point feature is improved.

[0115] Embodiment four

[0116] Figure 4 is a structural schematic diagram of the white balance correction device in the embodiment four of the application. The embodiment can be applied to the case that the image obtained by the camera is subjected to white balance correction processing. As shown in Figure 4 , the device comprises:

[0117] The weight map determination module 410 is configured to input the to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image.

[0118] The illumination color determination module 420 is configured to determine an illumination color parameter of the to-be-corrected image according to the achromatic point weight map.

[0119] The white balance correction module 430 is configured to perform white balance correction on the to-be-corrected image according to the illumination color parameter to obtain a corrected image.

[0120] The embodiment of the present application determines an achromatic point weight map of a to-be-corrected image based on a pre-trained achromatic point model, obtains an illumination color of the to-be-corrected image according to the achromatic point weight map, and finally performs white balance correction according to the illumination color, thereby realizing white balance correction on the to-be-corrected image. The white balance correction method of the embodiment of the present application does not limit the scene of the to-be-corrected image, can improve the adaptability of white balance correction, and improves the accuracy of the image correction result.

[0121] Optionally, the device comprises a model training module configured to train the achromatic point model, comprising:

[0122] A sample image acquisition unit is configured to acquire a training sample set color image.

[0123] A sample weight map determination unit is configured to determine a sample achromatic point weight map of the training sample set color image by using a pre-constructed network.

[0124] A sample illumination color determination unit is configured to determine a sample illumination color parameter of the training sample set color image according to the sample achromatic point weight map.

[0125] A sample corrected image determination unit is configured to determine a sample corrected image of the training sample set color image according to the sample illumination color parameter.

[0126] A network parameter optimization unit is configured to perform back propagation on the achromatic point model according to image information of the sample corrected image and the sample achromatic point weight map, so as to optimize parameters in the network.

[0127] Optionally, the training sample set color image does not comprise a labeled illumination color parameter.

[0128] Correspondingly, the network parameter optimization unit comprises:

[0129] A loss function construction subunit is configured to construct a loss function for unsupervised training.

[0130] an error value determination subunit configured to determine an error value of the sample corrected image based on the loss function and according to image information of the sample corrected image and the sample achromatic point weight map;

[0131] a back propagation subunit configured to perform back propagation on the achromatic point model according to the error value to optimize parameters in the network.

[0132] Optionally, the loss function of the unsupervised training is constructed by the following formula:

[0133]

[0134]

[0135]

[0136] wherein I represents a sample corrected image to be calculated for an error value; i represents a serial number of a local image block pre-divided in the sample corrected image; n represents that the sample corrected image is divided into n local image blocks; p i represents a local contrast of the i-th local image block; L i represents a brightness value of the i-th local image block; t represents a serial number of three channels of an image; SD t represents a local variance of the local image block in the t-th channel; represents a local mean square error of the local image block in three channels.

[0137] Optionally, the error value determination subunit comprises:

[0138] according to image information of the sample corrected image and the sample achromatic point weight map, a correction weight image is determined by the formula

[0139] based on the loss function and the correction weight image, an error value of the sample corrected image is determined;

[0140] wherein is a t-th channel value of the correction weight image at a (x, y) pixel point, ω(x, y) is an achromatic point weight value of the (x, y) pixel point determined according to the sample achromatic point weight map, white i (x, y) is a t-th channel value of the sample corrected image at the (x, y) pixel point, and Σω(x, y) is a weight sum of the sample achromatic point weight map.

[0141] Optionally, the pre-constructed network at least comprises a convolution layer, an activation function layer, a pooling layer and a deconvolution layer, so that a color image of an input training sample set and a sample achromatic point weight map output have the same resolution.​

[0142] Optionally, the illumination color parameter e = [r, g, b] is determined by the following formula:

[0143]

[0144]

[0145]

[0146] wherein, ω(x, y) is the achromatic point weight value at the (x, y) pixel point determined according to the achromatic point weight map, and Σω(x, y) is the weight sum of the achromatic point weight map, f R (x, y) is the R channel value at the (x, y) pixel point in the to-be-corrected image, f G (x, y) is the G channel value at the (x, y) pixel point in the to-be-corrected image, f B (x, y) is the B channel value at the (x, y) pixel point in the to-be-corrected image.

[0147] Optionally, the corrected image is determined by the following formula:

[0148]

[0149] wherein, white R (x, y) is the R channel value at the (x, y) pixel point in the corrected image, white G (x, y) is the G channel value at the (x, y) pixel point in the corrected image, white B (x, y) is the B channel value at the (x, y) pixel point in the corrected image.

[0150] The white balance correction device provided in the embodiments of the present application can execute the white balance correction method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the white balance correction method.

[0151] Embodiment five

[0152] Figure 5 is a structural schematic diagram of a device provided in embodiment five of the present application. Figure 5 A block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present application is shown. Figure 5 The device 12 shown is merely an example and should not be taken as limiting the functionality or applicability of embodiments of the present application.

[0153] As Figure 5As shown, the device 12 is in the form of a general- purpose computer. The components of the device 12 can include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 to the processing unit 16.

[0154] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics bus (e.g., an Accelerated Graphics Port, or AGP bus) and a local bus using any of a variety of bus architectures. By way of example, these architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0155] The device 12 typically includes a variety of computer system readable media. Such media can be any available media that is located either internally or externally to the device 12, including both volatile and nonvolatile media, removable and non-removable media.

[0156] The system memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 can be provided for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 5 Not shown, a removable / non-removable interface can also be coupled to the bus 18 through which a user might, for example, insert and / or remove non-removable, non-volatile media and / or insert and / or remove removable, non-volatile media. Figure 5 A magnetic hard disk drive, for example, can be provided for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Although not specifically shown, such can be provided in the device 12. In these instances, one or more data media interfaces can be provided in the device 12 through which a user might, for example, insert and / or remove a non-removable, non-volatile media and / or insert and / or remove a removable, non-volatile media (such as a

[0157] A program / utility 40 having a set (at least one) of program modules 42 can be stored in, for example, the system memory 28 by way of example, such program modules 42 include an operating system, one or more application programs, other program modules, and program data, each of which or a combination can include implementation of the network environment. The program modules 42 generally carry out the functions and / or methodologies of embodiments of the present application.

[0158] Device 12 can also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; other devices that enable a user to interact with device 12; and / or any devices (e.g., a network card, a modem, etc.) that enable device 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 22. Still yet, device 12 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through network adapter 20. As Figure 5 illustrated, network adapter 20 communicates with the other components of device 12 via bus 18. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with device 12. These include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0159] Processing unit 16 can execute a program from a system memory 28 in order to perform various processing functions and data processing, such as implementing the white balance correction method provided by embodiments of the present application, including:

[0160] inputting a to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image;

[0161] determining an illumination color parameter of the to-be-corrected image according to the achromatic point weight map;

[0162] performing white balance correction on the to-be-corrected image according to the illumination color parameter to obtain a corrected image.

[0163] Embodiment Six

[0164] Embodiment Six of the present application also provides a computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the white balance correction method provided by embodiments of the present application, including:

[0165] inputting a to-be-corrected image into a pre-trained achromatic point model to obtain an achromatic point weight map of the to-be-corrected image;

[0166] determining an illumination color parameter of the to-be-corrected image according to the achromatic point weight map;

[0167] performing white balance correction on the to-be-corrected image according to the illumination color parameter to obtain a corrected image.

[0168] The computer storage media of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable media can be computer-readable signal media or computer-readable storage media. The computer-readable storage media, for example, can be, but is not limited to, an electrical, a magnetic, an optical, an electromagnetic, an infrared, or a semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media can be any tangible medium that contains or stores a program in a form that can be used by or in connection with an instruction execution system, apparatus, or device.

[0169] The computer-readable signal media can include a computer-readable program code in a baseband or propagated as a carrier wave in a propagation medium. Such a propagated signal can take a wide variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium that can be used to carry or store program code for use by or in connection with an instruction execution system, apparatus, or device.

[0170] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the above.

[0171] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments of the present application, electronic mail (email) can be utilized as the

[0172] Note that the above merely describes the preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, reconfigurations and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the appended claims.

Claims

1. A white balance correction method characterized by, The method comprises: inputting the image to be corrected into a pre-trained achromatic point model to obtain an achromatic point weight map of the image to be corrected; wherein the training process of the achromatic point model comprises: obtaining a training sample set color image; determining a sample achromatic point weight map of the training sample set color image through a pre-constructed network; determining a sample illumination color parameter of the training sample set color image according to the sample achromatic point weight map; determining a sample corrected image of the training sample set color image according to the sample illumination color parameter; performing back propagation on the achromatic point model according to image information of the sample corrected image and the sample achromatic point weight map to optimize parameters in the network, comprising: constructing a loss function for unsupervised training through the following formula: wherein I represents a sample corrected image whose error value is to be calculated; i represents a serial number of a local image block pre-divided in the sample corrected image; n represents a number of dividing the sample corrected image into n local image blocks; p i represents a local contrast of the i-th local image block; L i represents a luminance value of the i-th local image block; t represents a three-channel serial number of the image; SD t represents a local variance of the local image block in the t channel; represents a local mean square deviation of the local image block on three channels. determining an illumination color parameter of the image to be corrected according to the achromatic point weight map; performing white balance correction on the image to be corrected according to the illumination color parameter to obtain a corrected image.

2. The method of claim 1, wherein, The training sample set color image does not include a labeled illumination color parameter; determining an error value of the sample corrected image according to the image information of the sample corrected image and the sample achromatic point weight map based on the loss function; performing back propagation on the achromatic point model according to the error value to optimize the parameters in the network.

3. The method of claim 2, wherein, determining an error value of the sample corrected image according to the image information of the sample corrected image and the sample achromatic point weight map based on the loss function, comprising: According to the sample correction image information and the sample achromatic point weight map, a correction weight image is determined by the formula t∈{R,G,B} determining the error value of the sample corrected image based on the loss function and the corrected weight image; wherein, is the t-channel value at the (x, y) pixel in the correction weight image, ω(x, y) is the achromatic point weight value at the (x, y) pixel determined according to the sample achromatic point weight image, white i (x, y) is the t-channel value at the (x, y) pixel in the sample correction image, and ∑ω(x, y) is the weight sum of the sample achromatic point weight image.

4. The method of claim 1, wherein, The pre-constructed network at least comprises a convolution layer, an activation function layer, a pooling layer and a deconvolution layer, so that the input training sample set color image and the output sample achromatic point weight map have the same resolution.

5. The method of claim 1, wherein, The illumination color parameter e = [r, g, b] is determined through the following formula: wherein, ω(x, y) is an achromatic point weight value at the (x, y) pixel point determined according to the achromatic point weight map, and ∑ω(x, y) is a weight sum of the achromatic point weight map, f R (x, y) is an R channel value at the (x, y) pixel point in the image to be corrected, f G (x, y) is an R channel value at the (x, y) pixel point in the image to be corrected, f B (x, y) is an R channel value at the (x, y) pixel point in the image to be corrected.

6. The method of claim 5, wherein, The corrected image is determined through the following formula: where white R (x,y) is the R channel value at the (x,y) pixel in the corrected image, white G (x,y) is the G channel value at the (x,y) pixel in the corrected image, white B (x,y) is the B channel value at the (x,y) pixel in the corrected image.

7. A white balance correction device characterized by comprising: The method comprises: a weight map determination module configured to input the image to be corrected into a pre-trained achromatic point model to obtain an achromatic point weight map of the image to be corrected; wherein the training process of the achromatic point model comprises: obtaining a training sample set color image; determining a sample achromatic point weight map of the training sample set color image through a pre-constructed network; determining a sample illumination color parameter of the training sample set color image according to the sample achromatic point weight map; determining a sample corrected image of the training sample set color image according to the sample illumination color parameter; performing back propagation on the achromatic point model according to image information of the sample corrected image and the sample achromatic point weight map to optimize parameters in the network, comprising: constructing a loss function for unsupervised training through the following formula: wherein I represents a sample corrected image whose error value is to be calculated; i represents a serial number of a local image block pre-divided in the sample corrected image; n represents a number of dividing the sample corrected image into n local image blocks; p i represents a local contrast of the i-th local image block; L i represents a luminance value of the i-th local image block; t represents a three-channel serial number of the image; SD t represents a local variance of the local image block in the t channel; represents a local mean square deviation of the local image block on three channels. an illumination color determination module configured to determine an illumination color parameter of the image to be corrected according to the achromatic point weight map; a white balance correction module configured to perform white balance correction on the image to be corrected according to the illumination color parameter to obtain a corrected image.

8. An apparatus, comprising: The method comprises: one or more processors; a storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the white balance correction method as claimed in any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the white balance correction method as claimed in any one of claims 1-6.

Images