White balance correction method and system based on eye white region segmentation and color compensation, electronic equipment and storage medium

Abstract

The invention discloses a white balance correction method and system based on eye white region segmentation and color compensation, electronic equipment and a storage medium, and relates to the field of computer vision, image processing and color correction, and the method comprises the steps: obtaining a portrait image of a user, and carrying out the detection of the portrait image through a face detection model obtained through pre-training, cutting out a face image; through a deep learning eye white segmentation model obtained through pre-training, performing eye white feature learning processing on the human face image cut from the human face image pair, and extracting an eye white region; using an RGB mode and a gray mode to sort pixels in the eye white area, taking out a pixel block closest to white according to a preset middle section percentage, and calculating related parameters of RGB three channels; and generating a white balance image of the final original image according to the RGB three-channel related parameters. According to the method, the visual quality of the image is improved, the color authenticity of the face image is enhanced, and the generated image has a more natural and accurate white balance effect.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision, image processing and color correction, in particular to a technology, system, electronic device and storage medium for segmenting white eye regions and correcting white balance based on face images. Background Art

[0002] In many image processing and computer vision applications, face detection and color correction are critical steps, especially in scenarios where color authenticity needs to be ensured. Traditional color correction methods rely on the color information of the entire image, which is difficult to process for local areas alone, and are often affected by ambient lighting and inaccurate white balance settings of the device, resulting in distorted face colors in the image. Summary of the invention

[0003] In order to alleviate or solve at least one aspect or at least one point of the above-mentioned problems, the present invention proposes a white balance correction method based on white-eye area segmentation and color compensation. The method realizes accurate detection and processing of the white-eye area in the face image through a series of image processing steps, and adjusts the white balance based on the extracted white-eye area, thereby improving the color accuracy of the image.

[0004] In a first aspect, the present invention provides a white balance correction method based on white eye area segmentation and color compensation, comprising:

[0005] Acquire a user's portrait image, detect the portrait image using a pre-trained face detection model, and crop a face image; the face detection model is trained based on multiple face images; the face image contains a rectangular facial area including eyes;

[0006] The pre-trained deep learning eye-white segmentation model is used to process the cropped face image with the eye-white feature, and the eye-white area is extracted; the eye-white segmentation model is trained based on the features of the eye-whites of multiple users;

[0007] Use the RGB mode and grayscale mode to sort the pixels in the white area of ​​the eye, extract the pixel block closest to white according to the preset middle segment percentage, and calculate the relevant parameters of the RGB three channels;

[0008] Generate a white balance image of the final original image based on the relevant parameters of the RGB three channels.

[0009] Optionally, the acquiring of the user's portrait image and the detecting of the portrait image by a pre-trained face detection model include:

[0010] It is necessary to judge the area of ​​the input portrait image in order to optimize the detection process;

[0011] The preprocessed portrait image is detected using the face detection model.

[0012] Optionally, the deep learning eye sclera segmentation model obtained through pre-training performs eye sclera feature processing on the cropped face image of the face image, and extracts the eye sclera area including:

[0013] Trained using real-time semantic segmentation technology to accurately extract the white of the eye area.

[0014] Optionally, the method further includes: performing eye white feature processing on the cropped face image by learning the eye white feature of the face image through the pre-trained deep learning eye white segmentation model, extracting the eye white area, and sorting the pixels in the eye white area using the RGB mode and the grayscale mode, taking out the pixel block closest to white according to the preset middle segment percentage, and calculating the RGB three-channel related parameters.

[0015] The segmented white of the eye area is processed by weight change, and the image erosion technology is applied to further optimize the boundary of the segmented area to extract the final white of the eye segmentation area.

[0016] Optionally, performing weight change processing on the segmented white eye region, applying image erosion technology to further optimize the boundary of the segmented region, and extracting the final white eye segmentation region includes:

[0017] Define a condition under which accurate localization of the white area of ​​the eye in the segmentation map is performed;

[0018] Use the image erosion function to erode the mask image.

[0019] Optionally, the RGB three-channel related parameters include: average value;

[0020] Generating a white balance image of the final original image according to the relevant parameters of the RGB three channels includes:

[0021] Calculate the compensation value of the RGB three channels according to the average value of the RGB three channels and the preset target value;

[0022] The RGB three channels of the original portrait image are compensated according to the RGB three-channel compensation values ​​to generate a white balance image of the final original image.

[0023] In a second aspect, the present invention provides a white balance correction system based on white eye area segmentation and color compensation, comprising:

[0024] An acquisition module is used to acquire a user's portrait image, detect the portrait image using a pre-trained face detection model, and crop a face image; the face detection model is trained based on multiple face images; the face image contains a rectangular facial area including eyes;

[0025] An extraction module, configured to process the cropped face image from the face image by learning the features of the white of the eye through a pre-trained deep learning white of the eye segmentation model, and extract the white of the eye area; the white of the eye segmentation model is trained based on the features of the white of the eye of multiple users;

[0026] A calculation module is used to sort the pixels in the white area of ​​the eye using the RGB mode and the grayscale mode, extract the pixel block closest to white according to the preset middle segment percentage, and calculate the relevant parameters of the RGB three channels;

[0027] The generation module is used to generate a white balance image of the final original image according to the relevant parameters of the RGB three channels.

[0028] In a third aspect, the present invention provides an electronic device comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to execute any of the steps of the white balance correction method based on white eye area segmentation and color compensation described above.

[0029] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of any of the above-mentioned white balance correction methods based on white eye area segmentation and color compensation.

[0030] The present invention provides a white balance correction method based on white eye area segmentation and color compensation. After detecting a human face and extracting the white eye area, the average values ​​of the RGB channels of the white eye area are first calculated, for example, the average values ​​of the RGB three channels are 140, 120, and 110. By setting the target RGB three-channel white balance values ​​to 128, 128, and 128, the compensation values ​​of each RGB channel are calculated to be -12, 8, and 18. These compensation values ​​are used to perform color adjustment on the entire face area to ensure that the value of each channel is compensated to be close to the target white balance. The method not only improves the visual quality of the image, but also enhances the color authenticity of the face image, and the generated image has a more natural and accurate white balance effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The present invention is a flowchart of a white balance correction method based on white eye area segmentation and color compensation according to an embodiment of the present invention.

[0032] Figure 2 The present invention provides a flowchart for constructing a white eye region segmentation model according to an embodiment of the present invention.

[0033] Figure 3 This is a basic network diagram of the face detection model according to an embodiment of the present invention.

[0034] Figure 4 Schematic diagram of a triplet loss function according to an embodiment of the present invention.

[0035] Figure 5 This is a diagram of a short-term densely connected network structure according to an embodiment of the present invention.

[0036] Figure 6 Schematic diagram of a simple pyramid pooling module according to an embodiment of the present invention.

[0037] Figure 7 Schematic diagram of a unified attention fusion module according to an embodiment of the present invention.

[0038] Figure 8 This is a white of eye segmentation diagram according to an embodiment of the present invention.

[0039] Fig. 9 This is an eye white corrosion diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention, and should not be construed as a limitation of the present invention. In the present invention, the same reference numerals represent the same or similar components.

[0041] The embodiment of the present invention provides a white balance correction method based on eye white area segmentation and color compensation, which can calculate the compensation value of RGB channels on the basis of detecting a human face and extracting the eye white area, thereby adjusting the color of the entire face area. Figures 1 to 9 As shown, the following steps are included:

[0042] Step S1: Input a portrait image, perform face detection on the portrait image, and crop out a rectangular facial area including the eyes. The face detection uses a face detection model to detect the face area in the portrait image. Portrait images refer to images with a person's face or body as the main subject, usually used to show a person's facial features, posture, expression and other details; face detection refers to the technology of automatically identifying and locating the face area in an image. Using the face detection model to detect the face area is one of the basic tasks in the field of computer vision, which is to determine whether there are faces in the image and the specific locations of these faces in the image; such as Figure 3As shown in the figure, the face detection model network architecture first inputs multiple face image data into the system through a batch input layer, and then extracts features through a convolutional neural network. In the convolutional neural network, high-level features of the face are gradually extracted through multiple layers of convolution, pooling, and nonlinear activation operations. After these features are processed at multiple levels by the deep network, a set of high-dimensional feature vectors are formed. In order to ensure that the feature vectors have a uniform scale, the network applies an L2 normalization operation after the output of the convolutional neural network. This process normalizes the modulus of the feature vector to 1, thereby obtaining a face embedding with a fixed length and scale. This normalized embedding vector facilitates similarity measurement in subsequent training and matching processes. In the training phase, Figure 4 As shown in the figure, the network uses the triplet loss function as the objective function to optimize the model's feature learning ability. The triplet loss minimizes the distance between the embeddings of the same face and increases the distance between the embeddings of different faces, so that the model can distinguish different individuals more accurately. The design of this loss function ensures that in the high-dimensional feature space, the embeddings of faces from the same person are close to each other, while the embeddings of faces from different people are far away from each other, thereby improving the accuracy and robustness of face recognition.

[0043] Step S11: Before performing face detection, the area of ​​the input image needs to be determined in order to optimize the detection process. Specifically, the resolution of the input image, i.e., the width and height, is first obtained, and the actual area of ​​the image is calculated, and a preset area value is set to determine whether the image needs to be reduced. If the image area exceeds the preset area value, it means that the image resolution is high. In order to reduce computing resource usage and increase detection speed, the image needs to be scaled, and the image needs to be scaled by calculating the multiple that needs to be scaled.

[0044] Step S12: Use face detection on the preprocessed image. When a face area is detected in the preprocessed image, a rectangular frame coordinate will be output. For the detected face, face detection will return the coordinates of the upper left corner and the lower right corner of the rectangular frame. If the preprocessed image is scaled, the detected face coordinates are based on the scale of the scaled image. When cropping the original image, these coordinates need to be enlarged according to the scaling ratio to obtain the rectangular frame coordinates of the face area in the original image. A rectangular area containing the face is cropped out of the portrait image based on the coordinate information. In this way, the cropped area obtained maintains the original resolution, reduces the face detection speed, and does not require repeated detection steps.

[0045] Step S2: Based on the pre-trained deep learning eye sclerotin segmentation model, the cropped face image is processed to extract the eye sclerotin area. The eye sclerotin segmentation model is a computer vision model that aims to automatically identify and extract the white part of the eye in an image. It learns the features of the eye sclerotin by training with annotated images so as to accurately segment the area in a new image. Figure 2 A flowchart illustrating a method for segmenting the white eye region of a face image based on a white eye segmentation model according to an example embodiment of the present invention:

[0046] Step S21: Collection of portrait data sets and preprocessing of data sets. Portrait images contain complete face areas. Use face detection to process portrait image data, crop the area containing the face from the original image according to the output face area rectangular coordinates, and organize the cropped face image. In the cropped face area, use the image annotation tool to mark the white area of ​​the face as white and other areas as black. This step ensures that the model can learn the characteristics of the white of the eye and save the annotation results. Process the annotated image, convert the non-black pixel values ​​in the annotated image to 1, traverse the annotated image pixels, set all non-black pixel values ​​to 1, and the final generated mask image will be used to train the model. The cropped face images and the corresponding annotated images are organized into a training data set. In order to evaluate the performance of the model, the data set is divided into a training set and a validation set, divided in a ratio of 8:2 to ensure that the training set contains enough samples to improve the generalization ability of the model.

[0047] Step S22: Building a white-of-the-eye segmentation model. Utilizing the characteristics of annotation accuracy and consistency of the training data set of the present invention, the white-of-the-eye segmentation model is trained using real-time semantic segmentation technology to accurately extract the white-of-the-eye area. By adopting a real-time semantic segmentation model, real-time processing can be achieved while ensuring accuracy. During the training process, the model learns based on a large amount of annotated data, so that it can accurately identify and segment the boundaries of the white-of-the-eye area and adapt to different individuals and various eye postures. This design not only improves the efficiency of segmentation, but also ensures the fluency and reliability of the model in practical applications. Specifically as follows:

[0048] Real-time semantic segmentation is a computer vision technology that aims to classify images pixel by pixel while maintaining sufficient processing speed for real-time applications. Semantic segmentation refers to the classification of each pixel in an image into a certain class of objects and the generation of a corresponding segmented image. Real-time means that the segmentation processing speed of the model is fast enough to respond in milliseconds.

[0049] The real-time semantic segmentation model is a lightweight semantic segmentation model based on convolutional neural networks, which aims to achieve real-time segmentation through a streamlined network structure and an innovative combination of modules. The structure of the lightweight semantic segmentation model includes an encoder, a decoder, and a segmentation head. The lightweight semantic segmentation model adopts a simple pyramid pooling module to aggregate global contextual information at a low computational cost, thereby improving the model's perception of the environmental background and enhancing the segmentation performance. At the same time, a flexible and lightweight decoder is used to reduce the computational burden of traditional decoders and improve reasoning efficiency. In addition, the lightweight semantic segmentation model also integrates a unified attention fusion module as a key component for enhanced feature representation. This module uses spatial and channel attention mechanisms to generate weights and fuses input features with these weights to help the model capture key semantic information more effectively. Specifically as follows:

[0050] The encoder uses the backbone network as the basic backbone network to extract high-level semantic features from the input image. Through a series of convolutional layers, pooling layers, and activation functions, the spatial dimensions [H, W] of the image are gradually reduced, and the semantic information in the image is enhanced, that is, the number of feature map channels increases. The encoder's task is to convert the original image into a low-resolution feature map with rich semantic information, including information such as the object, texture, and structure of the image.

[0051] The backbone network is a short-term densely connected network, a lightweight convolutional neural network that aims to improve the model reasoning speed while ensuring the effectiveness of feature extraction. Figure 5 As shown in Figure 1, the backbone network is divided into 6 stages, of which the first 5 stages are mainly used as the backbone for feature extraction and segmentation. The output feature map of each stage gradually decreases in size, while the number of channels gradually increases. The size of the output feature map of the i-th stage is H×W of the original input. The number of channels of the feature map increases with the number of stages, and the number of channels in the i-th stage is 16×2 i. The number of channels is gradually increased in each stage to enhance the network's ability to represent features. When the number of channels of the output feature map of the second stage is greater than 64, a "final convolution layer" is added to the output of the fifth stage. The function of this layer is to ensure that the number of channels of the feature map output by the fifth stage is not less than 1024, so as to better perform subsequent processing. A short dense connection module is used inside each stage. The core design of this module is to achieve more effective information transfer through dense connections. The receptive field of the low-level convolution in the module is small, so more channels are required to extract detailed information. The receptive field of the high-level convolution is larger, and only a small number of channels are required to extract sufficient semantic information. A detailed guidance training branch is added. During training, the loss is calculated for the feature maps output by the previous stages to improve the ability of the low-level convolution to extract image details. During inference, the feature map of the low-level convolution is directly fused with the high-level convolution feature map containing more semantic information. The calculation process is represented by expression (1):

[0052] L detail (p d ,g d )=L dice (p d ,g d )+L bce (p d ,g d )

[0053]

[0054] L detail represents the detail loss function; p d Represents the detailed information of each pixel or region in the predicted distribution or feature map; g d represents the true label or target value; L dice represents the Dice loss function; L bce represents the binary cross entropy loss; Represents the predicted value of the i-th pixel position; The true label value of the i-th pixel position; d represents the detail layer; H×W represents the resolution of the feature map; ε represents the smoothing term to prevent the denominator from being zero;

[0055] The decoder is the part corresponding to the encoder, which is responsible for restoring the low-resolution feature map generated by the encoder to the resolution of the original image and performing pixel-level classification. The decoder usually includes upsampling layers and convolution layers to gradually restore the details of the feature map and generate a feature map of the same size as the input image. The decoder maps the semantic features extracted by the encoder to the pixel level in order to classify each pixel.

[0056] The segmentation head is a fully connected layer or convolutional layer that transforms the feature map output by the decoder into the final segmentation structure.

[0057] Simple pyramid pooling module such as Figure 6 As shown in the figure, the pyramid pooling module is used to fuse the input features. The pyramid pooling module contains three global average pooling operations, and the corresponding pooling kernel sizes are 1x1, 2x2, and 4x4 respectively. The output features undergo 1x1 convolution and upsampling operations. These upsampled features are added together, and a 3x3 convolution operation is applied to generate fine features.

[0058] The flexible lightweight decoder is a module for generating high-quality output feature maps. It aims to accelerate the reasoning of the model by reducing the amount of computation and parameters, while being able to adapt to a variety of input conditions and task requirements. The main function of the flexible lightweight decoder is to gradually increase the spatial size [H, W] of the feature map while gradually reducing the number of feature channels during the operation of the decoder, balancing the computational complexity of the encoder and decoder, making the overall model more efficient.

[0059] Unified attention fusion module such as Figure 7 As shown in Figure 1, it is a structure for fusing multiple feature maps. It uses the spatial and channel attention mechanism to generate a weight matrix α, and multiplies and fuses the input features with the weight matrix to obtain the fused features. This enables the model to better capture the key semantic information in the image and improve the segmentation accuracy. Expression (2) represents its calculation process:

[0060]

[0061] F high represents the output of the deep module; F low Indicates the corresponding part of the encoder; F up It is represented by the upsampled feature; α represents the attention weight coefficient; Upsample(F high ) represents the high-resolution feature map F high The upsampling operation; (1-α) reverse attention weight represents the low-resolution feature map F during the fusion process low The importance weight of .

[0062] The spatial attention module uses the spatial relationship between pixels to generate a weight that represents the importance of each pixel in the input feature.

[0063] The channel attention module uses the relationship between channels to generate weights, which indicate the importance of each channel in the input features. The channel attention module uses average pooling and maximum pooling operations to compress the spatial dimensions of the input features.

[0064] Step S23: Convolutional neural network parameter analysis. The present invention uses a method for convolutional network parameter analysis through a sclera segmentation model. The method evaluates the size and computational efficiency of the overall model by calculating the parameter amount and floating-point operation amount of each layer. In addition, the sorted face images and annotated pictures are used as training data, combined with optimized hyperparameters such as learning rate and batch size to accelerate model convergence, and a dynamic learning rate adjustment strategy is adopted to adapt to the training stage. Combined with data enhancement techniques such as random cropping, rotation and flipping, the diversity of training data is increased and the generalization ability of the model is improved. Through the above convolutional network parameter analysis method, the present invention can effectively optimize the performance of the sclera segmentation model in the sclera segmentation task, reduce computational complexity, and improve segmentation accuracy, thereby achieving real-time and efficient sclera segmentation.

[0065] Step S24: training of the sclera segmentation model. The cropped face image is input into the trained sclera segmentation model. The model will use its convolutional neural network structure to perform forward inference on the image and use multiple convolutional layers to extract multi-scale features of the image, highlighting the details of the edges, i.e. the sclera area. The sclera segmentation model reduces the resolution of the feature map by downsampling to obtain a wider range of contextual information, and then restores it to the resolution of the original image by upsampling to accurately locate the sclera area. After a series of convolution, downsampling, and upsampling operations, the sclera segmentation model outputs a segmentation map of the same size as the input image. The value of each pixel indicates the probability that the position belongs to the sclera area. The probability map is converted into a binary segmentation map by a threshold, and the final generated sclera segmentation map is as follows: Figure 8 shown.

[0066] Step S3: Perform weight change processing on the segmented white eye area, apply image erosion technology to further optimize the boundary of the segmented area, and extract the final white eye segmentation area.

[0067] Step S31: Define a condition for accurately locating the pixels representing the white of the eye area in the segmentation map. This condition is usually achieved by comparing the color channel values ​​of each pixel. For example, if we know that the white of the eye area is represented by a specific color value in the segmentation map, we can filter the pixels in the image by setting such a condition. Using the conditional selection function, find the pixel coordinates that meet the condition according to the previously defined condition. The function returns the indexes of all pixels that meet the condition. These indexes will provide us with the specific location of the white of the eye area in the image. Based on these coordinates, create a Boolean mask to mark the white of the eye area in the segmentation map. Initialize an all-zero array of the same size as the segmentation map, and use the pixel coordinates found previously to set the value of the corresponding position to 255 to mark the white of the eye area. The conditional selection function is a function used to select or find array elements based on conditions.

[0068] Step S32: Using the image erosion function to perform an erosion operation on the mask image is an important step in image processing. Its main purpose is to optimize the boundary of the white area of ​​the eye so as to more accurately determine that the segmentation range is within the white area of ​​the eye. Fig. 9 As shown. The erosion operation affects the edge of the image, making the boundary of the white area of ​​the eye smoother and avoiding unnecessary noise. The erosion operation is performed by moving a structuring element on the mask image. The size and shape of this structuring element are set according to the specific needs of the image. For example, selecting a 3x3 square can shrink the edge of the target area inward, thereby streamlining the boundary. This shrinking effect can effectively remove small noise and irregular boundaries in the image, ensuring that the final segmented area is clearer. During the erosion operation, the value of each pixel depends on the minimum value of its surrounding pixels. If one or more pixels in the area covered by the kernel have a value of 0 (black), the pixel value after erosion will also become 0. This feature shrinks the edges of the white area in the image, thereby effectively removing those tiny, irregular noises, and ultimately ensuring that the obtained white area has good coherence and integrity.

[0069] Step S33: After the erosion operation, detect the pixel values ​​that meet the conditions in the mask, convert the eroded mask image into a grayscale image, and then use the conditional selection function to find the grayscale value in the added white area pixel position. Extract the pixel value corresponding to the white area and convert it to a floating point type for distance calculation, define the ideal white pixel value, calculate the distance between each extracted pixel value and the ideal white value, and obtain the color difference distance of each pixel through the matrix norm function. The matrix norm function is used to calculate the size or length of the matrix. The size of the matrix is ​​measured by different types of norms.

[0070] Step S4: Use the RGB mode and the grayscale mode to sort the pixels in the white area of ​​the eye, take out the pixel block closest to white according to the preset middle segment percentage, and calculate the average value of the RGB three channels.

[0071] Step S41: In RGB mode, we extract all pixel values ​​in the white area of ​​the eye and then sort them by the values ​​of the color channels (red, green, and blue). Subsequently, in grayscale mode, a weighted average method is usually used to further filter and sort the pixels. Weighted average is to multiply a set of values ​​by the corresponding weights, then sum them, and then divide the result by the sum of all weights. As shown in expression (3):

[0072]

[0073] x i represents the i-th data point; w i represents the weight of the i-th data; n is the total number of data points;

[0074] Step S42: Set a preset middle segment percentage. This percentage is used to determine the range of color difference distances in order to filter out the pixels closest to the ideal white pixel value. The color difference distance between each extracted pixel value and the ideal white pixel value is calculated through the previous processing steps. The threshold of the distance is calculated using the statistical function to filter out the pixels closest to white. For a deeper analysis, we set two additional thresholds, which are used to filter out pixels within a certain range of distances, that is, pixels in the middle range in color. The statistical function is also used to calculate these two thresholds. With these thresholds, we can create a Boolean array to filter out pixels that meet the conditions. First, filter out all pixels with a color difference distance less than or equal to, which are our pixels closest to white. Filter out those pixels whose color difference distance is between the two additional thresholds, which are in the middle range in color. It can accurately find the pixels closest to white and extract the middle pixels with color differences within a certain range. The statistical function is used to calculate the specified percentile of a given array.

[0075] Step S43: averaging the red, green and blue channel values ​​of the selected pixel respectively to obtain the average color value of the pixel closest to white and the middle pixel area with color difference within a certain range.

[0076] Step S5: Calculate the RGB three-channel compensation value based on the RGB three-channel average value and the preset target value, aiming to improve the overall color performance of the image by adjusting the color channel of the image to make it closer to the target color.

[0077] Step S51: define a preset target value, which usually represents an ideal color characteristic. By averaging the RGB channel values ​​of the selected pixel, we can get the average color value of the current image area. In order to make the average color value of the image closer to the preset target value, we need to calculate the compensation value. The compensation value can be obtained by subtracting the target value from the average value, and the compensation value is equal to the target value minus the average value.

[0078] Step S6: Compensate the RGB channels of the original portrait image according to the RGB channel compensation values ​​to generate a white balance image of the final original image. A white balance image is an image that is processed so that the white and gray parts of the image appear to be truly white and gray visually. White balance is a color correction process in image processing. By adjusting the relative intensity of the red, green, and blue (RGB) channels of the image, the color temperature effect of the light source is eliminated so that white objects in the image appear to be truly white. The goal is to make images taken under different light sources consistent in color.

[0079] Step S61: After obtaining the compensation value, we need to apply it to the RGB channel of the image to adjust the color of the image. This process involves adjusting the image pixel by pixel. We can add the compensation value to each pixel value of the original image and ensure that the adjusted value does not exceed 255 (the maximum value of the RGB color channel) to avoid overflow. The limit function needs to be used to ensure the legitimacy of the value. After the adjustment is completed, recalculate the average RGB value of the adjusted pixel to verify whether it is closer to the preset target value. By calculating the color value of the adjusted image, it can be ensured that the compensation is effective. The adjusted image is the final white balance image. The limit function is used to limit the values ​​in the array so that all elements in the array are limited between the specified minimum and maximum values.

[0080] The present invention provides a white balance correction method based on eye white area segmentation and color compensation. After detecting a human face and extracting an eye white area, firstly, the RGB channel average values ​​of the eye white area are calculated, for example, the average values ​​of the RGB three channels are 140, 120, and 110. By setting the target RGB three-channel white balance values ​​to 128, 128, and 128, the compensation values ​​of each RGB channel are calculated to be -12, 8, and 18. These compensation values ​​are used to perform color adjustment on the entire face area to ensure that the value of each channel is compensated to be close to the target white balance. The method not only improves the visual quality of the image, but also enhances the color authenticity of the face image, and the generated image has a more natural and accurate white balance effect.

[0081] The present invention provides a white balance correction system based on white eye area segmentation and color compensation, comprising:

[0082] The acquisition module is used to acquire the user's portrait image, detect the portrait image through a pre-trained face detection model, and crop the face image; the face detection model is trained based on multiple face images; the face image contains a rectangular facial area including the eyes;

[0083] An extraction module is used to process the cropped face image with the learned eye sclera features by using a pre-trained deep learning eye sclera segmentation model, and extract the eye sclera area; the eye sclera segmentation model is trained based on the eye sclera features of multiple users;

[0084] A calculation module is used to sort the pixels in the white area of ​​the eye using the RGB mode and the grayscale mode, extract the pixel block closest to white according to the preset middle segment percentage, and calculate the relevant parameters of the RGB three channels;

[0085] The generation module is used to generate a white balance image of the final original image according to the relevant parameters of the RGB three channels.

[0086] The present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to execute any step of the above-mentioned white balance correction method based on white eye area segmentation and color compensation.

[0087] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes any of the steps of the above-mentioned white balance correction method based on white eye area segmentation and color compensation.

[0088] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes may be made to the embodiments and combinations of elements may be made without departing from the principles and spirit of the invention, the scope of the invention being defined by the appended claims and their equivalents.

Claims

1. A white balance correction method based on white eye area segmentation and color compensation, characterized in that: include: Acquire a user's portrait image, detect the portrait image using a pre-trained face detection model, and crop a face image; the face detection model is trained based on multiple face images; the face image contains a rectangular facial area including eyes; Using a pre-trained deep learning eye-white segmentation model, the cropped face image is subjected to eye-white feature learning processing to extract the eye-white area; the eye-white segmentation model is trained based on the eye-white features of multiple users; Use the RGB mode and grayscale mode to sort the pixels in the white area of ​​the eye, extract the pixel block closest to white according to the preset middle segment percentage, and calculate the relevant parameters of the RGB three channels; Generate a white balance image of the final original image based on the relevant parameters of the RGB three channels.

2. The white balance correction method based on white eye area segmentation and color compensation according to claim 1, characterized in that: The acquiring of the user's portrait image and the detection of the portrait image by a pre-trained face detection model include: It is necessary to judge the area of ​​the input portrait image in order to optimize the detection process; The preprocessed portrait image is detected using the face detection model.

3. The white balance correction method based on white eye area segmentation and color compensation according to claim 1, characterized in that: The deep learning eye sclera segmentation model obtained by pre-training performs eye sclera feature processing on the cropped face image of the face image, and extracts the eye sclera area including: Trained using real-time semantic segmentation technology to accurately extract the white of the eye area.

4. The white balance correction method based on white eye area segmentation and color compensation according to claim 1, characterized in that: The method further includes: performing eye white feature processing on the cropped face image by using the pre-trained deep learning eye white segmentation model, extracting the eye white area, sorting the pixels in the eye white area using the RGB mode and the grayscale mode, taking out the pixel block closest to white according to the preset middle segment percentage, and calculating the RGB three-channel related parameters. The segmented white of the eye area is processed by weight change, and the image erosion technology is applied to further optimize the boundary of the segmented area to extract the final white of the eye segmentation area.

5. The white balance correction method based on white eye area segmentation and color compensation according to claim 4, characterized in that: The weight change processing is performed on the segmented white eye area, and the image erosion technology is applied to further optimize the boundary of the segmented area, and the final white eye segmentation area is extracted, which includes: Define a condition under which accurate localization of the white area of ​​the eye in the segmentation map is performed; Use the image erosion function to erode the mask image.

6. The white balance correction method based on white eye area segmentation and color compensation according to claim 1, characterized in that: The RGB three-channel related parameters include: average value; Generating a white balance image of the final original image according to the relevant parameters of the RGB three channels includes: Calculate the compensation value of the RGB three channels according to the average value of the RGB three channels and the preset target value; The RGB three channels of the original portrait image are compensated according to the RGB three-channel compensation values ​​to generate a white balance image of the final original image.

7. A white balance correction system based on white eye area segmentation and color compensation, characterized in that: include: An acquisition module is used to acquire a user's portrait image, detect the portrait image using a pre-trained face detection model, and crop a face image; the face detection model is trained based on multiple face images; the face image contains a rectangular facial area including eyes; An extraction module, used to process the cropped face image from the face image by learning the features of the white of the eye through a pre-trained deep learning white of the eye segmentation model, and extract the white of the eye area; the white of the eye segmentation model is trained based on the features of the white of the eye of multiple users; A calculation module is used to sort the pixels in the white area of ​​the eye using the RGB mode and the grayscale mode, extract the pixel block closest to white according to the preset middle segment percentage, and calculate the relevant parameters of the RGB three channels; The generation module is used to generate a white balance image of the final original image according to the relevant parameters of the RGB three channels.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to execute the steps of the white balance correction method based on white eye area segmentation and color compensation as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the white balance correction method based on white eye area segmentation and color compensation as claimed in any one of claims 1 to 6.

Images