Portrait lab color correction method and device based on convolutional neural network, medium and equipment

Abstract

The application discloses a portrait LAB color correction method based on a convolutional neural network, and comprises the following steps: constructing a face analysis network, wherein the face analysis network adopts a BiseNet (Bilateral Segmentation Network), and a channel fusion down-sampling structure is added to each pyramid sampling layer of a context path component of the BiseNet; training the face analysis network by using a training sample set to obtain a face analysis model; performing face analysis on a face image by using the face analysis model to obtain face subdivision parts and part identification information; performing LAB color cast detection on preselected face subdivision parts to obtain LAB color casts of each preselected face subdivision part; obtaining a color cast part according to the LAB color casts, and correcting the color cast part; and merging each face subdivision part according to the part identification information to obtain a corrected face image. The application solves the problems of low precision, color correction and low shooting quality of a certificate photo in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, medium and equipment for portrait LAB color correction based on a convolutional neural network. Background Art

[0002] When taking ID photos with ID photo equipment, users often expose areas such as their neck, eyebrows, ears, and face, requiring color correction for these exposed areas. For example, if someone's facial skin is dark and tanned, color correction may be needed to brighten it. Alternatively, if someone's face is heavily made up, the frontal light hitting their face makes their skin brighter, but their ears and neck remain dark, requiring separate color correction for these areas. During facial skin correction, color correction of areas such as the eyes and eyebrows is generally undesirable, as this can easily be mistaken for wearing cosmetic contact lenses, resulting in a failure in the image detection process.

[0003] However, due to the relatively small size of various facial regions and the difficulty in segmentation, existing technologies are unable to accurately detect areas requiring color correction. In the prior art, CN 110796645 A discloses a method for assessing the quality of ID photos, which includes obtaining an adversarial neural network for portrait analysis. The method obtains portrait subregions, including a head region, a clothing region, and a background region, and separately judges the quality of the head region, the clothing region, and the background region, outputting an assessment result. The aforementioned document primarily assesses image quality for regions such as the face, clothing, and background, but does not perform more refined segmentation and color correction on smaller regions such as the eyes, nose, neck, facial skin, and ears. As a result, the quality of ID photos is low, and users need to take multiple photos to obtain a satisfactory ID photo. Summary of the Invention

[0004] The embodiments of the present invention provide a convolutional neural network-based portrait LAB color correction method, device, medium and equipment to solve the problems of accuracy and color correction when cutting the face part of ID photos in the existing technology, as well as the problem of low quality of ID photo shooting.

[0005] A convolutional neural network-based portrait LAB color correction method, the method comprising:

[0006] Constructing a face parsing network, wherein the face parsing network adopts a bilateral segmentation network BiseNet, wherein the bilateral segmentation network BiseNet includes a context path component, and each pyramid sampling layer of the context path component includes a channel fusion downsampling structure;

[0007] Obtaining a preset training sample set, and using the training sample set to train the face parsing network to obtain a face parsing model;

[0008] Acquire a facial image to be corrected, perform facial analysis on the facial image using the trained facial analysis model, and obtain facial segmentation and segment identification information of the facial image;

[0009] Performing LAB color shift detection on pre-selected facial segments to obtain the LAB color shift of each pre-selected facial segment;

[0010] Obtaining a color deviation portion according to the LAB color deviation, and correcting the color deviation portion;

[0011] After correction, each facial segment is merged according to the segment identification information to obtain a corrected face image.

[0012] Optionally, in the channel fusion downsampling structure, after obtaining the input feature map, each pyramid sampling layer downsamples the input feature map through an n-fold downsampling module to obtain a first feature map, and extracts features from the input feature map through a convolution structure to obtain a second feature map, and merges the first feature map and the second feature map to obtain a convolution feature map;

[0013] Among them, the convolution structure adopts a size of The step size is Convolution is performed with the convolution kernel.

[0014] Optionally, obtaining a preset training sample set includes:

[0015] Obtain a preset face image sample set;

[0016] Using a semantic segmentation and annotation tool to perform pixel-level annotation on each face image sample in the face image sample set to obtain each face segment and face annotation information of each face image sample;

[0017] The labeled face image sample set is divided into a training sample set, a test sample set, and a verification sample set according to a preset ratio.

[0018] Optionally, the adopting the training sample set to train the face parsing network to obtain a face parsing model includes:

[0019] Input M1 facial image samples in the training sample set as a batch into the face parsing network, optimize the bilateral neural network loss function of each batch using stochastic gradient descent, and perform backpropagation. Stop the iteration when the loss cost of the face parsing network drops to a preset accuracy.

[0020] When training each batch, M2 face image samples in the verification sample set are input as a batch into the face parsing network for training verification;

[0021] After the training is completed, M3 face image samples in the test sample set are input as a batch into the face parsing network for testing and verification.

[0022] Optionally, performing LAB color shift detection on preselected facial subdivisions to obtain the LAB color shift of each preselected facial subdivision includes:

[0023] Converting the facial image from RGB color space to LAB color space;

[0024] Traverse each pre-selected facial segment and extract the pixel grayscale values ​​of the A channel and B channel in the LAB color space;

[0025] The LAB color shift of each facial segment is calculated using the equivalent circle, where the calculation formula is:

[0026]

[0027] In the above formula, N represents the number of pixels in a facial segment, a represents the grayscale value of the A channel of the i-th pixel, b represents the grayscale value of the B channel of the i-th pixel, and K represents the color shift factor. The larger the K value, the more severe the color shift.

[0028] Optionally, obtaining a color deviation portion according to the LAB color deviation and correcting the color deviation portion includes:

[0029] Obtain the color shift location according to the color shift factor K;

[0030] For each color deviation area, calculate the average LAB grayscale value of all pixels in the color deviation area;

[0031] Subtract the average LAB grayscale value from the compliance mean of the facial skin color to obtain the compensated mean of the color-shifted area;

[0032] Traverse each pixel in the color shift area, add the LAB grayscale value of the pixel to the compensation mean, and obtain the corrected grayscale value of the pixel.

[0033] Optionally, after obtaining the corrected grayscale value of the pixel, the method further includes:

[0034] The α channel is used to combine the LAB grayscale value and the corrected grayscale value of each pixel in the color shift area to obtain the target grayscale value of the pixel.

[0035] Optionally, the face parsing network uses cross entropy as a training loss function.

[0036] A portrait LAB color correction device based on a convolutional neural network, the device comprising:

[0037] A construction module is used to construct a face parsing network, wherein the face parsing network adopts a bilateral segmentation network BiseNet, and the bilateral segmentation network BiseNet includes a context path component, and a channel fusion downsampling structure is added to each pyramid sampling layer of the context path component;

[0038] A training module, configured to obtain a preset training sample set, and train the face parsing network using the training sample set to obtain a face parsing model;

[0039] An analysis module is used to obtain a facial image to be corrected, perform facial analysis on the facial image using the trained facial analysis model, and obtain facial segmentation and segment identification information of the facial image;

[0040] A detection module is used to perform LAB color shift detection on preselected facial segments to obtain the LAB color shift of each preselected facial segment;

[0041] a correction module, configured to obtain a color deviation portion according to the LAB color deviation and correct the color deviation portion;

[0042] The merging module is used to merge the facial subdivisions according to the part identification information after correction to obtain a corrected face image.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned convolutional neural network-based portrait LAB color correction method.

[0044] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the convolutional neural network-based portrait LAB color correction method as described above is implemented.

[0045] The embodiment of the present invention constructs a face parsing network, which uses a bilateral segmentation network (BiseNet) and includes a context path component. Each pyramid sampling layer of the context path component incorporates a channel fusion downsampling structure. A preset training sample set is obtained and the face parsing network is trained using the training sample set to obtain a face parsing model. A face image to be corrected is obtained and the face image is parsed using the trained face parsing model to obtain facial segments and segment identification information of the face image. LAB color shift detection is performed on each facial segment to obtain the LAB color shift of each facial segment. Each facial segment is corrected based on the LAB color shift and the corrected facial segments are merged according to the segment identification information to obtain a corrected face image. The present invention uses a convolutional neural network to perform finer segmentation on ID photos, effectively improving the segmentation accuracy of each facial region in the ID photo, thereby improving the pertinence and accuracy of color correction and the quality of the ID photo. Users can quickly obtain high-quality ID photos. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 This is a flowchart of a portrait LAB color correction method based on a convolutional neural network provided by one embodiment of the present invention;

[0048] Figure 2 is a schematic diagram of a face parsing network provided by an embodiment of the present invention;

[0049] Figure 3 is a schematic diagram of a channel fusion downsampling structure provided by an embodiment of the present invention;

[0050] Figure 4 This is a flowchart for implementing step S104 in the portrait LAB color correction method based on a convolutional neural network provided by one embodiment of the present invention;

[0051] Figure 5 This is a flowchart for implementing step S105 in the portrait LAB color correction method based on a convolutional neural network provided by one embodiment of the present invention;

[0052] Figure 6 1 is a schematic structural diagram of a convolutional neural network-based portrait LAB color correction device provided by one embodiment of the present invention;

[0053] Figure 7 FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] This invention uses a convolutional neural network to perform finer segmentation of ID photos, effectively improving the accuracy of facial segmentation and the targeted nature of color correction. Furthermore, it uses color cast detection to identify color-shifted areas in facial images for correction, increasing the accuracy of color cast correction and ultimately improving the quality of ID photos. This eliminates the need for multiple shots to produce satisfactory ID photos, saving time and improving photo quality.

[0056] The following is a detailed description of the portrait LAB color correction method based on convolutional neural network provided in this embodiment. Figure 1 As shown, the portrait LAB color correction method based on convolutional neural network includes:

[0057] In step S101, a face parsing network is constructed. The face parsing network adopts a bilateral segmentation network BiseNet. The bilateral segmentation network BiseNet includes a context path component. A channel fusion downsampling structure is added to each pyramid sampling layer of the context path component.

[0058] In an embodiment of the present invention, the face parsing network is used to segment facial regions in a face image. Optionally, as a preferred embodiment of the present invention, the face parsing network is derived by improving the bilateral segmentation network BiseNet to accommodate more refined segmentation in the context of ID photos.

[0059] The existing bilateral segmentation network BiseNet includes two components, namely the spatial path component (SpatialPath, referred to as SP) and the context path component (Context Path, referred to as CP). Among them, the spatial path component is used to obtain low-level features of the image to solve the problem of spatial information loss in the deep learning network; the context path component is used to solve the problem of reduced receptive field. In the existing bilateral segmentation network BiseNet, the context path component adopts multiple pyramid sampling layers. However, the pyramid sampling layer will reduce the classification and segmentation accuracy of local pixels when the sampling space loss is large. Therefore, in order to make the pixel classification more refined, the embodiment of the present invention focuses on optimizing the context path component and adding a channel fusion downsampling structure to each pyramid sampling layer. For ease of understanding, Figure 2 Schematic diagram of a face parsing network provided by an embodiment of the present invention. Figure 2 In the context path construction, four pyramid sampling layers are included, namely 4-fold downsampling, 8-fold downsampling, 16-fold downsampling, and 32-fold downsampling. Each pyramid sampling layer is added with a convolution structure to form a channel fusion downsampling structure.

[0060] Optionally, for ease of understanding, Figure 3 This is a schematic diagram of a channel fusion downsampling structure provided by an embodiment of the present invention. In the channel fusion downsampling structure, after obtaining an input feature map, each pyramid sampling layer downsamples the input feature map through an n-fold downsampling module to obtain a first feature map, and extracts features from the input feature map through a convolution structure to obtain a second feature map. The first feature map and the second feature map are merged to obtain a convolution feature map.

[0061] Among them, the convolution structure adopts a size of The step size is Convolution is performed with the convolution kernel of n. n represents the downsampling multiple.

[0062] By adopting the above-mentioned channel fusion downsampling structure, the embodiments of the present invention solve the problem of reduced local pixel segmentation accuracy when the pyramid sampling layer has a large sampling space loss. This effectively improves the segmentation accuracy of local pixels in the pyramid sampling layer, thereby improving the segmentation accuracy of each facial area in the ID photo.

[0063] In step S102, a preset training sample set is obtained, and the face parsing network is trained using the training sample set to obtain a face parsing model.

[0064] Optionally, as a preferred example of the present invention, the step S102 of obtaining a preset training sample set includes:

[0065] In step S201, a preset face image sample set is obtained.

[0066] For example, here, the embodiment of the present invention collects 5 photos of 1,000 volunteers in different postures taken by ID photo shooting equipment, for a total of 5,000 face images, to form a face image sample set.

[0067] In step S202, a semantic segmentation and annotation tool is used to perform pixel-level annotation on each face image sample in the face image sample set to obtain each facial segment and facial annotation information of each face image sample.

[0068] For the 5,000 collected facial images, this embodiment of the present invention performs pixel-level annotation on each facial image sample and segments each facial image sample according to a preset facial segment annotation sequence. The facial segment annotation sequence can be left eye, right eye, nose, upper lip, lower lip, left ear, right ear, left eyebrow, right eyebrow, exposed facial skin, neck, and exposed skin below the neck. The segmented facial image samples are labeled with different colors for each facial segment.

[0069] In step S203, the labeled face image sample set is divided into a training sample set, a test sample set, and a verification sample set according to a preset ratio.

[0070] Continuing with the previous example, after labeling, all 5,000 face images are divided into a training sample set, a test sample set, and a validation sample set according to a preset ratio of 7:2:1. The training sample set is used to train the face parsing network, the test sample set is used to test the face parsing network, and the validation sample set is used to verify the face parsing network.

[0071] After the face parsing network is constructed, the face parsing network is trained. Optionally, as a preferred example of the present invention, the step S102 of training the face parsing network using the training sample set to obtain a face parsing model includes:

[0072] In step S301, L facial image samples in the training sample set are input into the face parsing network as a batch, and the bilateral neural network loss function of each batch is optimized using the stochastic gradient descent method, and backpropagation is performed. The iteration is stopped when the loss cost of the face parsing network drops to a preset accuracy.

[0073] Here, the stochastic gradient descent (SGD) method is used. The embodiment of the present invention uses the SGD optimizer to optimize the bilateral neural network loss function of each batch.

[0074] In step S302, when training each batch, M face image samples in the verification sample set are input as a batch into the face parsing network for training verification.

[0075] In step S303, after the training is completed, the N face image samples in the test sample set are input as a batch into the face parsing network for testing and verification.

[0076] Here, the embodiment of the present invention uses 128 face image samples per batch to iteratively train the face parsing network. After stopping the iteration, the face parsing network is tested with 64 face image samples per batch to observe the accuracy and recall of the test set. After the training is completed, 32 face image samples are used as a batch for testing and verification to observe the generalization of the face parsing network. Optionally, as a preferred example of the present invention, the face parsing network uses cross entropy as the training loss function. It is calculated as follows:

[0077]

[0078] Among them, y is the true label value, is the predicted label value.

[0079] The trained face parsing network is referred to as the face parsing model. This model is used to segment the face image to be corrected into subdivided parts, and then perform LAB color correction on each subdivided part. By using the improved bilateral segmentation network (BiseNet), the accuracy of segmenting each facial region in ID photos is improved.

[0080] In step S103, a facial image to be corrected is obtained, and facial analysis is performed on the facial image using the trained facial analysis model to obtain facial segmentation parts and part identification information of the facial image.

[0081] Here, in this embodiment of the present invention, the facial image to be corrected is input into the trained facial parsing model described above. The facial parsing model is then used to segment and label the facial segments. These segments include, but are not limited to, the left eye, right eye, nose, upper lip, lower lip, left ear, right ear, left eyebrow, right eyebrow, exposed facial skin, neck, and exposed skin below the neck.

[0082] In step S104, LAB color shift detection is performed on the preselected facial subdivision parts to obtain the LAB color shift of each preselected facial subdivision part.

[0083] Optionally, in actual applications, LAB color shift detection is typically performed on several designated facial segments to speed up color shift detection and LAB color correction for ID photos. For example, in one specific embodiment, LAB color shift detection is not performed on the left eye, right eye, upper lip, lower lip, left eyebrow, and right eyebrow. However, LAB color shift detection is performed on the nose, left ear, right ear, exposed facial skin, neck, and exposed skin below the neck. These segments are then pre-selected as facial segments.

[0084] Alternatively, as a preferred example of the present invention, Figure 4 As shown, the LAB color shift detection is performed on the preselected facial subdivision parts in step 104 to obtain the LAB color shift of each preselected facial subdivision part, including:

[0085] In step S401, the face image is converted from the RGB color space to the LAB color space.

[0086] Here, the embodiment of the present invention first performs color conversion on the facial image to be corrected, converting the RGB color space into the LAB color space.

[0087] In step S402, each pre-selected facial segment is traversed to extract the pixel grayscale values ​​of the A channel and the B channel in the LAB color space.

[0088] Through the above steps, a facial image to be corrected corresponds to a set of preselected facial segments. After color space conversion, the embodiment of the present invention performs color extraction on each preselected facial segment, obtaining the grayscale values ​​of each pixel in the preselected facial segment in the A and B channels.

[0089] In step S403, the LAB color shift of each facial segment is calculated using the equivalent circle, where the calculation formula is:

[0090]

[0091] In the above formula, N represents the number of pixels in a facial segment, a represents the grayscale value of the A channel of the i-th pixel, b represents the grayscale value of the B channel of the i-th pixel, and K represents the color shift factor. The larger the K value, the more severe the color shift.

[0092] Here, using the equivalent circle to calculate color shift refers to calculating the ratio of the image's average chromaticity to the chromaticity center distance. This effectively improves the accuracy of color shift detection and produces a more precise LAB color shift. Step S403 above determines the LAB color shift for each preselected facial segment. The color shift factor K serves as the criterion for determining whether a facial segment exhibits color shift, with a critical value of 1. When the K value for a facial segment is greater than 1, it indicates that color shift exists in that segment and requires LAB color correction.

[0093] In step S105 , a color shift portion is obtained according to the LAB color shift, and the color shift portion is corrected.

[0094] Here, this embodiment of the present invention determines whether the color shift factor K for each facial segment is greater than or equal to a threshold value of 1. If so, the facial segment is used as the color shift area to be corrected and LAB color shift correction is performed. If not, LAB color shift correction is not performed.

[0095] Alternatively, as a preferred example of the present invention, Figure 5 As shown, the step S105 of obtaining the color deviation portion according to the LAB color deviation and correcting the color deviation portion includes:

[0096] In step S501 , a color shift location is obtained according to the color shift factor K.

[0097] The color shift portion refers to a face segment where a color shift factor K is greater than or equal to a critical value of 1.

[0098] In step S502 , for each color shift area, the average LAB grayscale value of all pixels in the color shift area is calculated.

[0099] For each color deviation portion, the embodiment of the present invention calculates the average grayscale value of the color deviation portion in the L channel, A channel and B channel based on the grayscale value of each pixel in the color deviation portion in the L channel, A channel and B channel. The calculation formula for the average grayscale value of the color shift part in the L channel is as follows:

[0100]

[0101] in, Indicates the average gray value of the color shift part in the L channel. It represents the gray value of the i-th pixel in the color shift area in the L channel, and N represents the number of pixels in the color shift area.

[0102] The calculation formula for the average grayscale value of the color shift part in channel A is as follows:

[0103]

[0104] in, Indicates the average gray value of the color shift part in channel A. It represents the gray value of the i-th pixel in the color shift area in channel A, and N represents the number of pixels in the color shift area.

[0105] The calculation formula for the average grayscale value of the color shift part in the B channel is as follows:

[0106]

[0107] in, Indicates the average gray value of the color shift part in the B channel. represents the grayscale value of the i-th pixel in the color-shifted area in the B channel, and N represents the number of pixels in the color-shifted area. It should be understood that the number of pixels in the color-shifted area is also the number of pixels in the facial subdivision area, so N is uniformly used in the embodiment of the present invention to represent it.

[0108] In step S503, the average LAB grayscale value is subtracted from the compliance average value of the facial skin color to obtain the compensated average value of the color-shifted portion.

[0109] Here, the compliance mean refers to the average LAB value of the facial skin color in the ID photo. Optionally, as a preferred embodiment of the present invention, the compliance mean can be (70.5, 13.5, 12.0). For each color-shifted area, the average LAB grayscale value of the color-shifted area is subtracted from the compliance mean to obtain the compensated mean value of the color-shifted area.

[0110] In step S504, each pixel in the color-shifted portion is traversed, and the LAB grayscale value of the pixel is added to the compensated mean value to obtain a corrected grayscale value of the pixel.

[0111] For a pixel in the color-shifted area, the embodiment of the present invention uses the compensated mean value to compensate and correct the LAB grayscale value of the pixel. Step S504 is repeated for each pixel, thereby completing the correction of all pixels in the color-shifted area and achieving independent LAB color correction for the color-shifted area.

[0112] Traversing each color deviation area, repeating the above steps S502, S503, and S504, thereby completing LAB color correction for all color deviation areas on the face image to be corrected. The embodiment of the present invention effectively improves the accuracy of LAB color correction by performing independent LAB color correction on each color deviation area.

[0113] Optionally, as another preferred example of the present invention, after obtaining the corrected grayscale value of the pixel in step S504, fusion correction may be performed on the pixels at the color shifted portion. The method further includes:

[0114] The α channel is used to combine the LAB grayscale value and the corrected grayscale value of each pixel in the color shift area to obtain the target grayscale value of the pixel.

[0115] Among them, when using the α channel to merge, the merging formula is as follows:

[0116] g(x)=(1-α)f0(x)+αf1(x)

[0117] Where f0(x) represents the LAB original grayscale value of the pixel, f1(x) represents the corrected grayscale value of the pixel, g(x) represents the target grayscale value of the pixel, and α represents the fusion ratio of the Alpha mask.

[0118] By further fusing and correcting the color deviation areas, it is helpful to improve the accuracy of correction, obtain ID photos that are more consistent with the skin color of normal faces, and improve the quality of ID photo shooting.

[0119] In step S106, after correction, each facial segment is merged according to the segment identification information to obtain a corrected facial image.

[0120] After correcting each color shift area, each corrected color shift area and the facial subdivision areas that do not need correction are merged according to the area identifier to obtain a face image in the LAB color space. The face image is then converted from the LAB color space to the RGB color space to obtain the final corrected face image.

[0121] In summary, this invention utilizes a convolutional neural network to achieve finer segmentation of ID photos, effectively improving the accuracy of segmentation of facial regions within photos and, consequently, enhancing the targeted color correction. Furthermore, this embodiment utilizes color cast detection to select areas with color casts for LAB color correction, effectively increasing the accuracy of LAB color correction and, consequently, improving the quality of ID photos. This eliminates the need for multiple takes to obtain satisfactory ID photos, saving time and improving the quality of the photos.

[0122] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] In one embodiment, the present invention further provides a portrait LAB color correction device based on a convolutional neural network, which corresponds one-to-one to the portrait LAB color correction method based on a convolutional neural network in the above embodiment. Figure 6 As shown, the portrait LAB color correction device based on convolutional neural network includes a construction module 61, a training module 62, a parsing module 63, a detection module 64, a correction module 65, and a merging module 66. The functional modules are described in detail as follows:

[0124] A construction module 61 is used to construct a face parsing network, wherein the face parsing network adopts a bilateral segmentation network BiseNet, and the bilateral segmentation network BiseNet includes a context path component, and a channel fusion downsampling structure is added to each pyramid sampling layer of the context path component;

[0125] A training module 62 is configured to obtain a preset training sample set, and train the face parsing network using the training sample set to obtain a face parsing model;

[0126] The parsing module 63 is used to obtain a facial image to be corrected, perform facial parsing on the facial image using the trained facial parsing model, and obtain facial segmentation and segment identification information of the facial image;

[0127] A detection module 64 is configured to perform LAB color shift detection on preselected facial segments to obtain the LAB color shift of each preselected facial segment;

[0128] A correction module 65 is configured to obtain a color deviation portion according to the LAB color deviation and correct the color deviation portion;

[0129] The merging module 66 is used to merge the facial subdivision parts according to the part identification information after correction to obtain a corrected facial image.

[0130] Optionally, in the channel fusion downsampling structure, after obtaining the input feature map, each pyramid sampling layer downsamples the input feature map through an n-fold downsampling module to obtain a first feature map, and extracts features from the input feature map through a convolution structure to obtain a second feature map, and merges the first feature map and the second feature map to obtain a convolution feature map;

[0131] Among them, the convolution structure adopts a size of The step size is Convolution is performed with the convolution kernel.

[0132] Optionally, the training module 62 includes:

[0133] An acquisition unit, configured to acquire a preset face image sample set;

[0134] a segmentation unit, configured to perform pixel-level annotation on each face image sample in the face image sample set using a semantic segmentation annotation tool, to obtain facial segmentation information and facial annotation information of each face image sample;

[0135] The division unit is used to divide the labeled face image sample set into a training sample set, a test sample set and a verification sample set according to a preset ratio.

[0136] Optionally, the training module 62 further includes:

[0137] A training unit is configured to input M1 facial image samples in a training sample set into the face parsing network as a batch, optimize the bilateral neural network loss function of each batch using a stochastic gradient descent method, and perform backpropagation, and stop iteration when the loss cost of the face parsing network drops to a preset accuracy;

[0138] A verification unit, configured to input M2 face image samples in the verification sample set as a batch into the face parsing network for training verification when training each batch;

[0139] The testing unit is used to input M3 face image samples in the test sample set as a batch into the face parsing network for testing and verification after the training is completed.

[0140] Optionally, the detection module 64 includes:

[0141] A conversion unit, configured to convert the facial image from an RGB color space to a LAB color space;

[0142] An extraction unit, configured to traverse each preselected facial segment and extract the pixel grayscale values ​​of the A channel and the B channel in the LAB color space;

[0143] The calculation unit is used to calculate the LAB color deviation of each facial segment using the equivalent circle, wherein the calculation formula is:

[0144]

[0145] In the above formula, N represents the number of pixels in a facial segment, a represents the grayscale value of the A channel of the i-th pixel, b represents the grayscale value of the B channel of the i-th pixel, and K represents the color shift factor. The larger the K value, the more severe the color shift.

[0146] Optionally, the correction module 65 includes:

[0147] A color shift location acquisition unit, configured to acquire the color shift location according to the color shift factor K;

[0148] an average grayscale value calculation unit, configured to calculate, for each color deviation portion, an average LAB grayscale value of all pixels within the color deviation portion;

[0149] a compensation mean value calculation unit, configured to subtract the average LAB grayscale value from the compliance mean value of the facial skin color to obtain a compensation mean value of the color-shifted portion;

[0150] The first correction unit is used to traverse each pixel in the color shift area, add the LAB grayscale value of the pixel to the compensation mean, and obtain the corrected grayscale value of the pixel.

[0151] Optionally, after obtaining the corrected grayscale value of the pixel, the correction module 65 further includes:

[0152] The second correction unit is used to combine the LAB grayscale value and the correction grayscale value of each pixel in the color shift area by using the α channel to obtain the target grayscale value of the pixel.

[0153] Optionally, the face parsing network uses cross entropy as a training loss function.

[0154] Regarding the specific definition of the portrait LAB color correction device based on a convolutional neural network, please refer to the definition of the portrait LAB color correction method based on a convolutional neural network above, and will not be repeated here. The various modules in the above-mentioned portrait LAB color correction device based on a convolutional neural network can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a portrait LAB color correction method based on a convolutional neural network is implemented.

[0156] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0157] Constructing a face parsing network, wherein the face parsing network adopts a bilateral segmentation network BiseNet, wherein the bilateral segmentation network BiseNet includes a context path component, and each pyramid sampling layer of the context path component includes a channel fusion downsampling structure;

[0158] Obtaining a preset training sample set, and using the training sample set to train the face parsing network to obtain a face parsing model;

[0159] Acquire a facial image to be corrected, perform facial analysis on the facial image using the trained facial analysis model, and obtain facial segmentation and segment identification information of the facial image;

[0160] Performing LAB color shift detection on pre-selected facial segments to obtain the LAB color shift of each pre-selected facial segment;

[0161] Obtaining a color deviation portion according to the LAB color deviation, and correcting the color deviation portion;

[0162] After correction, each facial segment is merged according to the segment identification information to obtain a corrected face image.

[0163] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0164] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0165] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A portrait LAB color correction method based on convolutional neural network, characterized in that: include: A face parsing network is constructed, wherein the face parsing network adopts a bilateral segmentation network BiseNet, and the bilateral segmentation network BiseNet includes a context path component, and a channel fusion downsampling structure is added to each pyramid sampling layer of the context path component; wherein, in the channel fusion downsampling structure, after obtaining an input feature map, each pyramid sampling layer downsamples the input feature map through an n-fold downsampling module to obtain a first feature map, and extracts features from the input feature map through a convolution structure to obtain a second feature map, and merges the first feature map and the second feature map to obtain a convolution feature map; the convolution structure adopts a size of ( ), the step size is Convolution is performed with the convolution kernel, n represents the downsampling multiple; Obtaining a preset training sample set, the training sample set comprising ID photo images annotated with facial subdivisions, the facial subdivisions comprising at least the neck and exposed skin below the neck; training the face parsing network using the training sample set to obtain a face parsing model; Acquire a facial image to be corrected, perform facial analysis on the facial image using the trained facial analysis model, and obtain facial segmentation and segment identification information of the facial image; Performing LAB color shift detection on preselected facial segments to obtain the LAB color shift of each preselected facial segment; wherein performing LAB color shift detection on the preselected facial segments to obtain the LAB color shift of each preselected facial segment includes: converting the facial image from RGB color space to LAB color space; traversing each preselected facial segment and extracting the pixel grayscale values ​​of the A channel and the B channel in the LAB color space; and calculating the LAB color shift of each facial segment using an equivalent circle, wherein the calculation formula is: , where K represents the color shift factor. The larger the K value, the more serious the color shift. Obtaining a color deviation portion based on the LAB color deviation and correcting the color deviation portion; the correction includes: calculating, for each color deviation portion, an average LAB grayscale value of all pixels within the color deviation portion; subtracting the average LAB grayscale value from the compliance mean of the facial skin color to obtain a compensated mean value for the color deviation portion; wherein the compliance mean value is (70.5, 13.5, 12.0); traversing each pixel in the color deviation portion, adding the LAB grayscale value of the pixel to the compensated mean value to obtain a corrected grayscale value for the pixel; After correction, each facial segment is merged according to the segment identification information to obtain a corrected face image.

2. The portrait LAB color correction method based on convolutional neural network according to claim 1, characterized in that: The step of obtaining a preset training sample set includes: Obtain a preset face image sample set; Using a semantic segmentation and annotation tool to perform pixel-level annotation on each face image sample in the face image sample set to obtain each face segment and face annotation information of each face image sample; The labeled face image sample set is divided into a training sample set, a test sample set, and a verification sample set according to a preset ratio.

3. The convolutional neural network-based portrait LAB color correction method according to claim 2, wherein: The face parsing network is trained using the training sample set to obtain a face parsing model, comprising: Input M1 facial image samples in the training sample set as a batch into the face parsing network, optimize the bilateral neural network loss function of each batch using stochastic gradient descent, and perform backpropagation. Stop the iteration when the loss cost of the face parsing network drops to a preset accuracy. When training each batch, M2 face image samples in the verification sample set are input as a batch into the face parsing network for training verification; After the training is completed, M3 face image samples in the test sample set are input as a batch into the face parsing network for testing and verification.

4. The portrait LAB color correction method based on convolutional neural network according to claim 1, characterized in that: After obtaining the corrected grayscale value of the pixel, the method further includes: use The channel combines the LAB grayscale value and the corrected grayscale value of each pixel in the color shift area to obtain the target grayscale value of the pixel.

5. The portrait LAB color correction method based on convolutional neural network according to claim 3, characterized in that: The face parsing network uses cross entropy as the training loss function.

6. A portrait LAB color correction device based on convolutional neural network, characterized in that: include: A construction module is used to construct a face parsing network, wherein the face parsing network adopts a bilateral segmentation network BiseNet, and the bilateral segmentation network BiseNet includes a context path component, and a channel fusion downsampling structure is added to each pyramid sampling layer of the context path component; wherein, in the channel fusion downsampling structure, after obtaining an input feature map, each pyramid sampling layer downsamples the input feature map through an n-fold downsampling module to obtain a first feature map, and extracts features from the input feature map through a convolution structure to obtain a second feature map, and merges the first feature map and the second feature map to obtain a convolution feature map; the convolution structure adopts a size of ( ), the step size is Convolution is performed with the convolution kernel, n represents the downsampling multiple; a training module configured to obtain a preset training sample set, the training sample set comprising ID photograph images annotated with facial subdivisions, the facial subdivisions comprising at least the neck and exposed skin below the neck; and train the face parsing network using the training sample set to obtain a face parsing model; An analysis module is used to obtain a facial image to be corrected, perform facial analysis on the facial image using the trained facial analysis model, and obtain facial segmentation and segment identification information of the facial image; A detection module is configured to perform LAB color shift detection on preselected facial segments to obtain the LAB color shift of each preselected facial segment. The method of performing LAB color shift detection on the preselected facial segments to obtain the LAB color shift of each preselected facial segment includes: converting the facial image from an RGB color space to a LAB color space; traversing each preselected facial segment to extract pixel grayscale values ​​of an A channel and a B channel in the LAB color space; and calculating a color shift factor K using an equivalent circle formula. A correction module is configured to obtain a color deviation portion based on the LAB color deviation and correct the color deviation portion; wherein the correction includes: for each color deviation portion, calculating the average LAB grayscale value of all pixels within the color deviation portion; subtracting the average LAB grayscale value from the compliance mean of the facial skin color to obtain a compensated mean value of the color deviation portion; wherein the compliance mean value is (70.5, 13.5, 12.0); traversing each pixel in the color deviation portion, adding the LAB grayscale value of the pixel to the compensated mean value to obtain a corrected grayscale value of the pixel; The merging module is used to merge the facial subdivisions according to the part identification information after correction to obtain a corrected face image.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the convolutional neural network-based portrait LAB color correction method according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the convolutional neural network-based portrait LAB color correction method according to any one of claims 1 to 5 is implemented.

Images