Face image processing method and device, computer device and storage medium

Abstract

The present disclosure provides a face image processing method and device, computer equipment and a storage medium, wherein the method comprises: obtaining a face image to be processed; adjusting the face image to be processed to generate a reference face image; determining target region information in the reference face image; fusing a target region image in the reference face image matching the target region information and the face image to be processed to generate a target face image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to a method, apparatus, computer device, and storage medium for facial image processing. Background Technology

[0002] With the development of artificial intelligence (AI) technology, neural networks are widely used in image processing scenarios, such as AI beautification. AI beautification generates beautified images by performing enhancement and makeup processing on existing images.

[0003] With an increasing number of user images showing high hairlines, hairline enhancement has become a demand in image beautification. Therefore, it is particularly important to propose a facial image processing method that meets this requirement. Summary of the Invention

[0004] This disclosure provides at least one facial image processing method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a facial image processing method, including:

[0006] Obtain the face image to be processed;

[0007] The face image to be processed is adjusted to generate a reference face image;

[0008] Determine the target region information in the reference face image;

[0009] The target region image that matches the target region information in the reference face image is fused with the face image to be processed to generate the target face image.

[0010] In one optional implementation, determining the target region information in the reference face image includes:

[0011] The reference face image and the face image to be processed are respectively subjected to face region segmentation processing to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different;

[0012] Based on the first segmented image and the second segmented image, the target region information of the area where the hairline is located in the reference face image is determined.

[0013] In one optional implementation, determining the target region information of the area where the hairline is located in the reference face image based on the first segmented image and the second segmented image includes:

[0014] The pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image are adjusted to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location;

[0015] Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located;

[0016] Based on the deviation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0017] In one optional implementation, determining the target region information of the area where the hairline is located in the reference facial image based on the deviation image includes:

[0018] The area where the hairline is located in the deviation image is dilated to generate a processed deviation image;

[0019] Generate a face mask image based on the first segmented image;

[0020] A hairline segmentation image is generated based on the processed deviation image and the face mask image;

[0021] Based on the hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0022] In one optional implementation, after generating the hairline segmentation image, the method further includes:

[0023] Based on the region containing the facial features in the first segmented image, a target image region containing the region containing the facial features is determined, and a processed first segmented image is generated based on the target image region; in the processed first segmented image, the pixel value at the pixel position corresponding to the target image region is zero.

[0024] An adjusted hairline segmentation image is generated based on the processed first segmentation image and the hairline segmentation image.

[0025] The step of determining the target region information of the area where the hairline is located in the reference face image based on the hairline segmentation image includes:

[0026] Based on the adjusted hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0027] In an optional implementation, before fusing the target region image in the reference face image that matches the target region information with the face image to be processed, the method further includes:

[0028] Determine the hair color information in the face image to be processed;

[0029] Based on the hair color information, the hair color of the reference face image is adjusted to generate an adjusted reference face image;

[0030] The step of fusing the target region image in the reference face image that matches the target region information with the face image to be processed to generate a target face image includes:

[0031] The target region image that matches the target region information in the adjusted reference face image is fused with the face image to be processed to generate the target face image.

[0032] In one optional implementation, adjusting the face image to be processed to generate a reference face image includes: adjusting the face image to be processed using a trained target neural network to generate a reference face image.

[0033] The target neural network is trained according to the following steps:

[0034] Multiple candidate face image pairs are obtained, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image;

[0035] For each candidate face image pair, a first reconstructed image of the first candidate face image and a second reconstructed image of the second candidate face image are determined in the candidate face image pair; and the region image of the area where the hairline is located in the second reconstructed image is fused with the first reconstructed image to generate a third reconstructed image, and the first reconstructed image and the third reconstructed image are determined as a reconstructed image pair;

[0036] Each of the reconstructed image pairs is determined as a training sample;

[0037] Using the training samples, the neural network to be trained is trained to obtain the target neural network.

[0038] In one optional implementation, determining the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair includes:

[0039] Determine the first noise data of the first candidate face image and the second noise data of the second candidate face image;

[0040] The second reconstructed image is generated based on the second noise data;

[0041] And, based on the first noise data, generate the first reconstructed image; or, determine the noise difference data between the second noise data and the first noise data, perform subtraction on the first noise data and the noise difference data to obtain third noise data, and use the third noise data to generate the first reconstructed image.

[0042] Secondly, embodiments of this disclosure also provide a facial image processing apparatus, comprising:

[0043] The acquisition module is used to acquire the face image to be processed;

[0044] The first generation module is used to adjust the face image to be processed and generate a reference face image;

[0045] A determining module is used to determine target region information in the reference face image;

[0046] The second generation module is used to fuse the target region image that matches the target region information in the reference face image with the face image to be processed to generate a target face image.

[0047] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0048] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation of the first aspect.

[0049] The facial image processing method, apparatus, computer device, and storage medium disclosed herein provide embodiments that adjust an acquired facial image to be processed using a target neural network to generate a reference facial image. Considering that the reference facial image differs from the facial image to be processed not only in the region information of specific areas (such as the hairline), but also in other region information such as background information, the user's facial features, skin color, and hair color, resulting in poor display effects of the reference facial image, this disclosure addresses this issue by determining target region information in the reference facial image, such as the target region information of the hairline area, and fusing the target region image matching the target region information in the reference facial image with the facial image to be processed to generate a target facial image. The target facial image is an image with the hairline adjusted. This ensures that the images of other regions in the target facial image, except for the hairline area, are identical to the facial image to be processed, while simultaneously achieving hairline adjustment, improving the hairline adjustment effect, and resulting in better display effects of the target facial image.

[0050] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0052] Figure 1 A flowchart of a facial image processing method provided by an embodiment of this disclosure is shown;

[0053] Figure 2 This illustration shows a schematic diagram of fusing a reference face image and a face image to be processed in the face image processing method provided in this embodiment of the present disclosure;

[0054] Figure 3 The illustration shows a second reconstructed image and a first reconstructed image generated using two methods in the face image processing method provided in this embodiment of the present disclosure;

[0055] Figure 4 A schematic diagram of a facial image processing apparatus provided in an embodiment of the present disclosure is shown;

[0056] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0058] With an increasing number of user images showing high hairlines, hairline enhancement has become a demand in image beautification. Based on this, this disclosure provides a facial image processing method that uses a target neural network to adjust the acquired facial image to generate a reference facial image, such as adjusting specific features like the hairline. Considering that the reference face image differs from the face image to be processed not only in the area where specific parts (such as the hairline) are located, but also in other areas such as background information, the user's facial features, skin color, and hair color, resulting in poor display of the reference face image, this disclosure addresses this issue by determining target area information in the reference face image, such as the area where the hairline is located. The target area image matching the target area information in the reference face image is then fused with the face image to be processed to generate a target face image. The target face image is the image with the adjusted hairline. This ensures that the images of other areas in the target face image, except for specific parts such as the hairline, are identical to the face image to be processed, while simultaneously achieving adjustment of specific parts, improving the hairline adjustment effect, and resulting in a better display of the target face image.

[0059] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0060] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0061] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0062] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0063] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0064] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0065] To facilitate understanding of this embodiment, a facial image processing method disclosed in this disclosure will first be described in detail. The subject executing this facial image processing method is generally a computer device with certain computing power.

[0066] The following describes the facial image processing method provided in this disclosure embodiment, taking the server as the execution subject as an example.

[0067] See Figure 1 The diagram shows a flowchart of a facial image processing method provided in this embodiment of the present disclosure. The method includes steps S101 to S104, wherein:

[0068] S101, Obtain the face image to be processed.

[0069] S102, adjust the face image to be processed to generate a reference face image.

[0070] S103, determine the target region information in the reference face image.

[0071] S104, the target region image that matches the target region information in the reference face image is fused with the face image to be processed to generate a target face image.

[0072] The following provides a detailed explanation of S101-S104.

[0073] For S101 and S102:

[0074] The face image to be processed can be any user's face image. In implementation, in response to an adjustment operation, the face image to be processed can be adjusted to generate a reference face image. Alternatively, a trained target neural network can be used to adjust the face image to be processed and generate a reference face image. That is, the acquired face image to be processed is input into the target neural network, which adjusts the face image to generate a reference face image. For example, the hairline of the face image to be processed can be adjusted, then the reference face image can be an image with the hairline supplemented (i.e., the hairline lowered). The target neural network is a trained network used for hairline adjustment. The network structure of the target neural network can be set as needed; for example, the target neural network can be a pix2pix network.

[0075] Regarding S103:

[0076] For example, in response to manual annotation, the area where the hairline is located can be annotated in a reference face image. Based on the annotation results, target region information in the reference face image, such as the target region information of the hairline area, can then be determined. Alternatively, the position of the eyebrows in the reference face image can be determined. Using this position as a reference, an annotation area in the reference face image can be determined according to a preset shape and size. This annotation area can then be identified as the hairline area, thereby obtaining the target region information of the hairline area in the reference face image. This target region information can be the positional information of the hairline area in the reference face image.

[0077] In one optional implementation, determining the target region information in the reference face image includes:

[0078] Step a1: Perform face region segmentation processing on the reference face image and the face image to be processed respectively to generate a first segmented image corresponding to the reference face image and a second segmented image corresponding to the face image to be processed; wherein, in the first segmented image and the second segmented image, the pixel values ​​corresponding to different semantic regions are different.

[0079] Step a2: Based on the first segmented image and the second segmented image, determine the target region information of the area where the hairline is located in the reference face image.

[0080] In implementation, the reference face image can be segmented to generate a first segmented image. For example, a face parsing tool can be used to segment the reference face image to generate the first segmented image. Alternatively, a face segmentation neural network can be used to segment the reference face image to generate the first segmented image. Similarly, the face image to be processed can be segmented in the same way to generate a second segmented image. In the first and second segmented images, the pixel values ​​corresponding to different semantic regions are different. The image size of the first segmented image can be the same as the image size of the reference face image, and the image size of the second segmented image can be the same as the image size of the face image to be processed.

[0081] For example, in the first segmented image, the pixel value corresponding to the eyebrow area can be s1, the pixel value corresponding to the eye area can be s2, the pixel value corresponding to the nose area can be s3, the pixel value corresponding to the lip area can be s4, the pixel value corresponding to the hair area can be s5, and the pixel value corresponding to the other areas on the entire face area other than the above-mentioned areas can be s6.

[0082] Then, based on the first and second segmented images, the target region information of the hairline area in the reference face image can be determined. For example, using the nose area as a reference, the first and second segmented images can be overlaid to determine the hair deviation area corresponding to the hair area in the first and second segmented images. This hair deviation area can then be identified as the hairline area in the reference face image, thus obtaining the target region information of the hairline area.

[0083] Here, by generating a first segmented image and a second segmented image, different semantic regions in the first segmented image and the second segmented image correspond to different pixel values. Therefore, based on the first segmented image and the second segmented image, the target region information of the area where the hairline is located in the reference face image can be determined more conveniently.

[0084] In one optional implementation, in step a2, based on the first segmented image and the second segmented image, the target region information of the area where the hairline is located in the reference face image is determined, specifically including:

[0085] Step a21: Adjust the pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location.

[0086] Step a22: Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located.

[0087] Step a23: Based on the deviation image, determine the target region information of the area where the hairline is located in the reference face image.

[0088] Considering that the hairline is generally located within the forehead area, to accurately determine the hairline's location and avoid adjustments to the areas of other facial features, a target baseline can be set based on the target area. For example, the target area could be the eyebrows or eyes. This is achieved by drawing a horizontal line at the target location as the target baseline. Then, the pixel values ​​of the pixels located below the target baseline in the first and second segmented images are adjusted to preset values, such as 0 or 1, resulting in the adjusted first and second segmented images.

[0089] Next, subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located. For example, the pixel value of the pixel located in the first row and first column of the adjusted first segmented image can be subtracted from the pixel value of the pixel located in the first row and first column of the adjusted second segmented image to obtain the pixel difference. This pixel difference is the pixel value of the pixel located in the first row and first column of the deviation image. Similarly, the pixel difference corresponding to each pixel position can be obtained, thus obtaining the deviation image.

[0090] When the image size is the same as that of the deviation image and the reference face image, the regional information of the area where the hairline is located in the deviation image can be determined as the target regional information of the area where the hairline is located in the reference face image.

[0091] Considering that the difference between the adjusted first segmented image and the adjusted second segmented image is that the hair is located in different areas, the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image are subtracted to generate a deviation image that includes the area where the hairline is located. Based on the deviation image, the target area information of the hairline in the reference face image can be determined more easily and efficiently.

[0092] In one optional implementation, determining the target region information of the area where the hairline is located in the reference face image based on the deviation image includes:

[0093] Step b1: Dilate the area where the hairline is located in the deviation image to generate a processed deviation image;

[0094] Step b2: Generate a face mask image based on the first segmented image; generate a hairline segmentation image based on the processed deviation image and the face mask image;

[0095] Step b3: Based on the hairline segmentation image, determine the target region information of the area where the hairline is located in the reference face image.

[0096] To generate an image with better hairline adjustment results, the hairline area in the biased image can be dilated to produce a processed biased image. In practice, convolution operations using convolution kernels can be used to dilate the biased image, resulting in the processed biased image. Alternatively, erosion and dilation operations can be performed on the hairline area in the biased image to generate the processed biased image.

[0097] Considering that the hairline is located on the face and hair, to mitigate interference from other areas besides the face and hair regions and to more accurately determine the hairline area, a face mask image can be generated based on the first segmentation image. In this face mask image, the pixel values ​​of the face and hair regions are 1, and the pixel values ​​of other regions are 0. The processed bias image and the face mask image are then multiplied by the pixel values ​​at corresponding pixel positions to generate the hairline segmentation image.

[0098] Based on the hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined. For example, the region information of the area where the hairline is located in the hairline segmentation image can be determined as the target region information.

[0099] Here, a face mask image is generated based on the first segmented image. Since the pixel values ​​of the areas other than the face and hair in the face mask image are zero, the pixel values ​​at corresponding pixel positions in the face mask image and the processed deviation image are multiplied. This can filter out the pixel information in the areas other than the face and hair in the processed deviation image, so as to more accurately determine the target area information in the future.

[0100] In one optional embodiment, after generating the hairline segmentation image, the method further includes: determining a target image region containing the facial features based on the facial feature region in the first segmentation image; generating a processed first segmentation image based on the target image region; wherein the pixel value at the pixel position corresponding to the target image region in the processed first segmentation image is zero; and generating an adjusted hairline segmentation image based on the processed first segmentation image and the hairline segmentation image.

[0101] The step of determining the target region information of the area where the hairline is located in the reference face image based on the hairline segmentation image includes: determining the target region information of the area where the hairline is located in the reference face image based on the adjusted hairline segmentation image.

[0102] After generating the hairline segmentation map, considering that the hairline area in the biased image has been dilated, which may cause the hairline area to occupy a large portion of the forehead area, the following problems may occur when fusing the target area image from the reference face image and the face image to be processed: the pixel information of the forehead area in the obtained target face image is more matched with the reference face image than the face image to be processed. Due to the deviation between the reference face image and the face image to be processed, there is a deviation in the forehead area between the target face image and the face image to be processed, resulting in poor display effect of the target face image.

[0103] To alleviate the aforementioned problems, this disclosure determines a target image region containing the facial features based on the area where the facial features are located in the first segmented image. For example, the area where the facial features are located in the first segmented image can be subjected to erosion and dilation processing to generate a dilated facial feature region, thus obtaining the target image region containing the facial features. Then, a processed first segmented image is generated based on the target image region, wherein the pixel value at the pixel position corresponding to the target image region in the first segmented image is zero. The processed first segmented image is then multiplied by the pixel value at the corresponding pixel position in the hairline segmented image to generate an adjusted hairline segmented image. This adjusted hairline segmented image is then used to determine the target region information of the hairline area in the reference face image, ensuring that the determined target region information occupies less of the forehead area, thus enabling the generation of a target face image with better display effect.

[0104] Regarding S104:

[0105] After obtaining the target region information, a target region image matching the target region information can be determined from the reference face image. This target region image can be the region image of the hairline area in the reference face image. Then, this target region image is fused with the face image to be processed to generate the target face image. For example, a local image matching the target region information can be determined from the face image to be processed, and this local image can be replaced with the target region image to generate the target face image.

[0106] See Figure 2 As shown, Figure 2 Image a shows the face image to be processed. Figure 2 Figure b shows a reference face image and a target region image. The face image to be processed is fused with the target region image to obtain the target face image, as shown below. Figure 2 The target face image shown in c.

[0107] In practice, a target face image can be generated using a hairline segmentation image (or an adjusted hairline segmentation image), the face image to be processed, and the target region image. The hairline segmentation image is a mask. For example, the target face image can be generated using the following formula:

[0108] blend_img=img_x*(1.0-mask)+img_y*mask

[0109] Where mask is the pixel value at the pixel position in the mask image, img_x is the pixel value at the same pixel position in the face image to be processed, img_y is the pixel value at the same pixel position in the reference face image, and blend_img is the pixel value at the same pixel position in the target face image.

[0110] Considering that the hair color of the reference face image may differ from that of the face image to be processed after the target neural network adjusts the hairline of the face image to be processed, in order to ensure that the hair color of the generated target face image is consistent with that of the face image to be processed, the hair color of the reference face image can be adjusted before image fusion.

[0111] In a specific implementation, before fusing the target region image that matches the target region information in the reference face image and the face image to be processed, the method further includes: determining the hair color information in the face image to be processed; adjusting the hair color of the reference face image based on the hair color information to generate an adjusted reference face image.

[0112] The step of fusing the target region image in the reference face image that matches the target region information with the face image to be processed to generate a target face image includes: fusing the target region image in the adjusted reference face image that matches the target region information with the face image to be processed to generate a target face image.

[0113] The hair color information in the face image to be processed is determined, for example, using histogram statistics. This hair color can then be transferred to a reference face image to generate an adjusted reference face image. Next, a target region image matching the target region information is determined from the adjusted reference face image. Finally, the target region image is fused with the face image to generate the target face image.

[0114] In implementation, the trained target neural network can be used to adjust the face image to be processed to generate a reference face image. Prior to implementation of this disclosure, a step of training the target neural network may also be included, and the process of training the target neural network is illustrated below.

[0115] In one optional implementation, training the target neural network includes:

[0116] Step c1: Obtain multiple candidate face image pairs, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image;

[0117] Step c2: For each candidate face image pair, determine the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair; and fuse the region image of the hairline area in the second reconstructed image with the first reconstructed image to generate a third reconstructed image, and determine the first reconstructed image and the third reconstructed image as a reconstructed image pair;

[0118] Step c3: Determine each of the reconstructed image pairs as training samples;

[0119] Step c4: Using the training samples, train the neural network to be trained to obtain the target neural network.

[0120] In step c1, each candidate face image pair includes a first candidate face image and a second candidate face image; the first candidate face image can be any face image, and the second candidate face image can be obtained by adjusting the hairline of the first candidate face image. In practice, the hairline of the first candidate face image can be adjusted in response to manual operation to generate the second candidate face image. Alternatively, a pix2pix network can be trained using samples to obtain a trained pix2pix network, which is used to adjust the hairline of the input image; then, the first candidate face image is input into the trained pix2pix network to generate the second candidate face image. Since the pix2pix network makes subtle adjustments to the input image, the trained pix2pix network can be used to adjust the first candidate face image multiple times to generate the second candidate face image.

[0121] In step c2, because the hairline adjustment in the second candidate face image is too heavy, making the second candidate face image unnatural and resulting in poor display effect, in order to obtain an image with a more natural hairline adjustment, the first and second candidate face images can be reconstructed. In practice, for each candidate face image pair, a neural network can be used to reconstruct the first and second candidate face images, generating a first reconstructed image and a second reconstructed image. For example, a neural network consisting of an encoder for editing (e4e) tool and a stylegan2 network can be used for image reconstruction. Specifically, e4e can convert the input image into noise data, and then input this noise data into stylegan2 to generate the reconstructed image corresponding to the input image.

[0122] In specific implementation, the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair can be determined according to the following steps:

[0123] Step c21: Determine the first noise data of the first candidate face image and the second noise data of the second candidate face image.

[0124] Step c22: Generate the second reconstructed image based on the second noise data.

[0125] Step c23, and generating the first reconstructed image based on the first noise data; or, determining the noise difference data between the second noise data and the first noise data, performing subtraction on the first noise data and the noise difference data to obtain third noise data, and generating the first reconstructed image using the third noise data.

[0126] For example, an image-to-noise tool can be used to convert the input image into noisy data. For instance, e4e can be used to process a first candidate face image to generate first noisy data, and e4e can be used to process a second candidate face image to generate second noisy data. The second noisy data can then be input into the StyleGAN2 network to generate a second reconstructed image, see [link to documentation]. Figure 3 As shown in c.

[0127] In one approach, the first noisy data can be input into the StyleGAN2 network to generate the first reconstructed image, see [link to relevant documentation]. Figure 3 As shown in Figure b. In another approach, considering that the hairline position in the reconstructed image is relatively similar to that in the second reconstructed image after directly inputting the first noise data into the StyleGAN2 network, to alleviate the above problem, a noise difference data can be determined between the second noise data and the first noise data. That is, the first noise data is subtracted from the second noise data to generate the noise difference data. Since the difference between the first candidate face image and the second candidate face image lies in the position of the hairline, the difference between the first noise data and the second noise data is also the difference in hairline position. Therefore, the noise difference data can characterize the hairline position difference feature between the first noise data and the second noise data. Then, the noise difference data is subtracted from the first noise data to obtain the third noise data. This third noise data characterizes the noise data corresponding to the image after the hairline is reversed (i.e., moved upward). The third noise data is then input into the StyleGAN2 network to generate the first reconstructed image. See [link to relevant documentation]. Figure 3 As shown in Figure a, compared with directly inputting the first noise data into the StyleGAN2 network to generate the reconstructed image, the hairline in the first reconstructed image is balder, and the difference in the position of the hairline between the first reconstructed image and the second reconstructed image is significant.

[0128] After obtaining the first and second reconstructed images, slight differences exist between them, such as skin color, hair color, and facial feature shape, aside from the difference in hairline position. To avoid interference from these differences in the neural network training process, a region image of the hairline area in the second reconstructed image can be identified. This region image is then fused with the first reconstructed image to generate the third reconstructed image. The first and third reconstructed images are then considered as a reconstructed image pair. This allows us to obtain the reconstructed image pair corresponding to each candidate face image pair.

[0129] The process of determining the region image of the hairline area in the second reconstructed image is the same as the process of determining the target region information of the hairline area in the reference face image in S103, and can be referred to the specific description of S103 above. The process of fusing the region image of the hairline area in the second reconstructed image with the first reconstructed image to generate the third reconstructed image can be referred to the specific description of S104.

[0130] For example, face region segmentation is performed on the first reconstructed image and the second reconstructed image respectively, generating a first reconstructed segmented image corresponding to the first reconstructed image and a second reconstructed segmented image corresponding to the second reconstructed image. Then, the pixel values ​​of pixels located below the target baseline in both the first and second reconstructed segmented images are adjusted to preset values, resulting in adjusted first and second reconstructed segmented images. The pixel values ​​at corresponding pixel positions in the adjusted first and second reconstructed segmented images are subtracted to generate a reconstruction deviation image containing the hairline region. The hairline region in the reconstruction deviation image is dilated to generate a processed reconstruction deviation image. Based on the first reconstructed segmented image, a reconstructed face mask is generated, and the pixel values ​​at corresponding pixel positions in the processed reconstruction deviation image and the reconstructed face mask are multiplied to generate a hairline reconstructed segmented image. Based on the region containing the facial features in the first reconstructed segmented image, a target image region containing the facial features is determined. Based on the target image region, a processed first reconstructed segmented image is generated. The processed first reconstructed segmented image is then multiplied by the pixel values ​​at corresponding pixel positions in the hairline reconstructed segmented image to generate an adjusted hairline reconstructed segmented image. Subsequently, based on the adjusted hairline reconstructed segmented image, the region image of the hairline area in the second reconstructed image is determined.

[0131] In steps c3 and c4, multiple reconstructed image pairs can be selected as training samples. These training samples are then used to train the neural network to be trained until the trained neural network meets the training cutoff conditions, thus obtaining the target neural network. The training cutoff conditions include the number of training iterations exceeding a set threshold, the network loss value being less than a set loss threshold, and neural network convergence. The network structure of the neural network to be trained can be a pix2pix network.

[0132] For example, the first reconstructed image can be input into the neural network to be trained to generate a predicted image. Based on the predicted image and the second reconstructed image, a loss value is determined. The network parameters of the neural network to be trained are then adjusted using the loss value to obtain the target neural network. The loss function can be set as needed.

[0133] An example is provided to illustrate a facial image processing method. First, the process of training the target neural network is explained:

[0134] Step 11: Obtain multiple candidate face image pairs, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image.

[0135] Step 12: For each candidate face image pair, input the first candidate face image into e4e to generate the first noise data, and input the second candidate face image into e4e to generate the second noise data.

[0136] Step 13: Input the second noise data into the StyleGAN2 network to generate the second reconstructed image. Alternatively, in one method, input the first noise data into the StyleGAN2 network to generate the first reconstructed image. In another method, subtract the first noise data from the second noise data to obtain noise difference data, then subtract the first noise data from the noise difference data to obtain third noise data; input the third noise data into the StyleGAN2 network to generate the first reconstructed image.

[0137] Step 14: Using a face segmentation tool such as faceparsing, perform face region segmentation on the first reconstructed image and the second reconstructed image respectively to generate a first reconstructed segmented image corresponding to the first reconstructed image and a second reconstructed segmented image corresponding to the second reconstructed image.

[0138] Step 15: Adjust the pixel values ​​of the pixels located below the eyebrow area in the first and second reconstructed segmentation images to zero, respectively, to obtain the adjusted first and second reconstructed segmentation images. Subtract the pixel values ​​at corresponding pixel positions in the adjusted first and second reconstructed segmentation images to generate a reconstruction deviation image that includes the area where the hairline is located.

[0139] Step 16: Dilate the area where the hairline is located in the reconstructed deviation image to generate a processed reconstructed deviation image. Based on the first reconstructed segmentation image, generate a reconstructed face mask image, and multiply the pixel values ​​at corresponding pixel positions in the processed reconstructed deviation image and the reconstructed face mask image to generate a hairline reconstructed segmentation image.

[0140] Step 17: Based on the region containing the facial features in the first reconstructed segmented image, determine the target image region containing the facial features. Based on the target image region, generate the processed first reconstructed segmented image. Multiply the pixel values ​​at corresponding pixel positions in the processed first reconstructed segmented image and the hairline reconstructed segmented image to generate the adjusted hairline reconstructed segmented image. Then, based on the adjusted hairline reconstructed segmented image, determine the region image of the area where the hairline is located in the second reconstructed image.

[0141] Step 18: Fuse the region image of the hairline area in the second reconstructed image with the first reconstructed image to generate a third reconstructed image. The first and third reconstructed images are then identified as a reconstructed image pair.

[0142] Step 19: Determine each reconstructed image pair as training samples; use the training samples to train the neural network to obtain the target neural network.

[0143] The application process of the target neural network will then be explained. Specifically, it includes:

[0144] Step 21: Obtain the face image to be processed.

[0145] Step 22: Use the target neural network to adjust the hairline of the face image to be processed and generate a reference face image.

[0146] Step 23: Perform face region segmentation processing on the reference face image and the face image to be processed respectively to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different.

[0147] Step 24: Adjust the pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image to preset values ​​respectively, to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location.

[0148] Step 25: Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located.

[0149] Step 26: Dilate the area containing the hairline in the deviation image to generate a processed deviation image. Based on the first segmentation image, generate a face mask image, and multiply the pixel values ​​at corresponding pixel positions in the processed deviation image and the face mask image to generate a hairline segmentation image; the pixel values ​​of the areas in the face mask image other than the face and hair are zero.

[0150] Step 27: Based on the area where the facial features are located in the first segmented image, determine the target image area containing the area where the facial features are located. Based on the target image area, generate the processed first segmented image. In the processed first segmented image, the pixel value at the pixel position corresponding to the target image area is zero. Multiply the processed first segmented image with the pixel value at the corresponding pixel position in the hairline segmented image to generate the adjusted hairline segmented image.

[0151] Step 28: Based on the adjusted hairline segmentation image, determine the target region information of the area where the hairline is located in the reference face image. Also determine the hair color information in the face image to be processed; based on the hair color information, adjust the hair color of the reference face image to generate the adjusted reference face image.

[0152] Step 29: Fuse the target region image that matches the target region information in the adjusted reference face image with the face image to be processed to generate the target face image.

[0153] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0154] Based on the same inventive concept, this disclosure also provides a facial image processing device corresponding to the facial image processing method. Since the principle of the device in this disclosure for solving the problem is similar to the facial image processing method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0155] Reference Figure 4 The diagram shown is a schematic representation of the architecture of a facial image processing device according to an embodiment of this disclosure. The device includes: an acquisition module 401, a first generation module 402, a determination module 403, and a second generation module 404; wherein,

[0156] The acquisition module 401 is used to acquire the face image to be processed;

[0157] The first generation module 402 is used to adjust the face image to be processed and generate a reference face image;

[0158] The determining module 403 is used to determine the target region information in the reference face image;

[0159] The second generation module 404 is used to fuse the target region image that matches the target region information in the reference face image with the face image to be processed to generate a target face image.

[0160] In an optional implementation, the determining module 403, when determining the target region information in the reference face image, is used to:

[0161] The reference face image and the face image to be processed are respectively subjected to face region segmentation processing to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different;

[0162] Based on the first segmented image and the second segmented image, the target region information of the area where the hairline is located in the reference face image is determined.

[0163] In an optional implementation, the determining module 403, when determining the target region information of the area where the hairline is located in the reference face image based on the first segmented image and the second segmented image, is used to:

[0164] The pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image are adjusted to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location;

[0165] Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located;

[0166] Based on the deviation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0167] In an optional implementation, the determining module 403, when determining the target region information of the area where the hairline is located in the reference face image based on the deviation image, is used to:

[0168] The area where the hairline is located in the deviation image is dilated to generate a processed deviation image;

[0169] Generate a face mask image based on the first segmented image;

[0170] A hairline segmentation image is generated based on the processed deviation image and the face mask image;

[0171] Based on the hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0172] In an optional implementation, the determining module 403, after generating the hairline segmentation image, is further configured to:

[0173] Based on the region containing the facial features in the first segmented image, a target image region containing the region containing the facial features is determined, and a processed first segmented image is generated based on the target image region; in the processed first segmented image, the pixel value at the pixel position corresponding to the target image region is zero.

[0174] An adjusted hairline segmentation image is generated based on the processed first segmentation image and the hairline segmentation image.

[0175] The determining module 403, when determining the target region information of the area where the hairline is located in the reference face image based on the hairline segmentation image, is used for:

[0176] Based on the adjusted hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0177] In an optional implementation, before fusing the target region image matching the target region information in the reference face image with the face image to be processed, the method further includes: an adjustment module 405, configured to:

[0178] Determine the hair color information in the face image to be processed;

[0179] Based on the hair color information, the hair color of the reference face image is adjusted to generate an adjusted reference face image;

[0180] The second generation module 404, when fusing the target region image that matches the target region information in the reference face image with the face image to be processed to generate the target face image, is used for:

[0181] The target region image that matches the target region information in the adjusted reference face image is fused with the face image to be processed to generate the target face image.

[0182] In one optional implementation, the first generation module 402, when adjusting the face image to be processed to generate a reference face image, is used to: adjust the face image to be processed using a trained target neural network to generate a reference face image.

[0183] The device further includes a training module 406, used to train the target neural network according to the following steps:

[0184] Multiple candidate face image pairs are obtained, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image;

[0185] For each candidate face image pair, a first reconstructed image of the first candidate face image and a second reconstructed image of the second candidate face image are determined in the candidate face image pair; and the region image of the area where the hairline is located in the second reconstructed image is fused with the first reconstructed image to generate a third reconstructed image, and the first reconstructed image and the third reconstructed image are determined as a reconstructed image pair;

[0186] Each of the reconstructed image pairs is determined as a training sample;

[0187] Using the training samples, the neural network to be trained is trained to obtain the target neural network.

[0188] In an optional implementation, the training module 406, when determining the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair, is configured to:

[0189] Determine the first noise data of the first candidate face image and the second noise data of the second candidate face image;

[0190] The second reconstructed image is generated based on the second noise data;

[0191] And, based on the first noise data, generate the first reconstructed image; or, determine the noise difference data between the second noise data and the first noise data, perform subtraction on the first noise data and the noise difference data to obtain third noise data, and use the third noise data to generate the first reconstructed image.

[0192] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0193] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 5 The diagram shows the structure of a computer device 500 provided in this embodiment of the present disclosure, including a processor 501, a memory 502, and a bus 503. The memory 502 stores execution instructions and includes main memory 5021 and external memory 5022. The main memory 5021, also called internal memory, is used to temporarily store computational data in the processor 501 and data exchanged with external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the main memory 5021. When the computer device 500 is running, the processor 501 and the memory 502 communicate through the bus 503, causing the processor 501 to execute the following instructions:

[0194] Obtain the face image to be processed;

[0195] The face image to be processed is adjusted to generate a reference face image;

[0196] Determine the target region information in the reference face image;

[0197] The target region image that matches the target region information in the reference face image is fused with the face image to be processed to generate the target face image.

[0198] In one possible design, the instructions executed by processor 501, including determining the target region information in the reference face image, include:

[0199] The reference face image and the face image to be processed are respectively subjected to face region segmentation processing to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different;

[0200] Based on the first segmented image and the second segmented image, the target region information of the area where the hairline is located in the reference face image is determined.

[0201] In one possible design, the instructions executed by processor 501, which include determining the target region information of the area where the hairline is located in the reference face image based on the first segmented image and the second segmented image, include:

[0202] The pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image are adjusted to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location;

[0203] Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located;

[0204] Based on the deviation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0205] In one possible design, the instructions executed by processor 501, which include determining the target region information of the area where the hairline is located in the reference facial image based on the deviation image, include:

[0206] The area where the hairline is located in the deviation image is dilated to generate a processed deviation image;

[0207] Generate a face mask image based on the first segmented image;

[0208] A hairline segmentation image is generated based on the processed deviation image and the face mask image;

[0209] Based on the hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0210] In one possible design, the instructions executed by processor 501, after generating the hairline segmentation image, further include:

[0211] Based on the region containing the facial features in the first segmented image, a target image region containing the region containing the facial features is determined, and a processed first segmented image is generated based on the target image region; in the processed first segmented image, the pixel value at the pixel position corresponding to the target image region is zero.

[0212] An adjusted hairline segmentation image is generated based on the processed first segmentation image and the hairline segmentation image.

[0213] The step of determining the target region information of the area where the hairline is located in the reference face image based on the hairline segmentation image includes:

[0214] Based on the adjusted hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

[0215] In one possible design, the instructions executed by processor 501, before fusing the target region image in the reference face image that matches the target region information with the face image to be processed, further include:

[0216] Determine the hair color information in the face image to be processed;

[0217] Based on the hair color information, the hair color of the reference face image is adjusted to generate an adjusted reference face image;

[0218] The step of fusing the target region image in the reference face image that matches the target region information with the face image to be processed to generate a target face image includes:

[0219] The target region image that matches the target region information in the adjusted reference face image is fused with the face image to be processed to generate the target face image.

[0220] In one possible design, the instruction executed by processor 501 to adjust the face image to be processed and generate a reference face image includes: adjusting the face image to be processed using a trained target neural network to generate a reference face image.

[0221] The target neural network is trained according to the following steps:

[0222] Multiple candidate face image pairs are obtained, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image;

[0223] For each candidate face image pair, a first reconstructed image of the first candidate face image and a second reconstructed image of the second candidate face image are determined in the candidate face image pair; and the region image of the area where the hairline is located in the second reconstructed image is fused with the first reconstructed image to generate a third reconstructed image, and the first reconstructed image and the third reconstructed image are determined as a reconstructed image pair;

[0224] Each of the reconstructed image pairs is determined as a training sample;

[0225] Using the training samples, the neural network to be trained is trained to obtain the target neural network.

[0226] In one possible design, the instructions executed by processor 501, including determining the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair, include:

[0227] Determine the first noise data of the first candidate face image and the second noise data of the second candidate face image;

[0228] The second reconstructed image is generated based on the second noise data;

[0229] And, based on the first noise data, generate the first reconstructed image; or, determine the noise difference data between the second noise data and the first noise data, perform subtraction on the first noise data and the noise difference data to obtain third noise data, and use the third noise data to generate the first reconstructed image.

[0230] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the facial image processing method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0231] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the facial image processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0232] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0233] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0234] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0235] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0236] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A facial image processing method, characterized in that, include: Obtain the face image to be processed; The face image to be processed is adjusted to generate a reference face image; The reference face image and the face image to be processed are respectively subjected to face region segmentation processing to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different; The pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image are adjusted to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location; Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located; The area where the hairline is located in the deviation image is dilated to generate a processed deviation image; Generate a face mask image based on the first segmented image; Based on the processed deviation image and the face mask image, the pixel information in the regions other than the face and hair in the processed deviation image is filtered out to generate a hairline segmentation image. Based on the region containing the facial features in the first segmented image, a target image region containing the region containing the facial features is determined, and a processed first segmented image is generated based on the target image region; in the processed first segmented image, the pixel value at the pixel position corresponding to the target image region is zero. An adjusted hairline segmentation image is generated based on the processed first segmentation image and the hairline segmentation image. Based on the adjusted hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

2. The method according to claim 1, characterized in that, Before fusing the target region image that matches the target region information in the reference face image with the face image to be processed, the method further includes: Determine the hair color information in the face image to be processed; Based on the hair color information, the hair color of the reference face image is adjusted to generate an adjusted reference face image; The step of fusing the target region image in the reference face image that matches the target region information with the face image to be processed to generate a target face image includes: The target region image that matches the target region information in the adjusted reference face image is fused with the face image to be processed to generate the target face image.

3. The method according to claim 1 or 2, characterized in that, The step of adjusting the face image to be processed to generate a reference face image includes: The target neural network obtained through training is used to adjust the face image to be processed, thereby generating a reference face image; The target neural network is trained according to the following steps: Multiple candidate face image pairs are obtained, wherein each candidate face image pair includes: a first candidate face image and a second candidate face image obtained by adjusting the hairline of the first candidate face image; For each candidate face image pair, a first reconstructed image of the first candidate face image and a second reconstructed image of the second candidate face image are determined in the candidate face image pair; and the region image of the area where the hairline is located in the second reconstructed image is fused with the first reconstructed image to generate a third reconstructed image, and the first reconstructed image and the third reconstructed image are determined as a reconstructed image pair; Each of the reconstructed image pairs is determined as a training sample; Using the training samples, the neural network to be trained is trained to obtain the target neural network.

4. The method according to claim 3, characterized in that, Determining the first reconstructed image of the first candidate face image and the second reconstructed image of the second candidate face image in the candidate face image pair includes: Determine the first noise data of the first candidate face image and the second noise data of the second candidate face image; The second reconstructed image is generated based on the second noise data; And, based on the first noise data, generate the first reconstructed image; or, determine the noise difference data between the second noise data and the first noise data, perform subtraction on the first noise data and the noise difference data to obtain third noise data, and use the third noise data to generate the first reconstructed image.

5. A facial image processing device, characterized in that, include: The acquisition module is used to acquire the face image to be processed; The first generation module is used to adjust the face image to be processed and generate a reference face image; A determining module is used to determine target region information in the reference face image; The second generation module is used to fuse the target region image that matches the target region information in the reference face image with the face image to be processed to generate a target face image. The determining module is further configured to: The reference face image and the face image to be processed are respectively subjected to face region segmentation processing to generate a first segmentation image corresponding to the reference face image and a second segmentation image corresponding to the face image to be processed; wherein, in the first segmentation image and the second segmentation image, the pixel values ​​corresponding to different semantic regions are different; The pixel values ​​of the pixels located below the target baseline in the first segmented image and the second segmented image are adjusted to preset values ​​to obtain the adjusted first segmented image and the adjusted second segmented image, wherein the target baseline is determined based on the target location; Subtract the pixel values ​​at corresponding pixel positions in the adjusted first segmented image and the adjusted second segmented image to generate a deviation image that includes the area where the hairline is located; The area where the hairline is located in the deviation image is dilated to generate a processed deviation image; Generate a face mask image based on the first segmented image; Based on the processed deviation image and the face mask image, the pixel information in the regions other than the face and hair in the processed deviation image is filtered out to generate a hairline segmentation image. Based on the region containing the facial features in the first segmented image, a target image region containing the region containing the facial features is determined, and a processed first segmented image is generated based on the target image region; in the processed first segmented image, the pixel value at the pixel position corresponding to the target image region is zero. An adjusted hairline segmentation image is generated based on the processed first segmentation image and the hairline segmentation image. Based on the adjusted hairline segmentation image, the target region information of the area where the hairline is located in the reference face image is determined.

6. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the facial image processing method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the facial image processing method as described in any one of claims 1 to 4.

Images