Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium. According to the method, an initial makeup trying image is generated through a first generation model, and the first generation model is formed by loading an actor feature adaptation module and a clothing feature adaptation module to a pre-trained image generation model according to different weight proportions; performing local redrawing on a face region in the initial makeup trying image to obtain a redrawn face region; fusing the redrawn face region with an initial makeup trying image to generate an intermediate makeup trying effect picture; performing color migration processing on a face region in the middle makeup trying effect picture based on face region color features of the initial makeup trying image; and fusing the face region subjected to color migration processing with the middle makeup trying effect picture to generate a target makeup trying effect picture. Therefore, the skin color difference caused by independent training of the feature model is fundamentally bridged, and perfect color transition between the face and the body area is realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the widespread application of artificial intelligence technology in the film and entertainment industry, virtual makeup try-on technology based on deep learning is gradually becoming an important auxiliary tool in film and television production. This technology can quickly generate makeup and clothing matching effects of different styles, greatly improving makeup try-on efficiency and reducing production costs. Especially in film and television shooting and advertising production fields that require frequent changes of styling, virtual makeup try-on technology has shown great application potential.

[0003] Currently, mainstream virtual makeup try-on solutions primarily employ generative adversarial networks (GANs) or diffusion models as their underlying architecture. They control the generated effects by separately training actor feature models and clothing feature models. The actor feature model focuses on learning the facial features of a specific actor, including details such as facial features and skin tone; while the clothing feature model primarily learns the visual features of various garments, such as style and texture. In practical applications, a basic try-on image is typically generated first using the clothing feature model, and then the facial regions in the image are redrawn and optimized using the actor feature model.

[0004] However, because actor feature models and clothing feature models are trained independently in existing technologies, the skin color features learned by the two differ significantly. When using the actor feature model to redraw an image generated by the clothing feature model, it results in a noticeable color difference between the redrawn facial area and the neck, body, and other parts of the original image. This color inconsistency creates a strong sense of visual disjointedness, severely affecting the realism and usability of the makeup trial effect. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method, apparatus, electronic device, and storage medium to solve the problem of inconsistent colors in makeup trial effect images generated in the prior art. The specific technical solution is as follows:

[0006] In a first aspect, this application provides an image processing method, comprising:

[0007] An initial makeup trial image is generated through a first generative model, which consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios.

[0008] The face region in the initial makeup trial image is partially redrawn to obtain the redrawn face region;

[0009] The redrawn face area is merged with the initial makeup trial image to generate an intermediate makeup trial effect image;

[0010] Based on the color features of the face region in the initial makeup trial image, color transfer processing is performed on the face region in the intermediate makeup trial effect image.

[0011] The color-transfer processed facial area is merged with the intermediate makeup trial image to generate the target makeup trial image.

[0012] In one possible implementation, the step of performing color transfer processing on the face region in the intermediate makeup effect image based on the color features of the face region in the initial makeup trial image includes:

[0013] The coordinates of the face detection box are determined based on the initial makeup trial image;

[0014] Based on the coordinates of the face detection box, a first face region image is extracted from the initial makeup trial image, and a second face region image is extracted from the intermediate makeup trial effect image based on the coordinates of the face detection box.

[0015] Using the first face region image as the chromaticity reference source image and the second face region image as the chromaticity transfer target image, color transfer is performed to obtain a color-corrected face region image.

[0016] In one possible implementation, determining the coordinates of the face detection box based on the initial makeup trial image includes:

[0017] The initial makeup trial image is processed using an object detection algorithm to obtain multiple object detection boxes;

[0018] The core character region is determined within the multiple target detection frames;

[0019] Copy the core character area to a blank image with the same resolution as the initial makeup trial image to generate an intermediate image;

[0020] Face detection is performed on the intermediate image to obtain face detection bounding boxes;

[0021] If multiple face detection boxes exist, the face detection box with the smallest center coordinate ordinate value is selected as the face detection box coordinate.

[0022] In one possible implementation, determining the core person region within the plurality of target detection boxes includes:

[0023] Based on multiple target detection boxes, a semantic segmentation algorithm is used to process the initial makeup trial image to generate a human semantic segmentation map;

[0024] Calculate the area of ​​the region enclosed by each target detection box;

[0025] Based on the human semantic segmentation map, the proportion of human semantic pixels within each target detection box is calculated.

[0026] Select the target detection box with the largest area that meets the preset threshold for the proportion of human semantic pixels from multiple target detection boxes, and use it as the core human region.

[0027] In one possible implementation, the step of extracting a first face region image from the initial makeup trial image based on the face detection box coordinates, and extracting a second face region image from the intermediate makeup trial effect image based on the face detection box coordinates, includes:

[0028] Based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the initial makeup trial image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a first face region image containing only the face region is extracted from the enlarged face image.

[0029] Furthermore, based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the intermediate makeup effect image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a second face region image containing only the face region is extracted from the enlarged face image.

[0030] In one possible implementation, the step of fusing the color-transfer processed facial region with the intermediate makeup trial image to generate the target makeup trial image includes:

[0031] The face mask is then shrunk inward to obtain an optimized face mask;

[0032] Based on the optimized face mask, the color-corrected face region image is correspondingly pasted onto the second face region image to obtain a pasted image;

[0033] The fitting image is scaled using a bilinear interpolation algorithm so that the size of the scaled fitting image matches the size of the face area in the intermediate makeup effect image.

[0034] Based on the coordinates of the face detection box, the scaled and fitted image is fused with the intermediate makeup effect image to generate the target makeup effect image.

[0035] In one possible implementation, the step of locally redrawing the facial region in the initial makeup trial image to obtain the redrawn facial region includes:

[0036] Face detection is performed on the initial makeup trial image to obtain the target face region image, and semantic segmentation is performed on the target face region image to obtain the target face mask;

[0037] The actor feature adaptation module is loaded into a pre-trained image generation model to form a second generation model, and the second generation model is used to perform local redrawing based on the target face mask to obtain the redrawn face region.

[0038] Secondly, this application provides an image processing apparatus, comprising:

[0039] The generation module is used to generate an initial makeup trial image through a first generation model, which consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios.

[0040] The redrawing module is used to locally redraw the face region in the initial makeup trial image to obtain the redrawn face region;

[0041] The first fusion module is used to fuse the redrawn face region with the initial makeup trial image to generate an intermediate makeup trial effect image.

[0042] The migration module is used to perform color migration processing on the face region in the intermediate makeup effect image based on the color features of the face region in the initial makeup trial image;

[0043] The second fusion module is used to fuse the color-transfer processed face area with the intermediate makeup effect image to generate the target makeup effect image.

[0044] In one possible implementation, the migration module is specifically used for:

[0045] The coordinates of the face detection box are determined based on the initial makeup trial image;

[0046] Based on the coordinates of the face detection box, a first face region image is extracted from the initial makeup trial image, and a second face region image is extracted from the intermediate makeup trial effect image based on the coordinates of the face detection box.

[0047] Using the first face region image as the chromaticity reference source image and the second face region image as the chromaticity transfer target image, color transfer is performed to obtain a color-corrected face region image.

[0048] In one possible implementation, the migration module is further configured to:

[0049] The initial makeup trial image is processed using an object detection algorithm to obtain multiple object detection boxes;

[0050] The core character region is determined within the multiple target detection frames;

[0051] Copy the core character area to a blank image with the same resolution as the initial makeup trial image to generate an intermediate image;

[0052] Face detection is performed on the intermediate image to obtain face detection bounding boxes;

[0053] If multiple face detection boxes exist, the face detection box with the smallest center coordinate ordinate value is selected as the face detection box coordinate.

[0054] In one possible implementation, the migration module is further configured to:

[0055] Based on multiple target detection boxes, a semantic segmentation algorithm is used to process the initial makeup trial image to generate a human semantic segmentation map;

[0056] Calculate the area of ​​the region enclosed by each target detection box;

[0057] Based on the human semantic segmentation map, the proportion of human semantic pixels within each target detection box is calculated.

[0058] Select the target detection box with the largest area that meets the preset threshold for the proportion of human semantic pixels from multiple target detection boxes, and use it as the core human region.

[0059] In one possible implementation, the migration module is further configured to:

[0060] Based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the initial makeup trial image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a first face region image containing only the face region is extracted from the enlarged face image.

[0061] Furthermore, based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the intermediate makeup effect image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a second face region image containing only the face region is extracted from the enlarged face image.

[0062] In one possible implementation, the second fusion module is specifically used for:

[0063] The face mask is then shrunk inward to obtain an optimized face mask;

[0064] Based on the optimized face mask, the color-corrected face region image is correspondingly pasted onto the second face region image to obtain a pasted image;

[0065] The fitting image is scaled using a bilinear interpolation algorithm so that the size of the scaled fitting image matches the size of the face area in the intermediate makeup effect image.

[0066] Based on the coordinates of the face detection box, the scaled and fitted image is fused with the intermediate makeup effect image to generate the target makeup effect image.

[0067] In one possible implementation, the redraw module is specifically used for:

[0068] Face detection is performed on the initial makeup trial image to obtain the target face region image, and semantic segmentation is performed on the target face region image to obtain the target face mask;

[0069] The actor feature adaptation module is loaded into a pre-trained image generation model to form a second generation model, and the second generation model is used to perform local redrawing based on the target face mask to obtain the redrawn face region.

[0070] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0071] Memory, used to store computer programs;

[0072] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0073] Fourthly, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0074] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform any of the image processing methods described above.

[0075] Beneficial effects of the embodiments in this application:

[0076] This application provides an image processing method, apparatus, electronic device, and storage medium. First, an initial makeup trial image is constructed using a dual-module generation model that integrates actor and costume features, preserving the dominance of costume features while also considering the actor's basic characteristics. Then, the facial region is specifically redrawn and optimized, and an intermediate effect image is generated through precise fusion. Further, using the facial region of the initial image as a color reference, directional color transfer is performed on the intermediate effect image. Finally, intelligent fusion technology ensures that the corrected facial region seamlessly integrates with the overall image. This progressive processing flow not only fully preserves the advantages of the original generation model but also fundamentally bridges the skin tone differences caused by independent training of feature models by introducing a color reference mechanism and transfer algorithm. This allows the final makeup trial effect to maintain stylistic consistency while achieving a perfect color transition between the face and body areas.

[0077] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0078] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0081] Figure 1 A flowchart illustrating an image processing method provided in an embodiment of this application;

[0082] Figure 2 A flowchart illustrating another image processing method provided in this application embodiment;

[0083] Figure 3 A flowchart illustrating yet another image processing method provided in this application embodiment;

[0084] Figure 4 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0085] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0087] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0088] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This method can be applied to one or more electronic devices such as smartphones, laptops, desktop computers, portable computers, and servers. Furthermore, the execution entity of this method can be hardware or software. When the execution entity is hardware, it can be one or more of the aforementioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the execution entity is software, this method can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are imposed here.

[0089] like Figure 1 As shown, the method specifically includes:

[0090] S101. Generate an initial makeup trial image through a first generation model. The first generation model consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios.

[0091] The feature adaptation module refers to the model fine-tuning component based on LoRA (Low-Rank Adaptation). Specifically, the actor feature adaptation module (Actor LoRA) is used to encode the actor's facial features and body contour parameters; the clothing feature adaptation module (Clothing LoRA) is used to encode the clothing's cutting structure and pattern details.

[0092] Image generation models refer to image generation models that are pre-trained on large-scale datasets (such as Flux large models).

[0093] Initial makeup test images refer to preliminary composite images that include basic costumes and actor body shapes.

[0094] In this embodiment, firstly, the actor's LoRA is set to a lower weight (e.g., 0.3), and the clothing's LoRA is set to a higher weight (e.g., 0.9), and these are connected in parallel to the Flux large model to construct a first generation model. Then, based on the prompts input by the user, the first generation model generates an initial makeup trial image under the control of differentiated weights. Thus, when generating the makeup trial image, the low-weight actor LoRA produces a "feature-inducing" effect, making the basic outline (face shape / body shape) of the generated character approximate the actor's features; the high-weight clothing LoRA (0.9) dominates the generation of details, ensuring that the clothing style is accurately expressed. This differentiated weight allocation strategy allows clothing details to be expressed first, while preserving the actor's basic outline features, thus avoiding mutual interference between the two types of features during the generation process from the source.

[0095] S102. The face region in the initial makeup trial image is partially redrawn to obtain the redrawn face region.

[0096] Local redrawing refers to an image processing technique that redraws or modifies a specific region (in this case, the face region) in an image.

[0097] Redrawing the face region refers to the new face image region obtained after local redrawing.

[0098] In this embodiment of the application, S102 may specifically include the following steps: performing face detection on the initial makeup trial image to obtain a target face region image, and performing semantic segmentation on the target face region image to obtain a target face mask; loading the actor feature adaptation module into a pre-trained image generation model to form a second generation model, and using the second generation model to perform local redrawing based on the target face mask to obtain a redrawn face region.

[0099] The target face region image refers to the face region image located and extracted from the initial makeup trial image through face detection. Semantic segmentation refers to the process of segmenting an image into pixel regions with specific semantic meaning. The target face mask refers to the binary image obtained through semantic segmentation used to identify the precise face region. The second generative model refers to a generative model specifically used for face region redrawing, consisting of an actor feature adaptation module loaded into a pre-trained image generation model.

[0100] The specific implementation process of this step is as follows: First, a face detection algorithm is used to process the initial makeup trial image to accurately locate the face region and extract the target face region image; then, a semantic segmentation model (such as UNet) is used to process the target face region image to generate an accurate target face mask; next, a second generation model is constructed, which focuses on preserving the actor's facial features by separately loading an actor feature adaptation module; finally, the second generation model is used to locally redraw the specified area based on the target face mask. This staged redrawing process can effectively avoid the interference of clothing features on the face region, ensuring that the generated facial features (including facial features, skin tone, etc.) accurately reflect the actor's real features, while maintaining a natural transition with the clothing region. This solves the problem of inconsistent colors between the face and body regions caused by independent training of feature modules, significantly improving the realism and visual effect of virtual makeup trial.

[0101] S103. The redrawn face area is fused with the initial makeup trial image to generate an intermediate makeup trial effect image.

[0102] Intermediate makeup preview image refers to the transitional image generated by merging the redrawn face area with the initial makeup preview image.

[0103] In this embodiment, firstly, the target face mask obtained based on semantic segmentation is subjected to edge expansion and blurring to obtain a transition mask. Then, a complementary mask (a binary image generated by inverting the transition mask) is generated based on the pixel value distribution of the transition mask. Next, an image scaling algorithm is used to adjust the resolution of the redrawn face region and the transition mask, thereby restoring the redrawn face region and the transition mask to the resolution of the face region in the initial makeup trial image. Finally, the redrawn face region and the initial makeup trial image are selectively fused according to the complementary mask. That is, for the face region marked by the transition mask, the corresponding pixel values ​​of the redrawn face image are used; for the non-face region marked by the complementary mask, the original pixel values ​​of the initial makeup trial image are retained. The transition region between the face region and the non-face region is processed using a gradient fusion. This achieves a natural fusion of the redrawn face features and the original clothing features, providing a high-quality intermediate image for subsequent color transfer processing.

[0104] S104. Based on the color features of the face region in the initial makeup trial image, perform color transfer processing on the face region in the intermediate makeup trial effect image.

[0105] Color transfer processing refers to the processing method of transferring color features from a source image to a target image.

[0106] In this embodiment, the face region of the initial makeup trial image is used as a color reference source to extract its color distribution features; the face region of the intermediate makeup trial effect image is analyzed; and color transfer algorithms such as Linear-Monge-Kantorovitch are used to transfer the reference color features to the target face region to achieve color correction.

[0107] S105. The face area processed by color transfer is merged with the intermediate makeup effect image to generate the target makeup effect image.

[0108] The target makeup trial image is the final generated makeup trial result image.

[0109] In this embodiment, firstly, edge optimization processing is performed on the face area after color transfer by shrinking the face mask inward to avoid edge defects. Then, scaling algorithms such as bilinear interpolation are used to adjust the image size. Finally, the processed area is accurately fused into the intermediate makeup effect image.

[0110] In this embodiment, firstly, an initial makeup trial image is constructed using a dual-module generation model that integrates actor and costume features, preserving the dominance of costume features while also considering the actor's basic characteristics. Then, targeted local redrawing and optimization of the facial region are performed, and an intermediate effect image is generated through precise fusion. Furthermore, using the facial region of the initial image as a color reference, directional color transfer is implemented on the intermediate effect image. Finally, intelligent fusion technology ensures a natural transition between the corrected facial region and the overall image. This progressive processing flow not only fully preserves the advantages of the original generation model but also fundamentally bridges the skin tone differences caused by independent training of the feature models by introducing a color reference mechanism and transfer algorithm. This allows the final makeup trial effect to maintain stylistic consistency while achieving a perfect color transition between the face and body areas.

[0111] See Figure 2 This is a flowchart illustrating another embodiment of the image processing method provided in this application. Figure 2 The process shown above Figure 1 Based on the illustrated process, this paper describes how to perform color transfer processing on the face region in the intermediate makeup effect image based on the color features of the face region in the initial makeup trial image. For example... Figure 2 As shown, the process may include the following steps:

[0112] S201. Determine the coordinates of the face detection box based on the initial makeup trial image.

[0113] Face detection bounding box coordinates refer to the vertex coordinates of the rectangle generated when locating a face region in an image using an object detection algorithm.

[0114] In this embodiment of the application, S201 may include the following steps: processing the initial makeup trial image using a target detection algorithm to obtain multiple target detection boxes; determining a core character region within the multiple target detection boxes; copying the core character region to a blank image with the same resolution as the initial makeup trial image to generate an intermediate image; performing face detection on the intermediate image to obtain face detection boxes; if multiple face detection boxes exist, selecting the face detection box with the smallest center coordinate ordinate value as the face detection box coordinate.

[0115] The target detection box refers to the rectangular region containing potential people identified by the target detection algorithm; the core person region refers to the region where the main person is located, determined by area calculation and semantic analysis; the intermediate image is a temporary image generated after extracting the core person region onto a blank background. The specific implementation process is as follows: First, the initial makeup trial image is processed using target detection algorithms such as YOLO v8 to identify all possible person regions and generate corresponding target detection boxes; then, the core person region is selected based on the area size of each detection box and the semantic segmentation results; next, the core person region is extracted onto a blank background to generate an intermediate image to eliminate other interference; finally, a dedicated face detection is performed on the intermediate image, and when multiple faces are present, the face with the highest position (smallest y-coordinate value) in the image is selected as the final target. This processing flow ensures the accuracy of face detection through a multi-level filtering mechanism, effectively avoiding false detection problems caused by multiple faces or background interference in complex scenes, providing a reliable face localization benchmark for subsequent color transfer processing, and significantly improving the stability and reliability of the entire makeup trial process.

[0116] The process of determining the core character region within multiple target detection boxes can specifically include the following steps: processing the initial makeup trial image using a semantic segmentation algorithm based on the multiple target detection boxes to generate a human semantic segmentation map; calculating the area of ​​the region enclosed by each target detection box; calculating the proportion of human semantic pixels within each target detection box according to the human semantic segmentation map; and selecting the target detection box with the largest area and a human semantic pixel proportion that meets a preset threshold from the multiple target detection boxes as the core character region.

[0117] A human semantic segmentation map refers to a binary image in which human pixels are accurately labeled using semantic segmentation algorithms. The region area refers to the total area of ​​the pixel region enclosed by the target detection box. The human semantic pixel percentage refers to the ratio of the number of pixels identified as human within the target detection box to the total number of pixels within the box. The specific implementation process is as follows: First, a semantic segmentation algorithm (such as the YOLOv8 semantic segmentation model) is used to process the initial makeup trial image to generate a semantic segmentation map identifying human regions. Then, for each target detection box, the area of ​​the rectangular region it encloses is calculated, and the proportion of pixels within the box identified as human by the semantic segmentation model is statistically analyzed. Finally, a reasonable percentage threshold (such as 60%) is set, and the detection boxes with the largest physical area and a human pixel percentage exceeding the threshold are selected as the core person region. This scheme, by combining the geometric features of the target detection boxes with pixel-level analysis of semantic segmentation, achieves accurate localization of the main person in the image, effectively solving problems such as multiple person interference and background misidentification that may exist in complex scenes, ensuring that subsequent processing always targets the correct person, and improving the accuracy and reliability of the entire makeup trial process.

[0118] S202. Extract a first face region image from the initial makeup trial image based on the face detection box coordinates, and extract a second face region image from the intermediate makeup trial effect image based on the face detection box coordinates.

[0119] The first face region image refers to the face region image cropped from the initial makeup trial image;

[0120] The second face region image refers to the corresponding face region image cropped from the intermediate makeup trial effect image.

[0121] In this embodiment, step S202 may include the following steps: extracting a rectangular region containing a face from the initial makeup trial image based on the face detection box coordinates; enlarging the rectangular region using an image scaling algorithm to obtain an enlarged face image; performing semantic segmentation on the enlarged face image using a face semantic segmentation model to generate a face mask; extracting a first face region image containing only the face region from the enlarged face image based on the face mask; and extracting a rectangular region containing a face from the intermediate makeup trial effect image based on the face detection box coordinates; enlarging the rectangular region using an image scaling algorithm to obtain an enlarged face image; performing semantic segmentation on the enlarged face image using a face semantic segmentation model to generate a face mask; and extracting a second face region image containing only the face region from the enlarged face image based on the face mask.

[0122] A face mask refers to a binary image generated through semantic segmentation that precisely identifies the face region. The first / second face region images refer to the clean face images extracted from the initial makeup trial image and the intermediate makeup trial effect image, respectively. The specific implementation process is as follows: First, based on the coordinates of the face detection box, rectangular regions containing the face are cropped from both the initial and intermediate makeup trial images. Then, image scaling algorithms such as Lanczos are used to enlarge the cropped regions to a preset resolution (e.g., multiples of 64). Next, a pre-trained face semantic segmentation model such as FaRL is used to process the enlarged image to generate a precise face mask. Finally, based on this mask, a valid region containing only the face is cropped from the enlarged image. This technical solution, through a standardized processing flow, ensures a strict correspondence in size and position between the face regions in the two images, providing accurate input data for face color transfer. It effectively solves the color transfer deviation problem caused by insufficient image resolution or background interference, significantly improving the accuracy and naturalness of subsequent color correction.

[0123] S203. Using the first face region image as the chromaticity reference source image and the second face region image as the chromaticity transfer target image, perform color transfer to obtain a color-corrected face region image.

[0124] A chromaticity reference source image is a reference image that provides color characteristics.

[0125] The color transfer target image refers to the target image to be color adjusted.

[0126] A color-corrected face region image refers to the image that has undergone color transfer processing.

[0127] In this embodiment, firstly, the color distribution features of the first face region image are extracted; then, the current color characteristics of the second face region image are analyzed; next, a color transfer algorithm (such as Linear-Monge-Kantorovitch) is used to transfer the reference color features to the target image; finally, the transfer result is smoothed to ensure a natural transition.

[0128] Figure 2The process described first employs a multi-level screening mechanism to accurately locate facial regions, combined with semantic segmentation analysis to ensure the selection of key figures within the image. Then, through a standardized image processing workflow, corresponding facial regions are extracted from both the initial makeup trial image and the intermediate final image, and their size is normalized using a professional algorithm. Finally, based on color transfer technology, the facial color features of the initial image are completely transferred to the intermediate final image. This solution maintains the accuracy of redrawing facial features while achieving a natural color transition between the face and body areas, ensuring the final makeup trial meets professional makeup standards. It significantly enhances the realism and visual effect of virtual makeup trials, providing reliable technical support for film and television-level makeup trial applications.

[0129] See Figure 3 This is a flowchart illustrating another embodiment of the image processing method provided in this application. Figure 3 The process shown above Figure 1 Based on the illustrated process, this section describes how to merge the color-transfer processed facial region with the intermediate makeup trial image to generate the target makeup trial image. For example... Figure 3 As shown, the process may include the following steps:

[0130] S301. The face mask is shrunk inward to obtain an optimized face mask.

[0131] An optimized face mask is a mask image generated by shrinking the boundaries of the original face mask inward through morphological processing.

[0132] In this embodiment, the initial face mask is first eroded to shrink the mask edge inward by several pixels, eliminating the transition area that may contain background impurities. This ensures that subsequent fusion only processes the clean face area and avoids edge defects.

[0133] S302. Based on the optimized face mask, the color-corrected face region image is correspondingly attached to the second face region image to obtain the attached image.

[0134] Image merging refers to the intermediate result of precisely aligning and fusing a color-corrected face image with a target region.

[0135] In this embodiment, based on the optimized face mask, a pixel-level blending algorithm is used to completely preserve the corrected face features in the area where the mask value is 1, and to achieve progressive fusion in the transition area to ensure a natural and smooth color transition.

[0136] S303. The fitting image is scaled using a bilinear interpolation algorithm so that the size of the scaled fitting image matches the size of the face area in the intermediate makeup effect image.

[0137] Bilinear interpolation is an image scaling method based on weighted calculation of neighboring pixels.

[0138] In this embodiment, firstly, the scaling ratio between the target size and the original image is calculated. Then, for each target pixel in the image, the four nearest neighbor source pixels around it are linearly weighted and calculated to achieve size adaptation while maintaining image clarity.

[0139] S304. Based on the coordinates of the face detection box, the scaled-down fitting image is fused with the intermediate makeup effect image to generate the target makeup effect image.

[0140] In this embodiment, the fusion position is first determined based on the coordinate information of the initial detection, and then advanced image fusion technology is used to seamlessly stitch the processed image with the original intermediate image, focusing on optimizing the texture and lighting consistency of the seam area.

[0141] Figure 3 The process described first eliminates edge imperfections through mask optimization, ensuring a natural fusion boundary. Then, precise fitting technology achieves pixel-level alignment between the color-corrected facial features and the target area. Next, adaptive scaling preserves image details while achieving size matching. Finally, an intelligent fusion algorithm ensures consistency in light and shadow texture between the facial area and the overall image. This systematic processing approach, while maintaining processing efficiency, enables the final generated makeup trial image to achieve professional-grade standards in edge smoothness, feature integrity, and visual uniformity, significantly enhancing the realism and usability of virtual makeup trials.

[0142] Based on the same technical concept, embodiments of this application also provide an image processing apparatus, such as... Figure 4 As shown, the device includes:

[0143] The generation module 41 is used to generate an initial makeup trial image through a first generation model, which is composed of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios.

[0144] The redrawing module 42 is used to locally redraw the face region in the initial makeup trial image to obtain the redrawn face region;

[0145] The first fusion module 43 is used to fuse the redrawn face region with the initial makeup trial image to generate an intermediate makeup trial effect image.

[0146] The migration module 44 is used to perform color migration processing on the face region in the intermediate makeup effect image based on the color features of the face region in the initial makeup image.

[0147] The second fusion module 45 is used to fuse the face region that has undergone color transfer processing with the intermediate makeup effect image to generate the target makeup effect image.

[0148] In one possible implementation, the migration module is specifically used for:

[0149] The coordinates of the face detection box are determined based on the initial makeup trial image;

[0150] Based on the coordinates of the face detection box, a first face region image is extracted from the initial makeup trial image, and a second face region image is extracted from the intermediate makeup trial effect image based on the coordinates of the face detection box.

[0151] Using the first face region image as the chromaticity reference source image and the second face region image as the chromaticity transfer target image, color transfer is performed to obtain a color-corrected face region image.

[0152] In one possible implementation, the migration module is further configured to:

[0153] The initial makeup trial image is processed using an object detection algorithm to obtain multiple object detection boxes;

[0154] The core character region is determined within the multiple target detection frames;

[0155] Copy the core character area to a blank image with the same resolution as the initial makeup trial image to generate an intermediate image;

[0156] Face detection is performed on the intermediate image to obtain face detection bounding boxes;

[0157] If multiple face detection boxes exist, the face detection box with the smallest center coordinate ordinate value is selected as the face detection box coordinate.

[0158] In one possible implementation, the migration module is further configured to:

[0159] Based on multiple target detection boxes, a semantic segmentation algorithm is used to process the initial makeup trial image to generate a human semantic segmentation map;

[0160] Calculate the area of ​​the region enclosed by each target detection box;

[0161] Based on the human semantic segmentation map, the proportion of human semantic pixels within each target detection box is calculated.

[0162] Select the target detection box with the largest area that meets the preset threshold for the proportion of human semantic pixels from multiple target detection boxes, and use it as the core human region.

[0163] In one possible implementation, the migration module is further configured to:

[0164] Based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the initial makeup trial image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a first face region image containing only the face region is extracted from the enlarged face image.

[0165] Furthermore, based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the intermediate makeup effect image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a second face region image containing only the face region is extracted from the enlarged face image.

[0166] In one possible implementation, the second fusion module is specifically used for:

[0167] The face mask is then shrunk inward to obtain an optimized face mask;

[0168] Based on the optimized face mask, the color-corrected face region image is correspondingly pasted onto the second face region image to obtain a pasted image;

[0169] The fitting image is scaled using a bilinear interpolation algorithm so that the size of the scaled fitting image matches the size of the face area in the intermediate makeup effect image.

[0170] Based on the coordinates of the face detection box, the scaled and fitted image is fused with the intermediate makeup effect image to generate the target makeup effect image.

[0171] In one possible implementation, the redraw module is specifically used for:

[0172] Face detection is performed on the initial makeup trial image to obtain the target face region image, and semantic segmentation is performed on the target face region image to obtain the target face mask;

[0173] The actor feature adaptation module is loaded into a pre-trained image generation model to form a second generation model, and the second generation model is used to perform local redrawing based on the target face mask to obtain the redrawn face region.

[0174] Based on the same technical concept, embodiments of this application also provide an electronic device, such as... Figure 5As shown, it includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0175] Memory 113 is used to store computer programs;

[0176] When processor 111 executes a program stored in memory 113, it performs the following steps:

[0177] An initial makeup trial image is generated through a first generative model, which consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios.

[0178] The face region in the initial makeup trial image is partially redrawn to obtain the redrawn face region;

[0179] The redrawn face area is merged with the initial makeup trial image to generate an intermediate makeup trial effect image;

[0180] Based on the color features of the face region in the initial makeup trial image, color transfer processing is performed on the face region in the intermediate makeup trial effect image.

[0181] The color-transfer processed facial area is merged with the intermediate makeup trial image to generate the target makeup trial image.

[0182] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0183] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0184] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0185] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0186] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image processing methods.

[0187] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image processing methods described above.

[0188] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0190] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0191] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image processing method, characterized in that, The method includes: An initial makeup trial image is generated through a first generative model, which consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios. The face region in the initial makeup trial image is partially redrawn to obtain the redrawn face region; The redrawn face area is merged with the initial makeup trial image to generate an intermediate makeup trial effect image; Based on the color features of the face region in the initial makeup trial image, color transfer processing is performed on the face region in the intermediate makeup trial effect image. The color-transfer processed facial area is merged with the intermediate makeup trial image to generate the target makeup trial image.

2. The method according to claim 1, characterized in that, The step of performing color transfer processing on the facial region in the intermediate makeup effect image based on the color features of the facial region in the initial makeup trial image includes: The coordinates of the face detection box are determined based on the initial makeup trial image; Based on the coordinates of the face detection box, a first face region image is extracted from the initial makeup trial image, and a second face region image is extracted from the intermediate makeup trial effect image based on the coordinates of the face detection box. Using the first face region image as the chromaticity reference source image and the second face region image as the chromaticity transfer target image, color transfer is performed to obtain a color-corrected face region image.

3. The method according to claim 2, characterized in that, Determining the coordinates of the face detection box based on the initial makeup trial image includes: The initial makeup trial image is processed using an object detection algorithm to obtain multiple object detection boxes; The core character region is determined within the multiple target detection frames; Copy the core character area to a blank image with the same resolution as the initial makeup trial image to generate an intermediate image; Face detection is performed on the intermediate image to obtain face detection bounding boxes; If multiple face detection boxes exist, the face detection box with the smallest center coordinate ordinate value is selected as the face detection box coordinate.

4. The method according to claim 3, characterized in that, The step of determining the core person region within the plurality of target detection boxes includes: Based on multiple target detection boxes, a semantic segmentation algorithm is used to process the initial makeup trial image to generate a human semantic segmentation map; Calculate the area of ​​the region enclosed by each target detection box; Based on the human semantic segmentation map, the proportion of human semantic pixels within each target detection box is calculated. Select the target detection box with the largest area that meets the preset threshold for the proportion of human semantic pixels from multiple target detection boxes, and use it as the core human region.

5. The method according to claim 2, characterized in that, The step of extracting a first face region image from the initial makeup trial image based on the face detection box coordinates, and extracting a second face region image from the intermediate makeup trial effect image based on the face detection box coordinates, includes: Based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the initial makeup trial image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a first face region image containing only the face region is extracted from the enlarged face image. Furthermore, based on the coordinates of the face detection box, a rectangular region containing the face is extracted from the intermediate makeup effect image; the rectangular region is enlarged using an image scaling algorithm to obtain an enlarged face image; a face semantic segmentation model is used to perform semantic segmentation on the enlarged face image to generate a face mask; based on the face mask, a second face region image containing only the face region is extracted from the enlarged face image.

6. The method according to claim 5, characterized in that, The step of fusing the color-transfer processed facial region with the intermediate makeup trial image to generate the target makeup trial image includes: The face mask is then shrunk inward to obtain an optimized face mask; Based on the optimized face mask, the color-corrected face region image is correspondingly pasted onto the second face region image to obtain a pasted image; The fitting image is scaled using a bilinear interpolation algorithm so that the size of the scaled fitting image matches the size of the face area in the intermediate makeup effect image. Based on the coordinates of the face detection box, the scaled and fitted image is fused with the intermediate makeup effect image to generate the target makeup effect image.

7. The method according to claim 1, characterized in that, The step of partially redrawing the face region in the initial makeup trial image to obtain the redrawn face region includes: Face detection is performed on the initial makeup trial image to obtain the target face region image, and semantic segmentation is performed on the target face region image to obtain the target face mask; The actor feature adaptation module is loaded into a pre-trained image generation model to form a second generation model, and the second generation model is used to perform local redrawing based on the target face mask to obtain the redrawn face region.

8. An image processing apparatus, characterized in that, The device includes: The generation module is used to generate an initial makeup trial image through a first generation model, which consists of an actor feature adaptation module and a clothing feature adaptation module loaded into a pre-trained image generation model with different weight ratios. The redrawing module is used to locally redraw the face region in the initial makeup trial image to obtain the redrawn face region; The first fusion module is used to fuse the redrawn face region with the initial makeup trial image to generate an intermediate makeup trial effect image. The migration module is used to perform color migration processing on the face region in the intermediate makeup effect image based on the color features of the face region in the initial makeup trial image; The second fusion module is used to fuse the color-transfer processed face area with the intermediate makeup effect image to generate the target makeup effect image.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the image processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image processing method according to any one of claims 1-7.

Images