Automatic half-length portrait changing method and system

Abstract

The present application provides a kind of automatic dressing method and system of half-length portrait, the method is used to execute the following steps: image input step, face detection step, portrait segmentation step, neck contour calculation step, virtual neck region boundary construction step, neck region inspection step, clothing fitting step, new image generation step, output step.The system includes image input module, face detection module, portrait segmentation module, neck contour calculation module, virtual neck region boundary construction module, neck region inspection module, clothing fitting module, new image generation module, output module.The present application can automatically detect and segment portrait, construct virtual neck region to meet fill clothes collar, so as to realize the intelligent fitting and dressing of clothing, improve user experience.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, specifically to a method and system for automatic clothing change from half-body portraits. It can automatically detect and segment the portrait, construct a virtual neck area, and realize intelligent clothing fitting and changing, applicable to multiple scenarios such as virtual try-on, online shopping, and entertainment interaction. Background Technology

[0002] With the rapid development of e-commerce and online social platforms, users' demand for virtual try-on and personalized image display is increasing. Existing portrait-changing technologies mainly rely on manual operation or simple image replacement. Some existing tools use skin tone detection methods in color spaces to detect the neck area point by point, which cannot adapt to the influence of different light sources, resulting in poor effects. Other makeup models that utilize existing stable diffusion technology alter facial features, leading to unrealistic results.

[0003] The shortcomings of existing technologies:

[0004] Complex operation: Users need to manually select and adjust clothing, which is cumbersome.

[0005] Unnatural effect: The integration of clothing and character often looks unnatural, with issues such as uneven edges and mismatched colors.

[0006] Poor adaptability: It is difficult to maintain consistency in the effect of changing clothes for portraits with different poses and backgrounds.

[0007] Therefore, developing an efficient, accurate, and natural automatic clothing replacement method and system for half-body portraits is of great practical significance. Summary of the Invention

[0008] This invention aims to solve the problems of complex operation, unnatural effects, poor adaptability and large computational load in the existing technology, and provides a method and system for automatic clothing change of half-body portraits. It can automatically detect and segment the portrait, construct a virtual neck area that fills the neckline of the clothes, realize intelligent clothing fit and change, and improve the user experience.

[0009] The present invention achieves the above objectives through the following technical solutions:

[0010] A method for automatically changing clothes in a half-body portrait includes:

[0011] Image input steps: Upload a half-body portrait photo through the system interface;

[0012] Face detection steps: Use a deep learning model to perform face detection on the uploaded half-body portrait photo and obtain facial key point information;

[0013] Human figure segmentation steps: Use a deep learning image segmentation model to accurately segment half-body portrait photos to separate key human body parts;

[0014] Neck contour calculation steps: Based on the segmented neck region mask, use a contour detection algorithm to calculate the neck contour;

[0015] Virtual neck region boundary construction steps: Based on the chin information of the facial key points and the calculated neck contour, construct a virtual neck region boundary to fill the neckline of the clothing.

[0016] Neck area inspection steps: Evaluate whether the image neck area meets the requirement of filling the virtual neck area of ​​the clothing neckline;

[0017] Clothing fitting steps: If the requirements are met, the hair, face and neck areas are preserved, and the new clothing is fitted to the neck at a suitable distance between the face and the neckline; if the requirements are not met, a new neck area is constructed, and a neck image close to the original face skin is generated through skin color transfer technology or image generation technology, and then the new clothing is fitted to the neck at a suitable distance between the face and the neckline.

[0018] New image generation step: Generate a new outfit image;

[0019] Output steps: Display the generated outfit images to the user through the system interface.

[0020] According to the present invention, an automatic face-changing method for half-body portrait photos is provided. In the face detection step, a pre-trained BlazeFace deep learning model is invoked. This model has been trained on a large amount of face image data and has the ability to automatically recognize faces and locate key points. The half-body portrait photo uploaded by the user is input into the BlazeFace model. The model extracts and processes features from the image through a convolutional neural network layer, automatically detects the face region in the photo, and further locates the key point information of the face. The key point information includes at least the contour points and feature point coordinates of the eyes, nose, mouth, chin, etc.

[0021] According to the present invention, an automatic clothing replacement method for half-body portrait photos is provided. In the portrait segmentation step, a multi-class image segmentation model pre-trained on a large-scale labeled portrait image dataset is used. This model has the ability to distinguish different semantic categories in the image. The half-body portrait photo with face detection completed is input into the model. The model automatically identifies and segments key semantic regions in the image, including at least hair, face, and neck, through multi-layer convolution and deconvolution operations combined with a context information fusion mechanism, and generates corresponding pixel-level masks. Each mask region marks the precise position and outline of the hair, face, and neck.

[0022] According to the present invention, an automatic clothing replacement method for half-body portraits is provided. In the neck contour calculation step, a neck region mask A is generated based on the output of a deep learning image segmentation model. This mask A is a binary image, where the pixel value of the neck region is 1 and the pixel value of the background region is 0. A contour detection algorithm is used to process the mask A. The algorithm principle is to detect the boundary point set of abrupt changes in grayscale value by scanning the grayscale value changes of adjacent pixels. The specific calculation formula is as follows:

[0023] For each pixel (x, y) in mask A, if there is at least one background pixel in its eight neighborhood and it is itself a pixel in the neck region, then the pixel is determined to be a contour point. All detected contour points are connected in sequence to form a closed neck contour curve B, which is mathematically represented as B={(x_i,y_i)|i=1,2,...,N}, where N is the total number of contour points and (x_i,y_i) is the coordinate of the i-th contour point.

[0024] According to the present invention, an automatic clothing replacement method for half-body portraits is provided. In the virtual neck region boundary construction step, a set of chin key points is extracted from the acquired facial key point information L, denoted as ChinPoints={( x c , y c ) i | i =1,..., n},in( x c , y c ) i Let be the coordinates of the i-th chin keypoint, and n be the total number of chin keypoints;

[0025] Polygon fitting is performed on the calculated neck contour B to obtain the polygon representation of the neck contour: NeckPolygon={( x n , y n ) j | j =1,..., m},in( x n , y n ) j Let be the coordinates of the j-th vertex of the neck contour polygon, and m be the total number of vertices;

[0026] Calculate the lowest point of the chin ,in The average x-coordinate of the key points of the chin. The minimum value of the y-coordinate of the key point of the chin;

[0027] Determine the neckline baseline: Using LowestChin as the reference point, construct a horizontal baseline CollarLine. y = y LowestChin + d ,in d This is the preset neckline height compensation value;

[0028] Constructing the virtual neck region boundary:

[0029] Calculate the set of intersection points (IntersectionPoints) between the neck contour polygon and the neckline baseline;

[0030] Sort IntersectionPoints and ChinPoints to obtain the ordered point set OrderedPoints;

[0031] The OrderedPoints are fitted using a spline curve interpolation algorithm to generate a smooth virtual neck region boundary curve C, denoted as:

[0032]

[0033] in t ∈[0,1] represents the curve parameters. N k ( t ) is a B-spline basis function. P k For control points.

[0034] According to the present invention, an automatic clothing-changing method for half-body portraits is provided, wherein in the neck area inspection step, the area of ​​the actual neck area A is calculated. S A The total number of pixels with a value of 1 in mask A is counted using the pixel counting method. ;

[0035] Calculate the area of ​​the region enclosed by the boundary C of the virtual neck region. S C The boundary curve C is calculated by integral using Green's formula:

[0036]

[0037] Define area matching index α :

[0038]

[0039] when α ≥ i When 1, the area matching requirement is met.

[0040] Polygonalization is performed on the actual neck contour B and the virtual neck boundary C to obtain a polygon. P B and P C ;

[0041] Calculate the Jaccard similarity coefficient between two polygons. β :

[0042]

[0043] Where || represents the area of ​​the polygon, when β ≥ i At time 2, the contour overlap requirement is met.

[0044] According to the present invention, an automatic clothing replacement method for half-body portrait photos is provided, which counts the number of key points located within the virtual neck region C from the acquired chin key point set ChinPoints. N in ;

[0045] Calculate key point coverage c :

[0046]

[0047] in n For the total number of key points on the chin, when c ≥ i At time 3, the key point coverage is deemed to meet the requirements;

[0048] Construct a comprehensive evaluation function F :

[0049]

[0050] in w 1, w 2, w 3 is the weighting coefficient, which satisfies w 1+ w 2+ w 3 = 1;

[0051] When the comprehensive evaluation value F ≥ i F If the image's neck region meets the requirement of filling the virtual neck region of the clothing collar, it is determined that the image does not meet the requirement.

[0052] According to the present invention, an automatic clothing changing method for half-body portraits is provided, wherein the clothing fitting step is specifically implemented as follows:

[0053] Direct bonding treatment when requirements are met:

[0054] When the comprehensive evaluation value F ≥ i F When this happens, perform the following operations:

[0055] Region Preservation: Based on the segmentation results of the portrait segmentation step, the hair region mask is preserved. H Face region mask F and original neck region mask A ;

[0056] Clothing positioning: Calculating the center point of the face C f coordinate:

[0057]

[0058] in( x fi , y fi ) represents the coordinates of facial key points, and n represents the number of key points;

[0059] Distance calculation: Determine the neckline baseline y = y collar Calculate the vertical distance from the center of the face to the neckline. d :

[0060]

[0061] Clothing deformation: according to d Values ​​of new clothing images S Perform perspective transformation, transformation matrix M Calculated using the following formula:

[0062]

[0063] in For the rotation angle of the garment, , The translation vector ensures that the lower edge of the garment is aligned with the neckline baseline.

[0064] The distorted clothing image S The Poisson blending of the region with the reserved region is represented as:

[0065]

[0066] Where Ω represents the area covered by clothing. I bg This is the original background image.

[0067] According to the present invention, an automatic clothing replacement method for half-body portraits is provided, including neck reconstruction processing when requirements are not met:

[0068] when F < i F When this happens, perform the following operations:

[0069] Geometric construction: Based on the virtual neck boundary C, extend outward by Δ w Pixel width constructs new neck region D :

[0070]

[0071] Skin color transfer: A skin color transfer algorithm based on histogram matching is used to calculate the original face region. F LAB color space histogram H F and new neck area D histogram H D Through optimal transport mapping T Achieve color conversion:

[0072]

[0073] in c ijk To meet transmission costs, and ;

[0074] Texture generation: Texture enhancement is performed on the areas after skin color migration using a Gabor filter bank. G λ,θ Extracting texture features:

[0075]

[0076] in x ′= x cos i + y sin i , y ′= x sin i + y cos i Natural textures are generated by optimizing filter parameters;

[0077] Clothing fit: for the reconstructed neck area D Repeat the clothing positioning, deformation, and blending steps to meet the requirements, except that the distance calculation is now based on the geometric center of the new neck region. CD :

[0078]

[0079] in m For the region D The number of pixels.

[0080] An automatic clothing-changing system for half-body portrait photos includes:

[0081] Image input module, used to upload half-body portrait photos through the system interface;

[0082] The face detection module is used to perform face detection on uploaded half-body portrait photos using a deep learning model and obtain facial key point information;

[0083] The portrait segmentation module is used to accurately segment half-body portrait photos using a deep learning image segmentation model to separate key human body parts;

[0084] The neck contour calculation module is used to calculate the neck contour based on the segmented neck region mask using a contour detection algorithm.

[0085] The virtual neck region boundary construction module is used to construct a virtual neck region boundary that fills the neckline of clothing based on the chin information of the facial key points and the calculated neck contour.

[0086] The neck region inspection module is used to evaluate whether the neck region of the image meets the requirement of filling the virtual neck region of the clothing neckline;

[0087] The clothing fitting module, if it meets the requirements, is used to preserve the hair, face and neck areas, and to fit the new clothing onto the neck at a suitable distance between the face and the neckline; if it does not meet the requirements, it is used to construct a new neck area, and to generate a neck image that is close to the original face skin through skin color transfer technology or image generation technology, and then fit the new clothing onto the neck at a suitable distance between the face and the neckline.

[0088] A new image generation module is used to generate new outfit images;

[0089] The output module is used to display the generated outfit images to the user through the system interface.

[0090] Therefore, compared with the prior art, the automatic clothing changing method and system for half-body portraits proposed in this invention have the following beneficial effects:

[0091] 1. This invention integrates fully automated functions such as image input, face detection, portrait segmentation, neck contour calculation, virtual neck region construction, clothing fitting, and new image generation. Users only need to upload a half-body portrait photo through the system interface, and the system can automatically complete all subsequent dressing steps without any manual intervention or complex operations from the user, greatly simplifying the dressing process and improving the user experience.

[0092] 2. By constructing a virtual neck area boundary that fills the neckline of the clothing, this invention ensures a perfect fit between the new clothing and the original portrait's neck area, avoiding issues of clothing floating or misalignment. Simultaneously, by combining clothing perspective transformation with Poisson fusion technology, a natural transition between the clothing and the original image background is achieved, resulting in a more realistic and natural image after the outfit change, significantly enhancing the user's visual experience.

[0093] 3. Addressing potential defects in the neck area of ​​a person's image (such as occlusion, shadows, incompleteness, etc.), this invention can intelligently reconstruct a neck area that matches the original face's skin tone and has a natural transition. Through skin tone transfer technology and image generation technology, the system can generate a virtual neck that matches the skin tone and texture of the neck area in the original image. This allows new clothing to fit more closely on the "reconstructed" neck, thus adapting to photos with more different states and facial features, improving the system's versatility and applicability.

[0094] 4. This invention utilizes a deep learning model for high-precision face detection and key point localization, accurately acquiring the positions of various feature points on the face. Simultaneously, by combining neck contour calculation and virtual neck region construction technology, the system can precisely determine the fit and orientation of clothing on the portrait, ensuring the accuracy of the dressing process and avoiding problems such as clothing misalignment or clipping.

[0095] 5. The dressing method of this invention is not only applicable to the replacement of a single style of clothing, but can also flexibly meet the dressing needs of different styles and materials of clothing. By adjusting the parameters of clothing perspective transformation and the Poisson fusion strategy, the system can ensure that various styles of clothing can naturally fit the portrait, thereby broadening the application scope of dressing technology and meeting the needs of more users.

[0096] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0097] Figure 1 This is a flowchart of an embodiment of the automatic clothing change method for half-body portraits according to the present invention.

[0098] Figure 2 This is a detailed flowchart of an embodiment of the automatic clothing change method for half-body portraits according to the present invention.

[0099] Figure 3 This is a schematic diagram of an embodiment of the automatic clothing changing system for half-body portraits according to the present invention. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0101] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0102] An embodiment of an automatic clothing change method for half-body portraits

[0103] See Figure 1 and Figure 2 This embodiment provides a method for automatic clothing changing in a half-body portrait, including:

[0104] Image input steps: Upload a half-body portrait photo through the system interface;

[0105] Face detection steps: Use a deep learning model to perform face detection on the uploaded half-body portrait photo and obtain facial key point information;

[0106] Human figure segmentation steps: Use a deep learning image segmentation model to accurately segment half-body portrait photos to separate key human body parts;

[0107] Neck contour calculation steps: Based on the segmented neck region mask, use a contour detection algorithm to calculate the neck contour;

[0108] Virtual neck region boundary construction steps: Based on the chin information of the facial key points and the calculated neck contour, construct a virtual neck region boundary to fill the neckline of the clothing.

[0109] Neck area inspection steps: Evaluate whether the image neck area meets the requirement of filling the virtual neck area of ​​the clothing neckline;

[0110] Clothing fitting steps: If the requirements are met, the hair, face and neck areas are preserved, and the new clothing is fitted to the neck at a suitable distance between the face and the neckline; if the requirements are not met, a new neck area is constructed, and a neck image close to the original face skin is generated through skin color transfer technology or image generation technology, and then the new clothing is fitted to the neck at a suitable distance between the face and the neckline.

[0111] New image generation step: Generate a new outfit image;

[0112] Output steps: Display the generated outfit images to the user through the system interface.

[0113] In the face detection step, the pre-trained BlazeFace deep learning model is invoked. This model has been trained on a large amount of face image data and has the ability to automatically recognize faces and locate key points. The half-body portrait photo uploaded by the user is input into the BlazeFace model. The model extracts and processes features from the image through convolutional neural network layers, automatically detects the face region in the photo, and further locates the key point information of the face. This key point information includes at least the contour points and feature point coordinates of the eyes, nose, mouth, chin and other parts.

[0114] In the portrait segmentation step, a multi-class image segmentation model pre-trained on a large-scale labeled portrait image dataset is used. This model has the ability to distinguish different semantic categories in the image. The half-body portrait photo with face detection completed is input into the model. The model automatically identifies and segments key semantic regions in the image, including at least hair, face, and neck, through multi-layer convolution and deconvolution operations combined with a context information fusion mechanism, and generates corresponding pixel-level masks. Each mask region marks the precise location and outline of hair, face, and neck.

[0115] In the neck contour calculation step, a neck region mask A is generated based on the output of a deep learning image segmentation model. This mask A is a binary image, where the pixel value of the neck region is 1 and the pixel value of the background region is 0. A contour detection algorithm (such as the cv::findContours function in the OpenCV library) is used to process mask A. The algorithm works by scanning the grayscale value changes of adjacent pixels to detect the set of boundary points where grayscale values ​​change abruptly. The specific calculation formula is as follows:

[0116] For each pixel (x, y) in mask A, if its eight neighbors contain at least one background pixel (value 0) and it is itself a neck region pixel (value 1), then the pixel is determined to be a contour point. All detected contour points are connected in sequence to form a closed neck contour curve B, which is mathematically represented as B={(x_i, y_i)|i=1,2,...,N}, where N is the total number of contour points and (x_i, y_i) is the coordinate of the i-th contour point.

[0117] In the virtual neck region boundary construction step, a set of chin key points is extracted from the acquired facial key point information L, denoted as ChinPoints={( x c , y c ) i | i =1,..., n},in( x c , y c ) i Let be the coordinates of the i-th chin keypoint, and n be the total number of chin keypoints;

[0118] Polygon fitting is performed on the calculated neck contour B to obtain the polygon representation of the neck contour: NeckPolygon={( x n , y n ) j | j =1,..., m},in( x n , y n ) j Let be the coordinates of the j-th vertex of the neck contour polygon, and m be the total number of vertices;

[0119] Calculate the lowest point of the chin ,in The average x-coordinate of the key points of the chin. The minimum value of the y-coordinate of the key point of the chin;

[0120] Determine the neckline baseline: Using LowestChin as the reference point, construct a horizontal baseline CollarLine. y = y LowestChin + d ,in d This is the preset neckline height compensation value;

[0121] Constructing the virtual neck region boundary:

[0122] Calculate the set of intersection points (IntersectionPoints) between the neck contour polygon and the neckline baseline;

[0123] Sort IntersectionPoints and ChinPoints to obtain the ordered point set OrderedPoints;

[0124] The OrderedPoints are fitted using a spline curve interpolation algorithm (such as B-spline) to generate a smooth virtual neck region boundary curve C, denoted as:

[0125]

[0126] in t ∈[0,1] represents the curve parameters. N k ( t ) is a B-spline basis function. P k Control points (including points in OrderedPoints);

[0127] In this embodiment, the final virtual neck region boundary C must satisfy:

[0128] Geometric constraints: Boundary curve C and neck contour B form a closed region below the neckline baseline;

[0129] Topological constraint: The union of the boundary curve C and the original neck contour B can completely cover the neckline area of ​​the clothing; Aesthetic constraint: The rate of change of curvature of the boundary curve C does not exceed a preset threshold to ensure visual naturalness.

[0130] During the neck region examination procedure, the area of ​​the actual neck region A is calculated. S A The total number of pixels with a value of 1 in mask A is counted using the pixel counting method. ;

[0131] Calculate the area of ​​the region enclosed by the boundary C of the virtual neck region. S C The boundary curve C is calculated by integral using Green's formula:

[0132]

[0133] Define area matching index α :

[0134]

[0135] when α ≥ i 1 ( i When 1 is a preset threshold (e.g., 0.85), the area matching requirement is considered met.

[0136] Polygonalization is performed on the actual neck contour B and the virtual neck boundary C to obtain a polygon. P B and P C ;

[0137] Calculate the Jaccard similarity coefficient between two polygons. β :

[0138]

[0139] Where || represents the area of ​​the polygon, when β ≥ i 2 ( i When 2 is a preset threshold (e.g., 0.75), the contour overlap is determined to meet the requirements.

[0140] From the obtained chin keypoint set ChinPoints, count the number of keypoints located within the virtual neck region C. N in ;

[0141] Calculate key point coverage c :

[0142]

[0143] in n For the total number of key points on the chin, when c ≥ i 3 ( i When 3 is a preset threshold (e.g., 0.9), the key point coverage is deemed to meet the requirements.

[0144] Construct a comprehensive evaluation function F :

[0145]

[0146] in w 1, w 2, w 3 is the weighting coefficient, which satisfies w 1+ w 2+ w 3 = 1;

[0147] When the comprehensive evaluation value F ≥ i F ( i F When a preset comprehensive threshold (e.g., 0.8) is set, the image's neck region is determined to meet the requirement of filling the virtual neck region of the clothing collar; otherwise, it is determined not to meet the requirement.

[0148] In this embodiment, the garment fitting step is specifically implemented as follows:

[0149] Direct bonding treatment when requirements are met:

[0150] When the comprehensive evaluation value F ≥ i F When this happens, perform the following operations:

[0151] Region Preservation: Based on the segmentation results of the portrait segmentation step, the hair region mask is preserved. H Face region mask F and original neck region mask A ;

[0152] Clothing positioning: Calculating the center point of the face C f coordinate:

[0153]

[0154] in( x fi , y fi ) represents the coordinates of facial key points, and n represents the number of key points;

[0155] Distance calculation: Determine the neckline baseline y = y collar (Taken from CollarLine), calculate the vertical distance from the center of the face to the neckline. d :

[0156]

[0157] Clothing deformation: according to d Values ​​of new clothing images S Perform perspective transformation, transformation matrix M Calculated using the following formula:

[0158]

[0159] in The rotation angle of the clothing (determined based on the direction the person's face is facing). , The translation vector ensures that the lower edge of the garment is aligned with the neckline baseline.

[0160] The distorted clothing image S The Poisson blending of the region with the reserved region is represented as:

[0161]

[0162] Where Ω represents the area covered by clothing. I bg This is the original background image.

[0163] Neck reconstruction treatment when requirements are not met:

[0164] whenF < i F When this happens, perform the following operations:

[0165] Geometric construction: Based on the virtual neck boundary C, extend outward by Δ w Pixel width constructs new neck region D :

[0166]

[0167] Skin color transfer: A skin color transfer algorithm based on histogram matching is used to calculate the original face region. F LAB color space histogram H F and new neck area D histogram H D Through optimal transport mapping T Achieve color conversion:

[0168]

[0169] in c ijk To meet transmission costs, and ;

[0170] Texture generation: Texture enhancement is performed on the areas after skin color migration using a Gabor filter bank. G λ,θ Extracting texture features:

[0171]

[0172] in x ′= x cos i + y sin i , y ′= x sin i + y cos i Natural textures are generated by optimizing filter parameters;

[0173] Clothing fit: for the reconstructed neck area D Repeat the clothing positioning, deformation, and blending steps to meet the requirements, except that the distance calculation is now based on the geometric center of the new neck region. CD :

[0174]

[0175] inm For the region D The number of pixels.

[0176] After completing the clothing fitting step, that is, after determining the relative position and transformation parameters of the new clothing image (denoted as S′) and the portrait preservation area (including the hair area mask H, the face area mask F, and the adapted neck area mask; if neck reconstruction exists, it is the reconstructed neck area D), the new image generation step performs the following operations:

[0177] Initialize a new image canvas: Create a new image canvas (denoted as Canvas) with the same size as the original input image, and set its background to transparent or a specific background color according to application requirements.

[0178] Layered fusion processing:

[0179] Bottom layer (background layer): If the original image background needs to be preserved, copy the original background image (denoted as Ibg) to the corresponding position on the Canvas; if the background needs to be replaced or set to transparent, skip this step.

[0180] Middle layer (preservation area layer): Based on the portrait segmentation results, the pixel values ​​of the hair region mask H, the face region mask F, and the adapted neck region (original or reconstructed) are directly copied to the corresponding positions on the Canvas to form the basic outline of the portrait.

[0181] Top layer (clothing layer): The new clothing image S′, after perspective transformation and deformation processing, is placed on the Canvas. Its position and orientation are determined by the calculated center point C of the human face. f The vertical distance d to the neckline, the rotation angle of the garment (determined based on the direction of the face), and the translation vector are all determined.

[0182] Specifically, the placement of the garment image S′ must ensure that its lower edge is aligned with the neckline baseline (denoted as CollarLine), and the transformation matrix... M It is calculated using the formula above.

[0183] Poisson Blending Process: To achieve a natural transition between the clothing and portrait retention areas, a Poisson fusion algorithm is used to process the boundary between the clothing layer and the portrait layer. Specifically, the fusion region Ω is defined as the boundary between the clothing-covered area and the portrait retention area, and seamless fusion is achieved through the aforementioned Poisson equation.

[0184] Output the new image: After the above layered blending and Poisson blending processes, the image on the Canvas is the final generated transformation image. This image can be displayed to the user through the system interface, or further post-processed (such as sharpening, color adjustment, etc.) can be performed before output.

[0185] An embodiment of an automatic clothing changing system for half-body portraits

[0186] like Figure 3 As shown in the figure, this embodiment provides an automatic clothing changing system for half-body portraits, including:

[0187] Image input module, used to upload half-body portrait photos through the system interface;

[0188] The face detection module is used to perform face detection on uploaded half-body portrait photos using a deep learning model and obtain facial key point information;

[0189] The portrait segmentation module is used to accurately segment half-body portrait photos using a deep learning image segmentation model to separate key human body parts;

[0190] The neck contour calculation module is used to calculate the neck contour based on the segmented neck region mask using a contour detection algorithm.

[0191] The virtual neck region boundary construction module is used to construct a virtual neck region boundary that fills the neckline of clothing based on the chin information of the facial key points and the calculated neck contour.

[0192] The neck region inspection module is used to evaluate whether the neck region of the image meets the requirement of filling the virtual neck region of the clothing neckline;

[0193] The clothing fitting module, if it meets the requirements, is used to preserve the hair, face and neck areas, and to fit the new clothing onto the neck at a suitable distance between the face and the neckline; if it does not meet the requirements, it is used to construct a new neck area, and to generate a neck image that is close to the original face skin through skin color transfer technology or image generation technology, and then fit the new clothing onto the neck at a suitable distance between the face and the neckline.

[0194] A new image generation module is used to generate new outfit images;

[0195] The output module is used to display the generated outfit images to the user through the system interface.

[0196] In summary, this invention integrates fully automated functions such as image input, face detection, portrait segmentation, neck contour calculation, virtual neck region construction, clothing fitting, and new image generation. Users only need to upload a half-body portrait photo through the system interface, and the system can automatically complete all subsequent dressing steps without any manual intervention or complex operations from the user, greatly simplifying the dressing process and improving the user experience.

[0197] Furthermore, by constructing a virtual neck area boundary that fills the neckline of the clothing, this invention ensures a perfect fit between the new clothing and the original portrait's neck area, avoiding issues of clothing floating or misalignment. Simultaneously, by combining clothing perspective transformation with Poisson fusion technology, a natural transition between the clothing and the original image background is achieved, resulting in a more realistic and natural image after the outfit change, significantly enhancing the user's visual experience.

[0198] Furthermore, addressing potential defects in the neck area of ​​a person's image (such as occlusion, shadows, incompleteness, etc.), this invention can intelligently reconstruct a neck area that matches the original face's skin tone and has a natural transition. Through skin tone transfer technology and image generation technology, the system can generate a virtual neck that matches the skin tone and texture of the neck area in the original image, allowing new clothing to fit more closely on the "reconstructed" neck. This adapts to photos with more different states and facial features, improving the system's versatility and applicability.

[0199] Furthermore, this invention utilizes deep learning models for high-precision face detection and key point localization, accurately acquiring the positions of various feature points on the face. Simultaneously, by combining neck contour calculation and virtual neck region construction technology, the system can precisely determine the fit and orientation of clothing on the portrait, ensuring the accuracy of the dressing process and avoiding problems such as clothing misalignment or clipping.

[0200] Furthermore, the dressing method of this invention is not only applicable to the replacement of a single style of clothing, but can also flexibly meet the dressing needs of different styles and materials of clothing. By adjusting the parameters of clothing perspective transformation and the Poisson fusion strategy, the system can ensure that various styles of clothing can naturally fit the portrait, thereby broadening the application scope of dressing technology and meeting the needs of more users.

[0201] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0202] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method of automatically changing a half-length portrait, characterized by, include: Image input steps: Upload a half-body portrait photo through the system interface; Face detection steps: Use a deep learning model to perform face detection on the uploaded half-body portrait photo and obtain facial key point information; Human figure segmentation steps: Use a deep learning image segmentation model to accurately segment half-body portrait photos to separate key human body parts; Neck contour calculation steps: Based on the segmented neck region mask, use a contour detection algorithm to calculate the neck contour; Virtual neck region boundary construction steps: Based on the chin information of the facial key points and the calculated neck contour, construct the virtual neck region boundary to fill the neckline of the clothing. Neck area inspection steps: Evaluate whether the image neck area meets the requirement of filling the virtual neck area of ​​the clothing neckline; Clothing fitting steps: If the requirements are met, the hair, face and neck areas are preserved, and the new clothing is fitted to the neck at a suitable distance between the face and the neckline; if the requirements are not met, a new neck area is constructed, and a neck image close to the original face skin is generated through skin color transfer technology or image generation technology, and then the new clothing is fitted to the neck at a suitable distance between the face and the neckline. New image generation step: Generate a new outfit image; Output steps: Display the generated outfit images to the user through the system interface; In the neck region checking step, the area of the actual neck region A is calculated S A The total number of pixels in mask A with value 1 is counted by pixel counting method, i.e. ; calculating the area of the region enclosed by the virtual neck region boundary C S C Green's theorem is used to integrate the boundary curve C: Define area matching index α : when α ≥ θ At time 1, the area matching requirement is met, where, θ 1 represents the first preset threshold; Polygonalization is performed on the actual neck contour B and the virtual neck boundary C to obtain a polygon. P B and P C ; Calculate the Jaccard similarity coefficient between two polygons. β : Where || represents the area of ​​the polygon, when β ≥ θ At time 2, the contour overlap requirement is met, where, θ 2 represents the second preset threshold; From the obtained chin keypoint set ChinPoints, count the number of keypoints located within the virtual neck region C. N in ; Calculate key point coverage γ : in n For the total number of key points on the chin, when γ ≥ θ At time 3, the key point coverage is deemed to meet the requirements, among which, θ 3 represents the third preset threshold. Construct a comprehensive evaluation function F : in w 1, w 2, w 3 is the weighting coefficient, which satisfies w 1+ w 2+ w 3 = 1; When the comprehensive evaluation value F ≥ θ F When the image's neck region meets the requirement of filling the virtual neck region of the clothing collar, it is determined that the requirement is not met. θ F This is the preset comprehensive threshold.

2. The method according to claim 1, characterized in that: In the face detection step, the pre-trained BlazeFace deep learning model is invoked. This model has been trained on a large amount of face image data and has the ability to automatically recognize faces and locate key points. The half-body portrait photo uploaded by the user is input into the BlazeFace model. The model extracts and processes features from the image through convolutional neural network layers, automatically detects the face region in the photo, and further locates the key point information of the face. This key point information includes at least the contour points and feature point coordinates of the eyes, nose, mouth, chin and other parts.

3. The method according to claim 1, characterized in that: In the portrait segmentation step, a multi-class image segmentation model pre-trained on a large-scale labeled portrait image dataset is used. This model has the ability to distinguish different semantic categories in the image. The half-body portrait photo with face detection completed is input into the model. The model automatically identifies and segments key semantic regions in the image, including at least hair, face, and neck, through multi-layer convolution and deconvolution operations combined with a context information fusion mechanism, and generates corresponding pixel-level masks. Each mask region marks the precise location and outline of hair, face, and neck.

4. The method according to claim 1, characterized in that: In the neck contour calculation step, a neck region mask A is generated based on the output of a deep learning image segmentation model. This mask A is a binary image, where the pixel value of the neck region is 1 and the pixel value of the background region is 0. A contour detection algorithm is used to process mask A. The algorithm principle is to detect the boundary point set of abrupt changes in gray value by scanning the gray value changes of adjacent pixels. The specific calculation formula is as follows: For each pixel (x, y) in mask A, if there is at least one background pixel in its eight neighborhood and it is itself a pixel in the neck region, then the pixel is determined to be a contour point. All detected contour points are connected in sequence to form a closed neck contour curve B, which is mathematically represented as B={(x_i,y_i)|i=1,2,...,N}, where N is the total number of contour points and (x_i,y_i) is the coordinate of the i-th contour point.

5. The method according to claim 4, characterized in that: In the virtual neck region boundary construction step, a set of chin key points is extracted from the acquired facial key point information L, denoted as ChinPoints={( x c , y c ) i | i =1,..., n },in( x c , y c ) i Let be the coordinates of the i-th chin keypoint, and n be the total number of chin keypoints; Polygon fitting is performed on the calculated neck contour B to obtain the polygon representation of the neck contour: NeckPolygon={( x n , y n ) j | j =1,..., m },in( x n , y n ) j Let be the coordinates of the j-th vertex of the neck contour polygon, and m be the total number of vertices; Calculate the lowest point of the chin ,in The average x-coordinate of the key points of the chin. The minimum value of the y-coordinate of the key point of the chin; Determine the neckline baseline: Using LowestChin as the reference point, construct a horizontal baseline CollarLine. y = y LowestChin + δ ,in δ This is the preset neckline height compensation value; Constructing the virtual neck region boundary: Calculate the set of intersection points (IntersectionPoints) between the neck contour polygon and the neckline baseline; Sort IntersectionPoints and ChinPoints to obtain the ordered point set OrderedPoints; The OrderedPoints are fitted using a spline curve interpolation algorithm to generate a smooth virtual neck region boundary curve C, denoted as: in t ∈[0,1] represents the curve parameters. N k ( t ) is a B-spline basis function. P k For control points.

6. The method according to claim 5, characterized in that, The specific implementation of the garment bonding step is as follows: Direct bonding treatment when requirements are met: When the comprehensive evaluation value F ≥ θ F When this happens, perform the following operations: Region Preservation: Based on the segmentation results of the portrait segmentation step, the hair region mask is preserved. H Face region mask F and original neck region mask A ; Clothing positioning: Calculating the center point of the face C f coordinate: in( x fi , y fi ) represents the coordinates of facial key points, and n represents the number of key points; Distance calculation: Determine the neckline baseline y = y collar Calculate the vertical distance from the center of the face to the neckline. d : Clothing deformation: according to d Values ​​of new clothing images S Perform perspective transformation, transformation matrix M Calculated using the following formula: in For the rotation angle of the garment, , The translation vector ensures that the lower edge of the garment is aligned with the neckline baseline. The distorted clothing image S The Poisson blending of the region with the reserved region is represented as: Where Ω represents the area covered by clothing. I bg This is the original background image.

7. The method according to claim 6, characterized in that: Neck reconstruction treatment when requirements are not met: when F < θ F When this happens, perform the following operations: Geometric construction: Based on the virtual neck boundary C, extend outward by Δ w Pixel width constructs new neck region D : Skin color transfer: A skin color transfer algorithm based on histogram matching is used to calculate the original face region. F LAB color space histogram H F and new neck area D histogram H D Through optimal transport mapping T Achieve color conversion: in c ijk To meet transmission costs, and ; Texture generation: Texture enhancement is performed on the areas after skin color migration using a Gabor filter bank. G λ,θ Extract texture feature: in x ′= x cos θ + y sin θ , y ′= x sin θ + y cos θ Natural textures are generated by optimizing filter parameters; Clothing fit: for the reconstructed neck area D Repeat the clothing positioning, deformation, and blending steps to meet the requirements, except that the distance calculation is now based on the geometric center of the new neck region. Cd : in m For the region D The number of pixels.

8. An automatic clothing-changing system for half-body portraits, characterized in that, include: Image input module, used to upload half-body portrait photos through the system interface; The face detection module is used to perform face detection on uploaded half-body portrait photos using a deep learning model and obtain facial key point information; The portrait segmentation module is used to accurately segment half-body portrait photos using a deep learning image segmentation model to separate key human body parts; The neck contour calculation module is used to calculate the neck contour based on the segmented neck region mask using a contour detection algorithm. The virtual neck region boundary construction module is used to construct a virtual neck region boundary that fills the neckline of clothing based on the chin information of the facial key points and the calculated neck contour. The neck region inspection module is used to evaluate whether the neck region of the image meets the requirement of filling the virtual neck region of the clothing neckline; The clothing fitting module, if it meets the requirements, is used to preserve the hair, face and neck areas, and to fit the new clothing onto the neck at a suitable distance between the face and the neckline; if it does not meet the requirements, it is used to construct a new neck area, and to generate a neck image that is close to the original face skin through skin color transfer technology or image generation technology, and then fit the new clothing onto the neck at a suitable distance between the face and the neckline. A new image generation module is used to generate new outfit images; The output module is used to display the generated outfit images to the user through the system interface; In the neck region inspection module, the area of ​​the actual neck region A is calculated. S A The total number of pixels with a value of 1 in mask A is counted using the pixel counting method. ; Calculate the area of ​​the region enclosed by the boundary C of the virtual neck region. S C The boundary curve C is calculated by integral using Green's formula: Define area matching index α : when α ≥ θ At time 1, the area matching requirement is met, where, θ 1 represents the first preset threshold; Polygonalization is performed on the actual neck contour B and the virtual neck boundary C to obtain a polygon. P B and P C ; Calculate the Jaccard similarity coefficient between two polygons. β : Where || represents the area of ​​the polygon, when β ≥ θ At time 2, the contour overlap requirement is met, where, θ 2 represents the second preset threshold; From the obtained chin keypoint set ChinPoints, count the number of keypoints located within the virtual neck region C. N in ; Calculate key point coverage γ : in n For the total number of key points on the chin, when γ ≥ θ At time 3, the key point coverage is deemed to meet the requirements, among which, θ 3 represents the third preset threshold. Construct a comprehensive evaluation function F : in w 1, w 2, w 3 is the weighting coefficient, which satisfies w 1+ w 2+ w 3 = 1; When the comprehensive evaluation value F ≥ θ F When the image's neck region meets the requirement of filling the virtual neck region of the clothing collar, it is determined that the requirement is not met. θ F This is the preset comprehensive threshold.

Images