A contour-preserving feature grid deformation system and method for virtual fitting

Abstract

Disclosed by the present invention is a contour-preserving feature grid deformation system and method for virtual fitting, including creating a human body model database; training a clothing category classification and key point detection model, a clothing segmentation model; extracting the clothing contour line and then subdividing the clothing area by a triangular grid, through similarity transformation and differential deformation of the clothing grid; constructing a linear least squares sparse system for the clothing contour line to solve for the new contour line; performing a penetration test on the closed contour points; iteratively executing the sparse system solution for contour preservation and the penetration test until the penetration test is passed, and performing clothing grid deformation with the contour line as the position constraint; decomposing the clothing picture into foreground and background according to the clothing segmentation result, and performing hierarchical fusion with the human body model to complete virtual fitting, solving the problem of contour distortion caused by severe deformation of clothing pictures in virtual fitting applications based on image processing, reducing the user's usage cost, and providing users with an economical, convenient, real and natural fitting experience.

Description

Technical Field

[0001] The present invention relates to a deformation system and method thereof, and more specifically, to a contour-preserving feature grid deformation system and method for virtual fitting, belonging to the technical field of image processing. Background Art

[0002] With the improvement of living standards, consumers have an urgent need for the economic convenience of clothing fitting. Virtual fitting displays the upper body effect of clothing in a digital way. Users can quickly switch clothing combinations without incurring the costs of purchase and transportation, improving and meeting the users' fitting experience and personalized requirements.

[0003] The image grid deformation technology for virtual fitting combines computer graphics, image processing, and machine learning to quickly register and fuse clothing models and human body models. In this image processing process, how to maintain the clothing contour pattern and provide users with a real upper body matching effect is the key point and difficulty in the research of virtual fitting technology. Summary of the Invention

[0004] In order to solve the above-mentioned problems in the prior art, the present invention provides a contour-preserving feature grid deformation system and method for virtual fitting, which has the technical characteristics of being able to quickly and efficiently obtain fitting results, while maintaining the original contour shape of the clothing, and providing users with an economic, convenient, real, and natural fitting experience.

[0005] To achieve the above object, the present invention is realized through the following technical solutions:

[0006] A contour-preserving feature grid deformation system for virtual fitting according to the present invention is characterized in that: it includes an electronic device 100, and the electronic device 100 includes an image deformation fitting device 110, a memory 111, a processor 112, and a communication unit 113; the memory 111, the processor 112, and the communication unit 113 are connected to realize data transmission or interaction;

[0007] The memory 111 is used to store programs, the processor 112 executes the programs after receiving execution instructions, and the communication unit 113 is used to establish a communication connection between the electronic device 100 and other devices through the network, and is used to receive and send data through the network;

[0008] The image deformation fitting device 110 includes a fitting model data creation module 1101, a clothing detection module 1102, and an image processing module 1103;

[0009] Among them, the fitting model data creation module 1101 is used to mark and save the positions and attributes of the key points and contour lines of the human model for subsequent fitting judgment of the worn clothing and calculation of the deformation target position;

[0010] The clothing detection module 1102 is used to classify the categories of the trial clothing and detect key points, as well as segment the clothing area and the back area of the clothing, so as to determine the clothing deformation strategy and key control points, and provide semantic information and geometric information for the clothing grid deformation;

[0011] The image processing module 1103 is used for image triangular mesh division, as well as global and local deformation registration and fusion of the clothing and the human body model. It iteratively conducts penetration tests and corrects by constructing and solving a linear sparse system that preserves contour features to obtain the trial fitting result.

[0012] A method for grid deformation that preserves contour features for virtual fitting according to the present invention is characterized in that the method includes the following steps:

[0013] Step 20: Before the fitting, create the trial fitting human body model data and save it to the trial fitting database for reuse in various categories of clothing; use the self-owned labeled data to train the clothing category classification and key point detection model, as well as the clothing segmentation model, for obtaining clothing semantic information and geometric information;

[0014] Step 21: Obtain the clothing picture to be tried on and the specified trial fitting model;

[0015] Step 22: Classify the category of the clothing picture and detect key points, as well as segment the clothing area and the back mask of the clothing;

[0016] Step 23: According to the constraints such as the corresponding key points and contour shapes of the clothing and the model, differentially deform the clothing picture and fuse it to the model to generate the virtual fitting result.

[0017] Preferably, the specific steps of creating the trial fitting human body model data and saving it to the trial fitting database in Step 20 include:

[0018] Step 24: Label the position and attributes of the key points of the human body model torso;

[0019] Step 25: Extract the contour line of the human body model and partition it according to the body part where the contour line is located;

[0020] Step 26: Save the labeling results of the key points and the contour line to the trial fitting database for convenient reuse in the fitting of different categories of clothing.

[0021] Preferably, the key points in Step 24 include the shoulders, elbows, wrists, armpits, hips, crotches, knees and ankles of the human body model, and record the coordinate positions of their human body images and their part names;

[0022] Specifically in Step 25, the human body contour line is divided into 10 regions including the left and right arms and the inner and outer sides of the left and right arms, the left and right legs and the inner and outer sides of the left and right legs, and the left and right trunks, and record the contour point positions and the partitions where the contour points are located;

[0023] Specifically in step 26, the relevant data of the mannequin is marked and saved as a binary file. Subsequently, according to the fitting mannequin specified by the user, the relevant data is read from this file.

[0024] Preferably, training the clothing category classification and key point detection model and the clothing segmentation model in step 20 specifically includes the following steps:

[0025] Step S27: Use Labelme software to mark the data of self-owned clothing pictures;

[0026] Step S28: Use the HRNet convolutional neural network to train the self-owned dataset to obtain a clothing classification and key point detection model;

[0027] Step S29: Use the MaskR-CNN convolutional neural network to train the self-owned dataset to obtain a mask segmentation model for the clothing area and the back area.

[0028] Preferably, marking the data of self-owned clothing pictures in step S27 includes 5 clothing category labels of upper clothes, dresses, coats, skirts, and trousers, and the key points of the shoulders, cuffs, hems, trouser legs, crotches, and skirt hems of each clothing category, as well as the clothing area of each category and the back areas of the necklines, cuffs, and hems.

[0029] Preferably, the said step S23 includes sub-steps S231, S232, S233, S234, S235, and S236;

[0030] Step S231: Subdivide the clothing picture by triangular meshes; extract the contour line of the clothing area according to the clothing segmentation mask. Take the points on the contour line, together with the clothing key points, as the set of grid points to be subdivided, and the clothing contour line as the grid edge set. Use the Triangle algorithm library to subdivide the clothing picture into a set of triangular units to improve the image processing efficiency;

[0031] Step S232: Roughly align the clothing picture and the mannequin through similarity transformation and local Laplacian differential deformation successively;

[0032] Step S233: Integrate the contour shape constraint, the target position constraint, and the length constraint, construct and solve a linear least squares sparse system to obtain the position of the new contour line that maintains the original contour shape features;

[0033] Step S234: Conduct a penetration test on the clothing closed contour points, and update the safe position of the penetration points as the position constraint condition of the sparse linear system. The clothing closed contour points should be outside the corresponding mannequin contour boundary. Otherwise, the human body image will pass through the clothing contour, resulting in a penetration phenomenon, which is visually unreasonable. The steps of the penetration test and updating the penetration point position are as follows:

[0034] Select the closed contour points of the clothing ;

[0035] Use the kd-tree to find The nearest point in the corresponding contour partition line segment of the mannequin ;

[0036] According to The line segment vectors with the adjacent front and rear points and Calculate the normal vector of the human contour line at ; ;

[0037] Connect and as the vector Calculate the cross product of the vectors ; If the vector The z component of then record the current point as a penetration point and update to where is the gap margin from the updated contour point to the mannequin contour, is The normal vector at the position, obtained by calculating the line segment vectors of with the adjacent front and rear points and Design the gap margin ; If then the closed contour point passes the penetration test;

[0038] Step S235: After confirming that all the closed contour points of the clothing pass the penetration test, use the position of the latest clothing contour line as the target constraint to differentially deform the clothing mesh and complete the processing of the clothing picture;

[0039] Step S236: Layer and fuse the processed clothing picture with the human mannequin image to complete virtual fitting.

[0040] Preferably, the step S232 includes sub-step S2321 and sub-step S2322;

[0041] Step 2321: Through the matching relationship between the corresponding points of the clothing and the mannequin, perform a similarity transformation on the clothing to achieve a global rough alignment. For upper clothes, coats and dresses, select their shoulder points and match them with the shoulder points of the mannequin to calculate the similarity transformation matrix; for half skirts and trousers, select their waistbands and match them with the waist contour points of the mannequin to calculate the similarity transformation matrix, and perform a similarity transformation on the clothing picture through this matrix to achieve a global rough alignment with the mannequin;

[0042] Step 2322: Through the corresponding point matching relationship between the clothing and the model, perform Laplacian differential deformation on the clothing grid to achieve local alignment; for tops, coats, and dresses, select their underarm points, elbow points, and mid-sleeve points as control points, and the underarm points, elbow points, and wrist points of the corresponding model as target positions; for skirts and trousers categories, select their waist points, mid-hem points, and crotch points as control points, and the waist points, ankle points, and crotch points of the corresponding model as target positions; perform Laplacian differential deformation on the clothing grid to achieve a rough alignment of the local positions;

[0043] The grid Laplacian differential deformation is to solve the least squares linear system, and the expression formula is as follows:

[0044] ;

[0045] Among them, and respectively represent the grid vertices before and after deformation, is the Laplacian coordinate of vertex , and its definition is:

[0046] ;

[0047] Among them represents the set of adjacent vertices of vertex , represents the position vector of the adjacent vertices of vertex , is the number of adjacent vertices of vertex .

[0048] Preferably, the step S233 may include sub-step S2331, sub-step S2332, sub-step S2333, and sub-step S2234;

[0049] Step S2331: Construct the clothing contour shape constraint; the clothing contour shape constraint is expressed as follows:

[0050] ;

[0051] Among them, and respectively represent the contour points before and after deformation, is the Laplacian coordinate of contour point , and its definition is:

[0052] ;

[0053] Among them represents the set of contour points within the n-Ring neighborhood of contour point , Indicates a point The position vector of the neighborhood contour point, is the vertex The number of inner contour points within the n-Ring neighborhood of, designed ;

[0054] Step S2332: Construct the position constraints of the key contour points of the clothing; the position constraints of the key contour points of the clothing are expressed as follows:

[0055] ;

[0056] Among them Indicates the contour points that need to be fixed for the clothing of the current category. For tops, coats, and dresses, the fixed contour points are the shoulder points, armpit points, elbow points, and hem points. For skirts and trousers, the fixed contour points are the waist points, hem points, cuff points, and crotch points;

[0057] Step S2333: Construct the length constraints of the key parts of the clothing; the length constraints of the key parts of the clothing are expressed as follows:

[0058] ;

[0059] Among them Indicates the connection line of the key part points; for tops, coats, and dresses, the key part is the sleeve width; for trousers, the key part is the leg width; there is no key part for skirts; and respectively represent the connection line vectors of the key part contour points before and after deformation, that is ;

[0060] Step S2334: Construct and solve the linear least squares sparse system; the linear least squares sparse system is expressed as follows:

[0061] ;

[0062] Among them , and respectively represent the weights of the clothing contour shape constraint, the clothing key contour point constraint, and the clothing key part length constraint, designed , , .

[0063] Beneficial effects: Compared with the existing fitting technologies, the present application creates a fitting model database, classifies clothing picture categories and detects key points, as well as segments the clothing area and the clothing back mask. According to the detected clothing information, through grid differential deformation, the clothing is registered and fused to the human model to complete the fitting process. By using the grid deformation method provided by the present application, the fitting result can be obtained quickly and efficiently, while maintaining the original contour shape of the clothing, providing users with an economical, convenient, realistic and natural fitting experience. Description of the Drawings

[0064] Figure 1 It is a schematic block diagram of the structure of the electronic device provided by the embodiment of the present application.

[0065] Figure 2 It is a schematic block diagram of the step flow of the contour-preserving feature grid deformation method for virtual fitting provided by the embodiment of the present application.

[0066] Figure 3 It is a schematic block diagram of other step flows of the contour-preserving feature grid deformation method for virtual fitting provided by the embodiment of the present application.

[0067] Figure 4 It is a schematic block diagram of other step flows of the contour-preserving feature grid deformation method for virtual fitting provided by the embodiment of the present application.

[0068] Figure 5 It is Figure 2 a schematic block diagram of the sub-step flow of step S23 in

[0069] Figure 6 It is Figure 5 a schematic block diagram of the sub-step flow of step S232 in

[0070] Figure 7 It is Figure 5 a schematic block diagram of the sub-step flow of step S233 in

[0071] Figure 8 It is a schematic block diagram of the image deformation fitting device provided by the embodiment of the present application.

[0072] Figure 9 It is a virtual fitting flow chart provided by the embodiment of the present application. Detailed Embodiments

[0073] The present invention will be further described below with reference to the accompanying drawings of the specification, but the present invention is not limited to the following embodiments. The described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and illustrated herein can be arranged and designed in various different configurations.

[0074] Accordingly, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the scope of protection of the present application.

[0075] The following will describe in detail the specific embodiments of the present application in conjunction with the accompanying drawings.

[0076] Please refer to Figure 1 , Figure 1 , which is a schematic block diagram of the structure of the electronic device 100. The electronic device 100 includes an image deformation virtual fitting device 110, a memory 111, a processor 112, and a communication unit 113.

[0077] The elements of the memory 111, the processor 112, and the communication unit 113 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0078] Among them, the memory 111 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. Among them, the memory 111 is used to store programs, and the processor 112 executes the programs after receiving execution instructions. The communication unit 113 is used to establish a communication connection between the electronic device 100 and other devices (such as user terminals) through a network, and is used to receive and send data through the network. For example, in this embodiment, the electronic device 100 performs data communication with external devices through the communication unit 113.

[0079] Please refer to Figure 2 , Figure 2 , which is a schematic block diagram of the steps of the grid deformation method for preserving contour features for virtual fitting provided by an embodiment of the present application. The present application provides a grid deformation method for preserving contour features for virtual fitting, which is applied to the electronic device 100. The method includes:

[0080] Step 21, obtaining a clothing picture to be virtually fitted and a specified virtual fitting model.

[0081] Step 22, classify the clothing picture categories and detect key points, and segment the clothing area and the clothing back mask.

[0082] Step 23, according to the constraints such as the corresponding key points and contour shapes of the clothing and the model, differentially deform the clothing picture and fuse it to the model to generate the virtual fitting result.

[0083] Please refer to Figure 3 , Figure 3 For other step process schematic block diagrams of the contour-preserving feature grid deformation method for virtual fitting provided in the embodiments of the present application. In this embodiment, human model data needs to be created before fitting and saved to the fitting database. The method includes:

[0084] Step 24, label the positions and attributes of the key points of the human model torso. The key points are specifically the shoulders, elbows, wrists, armpits, hips, crotches, knees and ankles of the human model, and record their coordinate positions and part names in the human body image;

[0085] Step 25, extract the contour line of the human model and partition it according to the body part where the contour line is located. Specifically, the human contour line is divided into 10 regions including the left (right) arm (inner) outer side, left (right) leg (inner) outer side, and left (right) torso, and record the contour point positions and the partitions where the contour points are located;

[0086] Step 26, save the key point and contour line annotation results to the fitting database for convenient reuse in fitting different categories of clothing. Specifically, the data related to the labeled human model is saved as a binary file, and subsequent relevant data is read from this file according to the user-specified fitting model.

[0087] Please refer to Figure 4 , Figure 4 For other step process schematic block diagrams of the contour-preserving feature grid deformation method for virtual fitting provided in the embodiments of the present application. In this embodiment, it is necessary to train a clothing category classification and key point detection model, and a mask segmentation model for the clothing area and the back area before fitting. The method includes:

[0088] Step S27, use the Labelme software to label the self-owned clothing picture data. Specifically, label 5 clothing category tags including tops, dresses, coats, skirts, and trousers, and key points such as the shoulders, cuffs, hems, trouser legs, crotches, and skirt hems of each clothing category, as well as the clothing areas and back areas such as the necks, cuffs, and hems of each category.

[0089] Step S28, use the HRNet convolutional neural network to train the self-owned data set to obtain a clothing classification and key point detection model;

[0090] Step S29: Train the self-owned dataset using the MaskR-CNN convolutional neural network to obtain the mask segmentation models for the clothing area and the back area;

[0091] Please refer to Figure 5 , Figure 5 which is Figure 2 the schematic block diagram of the sub-step process of step S23 in

[0092] Step S231: Subdivide the clothing picture with triangular meshes. Extract the contour line of the clothing area according to the clothing segmentation mask. Take the points on the contour line, together with the clothing key points, as the set of grid points to be subdivided, and the clothing contour line as the grid edge set. Use the Triangle algorithm library to subdivide the clothing picture into a set of triangular units to improve the image processing efficiency.

[0093] Step S232: Roughly align the clothing picture and the human model through similarity transformation and local Laplacian differential deformation in sequence.

[0094] Step S233: Integrate the contour shape constraint, the target position constraint, and the length constraint, construct and solve the linear least squares sparse system to obtain the position of the new contour line that maintains the original contour shape features.

[0095] Step S234: Conduct a penetration test on the clothing closed contour points and update the safe position of the penetration points as the position constraint condition of the sparse linear system. The clothing closed contour points should be outside the corresponding human model contour boundary. Otherwise, the human body image will pass through the clothing contour, resulting in a penetration phenomenon and causing visual unreasonableness. The steps of the penetration test and updating the penetration point position are as follows:

[0096] Select the clothing closed contour points ;

[0097] Use the kd-tree to find the nearest point in the corresponding human model contour partition line segment ;

[0098] According to and the line segment vectors of the adjacent front and rear points and , calculate the normal vector of the human contour line at ;

[0099] Connect and as the vector , calculate the cross product of the vectors ; If the vector the z-component , then record the current point as the penetration point and update to , where is the clearance margin between the updated contour point and the model contour, is the position normal vector, which is calculated from the line segment vector between and the adjacent front and rear points and . Design the clearance margin ; if , then the closed contour point passes the penetration test.

[0100] Step S235, after confirming that all the closed contour points of the clothing pass the penetration test, taking the position of the latest clothing contour line as the target constraint, perform differential deformation on the clothing mesh to complete the processing of the clothing picture.

[0101] Step S236, layer and fuse the processed clothing picture with the human model image to complete virtual fitting.

[0102] Please refer to Figure 6 , Figure 6 is Figure 5 the schematic block diagram of the sub-step process of step S232 in

[0103] Step 2321, through the matching relationship of corresponding points between the clothing and the model, perform similarity transformation on the clothing to achieve global rough alignment. For tops, coats and dresses, select their shoulder points and match them with the model's shoulder points to calculate the similarity transformation matrix; for skirts and trousers, select their waistbands and match them with the model's waist contour points to calculate the similarity transformation matrix. Perform similarity transformation on the clothing picture through this matrix to achieve global rough alignment with the model.

[0104] Step 2322, through the matching relationship of corresponding points between the clothing and the model, perform Laplacian differential deformation on the clothing mesh to achieve local alignment. For tops, coats and dresses, select their underarm points, elbow points and mid-points of the cuffs as control points, and the corresponding underarm points, elbow points and wrist points of the model as the target positions; for skirts and trousers, select their waistband points, mid-points of the trouser legs and crotch points as control points, and the corresponding waist points, ankle points and crotch points of the model as the target positions. Perform Laplacian differential deformation on the clothing mesh to achieve rough alignment of local positions.

[0105] The essence of the Laplacian differential deformation of the mesh is to solve the least squares linear system, and the expression formula is as follows:

[0106] ;

[0107] Among them, and respectively represent the grid vertices before and after deformation, is the Laplacian coordinate of vertex , and its definition is:

[0108] ;

[0109] where represents the set of adjacent vertices of vertex , represents the position vector of the adjacent vertices of vertex , is the number of adjacent vertices of vertex .

[0110] Please refer to Figure 7 , Figure 7 is Figure 5 a schematic block diagram of the sub-step process of step S223 in . In this embodiment, step S233 may include sub-steps S2331, S2332, S2333, and S2234.

[0111] Step S2331, constructing the clothing contour shape constraint. The clothing contour shape constraint is expressed as follows:

[0112] ;

[0113] where and respectively represent the contour points before and after deformation, is the Laplacian coordinate of contour point , and its definition is:

[0114] ;

[0115] where represents the set of contour points within the n-Ring neighborhood of contour point , represents the position vector of the neighboring contour points of point , is the number of contour points within the n-Ring neighborhood of vertex , and design .

[0116] Step S2332, constructing the clothing key contour point position constraint. The clothing key contour point position constraint is expressed as follows:

[0117] ;

[0118] where Indicates the contour points that need to be fixed for the current category of clothing. For tops, coats, and dresses, the fixed contour points are the shoulder points, underarm points, elbow points, and hem points. For skirts and pants, the fixed contour points are the waist points, hem points, cuff points, and crotch points.

[0119] Step S2333, construct the length constraints of the key parts of the clothing. The length constraints of the key parts of the clothing are expressed as follows:

[0120] ;

[0121] where represents the connection line of the key part points; for tops, coats, and dresses, the key part is the sleeve width; for pants, the key part is the leg width; there is no key part for skirts. and respectively represent the connection vectors of the key part contour points before and after deformation, that is, .

[0122] Step S2334, construct and solve the linear least squares sparse system. The linear least squares sparse system is expressed as follows:

[0123] ;

[0124] where , and respectively represent the weights of the clothing contour shape constraint, the clothing key contour point constraint, and the clothing key part length constraint. In the embodiments of the present application, , , .

[0125] Use the QR decomposition method to solve the linear least squares sparse system. QR decomposition can simplify the complex least squares problem into an upper triangular system of equations that is easy to solve by decomposing the matrix into an orthogonal matrix and an upper triangular matrix, and has numerical stability and high efficiency.

[0126] Please refer to Figure 9 , Figure 9 which is the virtual fitting flowchart provided by the embodiments of the present application. In this embodiment, a clothing picture of a dress category is tried on a female model. In other embodiments, the clothing category can also be tops, coats, skirts, and pants, and the fitting object can be any model in the fitting database.

[0127] In summary, the present application provides a contour feature preserving mesh deformation method for virtual fitting, the method comprising: creating a human model database; training a clothing category classification and key point detection model, and a clothing segmentation model to obtain semantic information and geometric information of a clothing image to be processed; extracting the clothing contour line and then subdividing the clothing area with a triangular mesh, respectively, through similarity transformation and clothing mesh differential deformation, to achieve global and local apparel and human model approximate alignment; integrating clothing contour line shape constraints, key point position constraints, and clothing key part length constraints, constructing a linear least squares sparse system about the clothing contour line, and solving a new contour line; performing a penetration test on closed contour points on the new contour line, and updating the contour point position constraints; iteratively executing the contour feature preserving sparse system solution and the penetration test until all closed clothing contour points pass the penetration test, and then performing clothing mesh deformation with the contour line as the position constraint; decomposing the clothing image into foreground and background according to the clothing segmentation result, and merging them with the human model in layers to complete virtual fitting. This solution solves the problem of contour distortion caused by severe deformation of clothing images in virtual fitting applications based on image processing, reduces user usage costs, and provides users with an economical, convenient, and real and natural fitting experience.

[0128] Finally, it should be noted that the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by ordinary technicians in this field should be considered as the protection scope of the present invention.

Claims

1. A contour feature-preserving mesh deformation method for virtual fitting, characterized in that The method comprises the following steps: Step 20: Before fitting, create fitting mannequin data and save it to the fitting database for reuse in various categories of clothing; use the self-annotated data to train clothing category classification and key point detection models, as well as clothing segmentation models, to obtain clothing semantic information and geometric information; Step 21: Obtain the clothing image to be tried on and designate the fitting model; Step 22: Classify clothing image categories and detect key points, as well as perform clothing area and clothing back mask segmentation; Step 23: According to the constraints of the key points and contour shapes of the clothing and the model, the clothing image is differentially deformed and fused to the model to generate a virtual fitting result; The step 23 includes sub-steps S231, S232, S233, S234, S235 and S236; Step S231: triangular mesh subdividing the clothing image; extracting the clothing area contour line according to the clothing segmentation mask, taking the points on the contour line together with the clothing key points as the grid point set to be subdivided, and the clothing contour line as the grid edge set, and using the Triangle algorithm library to subdivide the clothing image into a set of triangular units to improve image processing efficiency; Step S232: Aligning the clothing image and the human model by similarity transformation and local Laplacian differential deformation; Step S233: Integrate the contour shape constraint, target position constraint and length constraint, construct and solve a linear least squares sparse system, and obtain a new contour line position that maintains the original contour shape features; Step S234: Perform a penetration test on the closed contour points of the clothing, and update the safe position of the penetration point as the position constraint of the sparse linear system. The closed contour points of the clothing should be outside the corresponding human model contour boundary, otherwise the human image will pass through the clothing contour, causing a penetration phenomenon, which is visually unreasonable. The steps of the penetration test and updating the penetration point position are as follows: Select the closed contour points of the garment ; Use kd-tree to find The closest point in the segment corresponding to the mannequin's outline ; according to Adjacent points and The line segment vector of the human body is calculated Normal vector ; connect and For vector , calculate the vector outer product ; If the vector The z component , then record the current point as the penetration point and update to ,in The clearance margin from the updated contour point to the model contour, for The position normal vector is given by Adjacent points and The line segment vector is calculated and the design clearance margin is ;like , then the closed contour point passes the mold penetration test; Step S235: after confirming that all closed contour points of the clothing have passed the mold penetration test, the clothing mesh is differentially deformed with the latest clothing contour line position as the target constraint to complete the clothing image processing; Step S236: The processed clothing image is layered and fused with the human model image to complete the virtual fitting.

2. The contour feature-preserving mesh deformation method for virtual fitting according to claim 1, characterized in that: The specific steps of creating the fitting mannequin data in step 20 and saving it to the fitting database include: Step 24: Mark the key points and attributes of the mannequin’s torso; Step 25: Extract the contour of the human model and divide it into different parts according to the body parts where the contour is located; Step 26: Save the key point and contour line annotation results to the fitting database for easy reuse in fitting of different categories of clothing.

3. The contour feature-preserving mesh deformation method for virtual fitting according to claim 2, characterized in that: In step 24, the key points include shoulders, elbows, wrists, armpits, hips, crotch, knees and ankles of the human model, and the coordinate positions and part names of the human body images are recorded; In step 25, the human body contour line is specifically divided into 10 regions, namely, the left and right arms and the inner and outer sides of the left and right arms, the left and right legs and the inner and outer sides of the left and right legs, and the left and right torsos, and the positions of the contour points and the partitions where the contour points are located are recorded; In step 26, the data related to the labeled human model is saved as a binary file, and the relevant data is subsequently read from the file according to the fitting model specified by the user.

4. A contour feature preserving mesh deformation method for virtual fitting according to claim 1 or 2, characterized in that: Training the clothing category classification and key point detection model and clothing segmentation model in step 20 specifically includes the following steps: Step S27: labeling own clothing image data using Labelme software; Step S28: Use the HRNet convolutional neural network to train the own data set to obtain clothing classification and key point detection models; Step S29: Use the MaskR-CNN convolutional neural network to train the own data set to obtain the mask segmentation model of the clothing area and the back area.

5. The contour feature-preserving mesh deformation method for virtual fitting according to claim 4, characterized in that: The labeled own clothing image data in step S27 include labels of five clothing categories, namely, tops, dresses, jackets, skirts, and pants, and key points of shoulders, cuffs, hems, trouser legs, crotches, and hems of each clothing category, as well as clothing areas of each category and back areas of necklines, cuffs, and hems.

6. The contour feature preserving mesh deformation method for virtual fitting according to claim 1, characterized in that: The step S232 includes sub-step S2321 and sub-step S2322; Step 2321: by matching the corresponding points of the clothing and the model, similarity transformation is performed on the clothing to achieve global alignment. For tops, coats and dresses, the shoulder points are selected and matched with the shoulder points of the model to calculate the similarity transformation matrix; for skirts and trousers, the waistband is selected and matched with the waist contour points of the model to calculate the similarity transformation matrix. The clothing image is similarly transformed through the matrix to achieve global alignment with the model; Step 2322: through the matching relationship between the corresponding points of the clothing and the model, the clothing mesh Laplacian differential deformation is performed to achieve local alignment; for tops, coats and dresses, the armpit points, elbow points and cuff midpoints are selected as control points, and the armpit points, elbow points and wrist points of the corresponding models are the target positions; for skirts and trousers, the waist points, trouser leg midpoints and crotch points are selected as control points, and the waist points, ankle points and crotch points of the corresponding models are the target positions; execute the clothing mesh Laplacian differential deformation to achieve local position alignment; The grid Laplacian differential deformation is to solve the least squares linear system, which is expressed as follows: ； in, and Represent the mesh vertices before and after deformation, is the vertex The Laplacian coordinates of are defined as: ； in Represents a vertex The set of adjacent vertices of Represents a vertex The position vectors of adjacent vertices, is the vertex The number of adjacent vertices.

7. The contour feature preserving mesh deformation method for virtual fitting according to claim 1, characterized in that: The step S233 includes sub-step S2331, sub-step S2332, sub-step S2333 and sub-step S2234; Step S2331: constructing clothing outline shape constraints; the clothing outline shape constraints are expressed as follows: ； in, and Represent the contour points before and after deformation, respectively. It is a contour point The Laplacian coordinates of are defined as: ； in Represents contour points The set of contour points in the n-Ring neighborhood of Indicate point The position vector of the neighborhood contour points, is the vertex The number of contour points in the n-Ring neighborhood, design ; Step S2332: constructing the position constraints of the key contour points of the clothing; the position constraints of the key contour points of the clothing are expressed as follows: ； in Indicates the fixed contour points required for the current category of clothing. For tops, coats, and dresses, the fixed contour points are the shoulder point, armpit point, elbow point, and hem point. For skirts and trousers, the fixed contour points are the waist point, hem point, leg point, and crotch point. Step S2333: constructing the length constraint of the key parts of the clothing; the length constraint of the key parts of the clothing is expressed as follows: ； in Indicates the connection line of key points; for tops, coats and dresses, the key point is the sleeve width; for trousers, the key point is the trouser leg width; there is no key point for skirts; and Respectively represent the line vectors of the contour points of the key parts before and after deformation, that is, ; Step S2334: construct and solve a linear least squares sparse system; the linear least squares sparse system is expressed as follows: ； in , and They represent the weights of clothing contour shape constraints, clothing key contour point constraints, and clothing key part length constraints, respectively. , , .

Images