Three-dimensional clothing reconstruction method based on spatial image

Abstract

The present invention discloses a three-dimensional clothing reconstruction method based on spatial images, and the steps include: step 1, obtaining images of multi-angle and different spatial series views through a hemispherical method, and obtaining an initial sparse point cloud through an augmented SFM (motion structure recovery) algorithm; step 2, reconstructing a dense point cloud by using a multi-view stereo vision technology for the sparse point cloud; step 3, separating the background of the dense point cloud by a method based on cylinder segmentation to obtain a 3D clothing point cloud; step 4, removing outliers and noise from the 3D clothing point cloud; step 5, synthesizing a 3D clothing surface through a rolling ball algorithm; step 6, trying on the 3D clothing surface on a virtual model. The present invention can quickly generate high-quality and realistic 3D digital clothing without manual measurement.

Description

Technical Field

[0001] The invention belongs to the field of clothing three-dimensional reconstruction of computer vision and develops a new 3D clothing three-dimensional reconstruction framework. Background Art

[0002] With the continuous development of computer technology, the impact of digital clothing design on my country's clothing industry is self-evident. Digital clothing is one of the important forms of my country's digital development and evolution. It can be used for online shopping, cultural protection, virtual try-on, digital games, digital museums and personalized clothing design. 3D clothing reconstruction is one of the key technologies for obtaining digital clothing. In the field of clothing, the existing clothing reconstruction methods mainly include two-dimensional and three-dimensional design. Two-dimensional design can provide a broader creative dimension for clothing design. The three-dimensional design in three-dimensional design is used in digital printing, flat pleating and large-profile three-dimensional cutting. However, these methods are difficult to achieve for some special clothing or museum's unique clothing, as they do not allow the use of touch or contact measurement to reconstruct, which is difficult to achieve. At the same time, the traditional clothing design method mainly relies on experienced technicians to manually measure to reconstruct clothing, which is time-consuming and inefficient. Summary of the invention

[0003] The purpose of the present invention is to provide a three-dimensional clothing reconstruction method based on spatial images, which solves the problem that the traditional method in the prior art relies on manual measurement to reconstruct clothing.

[0004] The technical solution adopted by the present invention is:

[0005] The three-dimensional clothing reconstruction method based on spatial images includes the following steps:

[0006] Step 1: Use a hemispherical method to obtain images of multiple spatial series views from different angles, and use an augmented SFM algorithm to obtain an initial sparse point cloud;

[0007] Step 2, reconstructing dense point cloud using multi-view stereo vision technology on the sparse point cloud;

[0008] Step 3, the background is separated from the dense point cloud by a cylinder segmentation method to obtain a 3D clothing point cloud;

[0009] Step 4, remove outliers and noise from the 3D clothing point cloud;

[0010] Step 5, synthesizing the 3D clothing surface through a rolling ball algorithm;

[0011] Step 6: Try on the 3D clothing surface on the virtual model.

[0012] The present invention is also characterized in that:

[0013] Step 1 includes:

[0014] Step 1.1, take a series of pictures evenly around the surface of the garment in one direction, and the angle between two adjacent pictures should not exceed 15°. Each picture taken must contain all the information of the subject at that angle;

[0015] Step 1.2, use SIFT algorithm to find feature points in different scale spaces;

[0016] Step 1.3, when the feature point is selected, the nearby data points are fitted by filtering, and the position, principal curvature and scaling of the data points are found to filter out the feature points with unstable edge response and low contrast;

[0017] Step 1.4, feature matching, rotate the coordinate axis to the direction of the key point, obtain the gradient information, and perform normalization. Feature matching is achieved by calculating the Euclidean distance between the descriptors of feature points in different images. When the Euclidean distance is less than the feature matching threshold, the match is successful.

[0018] Step 1.5, geometric transformation, using epipolar geometry for the same spatial point, the geometric relationship between different camera positions and the projection on the camera image plane;

[0019] Step 1.6, image registration:

[0020] First, the camera position and orientation are obtained by decomposing the matrix, the camera projection matrix is ​​obtained by decomposing the basic matrix, and the intersection of the back-projection ray of the image between at least two images is calculated to realize the triangulation of 3D points to add the registration image. Each time a registration image is added, an optical adjustment optimization is required.

[0021] Secondly, the acquired series of images are registered and merged in sequence through augmented SFM, and each time a view is added, the changes in the position and direction of each image are linked to the previous image, and then the 3D data points in the same reference frame are calculated to finally obtain a sparse point cloud.

[0022] Step 2 is as follows: first, calculate the Harris corner points of multiple images, and use the feature point detection of the difference pyramid to obtain the sparse patches of the salient areas of the image, and use these matched initial feature points as the initial patches; second, expand the pixels near the sparse point cloud patches to obtain the corresponding relationship and obtain a dense point cloud; finally, use the camera internal and external parameters obtained by SFM to obtain the epipolar geometric constraints and filter out the wrong matching pairs;

[0023] The above process is iterated to finally obtain a high-quality 3D dense point cloud. The dense 3D point cloud data is downsampled using a voxel grid downsampling algorithm to optimize the data.

[0024] Step 3 is as follows: the optimized dense point cloud is separated from the background by a cylindrical segmentation method to obtain a 3D clothing point cloud; for upper body or one-piece clothing, the cylindrical segmentation algorithm is used for segmentation, and then the Euclidean clustering algorithm is used to separate the upper body clothing area and the floor to obtain a 3D clothing point cloud; for lower body clothing, the lower body cannot be completely separated from the background points, so a method combining the cylindrical segmentation algorithm and the straight-through filter algorithm is used for segmentation.

[0025] Step 3: For tops or one-piece garments, the specific steps are as follows:

[0026] Step 3.1, randomly select at least three non-collinear points from the 3D clothing point cloud, and estimate the 2D plane model by iteratively searching for the optimal parameters, as defined by formula (6):

[0027] ax+by+cz=d,a 2 +b 2 +c 2 =1,d>0 (6)

[0028] Among them, (a, b, c) represents the plane normal vector required to obtain the plane model; d is the distance from the origin to the plane;

[0029] Secondly, the unit normal vector of each point in the 3D clothing point cloud is calculated to create the initial value of the cylindrical longitudinal axis vector (a, b, c) of the 3D cylindrical model. The created 3D cylindrical model is perpendicular to the estimated 2D plane model and parallel to the Z axis. The specific position of the cylinder is determined by the parametric equation formula (7):

[0030] f=(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 (7)

[0031] Among them, f represents the obtained cylinder; (x1, y1, z1) are the coordinates of the center of the circle; r is the radius of the cylinder base; (a, b, c) are the cylinder axis vectors;

[0032] Finally, adjust the four parameters of normal number, normal weight, 3D cylinder radius and distance from the point inside the 3D cylinder model to the clothing point cloud to create the final 3D cylinder point cloud for different 3D clothing point clouds;

[0033] Step 3.2, after cylinder segmentation, use the Euclidean clustering algorithm to separate the upper body clothing area and the floor: First, randomly select a point x from the remaining points i, through the threshold of the Euclidean distance dL, use Kd-Tree to find the K nearest points, as shown in formula (8). The selected K nearest points need to be found by looking up x i K2 points whose Euclidean distance to the K closest points is less than a fixed threshold are further grouped. Finally, K2 points are retained and other points are deleted.

[0034]

[0035] Among them, x i represents a randomly selected point; y i represents the k nearest points; dL is the Euclidean distance.

[0036] The pass filter algorithm in step 3 is as shown in formula (9):

[0037] pc passthrough =Ylimit(min,max) (9)

[0038] Among them, pc represents the 3D point cloud that needs to be retained; Y represents the filter area; (min, max) represents the filter range.

[0039] Step 4 includes:

[0040] Step 4.1, use the Statistical Outlier Removal algorithm to calculate the Euclidean distance dl and remove outliers from the 3D clothing point cloud;

[0041] Step 4.2: Use PointCleanNet algorithm to remove noise and convert the 3D point cloud P with noise points into i Input from STN (Spatial Transfer Network) to extract the feature h(p) of each point j ), and aggregate all features into the global feature H, and estimate the displacement vector d of each point from the global feature i , each point is translated to the original clean surface and RGB is added to each point in the 3D clothing point cloud after PointCleanNet removes noise points to form a texture.

[0042] Step 5 includes: synthesizing the 3D clothing surface through a rolling ball algorithm, that is, starting from a seed triangle and a ball with a predefined radius r, if the ball contains three points with the same normal vector during the rolling process, a seed triangle is formed; secondly, starting from the seed triangle, the ball rotates around an edge until it touches another nearby point to form another triangle; the process continues until all edges can be reached, and finally, the process of changing the radius of the sphere is repeated to deal with uneven sampling density to obtain the 3D clothing surface.

[0043] Step 6 is as follows: Use the open source modeling software Maya to demonstrate the virtual try-on of clothing, use the human body model as a collision object, add dynamic properties to the 3D clothing surface, and ensure that each 3D clothing model becomes a canvas. At the same time, add Point-to-Surface constraints to the 3D clothing model and the human body model during the try-on process of the lower body clothing, bind the 3D clothing model to the human body model, and finally use the material editor in Maya to customize the clothing material for the 3D clothing surface to apply the two-dimensional image of the clothing to the model to form a texture.

[0044] The beneficial effects of the present invention are:

[0045] The 3D clothing point cloud is separated from the background by point cloud segmentation and clustering algorithms, which can be adapted to different types of clothing. The algorithm and statistical model based on deep learning can effectively remove the noise and outliers of the 3D clothing point cloud. The rolling ball algorithm is used to reconstruct the mesh from the 3D clothing point cloud to obtain high-quality and realistic 3D clothing surfaces. The present invention can quickly generate high-quality and realistic 3D digital clothing without manual measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 The dense point cloud result image obtained in Example 1 of the present invention;

[0048] Figure 3 is the 3D clothing point cloud result image obtained in Example 1 of the present invention;

[0049] Figure 4 This is a result diagram after removing outliers and removing noise in Example 1 of the present invention;

[0050] Figure 5 In Example 1 of the present invention, Figure 4 The result image after adding texture;

[0051] Figure 6 It is a two-dimensional image diagram of the clothing prototype;

[0052] Figure 7 is a graph of the reconstructed surface result obtained in Example 1 of the present invention;

[0053] Figure 8 This is an experimental diagram of virtual fitting of models in different postures according to Example 1 of the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] The three-dimensional clothing reconstruction method based on space images of the present invention comprises the following steps:

[0056] Step 1: Use a hemispherical method to obtain images of multiple spatial series views from different angles, and use an augmented SFM (Structure from Motion) algorithm to obtain an initial sparse point cloud;

[0057] Step 2, reconstructing dense point cloud using multi-view stereo vision technology on the sparse point cloud;

[0058] Step 3, the background is separated from the dense point cloud by a cylinder segmentation method to obtain a 3D clothing point cloud;

[0059] Step 4, remove outliers and noise from the 3D clothing point cloud;

[0060] Step 5, synthesizing the 3D clothing surface through a rolling ball algorithm;

[0061] Step 6: Try on the 3D clothing surface on the virtual model.

[0062] Step 1 is as follows:

[0063] Step 1.1, take a series of pictures evenly around the surface of the clothing in one direction, and the angle between two adjacent pictures shall not exceed 15°. Each picture taken must contain all the information of the subject at that angle.

[0064] Step 1.2: The first step of motion structure recovery is feature extraction. SIFT (Scale Invariant Feature Transform Matching Algorithm) can find feature points in different scale spaces. The scale space of the image can be obtained by Gaussian blurring the image. The process is defined as a function P(x, y, σ), which is obtained by convolving the original image L(x, y) with a Gaussian filter Q(x, y, σ), as shown in formula (1), where the two-dimensional filter of Q(x, y, σ) is as shown in formula (2).

[0065] P(x,y,σ)=Q(x,y,σ)*L(x,y) (1)

[0066]

[0067] Where σ is the scale space factor, which is the standard deviation of the Gaussian distribution. P(x,y,σ) is the Gaussian scale space of the image. L(x,y) is the original image.

[0068] When the feature point is selected, the nearby data points can be fitted by filtering, and the location, principal curvature and scaling of the data point can be found. The feature points with unstable edge response and low contrast can also be screened out. This process is the Taylor formula expansion on the Gaussian difference, as shown in formula (3).

[0069]

[0070] Where D is the Gaussian difference, Δx is the offset of the sample point, and x is the extreme point on D(x). Taking the derivative of (3) and setting the derivative to 0, we get formula (4).

[0071]

[0072] Substitute Δx into the Taylor expansion of D(x) to obtain formula (5).

[0073]

[0074] T is the contrast threshold, if Then the feature point is retained, otherwise it is removed.

[0075] Step 1.4, feature matching is to find the correspondence between the same objects in different images. The scale, direction and position information of each point extracted by the feature are represented by a set of vectors, called feature descriptors. In order to maintain its rotation invariance, the coordinate axis is rotated to the direction of the key point, the gradient information is obtained, and normalized. Feature matching is achieved by calculating the Euclidean distance between the descriptors of feature points in different images. When the Euclidean distance is less than the threshold of feature matching, the match is successful.

[0076] Step 1.5, geometric transformation is to ensure that the matching points found by feature extraction and feature matching correspond to each other in the real scene. Epipolar geometry is used for the geometric relationship between the same spatial point, different camera positions and the projection on the camera image plane. The essential matrix is ​​a matrix containing the epipolar geometric relationship. If the camera calibration internal parameters are known, the epipolar geometric essential matrix is ​​used to describe the camera movement, and the relationship between the binocular images on the two epipolar lines is obtained for verification.

[0077] Step 1.6, image registration is also a very important step in reconstruction. In the reconstruction process, the process of adding new images is also called registering images. Its camera posture can be obtained by corresponding to the reconstructed 3D points. The position and direction of the camera are obtained by decomposing the matrix, the projection matrix of the camera is obtained by decomposing the basic matrix, and the intersection of the back-projection ray of the image between at least two images is calculated to realize the triangulation process of the 3D points. Due to the error of the estimation method, BA (optical adjustment) optimization is required for each registration image added.

[0078] Once the basic matrix between multiple images is found and the corresponding 3D data points are found, the position of the camera in the same reference frame can be found. Therefore, the series of images acquired are registered and merged in sequence through the augmented SFM, and each time a view is added, the changes in the position and direction of each image are linked to the previous image, and then the 3D data points in the same reference frame are calculated, and finally a sparse point cloud is obtained.

[0079] Step 2, the initial sparse point cloud is reconstructed into a dense point cloud using multi-view stereo vision technology. Each pixel in the two-dimensional image is matched, and the area near the key point in each image plane is explored to make the density of the three-dimensional point cloud closer to the clarity of the image display. First, the Harris corner points of multiple images are calculated, and the feature point detection of the differential pyramid is used to obtain the sparse patches of the salient areas of the image, and these matched initial feature points are used as the initial patches; secondly, the pixels near the sparse point cloud patches are expanded to obtain the corresponding relationship to obtain a dense point cloud; finally, the camera internal and external parameters obtained by SFM are used to obtain the epipolar geometric constraints to filter out the wrong matching pairs.

[0080] In order to obtain better reconstruction effect, the above process can be iterated to finally obtain high-quality 3D dense point cloud, and the dense 3D point cloud data is downsampled using voxel grid downsampling algorithm. Downsampling constructs a three-dimensional voxel grid, and in each voxel, the centroid of all points in the voxel is used to approximate the other points in the voxel, so that all points in the voxel are represented by one centroid. Downsampling can achieve the effect of filtering, effectively reduce data, and improve the computational efficiency for the experiment.

[0081] Step 3, cylindrical segmentation is used to segment the model of wearing clothing (model head and model internal support), and cylindrical segmentation is based on the RANSAC (random sampling consensus) algorithm. In this step, the background of the dense point cloud is separated by a cylindrical segmentation-based method to obtain a 3D clothing point cloud; for red polka dot dresses, short-sleeved denim jackets, dresses and other tops or jumpsuits, the cylindrical segmentation algorithm is used for segmentation, and then the Euclidean clustering algorithm is used to separate the upper body clothing area and the floor to obtain a 3D clothing point cloud. Cylindrical segmentation is used to segment the model of wearing clothing (model head and model internal support), and the Euclidean clustering algorithm separates the upper body clothing area and the floor of the 3D point cloud to obtain a complete 3D clothing point cloud. Since cylindrical segmentation excessively removes useful points on the pants, the integrity of the pants is destroyed;

[0082] The specific steps for tops or one-piece clothing are as follows:

[0083] In step 3.1, at least three non-collinear points are first randomly selected from the 3D clothing point cloud, and the 2D plane model is estimated by iteratively searching for the optimal parameters, as defined in formula (6).

[0084] ax+by+cz=d,a 2 +b 2 +c 2 =1,d>0 (6)

[0085] Among them, (a, b, c) represents the plane normal vector required to obtain the plane model; d is the distance from the origin to the plane.

[0086] Secondly, the unit normal vector of each point in the 3D clothing point cloud is calculated to create the initial value of the cylindrical longitudinal axis vector (a, b, c) of the 3D cylindrical model. The created 3D cylindrical model is perpendicular to the estimated 2D plane model and parallel to the Z axis. Since the 3D clothing point cloud is parallel to the Y axis, (a, b, c) is converted to be parallel to the Y axis, which makes the 3D cylindrical model perpendicular to the (X, Z) plane. The parametric equation determines the specific position of the cylinder, which is defined by formula (7).

[0087] f=(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 (7)

[0088] Among them, f represents the obtained cylinder; (x1, y1, z1) are the coordinates of the center of the circle; r is the radius of the cylinder base; (a, b, c) are the cylinder axis vectors.

[0089] Finally, adjust the four parameters of normal number, normal weight, 3D cylinder radius and distance from the point inside the 3D cylinder model to the clothing point cloud to create the final 3D cylinder point cloud for different 3D clothing point clouds.

[0090] Step 3.2, after cylinder segmentation, use the Euclidean clustering algorithm to separate the upper body clothing area and the floor. First, randomly select a point x from the remaining points i , using the threshold of the Euclidean distance dL, use Kd-Tree to find the K nearest points, as shown in formula (8). The selected K nearest points need to be found by looking up x i K2 points whose Euclidean distance to the K closest points is less than a fixed threshold are further grouped. Finally, K2 points are retained and other points are deleted.

[0091]

[0092] Among them, x i represents a randomly selected point; y i represents the k nearest points; dL is the Euclidean distance.

[0093] For lower body clothing such as short skirts and pants, the lower body clothing cannot be completely separated from the background points. Therefore, the short skirts and pants are segmented by combining the cylindrical segmentation algorithm and the straight-through filter algorithm: the straight-through filter is added to the cylindrical segmentation, and the filtering area and range of the straight-through filter are set during the cylindrical segmentation process to retain the complete 3D clothing point cloud.

[0094] The pass filter algorithm is as shown in formula (9):

[0095] pc passthrough =Ylimit(min,max) (9)

[0096] Among them, pc represents the 3D point cloud that needs to be retained; Y represents the filter area; (min, max) represents the filter range.

[0097] Step 4: Since the segmented 3D clothing point cloud contains outliers and noise, they are removed by using Statistical OutlierRemoval and PointCleanNet algorithms based on deep learning. Specifically:

[0098] Step 4.1, Statistical Outlier Removal calculates the Euclidean distance dl of each point in the input 3D clothing point cloud, as shown in formula (10). If dl does not obey the Gaussian distribution, the corresponding point is regarded as an outlier and needs to be removed, and vice versa.

[0099]

[0100] Among them, p i represents a randomly selected point, q i represents the k closest points.

[0101] Step 4.2, the PointCleanNet algorithm is based on PCPNet, which estimates the displacement vector of each point and projects the noise point onto the original clean surface. i Input from STN (Spatial Transfer Network) to extract the feature h(p) of each point j ), and aggregate all features into the global feature H, as shown in formula (11). Estimate the displacement vector d of each point from the global feature i , each point is translated to the original clean surface and RGB is added to each point of the 3D clothing point cloud after PointCleanNet removes noise points to form a texture.

[0102] H(P i )=∑p j ∈P i h(p j ) (11)

[0103] Among them, H represents the global feature; h(p j ) represents point p j Characteristics of P i Represents the model output point; d i Represents the displacement vector for each point.

[0104] Step 5. Once a clean 3D garment point cloud is obtained, the mesh of the garment surface can be reconstructed using the BPA (rolling ball method). Depending on the density of the 3D garment point cloud, the BPA starts with a seed triangle and a ball with a predefined radius r. If the ball contains three points with the same normal vector during the rolling process, a seed triangle is formed. Secondly, starting from the seed triangle, the ball rotates around one edge until it touches another nearby point, forming another triangle. The process continues until all edges can be reached. Finally, the process of changing the radius of the sphere is repeated to deal with uneven sampling density. Compared with the garment prototype, the rolling ball method can restore high-quality garments and provide realistic 3D garment surfaces for virtual fitting applications.

[0105] Step 6. In order to perform dynamic physical simulation of 3D clothing on the virtual model, the open source modeling software Maya is used to demonstrate the virtual try-on of clothing. The human body model is used as a collision object, and physical properties are added to the 3D clothing models (red dot skirt, short-sleeved denim jacket, dress, short skirt and trousers), ensuring that each 3D clothing model becomes a canvas for simulating virtual fitting with the human body model in the software. At the same time, Point-to-Surface constraints are added to the 3D clothing model and the human body model during the fitting process of the short skirt and trousers, and the 3D clothing model is bound to the human body model to prevent the 3D clothing from falling off during the fitting process. Finally, the two-dimensional image of the clothing is applied to the model through the material editor and the clothing material is customized to form a texture.

[0106] The 3D clothing reconstruction model obtained by the above steps and the virtual try-on effect experiment show that the reconstruction method is feasible.

[0107] Embodiment 1,

[0108] Example 1: A red polka dot dress, a short-sleeved denim jacket, a dress, a short skirt and trousers are respectively reconstructed using the method of the present invention;

[0109] Execute step 1, obtain 150 images for each clothing type, and set the contrast threshold T to 0.5;

[0110] Execute step 2 to expand the pixels near the sparse point cloud patch to obtain the corresponding relationship and obtain a dense point cloud. Figure 2 .

[0111] Execute step 3, where the following parameters of the point cloud cylindrical segmentation can be changed to obtain the 3D clothing point cloud (red polka dot dress, dress, short sleeves and denim jacket), as shown in Table 1. The 3D clothing point cloud (trousers and short skirt) obtained by changing the parameters of the straight-through filter is shown in Table 2. The result is shown in Figure 2. Figure 3 shown.

[0112] Table 1: Parameters for 3D point cloud cylindrical segmentation

[0113] clothing Number of Normals Weight radius distance Red Polka Dots 100 0.18 0,2 0.85 dress 80 0.11 0,4 1.30 Short sleeve 100 0.17 0,2 0.55 Denim jacket 100 0.13 0,2 0.40

[0114] Table 2: Parameters of 3D point cloud passthrough filtering

[0115] clothing Filter Area Filter range trousers y -10,1.4 Short skirt y -5,0.8

[0116] Execute step 4, where the parameters of the PointCleanNet algorithm are batch size, epoch and learning rate are 64, 2000 and 0.00000001 respectively. The denoising result is Figure 4 As shown, add texture as Figure 5 shown.

[0117] Execute step 5, where the radius predefined by the rolling ball method is 30. The prototype image of the garment is Figure 6 The reconstructed surface result is Figure 7 shown.

[0118] Execute step 6, where Point-to-Surface constraints are added to the 3D clothing surface and the human body model during the skirt and pants fitting process. The result is Figure 8 shown.

Claims

1. A three-dimensional clothing reconstruction method based on spatial images, characterized in that the steps include: Step 1: Use a hemispherical method to obtain images of multiple spatial series views from different angles, and use an augmented SFM algorithm to obtain an initial sparse point cloud; Step 2, reconstructing dense point cloud using multi-view stereo vision technology on the sparse point cloud; Step 3, the background is separated from the dense point cloud by a cylinder segmentation method to obtain a 3D clothing point cloud; The step 3 is specifically as follows: the background is separated from the dense point cloud by a method based on cylindrical segmentation to obtain a 3D clothing point cloud; for upper garments or one-piece garments, after segmentation by a cylindrical segmentation algorithm, the upper body clothing area and the floor are separated by a Euclidean clustering algorithm to obtain a 3D clothing point cloud; for lower body garments, the lower garments and the background points cannot be completely separated, so a method combining a cylindrical segmentation algorithm and a straight-through filter algorithm is used for segmentation; Step 3: For tops or one-piece garments, the specific steps are as follows: Step 3.1, randomly select at least three non-collinear points from the 3D clothing point cloud, and estimate the 2D plane model by iteratively searching for the optimal parameters, as defined in formula (6): (6) Among them, (a, b, c) represents the plane normal vector required to obtain the plane model; d is the distance from the origin to the plane; Secondly, the unit normal vector of each point in the 3D clothing point cloud is calculated to create the initial value of the cylindrical longitudinal axis vector (a, b, c) of the 3D cylindrical model. The created 3D cylindrical model is perpendicular to the estimated 2D plane model and parallel to the Z axis. The specific position of the cylinder is determined by the parametric equation formula (7): (7) Among them, f represents the obtained cylinder; (x1, y1, z1) are the coordinates of the center of the circle; r is the radius of the cylinder base; (a, b, c) are the cylinder axis vectors; Finally, adjust the four parameters of normal number, normal weight, 3D cylinder radius and distance from the point inside the 3D cylinder model to the clothing point cloud to create the final 3D cylinder point cloud for different 3D clothing point clouds; Step 3.2, after cylinder segmentation, use the Euclidean clustering algorithm to separate the upper body clothing area and the floor: First, randomly select a point from the remaining points , through the Euclidean distance The threshold is used to find the K nearest points using Kd-Tree, as shown in formula (8). The selected K nearest points need to be found by The Euclidean distance between the K closest points is less than a fixed threshold Click to further group, and finally, points are retained, others are deleted; (8) in, represents a randomly selected point; represents the k closest points; is the Euclidean distance; Step 4, remove outliers and noise from the 3D clothing point cloud; Step 5, synthesizing the 3D clothing surface through a rolling ball algorithm; Step 6: Try on the 3D clothing surface on the virtual model.

2. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The step 1 comprises: Step 1.1, take a series of pictures evenly around the surface of the garment in one direction, and the angle between two adjacent pictures should not exceed 15°. Each picture taken must contain all the information of the subject at that angle; Step 1.2, use SIFT algorithm to find feature points in different scale spaces; Step 1.3, when the feature point is selected, the nearby data points are fitted by filtering, and the position, principal curvature and scaling of the data points are found to filter out the feature points with unstable edge response and low contrast; Step 1.4, feature matching, rotate the coordinate axis to the direction of the key point, obtain the gradient information, and perform normalization. Feature matching is achieved by calculating the Euclidean distance between the descriptors of feature points in different images. When the Euclidean distance is less than the feature matching threshold, the match is successful. Step 1.5, geometric transformation, using epipolar geometry for the same spatial point, the geometric relationship between different camera positions and the projection on the camera image plane; Step 1.6, image registration: First, the camera position and orientation are obtained by decomposing the matrix, the camera projection matrix is ​​obtained by decomposing the basic matrix, and the intersection of the back-projection ray of the image between at least two images is calculated to realize the triangulation of 3D points to add the registration image. Each time a registration image is added, an optical adjustment optimization method is required. Secondly, the acquired series of images are registered and merged in sequence through augmented SFM, and each time a view is added, the changes in the position and direction of each image are linked to the previous image, and then the 3D data points in the same reference frame are calculated to finally obtain a sparse point cloud.

3. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The step 2 is specifically as follows: first, calculating the Harris corner points of multiple images, and using the feature point detection of the difference pyramid to obtain the sparse patches of the salient areas of the images, and using these matched initial feature points as the initial patches; Secondly, the pixels near the sparse point cloud patch are expanded to obtain the corresponding relationship and obtain a dense point cloud. Finally, the camera internal and external parameters obtained by SFM are used to obtain the epipolar geometric constraints to filter out the wrong matching pairs. The above process is iterated to finally obtain a high-quality 3D dense point cloud. The dense 3D point cloud data is downsampled using a voxel grid downsampling algorithm to optimize the data.

4. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The pass filter algorithm in step 3 is as shown in formula (9): (9) Among them, pc represents the 3D point cloud that needs to be retained; Y represents the filter area; (min, max) represents the filter range.

5. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The step 4 comprises: Step 4.1: Calculate the Euclidean distance using the Statistical Outlier Removal algorithm , remove outliers from 3D clothing point cloud; Step 4.2: Use PointCleanNet algorithm to remove noise and transform the 3D point cloud with noise points into Input from STN (Spatial Transfer Network) to extract features for each point , and aggregate all features into global features , estimate the displacement vector of each point from the global features , each point is translated to the original clean surface and RGB is added to each point in the 3D clothing point cloud after PointCleanNet removes noise points to form a texture.

6. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The step 5 comprises: synthesizing the 3D clothing surface by a rolling ball algorithm, i.e. starting from a seed triangle and a ball with a predefined radius r, if the ball contains three points with the same normal vector during the rolling process, a seed triangle is formed; secondly, starting from the seed triangle, the ball rotates around an edge until it contacts another nearby point to form another triangle; the process continues until all edges can be reached, and finally, the process of changing the radius of the sphere is repeated to deal with uneven sampling density to obtain the 3D clothing surface.

7. The method for reconstructing three-dimensional clothing based on spatial images according to claim 1, characterized in that: The step 6 specifically includes: using the open source modeling software Maya to display the virtual try-on of clothing, taking the human body model as the collision object, adding dynamic properties to the 3D clothing surface, ensuring that each 3D clothing model becomes a canvas, and at the same time, adding Point-to-Surface constraints to the 3D clothing model and the human body model during the try-on process of the lower body clothing, binding the 3D clothing model to the human body model, and finally, in Maya, the 3D clothing surface being tried on is processed through the material editor and the clothing material is customized, and the two-dimensional image of the clothing is applied to the model to form a texture.

Images