Three-dimensional image reconstruction method and device based on sparse view and computer equipment

Abstract

The invention relates to the technical field of computer vision, and discloses a sparse view-based three-dimensional image reconstruction method and device and computer equipment, and the method comprises the steps: obtaining a multi-view image of a target human body wearing clothes; fitting a three-dimensional human body template of the target human body; extracting a multi-view clothing image; according to the multi-view garment image and the three-dimensional human body template, generating an initial Gaussian point cloud of the garment; constructing a boundary range for the initial Gaussian point cloud, and removing free Gaussian points outside the boundary range; constructing a loss function of two-dimensional Gaussian sputtering, performing multiple optimization iteration training on the initial Gaussian point cloud, and continuously removing free Gaussian points outside the boundary range in the optimization iteration training process to obtain a reconstructed garment model; generating a three-dimensional human body model with clothes according to the three-dimensional human body template and the reconstructed clothes model; according to the method, the quality and availability of three-dimensional human body model reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a sparse view-based three-dimensional image reconstruction method, device and computer equipment. Background Art

[0002] Creating high-quality 3D models of the clothed human body is a challenging fundamental research task in computer vision, with important applications in virtual reality (VR), telepresence, and digital content creation. Traditional methods for constructing clothed human 3D models, such as multi-camera capture and manual modeling, are not only resource-intensive but also suffer from poor scalability. Recent research has focused on automating the modeling process using images or videos from sparse viewpoints. However, this field still faces many core challenges.

[0003] Three-dimensional modeling methods for clothed humans are mainly divided into explicit and implicit representation methods. Explicit representation methods often rely on scanner-based systems or parametric models, such as SMPL (Skinned Multi-Person Linear Model). Although these mesh-based representation methods perform well in terms of geometric controllability, they are highly dependent on template accuracy and have obvious shortcomings when capturing complex clothing geometries (such as clothing wrinkles). Implicit representation methods use continuous neural representations to try to break through the limitations of templates. For example, the early NeRF (Neural Radiance Field) and SDF (Signed Distance Function) model surfaces through volume rendering or signed distance fields, but they are accompanied by high computational costs and limited editability. Recently, 3DGS (3DGaussian Splatting) has emerged as a promising alternative to NeRF-based methods. It represents the scene through explicit and learnable three-dimensional Gaussians, achieves excellent rendering efficiency (with real-time performance) compared to NeRF, and supports direct geometric operations in human body modeling. Therefore, it is increasingly used in high-fidelity virtual human generation.

[0004] However, existing technologies have serious multi-view inconsistency issues in 3D image reconstruction. Taking 3DGS as an example, Figure 1 As shown, due to the multi-view inconsistency of the 3D Gaussian, it cannot accurately represent the surface, resulting in fragmentation of the 3D image surface (such as Figure 1 Under sparse observation conditions, key geometric constraints lose their effectiveness due to unreliable gradient estimation, resulting in holes on the surface of the 3D image (as shown in the upper large box of Figure 1 (as shown in the small box in the middle of the ). Summary of the Invention

[0005] Based on this, it is necessary to address the problem of multi-perspective inconsistency in existing three-dimensional image reconstruction methods, which leads to surface fragmentation and the appearance of holes in the three-dimensional image. A three-dimensional image reconstruction method, device and computer equipment method based on sparse views are proposed.

[0006] A first aspect of the present invention provides a 3D image reconstruction method based on a sparse view, the 3D image reconstruction method based on a sparse view comprising:

[0007] Acquire multi-view images of a target human body wearing clothing;

[0008] Fitting a three-dimensional human body template M(β,θ) of the target human body according to the multi-view images to obtain shape parameters β and posture parameters θ of the three-dimensional human body template;

[0009] Extracting a multi-view clothing image from the multi-view image;

[0010] According to the multi-view clothing image, a sub-grid is extracted based on the three-dimensional human body template M(β,θ) as an initial clothing model G(β,θ,D), and the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values;

[0011] Constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range;

[0012] Constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0013] A three-dimensional human body model wearing clothing is generated based on the three-dimensional human body template and the reconstructed clothing model.

[0014] Furthermore, the step of extracting subgrids as initial clothing models G(β,θ,D) based on the three-dimensional human body template M(β,θ) according to the multi-view clothing images includes:

[0015] Fitting a clothing style template in a T-pose according to the multi-view clothing image to obtain a clothing style parameter D;

[0016] According to the three-dimensional human body template and the clothing style template, an initial clothing model G(β,θ,D) with a posture is obtained by fitting;

[0017] Furthermore, the step of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range includes:

[0018] A boundary range is constructed according to the initial clothing model G(β, θ, D), and free Gaussian points outside the boundary range are cleared.

[0019] Furthermore, the three-dimensional human body template is generated by fitting the multi-view image through the SMPL model, and the step of fitting an initial clothing model G(β, θ, D) with a posture according to the three-dimensional human body template and the clothing style template includes:

[0020] Associating the clothing style template with the SMPL model to associate clothing vertices i∈{1...m} of the clothing style template with body vertices j∈{1...n} of the three-dimensional human body template;

[0021] Get the sub-grid T of the clothing style template in T posture G (β,θ,D)=IT(β,θ)+D, where T(β,θ) is the unweighted SMPL function and I is the indicator matrix;

[0022] Each clothing vertex i∈{1...m} is deformed according to its associated body vertex j∈{1...n}, and the skinning calculation of the SMPL model is applied to each clothing vertex to obtain the initial clothing model with posture, which is expressed as G(β,θ,D)=W(T G (β,θ,D),J(β),θ,W), where W(·) is the learned skinning function of the SMPL model and W is the blending weight of the skeleton J(·).

[0023] Furthermore, the step of constructing a boundary range according to the initial clothing model G(β, θ, D) and removing free Gaussian points outside the boundary range includes:

[0024] Get the boundary B of the initial clothing model G(β,θ,D) = {x min ,x max ,y min ,y max ,z min ,z max}, taking boundary B as the boundary range;

[0025] Within the boundary B, the sampling points in the x, y, and z axes are divided with a resolution of N to form a spatial grid, where x i ,y j , z k are equally spaced sampling points on the x, y, and z axes respectively. The set of coordinate points in the spatial grid is represented as

[0026] in represents a set of integers;

[0027] For each sampling point s∈S in the spatial grid, the KD tree acceleration method is used to calculate the nearest distance d(s,G) from the sampling point to the boundary B of the initial clothing model G(β,θ,D), and a distance field is constructed based on the distance values ​​of all the sampling points. The distance field is expressed as SDF(S)={d(s,G)|s∈S}, where SDF(S) is the nearest distance field from each sampling point in the spatial grid to the boundary B;

[0028] The discrete distance field SDF(S) is converted into a continuous distance field function Φ(x,y,z) through three-dimensional interpolation;

[0029] For any Gaussian point p=(x p ,y p ,z p )∈P, calculate the distance Φ(p) from the Gaussian point to the boundary B by interpolation, expressed as Φ(p)=Interpolate(SDF(S),p), where P is the set of all Gaussian points;

[0030] Compare the distance Φ(p) of each Gaussian point with the preset distance threshold τ;

[0031] If Φ(p) is greater than or equal to τ, the corresponding Gaussian point is determined to be a free Gaussian point, and the free Gaussian point is removed.

[0032] Furthermore, the step of constructing a loss function of two-dimensional Gaussian sputtering includes:

[0033] Set the loss function to L = L C +γ d L d +γ n L n +γ depth L depth +γ CD L CD , where L c is the rendering loss function, L d is the depth distortion loss function, L n is the normal consistency loss function, L depth is the depth error loss function, L CD is the chamfer distance loss function, γ c , γ d , γ n , γ depth , γ CD They are the corresponding loss functions L c , Ld 、L n 、L depth 、L CD The weight coefficient of .

[0034] Furthermore, the L CD is expressed as:

[0035]

[0036] Among them, G is the point set of the initial clothing model G(β,θ,D), G={g1,g2,...,g m}; P is the set of all Gaussian points of the clothing model during the iterative training process, P = {p1, p2, ..., p n}.

[0037] Furthermore, the L depth is expressed as:

[0038] L depth =‖DD gt ‖,

[0039] Among them, D is the rendering depth value of the clothing model during the iterative training process, D gt is a depth value obtained from the multi-view image through monocular depth estimation.

[0040] Furthermore, the step of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range includes:

[0041] According to the multi-view clothing image, respectively obtain the longest distance values ​​of the clothing in the x-axis, y-axis, and z-axis directions;

[0042] According to the longest distance values ​​in the x, y, and z axis directions, an ellipsoid is set, and the ellipsoid is used as a boundary range to surround the clothing;

[0043] The free Gaussian points outside the ellipsoid in the initial Gaussian point cloud are cleared.

[0044] A second aspect of the present invention provides a 3D image reconstruction apparatus based on a sparse view, the 3D image reconstruction apparatus based on a sparse view comprising:

[0045] A multi-view image acquisition module is used to acquire multi-view images of a target human body wearing clothing;

[0046] A 3D human body fitting module is used to fit a 3D human body template M(β,θ) of the target human body according to the multi-view images, and obtain shape parameters β and posture parameters θ of the 3D human body template;

[0047] A multi-view clothing extraction module, configured to extract a multi-view clothing image from the multi-view image;

[0048] An initial point cloud generation module is used to extract a sub-grid as an initial clothing model G(β,θ,D) based on the three-dimensional human body template M(β,θ) according to the multi-view clothing image, and initialize the three-dimensional Gaussian point cloud position to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values;

[0049] An outlier removal module, configured to construct a boundary range for the initial Gaussian point cloud and remove free Gaussian points outside the boundary range;

[0050] An optimization module is used to construct a loss function of two-dimensional Gaussian sputtering, perform multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously remove free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0051] The three-dimensional image generation module is used to generate a three-dimensional human body model wearing clothing based on the three-dimensional human body template and the reconstructed clothing model.

[0052] A third aspect of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above-mentioned sparse view-based three-dimensional image reconstruction method.

[0053] The sparse view-based three-dimensional image reconstruction method, device and computer equipment of the present invention obtain multi-view images of a target human body wearing clothing; fit a three-dimensional human body template M(β,θ) of the target human body according to the multi-view images to obtain shape parameters β and posture parameters θ of the three-dimensional human body template; extract a multi-view clothing image from the multi-view clothing image; extract a sub-grid as an initial clothing model G(β,θ,D) based on the three-dimensional human body template M(β,θ) according to the multi-view clothing image, and initialize the position of the three-dimensional Gaussian point cloud to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian point p are initialized to arbitrary values; construct a boundary range for the initial Gaussian point cloud, and remove free Gaussian points outside the boundary range; construct A loss function of two-dimensional Gaussian sputtering is established, and the initial Gaussian point cloud is optimized and iteratively trained multiple times according to the loss function. Free Gaussian points outside the boundary range are continuously cleared during the optimization iterative training process until a preset optimization cutoff condition is reached to obtain a reconstructed clothing model; a three-dimensional human body model wearing clothing is generated according to the three-dimensional human body template and the reconstructed clothing model; thereby efficiently and accurately reconstructing a three-dimensional human body model wearing clothing from sparse multi-view images, being able to better handle complex clothing shapes and textures, and effectively removing imaging defects caused by multi-view inconsistency by constructing a boundary range and a loss function for step-by-step optimization, thereby improving the quality and usability of the three-dimensional human body model reconstruction, and being widely used in virtual reality, clothing design, film and television production and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] in:

[0056] Figure 1 A three-dimensional image obtained by using a three-dimensional image reconstruction method in the prior art;

[0057] Figure 2 is a flowchart of a 3D image reconstruction method based on sparse views in one embodiment;

[0058] Figure 3 is a structural block diagram of a 3D image reconstruction device based on sparse views in one embodiment;

[0059] Figure 4 FIG. 1 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] like Figure 2 As shown, in one embodiment, a 3D image reconstruction method based on sparse views is provided, which specifically includes the following steps:

[0062] S1: Acquire multi-view images of the target human body wearing clothing;

[0063] S2: fitting a 3D human body template M(β,θ) of the target human body according to the multi-view images, and obtaining shape parameters β and posture parameters θ of the 3D human body template;

[0064] S3: extracting a multi-view clothing image from the multi-view image;

[0065] S4: According to the multi-view clothing image, a sub-grid is extracted based on the three-dimensional human body template M(β,θ) as an initial clothing model G(β,θ,D), and the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian point p are initialized to arbitrary values;

[0066] S5: constructing a boundary range for the initial Gaussian point cloud, and removing free Gaussian points outside the boundary range;

[0067] S6: constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0068] S7: Generate a three-dimensional human body model wearing clothing based on the three-dimensional human body template and the reconstructed clothing model.

[0069] In this embodiment, in step S1, the multi-perspective image refers to a set of two-dimensional images captured from multiple different angles of the same target person wearing clothing. Multi-perspective images allow the target person and the clothing worn by the target person to be observed from different perspectives. Multiple cameras can be arranged around the target person at different angles, and while the target person remains stationary or in a specific motion, each camera is synchronously triggered to capture images, thereby acquiring images of the target person wearing clothing from different perspectives. Alternatively, a single camera can be used while the target person remains stationary and moved around the target person to capture images from different perspectives, thereby acquiring multi-perspective images.

[0070] In the above step S2, the three-dimensional human body template is a standard human body model with different shapes and postures. The shape parameter β and the posture parameter θ are parameters used to quantitatively describe the specific state of the three-dimensional human body template. Different β and θ values ​​correspond to different human body shapes and postures. Using an existing three-dimensional human body template fitting algorithm, such as the SMPL algorithm, the multi-view image is input into the algorithm. The algorithm will match and adjust with the existing three-dimensional human body template library by analyzing the human body contour, joint points and other features in the image. The shape parameter β is mainly used to describe the body shape characteristics such as fatness, thinness, height, etc. of the human body, and the posture parameter θ is used to describe the bending, rotation and other posture conditions of the human joints. By continuously optimizing the matching process, the three-dimensional human body template that best matches the target human body and its corresponding shape parameter β and posture parameter θ are finally determined.

[0071] In step S3, image segmentation technology, such as the YOLO model, is used. This deep learning-based semantic segmentation algorithm is used to input the multi-view images into a trained model. The model can then identify the human body and clothing in the image and segment the clothing, thereby obtaining clothing images from each perspective. This allows for the acquisition of multi-view clothing images containing only clothing, eliminating interference from other human body parts and allowing for the focus on reconstructing the clothing model.

[0072] In the above step S4, the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values. The step further includes:

[0073] Generate a Gaussian point cloud for each view: For each multi-view clothing image, the image is first normalized to unify parameters such as brightness and contrast to eliminate image inconsistencies caused by differences in shooting environment. Then, a two-dimensional Gaussian sputtering algorithm is used for processing. Specifically, the center point, tangent vector, and scaling factor determine the two-dimensional Gaussian form. The expression is defined on the local tangent plane, and the Gaussian value is calculated using a standard Gaussian function. After a homogeneous coordinate transformation, the two-dimensional Gaussian is projected onto the image plane to establish a connection from world space to screen space. When calculating the inverse of the projection transformation, a biplane parameterization is used to calculate the intersection of the ray and the sputtering due to numerical issues. Voxel alpha blending is used to integrate the alpha-weighted appearance color to complete the final processing of the two-dimensional Gaussian sputtering.

[0074] Merging Gaussian point clouds from each view: After generating Gaussian point clouds for each view, they are merged into a single initial Gaussian point cloud. In this initial point cloud, each Gaussian point p represents a feature of the garment at a specific location. The color and transparency of each Gaussian point p are initialized to arbitrary values. In subsequent optimization iterations, the color and transparency of these Gaussian points are adjusted and optimized based on the actual garment color and transparency information, as well as the degree of match with the multi-view images.

[0075] In step S5, the boundary range is used to define the garment region represented by the initial Gaussian point cloud. Free Gaussian points are those that significantly deviate from the main point cloud distribution and are located outside the boundary range, possibly due to multi-view inconsistencies. The boundary range is determined by calculating the minimum circumscribed ellipsoid of the initial Gaussian point cloud or other suitable boundary shape. Gaussian points outside the boundary range are removed from the point cloud, making the point cloud more compact and accurately representing the garment shape.

[0076] In the above step S6, the loss function usually takes into account factors such as the shape matching degree of the point cloud and the consistency with the original multi-view clothing image. For example, the difference between the optimized point cloud and the theoretical point cloud obtained by back-projection of the original image is calculated as the loss value. In each iterative training, the parameters such as the position, size and color of the Gaussian points are adjusted according to the results calculated by the loss function. At the same time, the newly generated free Gaussian points outside the boundary range are continuously checked and cleared. The preset optimization cutoff condition can be that the loss value converges to below a certain threshold or reaches the set maximum number of iterations. After multiple optimization iterations, the Gaussian point cloud gradually converges to a state that is highly matched with the original clothing shape and appearance, thereby obtaining an accurate reconstructed clothing model.

[0077] In step S7, the reconstructed clothing model is adapted and fused to the shape and posture of the 3D human body template. Specifically, based on the surface topology of the 3D human body template, the point cloud or mesh of the reconstructed clothing model is deformed and fitted so that it accurately overlays the 3D human body template, ultimately generating a complete 3D human body model wearing clothing that conforms to actual wearing effects.

[0078] The sparse view-based 3D image reconstruction method of this embodiment, through the aforementioned steps, efficiently and accurately reconstructs a 3D human model wearing clothing from sparse multi-view images. Compared to traditional methods, this embodiment provides more flexible and accurate clothing model reconstruction, better handling complex clothing shapes and textures. By constructing boundary ranges and loss functions for step-by-step optimization, it effectively removes imaging defects caused by multi-view inconsistencies, improving the quality and usability of 3D human model reconstruction. It is widely applicable to various fields, including virtual reality, fashion design, and film and television production.

[0079] In some specific embodiments, the step S4 of extracting a sub-grid as an initial clothing model G(β, θ, D) based on a three-dimensional human body template M(β, θ) according to the multi-view clothing image includes:

[0080] S401: fitting a clothing style template in a T-pose according to the multi-view clothing image to obtain a clothing style parameter D;

[0081] S402: fitting an initial clothing model G(β, θ, D) with a posture according to the 3D human body template and the clothing style template;

[0082] In this embodiment, in the above step S401, the T posture is a standard posture similar to that of a human body in the shape of the letter "T". The clothing style parameter D is a set of numerical values ​​that quantitatively describe the specific characteristics of the clothing style template. Different D values ​​correspond to different clothing styles, sizes, etc. The T posture is a standard human posture. Fitting a clothing style template under this posture helps to simplify calculations and improve the versatility of the template. The above clothing style template does not have the posture of the target human body. Through a clothing style fitting algorithm, such as a clothing fitting method based on machine learning in the prior art, during the fitting process, clothing features (such as outlines, wrinkles, texture distribution, etc.) in the multi-perspective clothing image are matched with a pre-established clothing style template library. By continuously adjusting and optimizing the matching parameters, the clothing style template in the T posture that best fits the multi-perspective clothing image is eventually obtained, and its corresponding clothing style parameter D is determined.

[0083] In step S402, the clothing style template is deformed and adapted to the shape and posture of the 3D human body template using a 3D model fitting method. Specifically, the size and weight of the garment are adjusted to fit the human body based on the human body shape parameter β. The garment is then bent and rotated to fit the human body based on the posture parameter θ. Through a series of geometric transformations and parameter adjustments, an initial garment model G(β, θ, D) is ultimately obtained, which accurately fits the 3D human body template with a specific shape and posture.

[0084] In some specific embodiments, the step S5 of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range includes:

[0085] S501: constructing a boundary range according to the initial clothing model G(β, θ, D), and removing free Gaussian points outside the boundary range.

[0086] In step S501, the boundary range can be determined by calculating the minimum bounding box of the initial clothing model G(β,θ,D) or using a convex hull algorithm based on its surface. After determining the boundary range, the initial Gaussian point cloud is traversed. Gaussian points outside the boundary range are determined to be free Gaussian points and removed from the point cloud. During the judgment process, the geometric relationship between the spatial coordinates of the Gaussian point and the boundary range can be used to determine whether the point is inside the rectangular bounding box.

[0087] This embodiment further constructs a boundary range based on the initial clothing model, and more accurately determines a reasonable boundary range based on the actual characteristics of the human body and clothing, so that the point cloud more accurately represents the shape of the clothing model, provides a more accurate boundary range for subsequent optimization iterative training, and improves the accuracy and quality of the reconstructed clothing model.

[0088] In some specific embodiments, the three-dimensional human body template is generated by fitting the multi-view image through the SMPL model, and the step S402 of fitting an initial clothing model G(β, θ, D) with a posture according to the three-dimensional human body template and the clothing style template includes:

[0089] S4021: Associating the clothing style template with the SMPL model, so as to associate the clothing vertex i∈{1...m} of the clothing style template with the body vertex j∈{1...n} of the three-dimensional human body template;

[0090] S4022: Obtain the sub-grid T of the clothing style template in T posture G (β,θ,D)=IT(β,θ)+D, where T(β,θ) is the unweighted SMPL function and I is the indicator matrix;

[0091] S5023: Each clothing vertex i∈{1...m} is deformed according to its associated body vertex j∈{1...n}, and the skinning calculation of the SMPL model is applied to each clothing vertex to obtain an initial clothing model with a posture, which is expressed as G(β,θ,D)=W(T G (β,θ,D),J(β),θ,W), where W(·) is the learned skinning function of the SMPL model and W is the blending weight of the skeleton J(·).

[0092] In this embodiment, in step S4021, the SMPL model is a three-dimensional parametric model of the human body based on statistical learning, capable of flexibly representing human bodies of various shapes and postures using a small number of parameters. Garment vertices are the basic geometric points that constitute the garment style template and are used to describe the shape of the garment; body vertices are the basic geometric points that constitute the three-dimensional human body template. The association operation establishes a correspondence between garment vertices and body vertices to facilitate subsequent deformation of the garment based on the human body's morphology. A garment style template has a series of m garment vertices; a three-dimensional human body template is generated using the SMPL model and has n body vertices. Using geometric matching or feature-based matching algorithms, for example, by calculating the similarity of geometric features such as spatial distances and normal vectors between garment vertices and body vertices, a correspondence between the two can be established. Associating garment vertices with corresponding body vertices enables subsequent deformation and adaptation of the garment based on the structure of the human body.

[0093] In step S4022, the unweighted SMPL function T(β,θ) is the function portion of the SMPL model that calculates basic human body geometry information based on shape and pose parameters. The indicator matrix I is a matrix used for specific selection or labeling operations, and is used to select human body parts related to clothing or label clothing vertices.

[0094] By multiplying the indicator matrix I with the unweighted SMPL function T(β,θ) and adding the clothing style parameter D, the sub-grid T of the clothing style template in T pose can be obtained. G (β,θ,D)=IT(β,θ)+D, combined with the style characteristics of the clothing itself, constructs a preliminary geometric representation of the clothing in the standard T posture.

[0095] In the above step S4023, based on the association relationship between clothing vertices and body vertices established in step S5021, for each clothing vertex i, the associated body vertex j is found. The skinning calculation of the SMPL model is based on the skeleton J(·) and the mixing weight W. The skeleton J(·) determines the position and posture of the human skeleton according to the posture parameter θ and shape parameter β of the human body, and the mixing weight W determines the degree of influence of the skeleton on different vertices. The learning skinning function W(·) performs deformation calculation on each clothing vertex based on the sub-grid TG(β,θ,D), the skeleton J(·) and the mixing weight W. Specifically, by distributing the movement and influence of the skeleton to each clothing vertex according to the mixing weight, the clothing vertex can be deformed accordingly according to the posture of the human body, and finally an initial clothing model G(β,θ,D) with posture is obtained.

[0096] In some specific embodiments, the step S501 of constructing a boundary range according to the initial clothing model G(β, θ, D) and removing free Gaussian points outside the boundary range includes:

[0097] S5011: Obtain the boundary B of the initial clothing model G(β,θ,D) = {x min ,x max ,y min ,y max ,z min ,z max}, taking boundary B as the boundary range;

[0098] S5012: Within the boundary B, the sampling points in the x, y, and z axes are divided at a resolution of N to form a spatial grid, where x i ,y j , z k are equally spaced sampling points on the x, y, and z axes respectively. The set of coordinate points in the spatial grid is represented as

[0099] in represents a set of integers;

[0100] S5013: For each sampling point s∈S in the spatial grid, the closest distance value d(s,G) from the sampling point to the boundary B of the initial clothing model G(β,θ,D) is calculated using the KD tree acceleration method, and a distance field is constructed based on the distance values ​​of all the sampling points. The distance field is expressed as SDF(S)={d(s,G)|s∈S}, where SDF(S) is the closest distance field from each sampling point in the spatial grid to the boundary B;

[0101] S5014: Convert the discrete distance field SDF(S) into a continuous distance field function Φ(x, y, z) by three-dimensional interpolation.

[0102] S5015: For any Gaussian point p=(x p ,y p ,z p )∈P, calculate the distance Φ(p) from the Gaussian point to the boundary B by interpolation, expressed as Φ(p)=Interpolate(SDF(S),p), where P is the set of all Gaussian points;

[0103] S5016: Compare the distance Φ(p) of each Gaussian point with a preset distance threshold τ;

[0104] S5017: If Φ(p) is greater than or equal to τ, the corresponding Gaussian point is determined to be a free Gaussian point, and the free Gaussian point is cleared.

[0105] In this embodiment, all vertex coordinates of the initial clothing model G(β,θ,D) are traversed. In the three-dimensional space, the minimum and maximum values ​​of the coordinate values ​​in the x-axis, y-axis, and z-axis directions are found respectively. For example, in the x-axis direction, the x coordinate values ​​of all vertices are compared to determine the minimum x min and the largest x max ; Similarly, determine the y direction of the y axis min 、y max and z in the z-axis direction min 、z max The range determined by these six values ​​constitutes the boundary B of the initial clothing model.

[0106] The distance field SDF(S) records the closest distance value from each sampling point in the spatial grid to the boundary B of the initial clothing model. The value in the distance field can be positive or negative, which is used to indicate whether the point is inside or outside the model.

[0107] The three-dimensional interpolation method described above estimates the values ​​of other points in a continuous space based on the values ​​of discrete points in three-dimensional space. The continuous distance field function Φ(x, y, z) expands the discrete distance field data into a function that varies continuously throughout the entire three-dimensional space. It can provide distance information from any point in space to the boundary of the initial clothing model.

[0108] The preset distance threshold τ is a manually set distance critical value used to determine whether a Gaussian point is a free point. By comparing with this threshold, it can be determined whether the Gaussian point is within a reasonable range. If the distance Φ(p) of a Gaussian point p is greater than or equal to the preset distance threshold τ, the Gaussian point is considered to be far away from the initial clothing model body and is a free Gaussian point. Removing the free Gaussian point from the Gaussian point set P completes the clearing operation, allowing the remaining Gaussian points to more accurately represent the shape of the clothing model, improving the accuracy and quality of clothing model reconstruction, and providing more accurate data for subsequent optimization iterative training.

[0109] This embodiment accurately considers the distance relationship between each point in space and the clothing model through a series of operations such as obtaining the model boundary, constructing the spatial grid, calculating the distance field, converting it into a continuous function, calculating the Gaussian point distance and comparing it with the threshold, thereby improving the accuracy and rationality of the judgment of free Gaussian points, effectively optimizing the initial Gaussian point cloud, and providing a high-quality data foundation for subsequent clothing model optimization iterative training, thereby improving the accuracy and quality of the final reconstructed clothing model and the three-dimensional human body model wearing clothing.

[0110] In some specific embodiments, the step S6 of constructing the loss function of two-dimensional Gaussian sputtering includes:

[0111] S601: Set the loss function to L = L C +γ d L d +γ n L n +γ depth L depth +γ CD L CD , where L c is the rendering loss function, L d is the depth distortion loss function, L n is the normal consistency loss function, L depth is the depth error loss function, L CD is the chamfer distance loss function, γ c , γ d , γ n , γ depth , γ CD They are the corresponding loss functions L c , L d , L n , L depth , L CD The weight coefficient of .

[0112] In this embodiment, the rendering loss function Lc is used to evaluate the visual difference between the rendered image and the real image, such as the degree of mismatch in color, brightness, texture, etc. The depth distortion loss function Ld is used to detect whether the deformation of the model in the depth direction meets the expectation and avoid unreasonable depth distortion. The normal consistency loss function Ln is used to ensure the correctness of the normal direction of the model surface. The depth error loss function L depth : The error between the depth value obtained by comparing the model rendering and the real depth value obtained from the multi-view image through monocular depth estimation. Chamfer distance loss function L CD: Measures the similarity between two point sets and evaluates the degree of matching of the model shape by calculating the minimum distance between the point sets. The weight coefficient is used to adjust the proportion of each loss function in the total loss function to balance the optimization requirements of different aspects.

[0113] This embodiment constructs a comprehensive loss function that can evaluate and optimize the clothing model generated by two-dimensional Gaussian sputtering from multiple dimensions, guiding the model to develop in a direction that is more in line with actual needs during the training process, and improving the reconstruction quality of the clothing model.

[0114] In some specific embodiments, L d =∑ i,j w i w j |z i -z j |, where w i and w j are the mixing weights of the i-th and j-th Gaussian points, respectively. The mixing weight is used to measure the relative importance of different Gaussian points in calculating the loss. The larger the value, the higher the weight of the corresponding Gaussian point in the loss calculation. i and z j are the depth values ​​of the i-th and j-th Gaussian points respectively. The depth value describes the distance between the Gaussian point and the camera in three-dimensional space. In this loss function, the depth distortion is measured by calculating the absolute value of the difference between the depth values ​​of different Gaussian points and combining it with the mixing weight. Among them, w i is the mixing weight of the i-th intersection, which is used to adjust the contribution of the intersection in the loss calculation. is the normal pointing to the camera. N is the normal estimated by the gradient in the depth map, and the normal direction of the scene surface is estimated by the gradient information of the depth map. Compare to measure the consistency of normals.

[0115] In some specific embodiments, the L CD is expressed as:

[0116]

[0117] Among them, G is the point set of the initial clothing model G(β,θ,D), G={g1,g2,...,g m}; P is the set of all Gaussian points of the clothing model during the iterative training process, P = {p1, p2, ..., p n By calculating L CDIt can effectively measure the shape difference between the initial clothing model and the Gaussian point set of the clothing model in iterative training, and promote the Gaussian point distribution of the clothing model in the training process to gradually approach the shape of the initial clothing model. For example, it can well restore the collar, cuff holes, and the seam gap in the middle of the pants, greatly improving the accuracy of the model.

[0118] In some specific embodiments, the L depth is expressed as:

[0119] L depth =‖DD gt ‖,

[0120] Among them, D is the rendering depth value of the clothing model during the iterative training process, D gt is the depth value obtained from the multi-view image by monocular depth estimation (for example, using the Depth-Anythingv2 model).

[0121]

[0122] Among them, j is the index item to be traversed, N is the set of index values ​​corresponding to each item involved in calculating the rendering depth, and d j is the depth value of each Gaussian point when observed from the i-th camera, α j is the weight, T j is a cumulative multiplication term, expressed as α k is the weight.

[0123] By calculating L depth , it can quantify the difference between the rendered depth and the real depth of the clothing model in iterative training, and guide the model to adjust its own depth information during the training process to make it more consistent with the actual situation, thereby improving the accuracy and authenticity of the clothing model in the depth dimension.

[0124] The synergistic effect of the above loss functions enables the clothing model to be continuously improved during the iterative training process, more accurately restoring the shape, depth and other features of real clothing, thereby ultimately generating a high-quality 3D human body model wearing clothing.

[0125] In some specific embodiments, the step S5 of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range includes:

[0126] S511: Obtaining the longest distance values ​​of the clothing in the x-axis, y-axis, and z-axis directions respectively according to the multi-view clothing image;

[0127] S512: setting an ellipsoid according to the longest distance values ​​in the x, y, and z axis directions, and using the ellipsoid as a boundary range to surround the garment;

[0128] S513: Clearing free Gaussian points outside the ellipsoid in the initial Gaussian point cloud.

[0129] In this embodiment, in step S511, the multi-view clothing images are preprocessed to unify the image coordinate system and scale. Then, for each image, the edge contour of the clothing is extracted using, for example, an edge detection algorithm. For the obtained edge contour, the two points with the largest x-coordinate difference on the contour are found in the x-axis direction, and their distance is calculated to obtain the distance value in the x-axis direction. Similarly, the same operation is performed in the y-axis and z-axis directions to obtain the distance values ​​in the y-axis and z-axis directions. All multi-view clothing images are traversed, and the distance values ​​in each direction are compared. The maximum value is selected and used as the longest distance value of the clothing in the x-, y-, and z-axis directions, respectively.

[0130] In step S512, the longest distances along the x, y, and z axes are known and are designated as a, b, and c. An ellipsoid is defined with the geometric center of the garment as its origin. The lengths of the three semi-axes of the ellipsoid correspond to the longest distances along the x, y, and z axes, respectively, completely enclosing the garment. This serves as the boundary for determining whether the Gaussian point is within the garment's bounds.

[0131] In the above step S513, the free Gaussian points outside the ellipsoid in the initial Gaussian point cloud are removed, so that the point cloud can more accurately represent the shape of the clothing model, improve the accuracy and quality of clothing model reconstruction, and provide a more accurate data basis for subsequent optimization iterative training.

[0132] This embodiment uses an ellipsoid as the boundary range, which is a more general constraint method. It no longer needs to calculate the accurate boundary range based on the initial clothing model. Although it contains less prior experience than the initial clothing model, it can still achieve a good spatial geometric constraint effect.

[0133] like Figure 3 As shown, in one embodiment, a 3D image reconstruction apparatus based on a sparse view is provided, and the 3D image reconstruction apparatus based on a sparse view includes:

[0134] A multi-view image acquisition module 10 is used to acquire multi-view images of a target human body wearing clothing;

[0135] A 3D human body fitting module 20 is configured to fit a 3D human body template M(β,θ) of a target human body according to the multi-view images, and obtain shape parameters β and posture parameters θ of the 3D human body template;

[0136] A multi-view clothing extraction module 30 is used to extract a multi-view clothing image from the multi-view image;

[0137] An initial point cloud generation module 40 is configured to extract a subgrid as an initial clothing model G(β, θ, D) based on a three-dimensional human body template M(β, θ) according to the multi-view clothing image, and initialize the three-dimensional Gaussian point cloud position to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a plurality of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values;

[0138] An outlier removal module 50 is configured to construct a boundary range for the initial Gaussian point cloud and remove free Gaussian points outside the boundary range;

[0139] an optimization module 60 for constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0140] The three-dimensional image generation module 70 is configured to generate a three-dimensional human body model wearing clothing based on the three-dimensional human body template and the reconstructed clothing model.

[0141] Furthermore, the initial point cloud generation module 40 includes:

[0142] a clothing style template fitting unit, configured to fit a clothing style template in a T posture according to the multi-view clothing image, and obtain a clothing style parameter D;

[0143] An initial clothing model fitting unit, configured to obtain an initial clothing model G(β, θ, D) having a posture according to the three-dimensional human body template and the clothing style template;

[0144] Furthermore, the outlier removal module 50 includes:

[0145] The first outlier removal unit is configured to construct a boundary range according to the initial clothing model G(β, θ, D) and remove free Gaussian points outside the boundary range.

[0146] Furthermore, the three-dimensional human body template is generated by fitting the multi-view image through the SMPL model, and the initial clothing model fitting unit includes:

[0147] An associating subunit, configured to associate the clothing style template with the SMPL model, so as to associate clothing vertices i∈{1…m} of the clothing style template with body vertices j∈{1…n} of the three-dimensional human body template;

[0148] The subgrid acquisition subunit is used to obtain the subgrid T of the clothing style template in the T posture G(β,θ,D)=IT(β,θ)+D, where T(β,θ) is the unweighted SMPL function and I is the indicator matrix;

[0149] The skinning subunit is used to deform each clothing vertex i∈{1…m} according to its associated body vertex j∈{1…n}, and apply the skinning calculation of the SMPL model to each clothing vertex to obtain an initial clothing model with a posture, which is expressed as G(β,θ,D)=W(T G (β,θ,D),J(β),θ,W), where W(·) is the learned skinning function of the SMPL model and W is the blending weight of the skeleton J(·).

[0150] Furthermore, the first outlier removal unit includes:

[0151] The first boundary acquisition subunit is used to obtain the boundary B of the initial clothing model G(β,θ,D)={x min ,x max ,y min ,y max ,z min ,z max}, taking boundary B as the boundary range;

[0152] The spatial grid division subunit is used to divide the sampling points in the x, y, and z axes within the boundary B with a resolution of N to form a spatial grid, where x i ,y j , z k are equally spaced sampling points on the x, y, and z axes respectively. The set of coordinate points in the spatial grid is represented as

[0153] in represents a set of integers;

[0154] A distance field construction subunit is used to calculate, for each sampling point s∈S in the spatial grid, the nearest distance value d(s,G) from the sampling point to the boundary B of the initial clothing model G(β,θ,D) by the KD tree acceleration method, and construct a distance field based on the distance values ​​of all the sampling points. The distance field is expressed as SDF(S)={d(s,G)|s∈S}, where SDF(S) is the nearest distance field from each sampling point in the spatial grid to the boundary B;

[0155] A continuous distance field calculation subunit, used to convert the discrete distance field SDF(S) into a continuous distance field function Φ(x, y, z) through a three-dimensional interpolation method;

[0156] The calculation subunit is used for any Gaussian point p=(x p ,y p ,zp )∈P, calculate the distance Φ(p) from the Gaussian point to the boundary B by interpolation, expressed as Φ(p)=Interpolate(SDF(S),p), where P is the set of all Gaussian points;

[0157] A comparison subunit, configured to compare the distance Φ(p) of each Gaussian point with a preset distance threshold τ;

[0158] The free point clearing subunit is used to determine that the corresponding Gaussian point is a free Gaussian point if Φ(p) is greater than or equal to τ, and clear the free Gaussian point.

[0159] Furthermore, the optimization module 60 includes:

[0160] Loss function setting unit, used to set the loss function to L = L C +γ d L d +γ n L n +γ depth L depth +γ CD L CD , where L c is the rendering loss function, L d is the depth distortion loss function, L n is the normal consistency loss function, L depth is the depth error loss function, L CD is the chamfer distance loss function, γ c , γ d , γ n , γ depth , γ CD They are the corresponding loss functions L c , L d , L n , L depth , L CD The weight coefficient of .

[0161] Furthermore, the L CD is expressed as:

[0162]

[0163] Among them, G is the point set of the initial clothing model G(β,θ,D), G={g1,g2,...,g m}; P is the set of all Gaussian points of the clothing model during the iterative training process, P = {p1, p2, ..., p n}.

[0164] Furthermore, the L depth is expressed as:

[0165] L depth =‖DD gt ‖,

[0166] Among them, D is the rendering depth value of the clothing model during the iterative training process, D gt is a depth value obtained from the multi-view image through monocular depth estimation.

[0167] Furthermore, the outlier removal module 50 includes:

[0168] a distance value determining unit, configured to obtain the longest distance values ​​of the clothing in the x-, y-, and z-axis directions respectively according to the multi-view clothing image;

[0169] An ellipsoid creation unit is used to set an ellipsoid according to the longest distance values ​​in the x, y, and z axis directions, and use the ellipsoid as a boundary range to surround the garment;

[0170] The second outlier removing unit is configured to remove free Gaussian points outside the ellipsoid in the initial Gaussian point cloud.

[0171] This embodiment efficiently and accurately reconstructs a 3D human model wearing clothing from sparse multi-view images. Compared to traditional devices, this embodiment provides more flexible and accurate clothing model reconstruction, better handling complex clothing shapes and textures. By constructing boundary ranges and loss functions for step-by-step optimization, it effectively removes imaging defects caused by multi-view inconsistencies, improving the quality and usability of 3D human model reconstruction. It can be widely used in fields such as virtual reality, fashion design, and film and television production.

[0172] Figure 4 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a three-dimensional image reconstruction method based on a sparse view. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a three-dimensional image reconstruction method based on a sparse view. Those skilled in the art will understand that Figure 4The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0173] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0174] Acquire multi-view images of a target human body wearing clothing;

[0175] Fitting a three-dimensional human body template M(β,θ) of the target human body according to the multi-view images to obtain shape parameters β and posture parameters θ of the three-dimensional human body template;

[0176] Extracting a multi-view clothing image from the multi-view image;

[0177] According to the multi-view clothing image, a sub-grid is extracted based on the three-dimensional human body template M(β,θ) as an initial clothing model G(β,θ,D), and the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values;

[0178] Constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range;

[0179] Constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0180] A three-dimensional human body model wearing clothing is generated based on the three-dimensional human body template and the reconstructed clothing model.

[0181] This embodiment efficiently and accurately reconstructs a 3D human model wearing clothing from sparse multi-view images. Compared to traditional methods, this embodiment provides more flexible and accurate clothing model reconstruction, better handling complex clothing shapes and textures. By constructing boundary ranges and loss functions for step-by-step optimization, it effectively removes imaging defects caused by multi-view inconsistencies, improving the quality and usability of 3D human model reconstruction. It can be widely used in fields such as virtual reality, clothing design, and film and television production.

[0182] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor performs the following steps:

[0183] Acquire multi-view images of a target human body wearing clothing;

[0184] Fitting a three-dimensional human body template M(β,θ) of the target human body according to the multi-view images to obtain shape parameters β and posture parameters θ of the three-dimensional human body template;

[0185] Extracting a multi-view clothing image from the multi-view image;

[0186] According to the multi-view clothing image, a sub-grid is extracted based on the three-dimensional human body template M(β,θ) as an initial clothing model G(β,θ,D), and the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values;

[0187] Constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range;

[0188] Constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model;

[0189] A three-dimensional human body model wearing clothing is generated based on the three-dimensional human body template and the reconstructed clothing model.

[0190] This embodiment efficiently and accurately reconstructs a 3D human model wearing clothing from sparse multi-view images. Compared to traditional methods, this embodiment provides more flexible and accurate clothing model reconstruction, better handling complex clothing shapes and textures. By constructing boundary ranges and loss functions for step-by-step optimization, it effectively removes imaging defects caused by multi-view inconsistencies, improving the quality and usability of 3D human model reconstruction. It can be widely used in fields such as virtual reality, clothing design, and film and television production.

[0191] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0192] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0193] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A 3D image reconstruction method based on sparse views, characterized in that: The sparse view-based three-dimensional image reconstruction method includes: Acquire multi-view images of a target human body wearing clothing; Fitting a three-dimensional human body template M(β,θ) of the target human body according to the multi-view images to obtain shape parameters β and posture parameters θ of the three-dimensional human body template; Extracting a multi-view clothing image from the multi-view image; According to the multi-view clothing image, a sub-grid is extracted based on the three-dimensional human body template M(β,θ) as an initial clothing model G(β,θ,D), and the three-dimensional Gaussian point cloud position is initialized to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values; Constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range; Constructing a two-dimensional Gaussian sputtering loss function, performing multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously removing free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model; A three-dimensional human body model wearing clothing is generated based on the three-dimensional human body template and the reconstructed clothing model.

2. The sparse view-based 3D image reconstruction method according to claim 1, wherein: The step of extracting subgrids as initial clothing models G(β,θ,D) based on the three-dimensional human body template M(β,θ) according to the multi-view clothing images includes: Fitting a clothing style template in a T-pose according to the multi-view clothing image to obtain a clothing style parameter D; According to the three-dimensional human body template and the clothing style template, an initial clothing model G(β,θ,D) with a posture is obtained by fitting; Furthermore, the steps of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range include: A boundary range is constructed according to the initial clothing model G(β, θ, D), and free Gaussian points outside the boundary range are cleared.

3. The sparse view-based 3D image reconstruction method according to claim 2, wherein: The three-dimensional human body template is generated by fitting the multi-view image through the SMPL model, and the step of fitting an initial clothing model G(β, θ, D) with a posture according to the three-dimensional human body template and the clothing style template includes: Associating the clothing style template with the SMPL model to associate clothing vertices i∈{1...m} of the clothing style template with body vertices j∈{1...n} of the three-dimensional human body template; Get the sub-grid T of the clothing style template in T posture G (β,θ,D)=IT(β,θ)+D, where T(β,θ) is the unweighted SMPL function and I is the indicator matrix; Each clothing vertex i∈{1...m} is deformed according to its associated body vertex j∈{1...n}, and the skinning calculation of the SMPL model is applied to each clothing vertex to obtain the initial clothing model with posture, which is expressed as G(β,θ,D)=W(T G (β,θ,D),J(β),θ,W), where W(·) is the learned skinning function of the SMPL model and W is the blending weight of the skeleton J(·).

4. The sparse view-based 3D image reconstruction method according to claim 2, wherein: The step of constructing a boundary range according to the initial clothing model G(β, θ, D) and removing free Gaussian points outside the boundary range includes: Get the boundary B of the initial clothing model G(β,θ,D) = {x min ,x max ,y min ,y max ,z min ,z max }, taking boundary B as the boundary range; Within the boundary B, the sampling points in the x, y, and z axes are divided with a resolution of N to form a spatial grid, where x i ,y j , z k are equally spaced sampling points on the x, y, and z axes respectively. The set of coordinate points in the spatial grid is represented as S={(x i ,y j ,z k )|x i ∈[x min ,x max ],y j ∈[y min ,y max ],z k ∈[z min ,z max ],i,j, 0≤i,j,k<N}, where represents a set of integers; For each sampling point s∈S in the spatial grid, the KD tree acceleration method is used to calculate the nearest distance d(s,G) from the sampling point to the boundary B of the initial clothing model G(β,θ,D), and a distance field is constructed based on the distance values ​​of all the sampling points. The distance field is expressed as SDF(S)={d(s,G)|s∈S}, where SDF(S) is the nearest distance field from each sampling point in the spatial grid to the boundary B; The discrete distance field SDF(S) is converted into a continuous distance field function Φ(x,y,z) through three-dimensional interpolation; For any Gaussian point p=(x p ,y p ,z p )∈P, calculate the distance Φ(p) from the Gaussian point to the boundary B by interpolation, expressed as Φ(p)=Interpolate(SDF(S),p), where P is the set of all Gaussian points; Compare the distance Φ(p) of each Gaussian point with the preset distance threshold τ; If Φ(p) is greater than or equal to τ, the corresponding Gaussian point is determined to be a free Gaussian point, and the free Gaussian point is removed.

5. The sparse view-based 3D image reconstruction method according to claim 2, wherein: The step of constructing a loss function of two-dimensional Gaussian sputtering comprises: Set the loss function to L = L C +γ d L d +γ n L n +γ depth L depth +γ CD L CD , where L c is the rendering loss function, L d is the depth distortion loss function, L n is the normal consistency loss function, L depth is the depth error loss function, L CD is the chamfer distance loss function, γ c , γ d , γ n , γ depth , γ CD They are the corresponding loss functions L c , L d , L n , L depth , L CD The weight coefficient of .

6. The sparse view-based 3D image reconstruction method according to claim 5, wherein: The L CD is expressed as: Among them, G is the point set of the initial clothing model G(β,θ,D), G={g1,g2,...,g m }; P is the set of all Gaussian points of the clothing model during the iterative training process, P = {p1, p2, ..., p n }.

7. The sparse view-based 3D image reconstruction method according to claim 5, wherein: The L depth is expressed as: Ldpth=‖D-Dgt‖, Among them, D is the rendering depth value of the clothing model during the iterative training process, D gt is a depth value obtained from the multi-view image through monocular depth estimation.

8. The sparse view-based 3D image reconstruction method according to claim 1, wherein: The step of constructing a boundary range for the initial Gaussian point cloud and removing free Gaussian points outside the boundary range includes: According to the multi-view clothing image, respectively obtain the longest distance values ​​of the clothing in the x-axis, y-axis, and z-axis directions; According to the longest distance values ​​in the x, y, and z axis directions, an ellipsoid is set, and the ellipsoid is used as a boundary range to surround the clothing; The free Gaussian points outside the ellipsoid in the initial Gaussian point cloud are cleared.

9. A 3D image reconstruction device based on sparse views, characterized in that: The sparse view-based three-dimensional image reconstruction device comprises: A multi-view image acquisition module is used to acquire multi-view images of a target human body wearing clothing; A 3D human body fitting module is used to fit a 3D human body template M(β,θ) of the target human body according to the multi-view images, and obtain shape parameters β and posture parameters θ of the 3D human body template; A multi-view clothing extraction module, configured to extract a multi-view clothing image from the multi-view image; An initial point cloud generation module is used to extract a sub-grid as an initial clothing model G(β,θ,D) based on the three-dimensional human body template M(β,θ) according to the multi-view clothing image, and initialize the three-dimensional Gaussian point cloud position to obtain an initial Gaussian point cloud, wherein the initial Gaussian point cloud includes a number of Gaussian points p, and the color and transparency of the Gaussian points p are initialized to arbitrary values; An outlier removal module, configured to construct a boundary range for the initial Gaussian point cloud and remove free Gaussian points outside the boundary range; An optimization module is used to construct a loss function of two-dimensional Gaussian sputtering, perform multiple optimization iterative training on the initial Gaussian point cloud according to the loss function, and continuously remove free Gaussian points outside the boundary range during the optimization iterative training process until a preset optimization cutoff condition is reached, thereby obtaining a reconstructed clothing model; The three-dimensional image generation module is used to generate a three-dimensional human body model wearing clothing based on the three-dimensional human body template and the reconstructed clothing model.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the sparse view-based three-dimensional image reconstruction method according to any one of claims 1 to 8.

Images