Monocular image human body reconstruction method and system based on implicit representation

Abstract

The invention discloses a monocular image human body reconstruction method and system based on implicit representation. Comprising the following steps: acquiring a real human body three-dimensional model, rendering to obtain a human body two-dimensional image, and randomly generating an index value; a Multi-view SMPLify method is used for the two-dimensional image of the human body, and a three-dimensional SMPL model of the human body is generated; a ray pair-based sampling method is used for the human body three-dimensional SMPL model, and a two-dimensional feature map representing three-dimensional features is obtained; finding a human body two-dimensional image at a corresponding angle according to the index value, and transmitting the human body two-dimensional image into an encoder for encoding to obtain a two-dimensional feature map with aligned pixels; and splicing the two two-dimensional feature maps, transmitting the spliced two-dimensional feature maps into a full-connection neural network, training an implicit function representing a human body curved surface by using deep ambiguity perception reconstruction loss, and outputting a reconstructed three-dimensional human body model. According to the method, the advantages of fitting any topological structure by comprehensively utilizing the regularization features and implicit functions of the human body model based on statistics can be comprehensively utilized, the feature map resolution and the training efficiency are improved, and the time overhead is reduced.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and in particular to a monocular image human body reconstruction method and system based on implicit representation. Background Art

[0002] Compared with text, image and other data, 3D data has a richer and more three-dimensional presentation and is more in line with the real world. Therefore, people's demand for the use of 3D technology is increasing day by day, the 3D technology industry is booming, and 3D reconstruction has become one of the important research topics in the field of computer vision. Among them, the reconstruction of 3D human body from monocular images has become one of the research hotspots of this topic with its application in various industries, such as medical, film and television, and games, and has received widespread attention.

[0003] At present, some models have been proposed for the reconstruction of clothed human bodies. Among these models, there are models based on explicit representation and models based on implicit representation. The methods based on explicit representation can be further divided into clothed human body reconstruction methods based on parametric models and clothed human body reconstruction methods based on non-parametric models. However, these methods have disadvantages such as difficulty in handling complex deformations and excessive memory requirements. The method based on implicit representation defines the human body surface as an implicit function, but at this stage it is still difficult to find an implicit function that accurately describes the human body, and the existing reconstruction effect is still not ideal.

[0004] One of the current existing technologies is a method for reconstructing dressed human bodies based on SMPL+D, proposed in the paper "Learning to reconstruct people in clothing from a single RGB camera". This method uses a relatively mature parametric human 3D model - the SMPL model, and adds offsets to it to characterize clothing and hair. The steps are: semantic segmentation and human joint point detection of the input image; estimation of SMPL parameters and offsets; prediction and reconstruction of 3D human bodies. The disadvantage of this method is that although the parametric human body model can depict the 3D morphology of the human body to a certain extent, this method has the problem of over-reliance on scanning 2D human body truth values ​​to train the network, and it is difficult to handle complex deformations such as long hair and skirts.

[0005] The second existing technology is a clothed human body reconstruction method based on pixel-aligned implicit function proposed in the paper "Pixel-aligned implicit function for high-resolution clothed human digitization". This method associates the pixels in the 2D image with the 3D human body information corresponding to the pixel, and based on this, proposes an end-to-end deep learning-based method to obtain the three-dimensional model of the human body and the texture information corresponding to the three-dimensional model from a single 2D image. The steps are: encoding of the two-dimensional feature map; acquisition of depth information; reconstruction and generation of the clothed three-dimensional human body. The disadvantage of this method is that although this method can reconstruct a rough three-dimensional model of the human body, it is difficult to reconstruct the rich details shown in the picture. The limitation of this method mainly comes from the conflict between the large-scale spatial information required for accurate prediction of the three-dimensional model of the human body and the higher-dimensional features required for fine surface details. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing methods and propose a monocular image human body reconstruction method and system based on implicit representation. The main problem solved by the present invention is how to comprehensively utilize the regularization characteristics of the statistical human body model and the advantages of implicit function fitting of arbitrary topological structures, and improve the feature map resolution and training efficiency, and reduce time overhead.

[0007] In order to solve the above problems, the present invention proposes a monocular image human body reconstruction method based on implicit representation, the method comprising:

[0008] Obtain a real human 3D model from the THuman dataset, process it, render it to obtain 2D images of the human body at different angles, and randomly generate index values;

[0009] For the two-dimensional human body images at different angles, a corresponding rough three-dimensional SMPL model of the naked human body is generated using a Multi-view SMPLify method;

[0010] For the rough naked human body three-dimensional SMPL model, a ray pair sampling method is used to obtain a two-dimensional feature map representing the three-dimensional features;

[0011] According to the randomly selected index value, a two-dimensional image of a human body at a corresponding angle is found, and the image is passed into an encoder for encoding to obtain a pixel-aligned two-dimensional feature map;

[0012] The two-dimensional feature map representing the three-dimensional features and the pixel-aligned two-dimensional feature map are spliced ​​and passed into a fully connected neural network, and the implicit function representing the human body surface is trained using the depth ambiguity-aware reconstruction loss to output a reconstructed three-dimensional human body model.

[0013] Preferably, a real human 3D model is obtained from the THuman dataset, processed, rendered to obtain 2D images of the human body at different angles, and index values ​​are randomly generated, specifically:

[0014] Each data item in the THuman dataset contains a human body texture surface mesh;

[0015] For the real human body three-dimensional model, a spherical harmonic global illumination model is used, the total number of viewing angles is n, and n uniformly distributed viewing angles are rendered under random illumination conditions to obtain a two-dimensional image under n viewing angles;

[0016] An index value is randomly generated in the range of 1 to n for each of the data items, and the two-dimensional image of the viewing angle corresponding to the index value is selected as training data, that is, the two-dimensional image of the human body at different angles.

[0017] Preferably, for the two-dimensional human body images at different angles, a Multi-view SMPLify method is used to generate a corresponding rough three-dimensional SMPL model of a naked human body, specifically:

[0018] Inputting the two-dimensional images of the human body at different angles into the convolutional neural network in the Multi-view SMPLify method, obtaining shape parameters and posture parameters of the SMPL model, and initializing the SMPL model;

[0019] In the SMPLify method, the SMPL model is optimized based on the two-dimensional images of the human body at different angles to obtain optimized shape parameters and posture parameters, and the optimized shape parameters and posture parameters are used to supervise the CNN model in the SMPLify method to train the SMPL model to form a self-supervisory framework to obtain the rough naked human three-dimensional SMPL model.

[0020] Preferably, for the rough naked human body three-dimensional SMPL model, a ray pair sampling method is used to obtain a two-dimensional feature map representing the three-dimensional features, specifically:

[0021] Projecting the rough naked human body three-dimensional SMPL model onto an image plane, so that pixels of the rough naked human body three-dimensional SMPL model and the two-dimensional images of the human body at different angles are aligned;

[0022] For each pixel, associate it with a ray, each ray records the index value of the image plane it passes through, obtains the three-dimensional coordinates of the three vertices of the corresponding image plane through the index value, and takes the average to obtain the coordinates of the image plane;

[0023] The two-dimensional feature map representing the three-dimensional feature is calculated according to the coordinates of the image plane and stored using a sparse matrix.

[0024] Preferably, according to the randomly selected index value, a two-dimensional image of the human body at a corresponding angle is found, and the image is passed into an encoder for encoding to obtain a pixel-aligned two-dimensional feature map, specifically:

[0025] The encoder uses an hourglass network with a downsampling-upsampling structure, which includes a convolutional layer, a normalization layer, a residual layer, a pooling layer, and an upsampling layer;

[0026] The two-dimensional image of the human body at the corresponding angle is decomposed and feature extracted by a convolution layer, and then normalized by a normalization layer. Then, the resolution is reduced by a pooling layer to obtain an image with a preliminary reduced resolution. The image with the preliminary reduced resolution is then passed backward through a residual layer to further reduce the resolution of the image. Then, the resolution of the image is amplified by an upsampling layer until it is consistent with the resolution of the original two-dimensional image of the human body at different angles, so as to obtain a pixel-aligned two-dimensional feature map.

[0027] Preferably, the two-dimensional feature map representing the three-dimensional features and the two-dimensional feature map aligned with pixels are spliced, passed into a fully connected neural network, and the implicit function representing the human body surface is trained using the depth ambiguity-aware reconstruction loss, and the reconstructed three-dimensional human body model is output, specifically:

[0028] Using the occupancy probability function F * Represents the human body surface. The training goal is to obtain the implicit function to fit F as much as possible. * ;

[0029] The implicit function is trained using multiple fully connected neural networks in series. The loss function used in the training is a deep inception-aware reconstruction loss. The loss function formula is as follows:

[0030]

[0031] Among them, n p is the number of sampling points, p i is the sampling point with index value i, F * (p i ) is p i The label value of the occupancy probability, Δp i =(0, 0, Δz i ) T is the compensating translation along the z-axis, calculated as follows:

[0032]

[0033]

[0034]

[0035] Where N(i) is the point p i The four nearest points, v j and are the j-th vertex of the predicted and true SMPL models, Z represents the depth value of the vertex, ω j→i It is based on the sample point p i Its adjacent SMPL vertex v j The mixing weight is defined by the distance between i is the weight normalization value;

[0036] After the training is completed, the trained implicit function F is obtained;

[0037] The value of the trained implicit function used to fit the human body surface is x, and the isosurface of F=x is taken to fit the human body surface, that is, the reconstructed three-dimensional human body model.

[0038] Accordingly, the present invention also provides a monocular image human body reconstruction system based on implicit representation, comprising:

[0039] A preprocessing unit is used to obtain a real human 3D model from the THuman data set, process it, render it to obtain a human 2D image at different angles, and randomly generate index values. For the human 2D images at different angles, a Multi-view SMPLify method is used to generate a corresponding rough naked human 3D SMPL model.

[0040] A feature extraction unit is used to obtain a two-dimensional feature map representing three-dimensional features for the rough three-dimensional SMPL model of naked human body by using a ray pair sampling method, find a two-dimensional image of human body at a corresponding angle according to the randomly selected index value, and pass it into an encoder for encoding to obtain a pixel-aligned two-dimensional feature map;

[0041] The human body model reconstruction unit is used to splice the two-dimensional feature map representing the three-dimensional features and the two-dimensional feature map aligned with pixels, pass them into a fully connected neural network, use the depth ambiguity perception reconstruction loss to train the implicit function representing the human body surface, and output the reconstructed three-dimensional human body model.

[0042] The implementation of the present invention has the following beneficial effects:

[0043] The present invention proposes a monocular image human body reconstruction network based on implicit representation. Based on a priori assumptions and implicit representation, the present invention performs three-dimensional reconstruction of the human body from a monocular image, which not only utilizes the regularization characteristics of the statistical human body model, but also utilizes the advantage that the implicit function can theoretically fit any topological structure. At the same time, the present invention uses a ray-based sampling method to obtain a set of two-dimensional feature maps to supplement the three-dimensional information. The obtained feature maps have higher resolution, less time overhead, and higher training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a monocular image human body reconstruction method based on implicit representation according to an embodiment of the present invention;

[0045] Figure 2 It is a structural diagram of a monocular image human body reconstruction system based on implicit representation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Figure 1 is a flow chart of a monocular image human body reconstruction method based on implicit representation according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0048] S1, obtains the real human 3D model from the THuman dataset, processes it, renders the human 2D images at different angles, and randomly generates index values;

[0049] S2, using the Multi-view SMPLify method to generate a corresponding rough naked human three-dimensional SMPL model for the two-dimensional human body images at different angles;

[0050] S3, using a ray pair sampling method to obtain a two-dimensional feature map representing three-dimensional features for the rough three-dimensional SMPL model of the naked human body;

[0051] S4, according to the randomly selected index value, find the two-dimensional image of the human body at the corresponding angle, and pass it into the encoder for encoding to obtain a pixel-aligned two-dimensional feature map;

[0052] S5, splicing the two-dimensional feature map representing the three-dimensional features and the pixel-aligned two-dimensional feature map, passing them into a fully connected neural network, using the depth ambiguity-aware reconstruction loss to train the implicit function representing the human body surface, and outputting a reconstructed three-dimensional human body model.

[0053] Step S1 is as follows:

[0054] S1-1, the THuman data set consists of data items, the data set contains about 7000 data items, each data item contains a human body texture surface mesh; for the real human body three-dimensional model, a spherical harmonic global illumination model is used, the total number of viewing angles is recorded as 60, and 60 uniformly distributed viewing angles are rendered under random lighting conditions to obtain two-dimensional images under 60 viewing angles;

[0055] S1-2, randomly generate an index value in the range of 1 to 60 for each data item, and select the two-dimensional image of the viewing angle corresponding to the index value as training data, that is, the two-dimensional image of the human body at different angles.

[0056] Step S2 is as follows:

[0057] S2-1, inputting the two-dimensional human body images at different angles into the convolutional neural network in the Multi-view SMPLify method, obtaining shape parameters and posture parameters of the SMPL model, wherein the shape parameter is a shape vector composed of 10 scalar values, and the posture parameter is a 24×3 posture vector, and initializing the SMPL model;

[0058] S2-2, in the SMPLify method, the SMPL model is optimized based on the two-dimensional images of the human body at different angles to obtain optimized shape parameters and posture parameters, and the optimized shape parameters and posture parameters are used to supervise the CNN model in the SMPLify method to train the SMPL model, forming a self-supervisory framework to obtain the rough naked human three-dimensional SMPL model.

[0059] Step S3 is as follows:

[0060] S3-1, projecting the rough naked human body three-dimensional SMPL model onto an image plane, so that pixels of the rough naked human body three-dimensional SMPL model and the two-dimensional images of the human body at different angles are aligned;

[0061] S3-2, for each pixel, associate it with a ray, each ray records the index value of the image plane it passes through, and obtains the three-dimensional coordinates of the three vertices of the corresponding image plane through the index value, and takes the average to obtain the coordinates of the image plane; the resolution of the two-dimensional image is 512×512, and a total of 512×512 rays are required; the number of image planes that the ray passes through is set to 6, and a face is represented by the average of the three-dimensional coordinates of the three vertices of the face, so 6×3=18 feature maps need to be calculated;

[0062] S3-3, calculating the two-dimensional feature map representing the three-dimensional features according to the coordinates of the image plane and storing it using a sparse matrix.

[0063] Step S4 is as follows:

[0064] S4-1, the encoder adopts an hourglass network with a downsampling-upsampling structure, which includes a convolution layer with a convolution kernel of 3×3, a normalization layer, a residual layer, a pooling layer, and an upsampling layer;

[0065] The two-dimensional image of the human body at the corresponding angle is decomposed and feature extracted by a convolution layer, and then normalized by a normalization layer. Then, the resolution is reduced by a pooling layer to obtain an image with a preliminary reduced resolution. The image with the preliminary reduced resolution is then passed backward through a residual layer to further reduce the resolution of the image. Then, the resolution of the image is amplified by an upsampling layer until it is consistent with the resolution of the original two-dimensional image of the human body at different angles, so as to obtain a pixel-aligned two-dimensional feature map.

[0066] Step S5 is as follows:

[0067] S5-1, using the occupancy probability function F * represents the human body surface. For a certain sampling point, if it is inside the human body surface, then F* takes the value of 1, otherwise it takes the value of 0. The training goal is to obtain the implicit function to fit F as much as possible. * ;

[0068] The implicit function is trained using multiple fully connected neural networks in series, and the dimensions of each fully connected neural network are 274, 1024, 512, 256, 128 and 1, respectively. The first dimension 274=256+18, 256 is the dimension of the two-dimensional feature map obtained by the encoder of the two-dimensional image, and 18 refers to the 18 corresponding feature maps obtained by ray sampling; the last dimension is the estimation of the occupancy probability value, which is a scalar value. The loss function used in the training uses a deep uprising-aware reconstruction loss, and the loss function formula is as follows:

[0069]

[0070] Among them, np is the number of sampling points, p i is the sampling point with index value i, F * (p i ) is p i The label value of the occupancy probability, Δp i =(0, 0, Δz i ) T is the compensating translation along the z-axis, calculated as follows:

[0071]

[0072]

[0073]

[0074] Where N(i) is the point p i The four nearest points, v j and are the j-th vertex of the predicted and true SMPL models, Z represents the depth value of the vertex, ω j→i It is based on the sample point p i Its adjacent SMPL vertex v j The mixing weight is defined by the distance between i is the weight normalization value;

[0075] After the training is completed, the trained implicit function F is obtained;

[0076] S5-2, the value of the trained implicit function used to fit the human body surface is recorded as 0.5, and the isosurface of F=0.5 is taken to fit the human body surface, that is, the reconstructed three-dimensional human body model.

[0077] Accordingly, the present invention also provides a monocular image human body reconstruction method system based on implicit representation, such as Figure 2 As shown, including:

[0078] Preprocessing unit 1 is used to obtain a real human 3D model from the THuman data set, process it, render it to obtain human 2D images at different angles, and randomly generate index values. For the human 2D images at different angles, a Multi-view SMPLify method is used to generate a corresponding rough naked human 3D SMPL model.

[0079] Specifically, each data item in the THuman dataset contains a human body texture surface mesh;

[0080] For the real human body three-dimensional model, a spherical harmonic global illumination model is used, the total number of viewing angles is n, and n uniformly distributed viewing angles are rendered under random illumination conditions to obtain a two-dimensional image under n viewing angles;

[0081] For each of the data items, an index value is randomly generated in the range of 1 to n, and the two-dimensional image of the viewing angle corresponding to the index value is selected as training data, that is, the two-dimensional image of the human body at different angles;

[0082] Inputting the two-dimensional images of the human body at different angles into the convolutional neural network in the Multi-view SMPLify method, obtaining shape parameters and posture parameters of the SMPL model, and initializing the SMPL model;

[0083] In the SMPLify method, the SMPL model is optimized based on the two-dimensional images of the human body at different angles to obtain optimized shape parameters and posture parameters, and the optimized shape parameters and posture parameters are used to supervise the CNN model in the SMPLify method to train the SMPL model to form a self-supervisory framework to obtain the rough naked human three-dimensional SMPL model.

[0084] The feature extraction unit 2 is used to obtain a two-dimensional feature map representing the three-dimensional features of the rough naked human body three-dimensional SMPL model by using a ray pair sampling method, find the two-dimensional image of the human body at the corresponding angle according to the randomly selected index value, and pass it to the encoder for encoding to obtain a pixel-aligned two-dimensional feature map;

[0085] Specifically, the rough naked human body three-dimensional SMPL model is projected onto an image plane, so that pixels of the rough naked human body three-dimensional SMPL model and the two-dimensional images of the human body at different angles are aligned;

[0086] For each pixel, associate it with a ray, each ray records the index value of the image plane it passes through, obtains the three-dimensional coordinates of the three vertices of the corresponding image plane through the index value, and takes the average to obtain the coordinates of the image plane;

[0087] Calculate the two-dimensional feature map representing the three-dimensional feature according to the coordinates of the image plane and store it using a sparse matrix;

[0088] The encoder uses an hourglass network with a downsampling-upsampling structure, which includes a convolutional layer, a normalization layer, a residual layer, a pooling layer, and an upsampling layer;

[0089] The two-dimensional image of the human body at the corresponding angle is decomposed and feature extracted by a convolution layer, and then normalized by a normalization layer. Then, the resolution is reduced by a pooling layer to obtain an image with a preliminary reduced resolution. The image with the preliminary reduced resolution is then passed backward through a residual layer to further reduce the resolution of the image. Then, the resolution of the image is amplified by an upsampling layer until it is consistent with the resolution of the original two-dimensional image of the human body at different angles, so as to obtain a pixel-aligned two-dimensional feature map.

[0090] A human body model reconstruction unit 3 is used to splice the two-dimensional feature map representing the three-dimensional features and the two-dimensional feature map aligned with pixels, pass them into a fully connected neural network, use the depth ambiguity perception reconstruction loss to train the implicit function representing the human body surface, and output the reconstructed three-dimensional human body model;

[0091] Specifically,

[0092] Using the occupancy probability function F * Represents the human body surface. The training goal is to obtain the implicit function to fit F as much as possible. * ;

[0093] The implicit function is trained using multiple fully connected neural networks in series. The loss function used in the training is a deep inception-aware reconstruction loss. The loss function formula is as follows:

[0094]

[0095] Among them, n p is the number of sampling points, p i is the sampling point with index value i, F * (p i ) is p i The label value of the occupancy probability, Δp i =(0, 0, Δz i ) T is the compensating translation along the z-axis, calculated as follows:

[0096]

[0097]

[0098]

[0099] Where N(i) is the point p i The four nearest points, v j and are the j-th vertex of the predicted and true SMPL models, Z represents the depth value of the vertex, ω j→i It is based on the sample point p i Its adjacent SMPL vertex vj The mixing weight is defined by the distance between i is the weight normalization value;

[0100] After the training is completed, the trained implicit function F is obtained;

[0101] The value of the trained implicit function used to fit the human body surface is x, and the isosurface of F=x is taken to fit the human body surface, that is, the reconstructed three-dimensional human body model.

[0102] Therefore, the present invention reconstructs the human body in three dimensions from a monocular image based on a priori assumptions and implicit representations, which not only utilizes the regularization characteristics of the human body model based on statistics, but also utilizes the advantage that implicit functions can theoretically fit any topological structure. At the same time, the present invention uses a ray-based sampling method to obtain a set of two-dimensional feature maps to supplement the three-dimensional information. The obtained feature maps have higher resolution, less time overhead, and higher training efficiency.

[0103] The above is a detailed introduction to the monocular image human body reconstruction method and system based on implicit representation provided by the embodiments of the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A monocular image human body reconstruction method based on implicit representation, characterized in that: The method comprises: Obtain a real human 3D model from the THuman dataset, process it, render it to obtain 2D images of the human body at different angles, and randomly generate index values; For the two-dimensional human body images at different angles, a corresponding rough three-dimensional SMPL model of the naked human body is generated using a Multi-view SMPLify method; For the rough naked human body three-dimensional SMPL model, a ray pair sampling method is used to obtain a two-dimensional feature map representing the three-dimensional features; According to the randomly selected index value, a two-dimensional image of a human body at a corresponding angle is found, and the image is passed into an encoder for encoding to obtain a pixel-aligned two-dimensional feature map; The two-dimensional feature map representing the three-dimensional features and the pixel-aligned two-dimensional feature map are spliced ​​and passed into a fully connected neural network, and the implicit function representing the human body surface is trained using the depth ambiguity-aware reconstruction loss to output a reconstructed three-dimensional human body model.

2. The method for monocular image human body reconstruction based on implicit representation as claimed in claim 1, characterized in that: The method obtains a real human 3D model from the THuman dataset, processes it, renders a human 2D image at different angles, and randomly generates index values, specifically: Each data item in the THuman dataset contains a human body texture surface mesh; For the real human body three-dimensional model, a spherical harmonic global illumination model is used, the total number of viewing angles is n, and n uniformly distributed viewing angles are rendered under random illumination conditions to obtain a two-dimensional image under n viewing angles; An index value is randomly generated in the range of 1 to n for each of the data items, and the two-dimensional image of the viewing angle corresponding to the index value is selected as training data, that is, the two-dimensional image of the human body at different angles.

3. The monocular image human body reconstruction method based on implicit representation as claimed in claim 1, characterized in that: The multi-view SMPLify method is used to generate a corresponding rough naked human body three-dimensional SMPL model for the two-dimensional human body images at different angles, specifically: Inputting the two-dimensional images of the human body at different angles into the convolutional neural network in the Multi-view SMPLify method, obtaining shape parameters and posture parameters of the SMPL model, and initializing the SMPL model; In the SMPLify method, the SMPL model is optimized based on the two-dimensional images of the human body at different angles to obtain optimized shape parameters and posture parameters, and the optimized shape parameters and posture parameters are used to supervise the CNN model in the SMPLify method to train the SMPL model to form a self-supervisory framework to obtain the rough naked human three-dimensional SMPL model.

4. The monocular image human body reconstruction method based on implicit representation as claimed in claim 1, characterized in that: The rough naked human body three-dimensional SMPL model is subjected to a ray pair sampling method to obtain a two-dimensional feature map representing three-dimensional features, specifically: Projecting the rough naked human body three-dimensional SMPL model onto an image plane, so that pixels of the rough naked human body three-dimensional SMPL model and the two-dimensional images of the human body at different angles are aligned; For each pixel, associate it with a ray, each ray records the index value of the image plane it passes through, obtains the three-dimensional coordinates of the three vertices of the corresponding image plane through the index value, and takes the average to obtain the coordinates of the image plane; The two-dimensional feature map representing the three-dimensional feature is calculated according to the coordinates of the image plane and stored using a sparse matrix.

5. The monocular image human body reconstruction method based on implicit representation as claimed in claim 1, characterized in that: According to the randomly selected index value, a two-dimensional image of the human body at a corresponding angle is found, and the image is passed into the encoder for encoding to obtain a pixel-aligned two-dimensional feature map, specifically: The encoder uses an hourglass network with a downsampling-upsampling structure, which includes a convolutional layer, a normalization layer, a residual layer, a pooling layer, and an upsampling layer; The two-dimensional image of the human body at the corresponding angle is decomposed and features are extracted through a convolution layer, and then normalized in a normalization layer. Then, the resolution is reduced through a pooling layer to obtain an image with a preliminary reduced resolution. The image with the preliminary reduced resolution is then passed backward through a residual layer to further reduce the resolution of the image. Then, the resolution of the image is amplified through an upsampling layer until it is consistent with the resolution of the original two-dimensional image of the human body at the corresponding angle, so as to obtain a pixel-aligned two-dimensional feature map.

6. The method for monocular image human body reconstruction based on implicit representation as claimed in claim 1, characterized in that: The two-dimensional feature map representing the three-dimensional feature and the pixel-aligned two-dimensional feature map are spliced, passed into a fully connected neural network, and the implicit function representing the human body surface is trained using the depth ambiguity perception reconstruction loss, and the reconstructed three-dimensional human body model is output, specifically: Using the occupancy probability function F * Represents the human body surface. The training goal is to obtain the implicit function to fit F as much as possible. * ; The implicit function is trained using multiple fully connected neural networks in series. The loss function used in the training is a deep inception-aware reconstruction loss. The loss function formula is as follows: Among them, n p is the number of sampling points, p i is the sampling point with index value i, F * (p i ) is p i The label value of the occupancy probability, Δp i =(0, 0, Δz i ) T is the compensating translation along the z-axis, calculated as follows: Where N(i) is the point p i The four nearest points, v j and are the j-th vertex of the predicted and true SMPL models, Z represents the depth value of the vertex, ω j→i It is based on the sample point p i Its adjacent SMPL vertex v j The mixing weight is defined by the distance between i is the weight normalization value; After the training is completed, the trained implicit function F is obtained; The value of the trained implicit function used to fit the human body surface is x, and the isosurface of F=x is taken to fit the human body surface, that is, the reconstructed three-dimensional human body model.

7. A monocular image human body reconstruction system based on implicit representation, characterized in that: The system comprises: A preprocessing unit is used to obtain a real human 3D model from the THuman data set, process it, render it to obtain a human 2D image at different angles, and randomly generate index values. For the human 2D images at different angles, a Multi-view SMPLify method is used to generate a corresponding rough naked human 3D SMPL model. A feature extraction unit is used to obtain a two-dimensional feature map representing three-dimensional features for the rough three-dimensional SMPL model of naked human body by using a ray pair sampling method, find a two-dimensional image of human body at a corresponding angle according to the randomly selected index value, and pass it into an encoder for encoding to obtain a pixel-aligned two-dimensional feature map; The human body model reconstruction unit is used to splice the two-dimensional feature map representing the three-dimensional features and the two-dimensional feature map aligned with pixels, pass them into a fully connected neural network, use the depth ambiguity perception reconstruction loss to train the implicit function representing the human body surface, and output the reconstructed three-dimensional human body model.

8. The monocular image human body reconstruction system based on implicit representation as claimed in claim 7, characterized in that: The preprocessing unit is used to obtain a real human 3D model from the THuman data set, process it, render it to obtain a human 2D image at different angles, and randomly generate index values. For the human 2D images at different angles, a Multi-view SMPLify method is used to generate a corresponding rough naked human 3D SMPL model, specifically: Each data item in the THuman dataset contains a human body texture surface mesh; For the real human body three-dimensional model, a spherical harmonic global illumination model is used, the total number of viewing angles is n, and n uniformly distributed viewing angles are rendered under random illumination conditions to obtain a two-dimensional image under n viewing angles; For each of the data items, an index value is randomly generated in the range of 1 to n, and the two-dimensional image of the viewing angle corresponding to the index value is selected as training data, that is, the two-dimensional image of the human body at different angles; Inputting the two-dimensional images of the human body at different angles into the convolutional neural network in the Multi-view SMPLify method, obtaining shape parameters and posture parameters of the SMPL model, and initializing the SMPL model; In the SMPLify method, the SMPL model is optimized based on the two-dimensional images of the human body at different angles to obtain optimized shape parameters and posture parameters, and the optimized shape parameters and posture parameters are used to supervise the CNN model in the SMPLify method to train the SMPL model to form a self-supervisory framework to obtain the rough naked human three-dimensional SMPL model.

9. The monocular image human body reconstruction system based on implicit representation as claimed in claim 7, characterized in that: The feature extraction unit is used to obtain a two-dimensional feature map representing the three-dimensional features of the rough naked human body three-dimensional SMPL model using a ray pair sampling method, find the human body two-dimensional image at the corresponding angle according to the randomly selected index value, and pass it to the encoder for encoding to obtain a pixel-aligned two-dimensional feature map, specifically: Projecting the rough naked human body three-dimensional SMPL model onto an image plane, so that pixels of the rough naked human body three-dimensional SMPL model and the two-dimensional images of the human body at different angles are aligned; For each pixel, associate it with a ray, each ray records the index value of the image plane it passes through, obtains the three-dimensional coordinates of the three vertices of the corresponding image plane through the index value, and takes the average to obtain the coordinates of the image plane; Calculate the two-dimensional feature map representing the three-dimensional feature according to the coordinates of the image plane and store it using a sparse matrix; The encoder uses an hourglass network with a downsampling-upsampling structure, which includes a convolutional layer, a normalization layer, a residual layer, a pooling layer, and an upsampling layer; The two-dimensional image of the human body at the corresponding angle is decomposed and feature extracted by a convolution layer, and then normalized by a normalization layer. Then, the resolution is reduced by a pooling layer to obtain an image with a preliminary reduced resolution. The image with the preliminary reduced resolution is then passed backward through a residual layer to further reduce the resolution of the image. Then, the resolution of the image is amplified by an upsampling layer until it is consistent with the resolution of the original two-dimensional image of the human body at different angles, so as to obtain a pixel-aligned two-dimensional feature map.

10. The monocular image human body reconstruction system based on implicit representation according to claim 7, characterized in that: The human body model reconstruction unit is used to splice the two-dimensional feature map representing the three-dimensional features and the two-dimensional feature map aligned with pixels, pass them into a fully connected neural network, use the depth ambiguity perception reconstruction loss to train the implicit function representing the human body surface, and output the reconstructed three-dimensional human body model, specifically: Using the occupancy probability function F * Represents the human body surface. The training goal is to obtain the implicit function to fit F as much as possible. * ; The implicit function is trained using multiple fully connected neural networks in series. The loss function used in the training is a deep inception-aware reconstruction loss. The loss function formula is as follows: Among them, n p is the number of sampling points, p i is the sampling point with index value i, F * (p i ) is p i The label value of the occupancy probability, Δp i =(0, 0, Δz i ) T is the compensating translation along the z-axis, calculated as follows: Where N(i) is the point p i The four nearest points, v j and are the j-th vertex of the predicted and true SMPL models, Z represents the depth value of the vertex, ω j→i It is based on the sample point p i Its adjacent SMPL vertex v j The mixing weight is defined by the distance between i is the weight normalization value; After the training is completed, the trained implicit function F is obtained; The value of the trained implicit function used to fit the human body surface is x, and the isosurface of F=x is taken to fit the human body surface, that is, the reconstructed three-dimensional human body model.

Images