132 results about "Monocular video" patented technology

The invention provides a 3D posture estimation method and system based on space-time double-flow intersection in the technical field of monocular video 3D human body posture estimation, and the method comprises the steps: S1, obtaining a large number of historical human body videos, carrying out the frame-by-frame processing of each historical human body video through a 2D human body posture estimation model, obtaining a 2D coordinate position, and carrying out the frame-by-frame processing of each historical human body video; constructing a joint space-time sequence data set; s2, creating a 3D human body posture estimation model based on a joint semantic coding module, a dynamic graph convolution module, a space-time double-flow cross fusion module and a 3D posture decoding module; s3, training the 3D human body posture estimation model through the joint space-time sequence data set and the loss function; and S4, acquiring a real-time human body video, and reasoning the real-time human body video through the 2D human body posture estimation model and the 3D human body posture estimation model to obtain a 3D human body posture estimation result. The method has the advantages that the precision and robustness of 3D human body posture estimation are greatly improved.

Gait characterization and analysis based on a monocular video of a walking subject are achieved, in various embodiments, by processing the video using one or more machine-learning models to predict three-dimensional (3D) keypoint coordinates for a set of anatomical keypoints of the subject, along with joint angles, body-segment rotations, and / or gait event classifications, for multiple video frames. The machine-learning models may be trained using ground-truth data acquired with a marker-based motion capture system and / or pressure-sensitive walkway.

The invention discloses a dynamic scene repairing method based on four-dimensional Gaussian sputtering, which is characterized in that efficient dynamic scene space repairing is realized based on a repairing framework of a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repairing model, and by counting the time frame number of Gaussian primitives in a mask area and removing Gaussian points exceeding a threshold value, the dynamic scene space repairing efficiency is improved. According to the method, high-efficiency dynamic object removal is realized, and a mask weighting loss function is introduced to guide the repair area to converge to real texture information in a subsequent time frame, so that the repair quality is improved. The invention provides a dynamic scene repair method based on four-dimensional Gaussian sputtering, and provides a repair framework based on a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repair model for removing and repairing a target object in a dynamic scene to realize three-dimensional repair of the dynamic scene.

The present disclosure describes techniques for processing monocular videos using three-dimensional gaussian splatting (3DGS). Spatial decomposition and temporal decomposition are performed on a monocular video to generate a plurality of clips. A first set of 3DGS representing foreground objects in each of the plurality of clips are initialized and optimized. A second set of 3DGS representing background in each of the plurality of clips are initialized and optimized. Two images are generated for each frame comprised in each of the plurality of clips based on the first set of 3DGS and the second set of 3DGS, respectively. Two images are merged to generate a resulting image for each frame in each of the plurality of clips. The resulting image accurately represents a corresponding frame in the monocular video.

Systems and methods are disclosed that utilize a triplane fusion architecture that achieves both state-of-the-art three-dimensional (3D) reconstruction accuracy and temporal consistency. For instance, recognizing the need to maintain both coherent identity and dynamic per-frame appearance to enable the best possible realism, the triplane fusion architecture may include an undistorter that removes view-dependent distortions from the raw triplane by using the triplane prior as a reference and a fuser that fuses the triplanes together to recover the occluded areas in the input frame by incorporating features from the triplane prior that is lifted from the frontal reference image. By using the undistorter, the fuser, and a volume renderer, the triplane fusion architecture allows for reconstruction of 3D portrait models from a monocular video stream of a user, which may be useful in various applications including 3D telepresence.

The invention discloses a three-dimensional human body posture estimation method and system, and relates to the technical field of computer vision, and the method comprises the steps: extracting two-dimensional human body posture key points from a monocular video picture sequence, and generating a two-dimensional human body posture key point sequence; projecting the two-dimensional key point sequence to a feature space through nonlinear high-dimensional mapping to generate a high-dimensional feature space matrix; and inputting the high-dimensional feature matrix into a three-dimensional human body posture recognition model fusing motion constraints and frequency division spatial-temporal features to obtain a three-dimensional human body posture key point sequence, and realizing three-dimensional human body posture estimation through three-dimensional coordinates. According to the method, the robustness and the detection precision of the monocular three-dimensional human body posture estimation method are improved. Error values of relative movement speed, skeleton length and skeleton direction of the key points are calculated, so that training is easier to converge, and the training process is more stable.

The invention relates to the technical field of three-dimensional reconstruction, and discloses a monocular depth guided object-level NeRF reconstruction method, which comprises the following steps: firstly, through monocular video sequence input, generating a frame-by-frame initial depth map by using a depth estimation module, and estimating a relative camera attitude between adjacent frames through a relative attitude estimation module; calculating the absolute attitude of the camera in combination with the initial depth map and the relative camera attitude; then constructing a geometrically enhanced NeRF model, and optimizing scene representation through multi-resolution hash position coding and spherical harmonic direction coding; introducing photometric loss, depth contrast loss and density loss to jointly optimize parameters of the NeRF model, and constraining geometric reconstruction of the object by using depth information; and finally, through a four-stage iterative training strategy, alternately optimizing depth estimation, a camera attitude and a NeRF model, and generating an object-level controllable three-dimensional model. According to the method, depth information is fully utilized to optimize the NeRF training process, so that end-to-end reconstruction from a monocular video to an object-level controllable model is completed.

The invention discloses a monocular video scene dynamic three-dimensional reconstruction method based on optical flow, and relates to the technical field of scene reconstruction. Calculating an optical flow of each pixel in each frame of image in the monocular video, and determining a dynamic region mask of the image; according to the dynamic region mask of each frame of image, determining a plurality of dynamic object instances with consistent time and space; performing four-dimensional Gaussian sputtering conversion on each frame of image to obtain four-dimensional Gaussian distribution representation; for any one dynamic object instance in any one frame of image, acquiring other images containing the dynamic object instance in different image frames, and according to the coordinates of the Gaussian point clouds of the image and other images, generating a visual angle point cloud, which is not observed in the image, of the object instance; and according to color information of each frame of Gaussian point cloud in the four-dimensional Gaussian distribution representation, performing color rendering on each frame of Gaussian point cloud generating the visual angle point cloud to obtain a reconstructed scene. The method can accurately realize scene reconstruction.

The invention provides a digital human body modeling system and method based on space-time consistency Gaussian sputtering, and the system comprises a Gaussian set construction module which is used for constructing an initial Gaussian set based on a collected monocular RGB video sequence of human body motions; the non-rigid deformation module is used for updating a pre-constructed standard Gaussian set by adopting a Gaussian decoder based on the initial Gaussian set to obtain a dynamic Gaussian set containing non-rigid deformation; the space-time constraint module is used for applying space-time consistency constraint to the dynamic Gaussian set to obtain the dynamic Gaussian set of the space-time consistency constraint; the rigid deformation module is used for obtaining an attitude-driven Gaussian set based on the rigid deformation of the linear binding skeleton; and the projection rendering module is used for rendering the attitude-driven Gaussian set into a Gaussian image under the current frame attitude. According to the method, a space-time consistency mechanism of motion perception is combined, and the consistency problem of non-rigid deformation in a monocular video is effectively solved.

The invention provides a dynamic three-dimensional scene reconstruction and rendering system and method and related equipment, and the method comprises the steps: receiving a monocular video sequence through an image encoder, and carrying out the visual feature extraction of each frame of image; the scene generator constructs a three-plane structure expression aligned with the view angle of the current camera based on the multi-frame visual features extracted by the image encoder, performs time coding on the multi-frame visual features and maps the multi-frame visual features to the three-plane structure expression to generate a three-dimensional scene feature expression; and the scene decoder decodes the three-dimensional scene feature representation and generates a composite image under a corresponding view angle based on a volume rendering mechanism, and the composite image can display geometric details of the scene under different view angles and maintain geometric consistency under different view angles. Modeling and rendering of the dynamic three-dimensional scene are achieved through the monocular video sequence, multi-view input or prior scene information does not need to be depended on, and the adaptive capacity and generalization performance of the system in a rapid change environment are effectively improved.

A monocular video-based three-dimensional reconstruction and depth prediction method and system for a pipeline are provided, relating to the technical field of deep learning and image processing. The method includes the following steps: performing feature extraction and matching and incremental reconstruction on collected image sequences of different pipeline scenes based on the COLMAP method to obtain a camera external parameter matrix corresponding to each image, and constructing a pipeline three-dimensional reconstruction dataset; training the Fast-MVSNet network and the PatchMatchNet network using the existing public datasets, and selecting an optimal network model from multiple trained network models for three-dimensional reconstruction and depth prediction of an actual pipeline scene. A holistic framework from a monocular video to a three-dimensional scene is constructed, which can effectively achieve three-dimensional reconstruction and depth prediction of a drainage pipeline and intuitively display internal structure information of the pipeline.

The invention relates to a real-time digital person generation method and system, and the method comprises the following steps: obtaining a monocular video of a target person, and extracting 3DMM information in the monocular video; performing Gaussian point initialization on the monocular video according to the 3DMM information to obtain Gaussian parameters in a standard space; extracting voice audio features in the monocular video, and inputting the voice audio features into the audio-motion model to obtain a universal face key point motion sequence; converting the universal face key point motion sequence into a target face key point motion sequence through a projection algorithm; inputting the target face key point motion sequence and the Gaussian parameters in the standard space into a face key point guided Gaussian deformation network to obtain Gaussian deformation parameters; and rendering the Gaussian deformation parameter into a corresponding video frame through a Gaussian rasterizer, thereby obtaining a digital person video of the target person. Compared with the prior art, the digital human video generation accuracy and creation flexibility are improved.

The invention provides a motion capture optimization method and a model training method based on a monocular video, and relates to the technical field of computers. The motion capture optimization method based on the monocular video comprises the following steps: inputting the monocular video into a grounding detection model to obtain a grounding detection result used for indicating the grounding probability of feet of a portrait; obtaining three-dimensional motion first data of the portrait according to the monocular video, and further obtaining a foot track of the portrait; according to the grounding probability, performing grounding optimization on the foot track, and performing jitter optimization on the foot track after grounding optimization; and according to the foot trajectory after jitter optimization, reconstructing the three-dimensional action first data through a reverse dynamics algorithm to obtain three-dimensional action second data of the portrait. According to the method provided by the invention, the problem that the accuracy of the three-dimensional motion data reconstructed by the inverse dynamics algorithm is reduced due to the fact that the phenomena of mold penetration and jitter possibly occur in the three-dimensional reconstruction process of the portrait foot is solved.

The invention relates to a monocular video three-dimensional human body high-quality reconstruction method based on Gaussian splashing and normal perception, and belongs to the field of computer aided design, graphics and computer vision. The method comprises the following steps: firstly, based on a monocular video frame in an input three-dimensional human body data set, guiding posture deformation through a time sequence, and carrying out whole body correlation deformation on a standard space posture to obtain an observation space human body posture; secondly, through normal perception Gaussian optimization, adaptive Gaussian density control and human body normal mapping supervision are carried out on the human body posture in the observation space, and a human body model of Gaussian representation is obtained; then, carrying out human body shadow feature learning on the input human body normal map by adopting an illumination model to obtain human body shadow features; and finally, light and shadow enhancement human body rendering is performed on the Gaussian representation human body model in combination with the human body light and shadow features and human body color information input into the monocular video, a human body rendering effect is output, and the quality and reality sense of three-dimensional human body reconstruction of the monocular video are improved.

The invention discloses a dynamic human body reconstruction method and system based on double-motion embedding and point cloud fusion, and relates to the field of digital human reconstruction processing. According to the method, the input frame sequence composed of three adjacent frames is intercepted from the monocular video and preprocessed to obtain necessary input parameters, then the dynamic human body reconstruction model is designed to process the input frame sequence and the input parameters to obtain the human body reconstruction image, the overall operation is fast and convenient, and the use effect is good. According to the dynamic human body reconstruction model designed by the invention, on one hand, a DMPF network part is adopted, dual-motion embedding is adopted to extract multi-modal motion features, and efficient fusion of 2D and 3D motion features is realized, so that richer motion information supervision is obtained, and on the other hand, a mixed point cloud encoder is adopted to fuse isolated point and overall point cloud features, so that the dynamic human body reconstruction model is more efficient in motion information supervision. Therefore, the dependency relationship of the local change of the human body surface on the global change is captured, and correct modeling of geometry and texture of the moving human body is further enhanced.

The invention discloses a monocular video three-dimensional human body posture estimation method based on a bidirectional state space model, and relates to the technical field of three-dimensional human body posture estimation. The method comprises the following steps: two-dimensional attitude sequence data preprocessing and position embedding: slicing a two-dimensional attitude sequence according to a specified length and performing spatial position embedding; expanding the state space model, designing a bidirectional calculation structure in space and time dimensions, and respectively learning a space relationship and a time relationship in combination with a space-time decoupling bidirectional scanning mechanism; post-processing the spatial-temporal characteristics and estimating the attitude; designing a loss function for model optimization, wherein the loss function comprises position loss and speed loss; and monocular video three-dimensional human body posture estimation is carried out based on the optimized bidirectional state space model. According to the method, the accuracy of 3D attitude estimation is improved by adopting key technologies such as extended bidirectional Mama, a space-time decoupling bidirectional scanning mechanism, a space extended bidirectional Mama block and a time extended bidirectional Mama block.

The invention belongs to the technical field of three-dimensional reconstruction, and particularly discloses a three-dimensional human body and scene interaction reconstruction method and system. The method is a three-dimensional human body and scene interactive reconstruction method for monocular video input, not only can quickly complete three-dimensional Gaussian reconstruction of a human body, a static scene and a moving object, but also can quickly complete the three-dimensional Gaussian reconstruction of the human body, the static scene and the moving object by introducing technical means such as posture correction, camera external parameter space regularization, joint reconstruction consistency loss and layered fusion rendering. And accurate alignment of the human body, the scene and the object in spatial positions, shielding relations and illumination styles is ensured, and dynamic interaction reconstruction conforming to real physical logic is realized. Finally, a joint rendering image with high fidelity, continuous time sequence and consistent structure can be generated, a stable and reliable three-dimensional expression capability is provided for complex human body actions, object interaction and scene understanding, and the three-dimensional interaction reconstruction quality and application value under the monocular video condition are greatly improved.

The invention provides a human body posture estimation device and method based on whole body pressure and a monocular video. The device comprises a video feature encoder used for extracting video features related to human body actions from a monocular front view angle video frame sequence of the human body actions; the pressure feature encoder is used for extracting whole body pressure features related to human body actions from the whole body ground pressure frame sequence; the feature fusion device is used for fusing the whole body pressure features and the video features to obtain fused features; and the action regression device is used for performing nonlinear regression calculation according to the fusion features to obtain posture and displacement parameters of the human body parameterized model. According to the method, the interaction relation between the human body and the ground is concerned, the pressure information and the monocular video information are fused, multi-dimensional representation of human body actions is achieved, the defect that a traditional monocular video human body posture estimation method neglects environment information is overcome, and the accuracy of human body posture estimation and the reasonability of interaction between the human body posture estimation method and the ground are remarkably enhanced.

The disclosure relates generally to methods and systems for real time video driven human 3-dimensional (3-D) posture estimation during physical activities. Conventional techniques do not exploit temporal information, they do not give smooth transition of postures over time. Furthermore, the techniques that exploit the temporal information suffer from higher time requirements due to two state computations. The present disclosure solves the technical problems in the art with the methods and systems for real time video driven human 3-D posture estimation during physical activities. The present invention discloses a smart-phone camera based automatic posture monitoring system designed with an auto-encoder based architecture. The disclosed auto-encoder based cross-modal method uses monocular video (2-D image sequences) from a single low-end mobile device (for example, smart-phone camera) for estimating human 3-D posture in real time (˜5 fps) with high accuracy (less than 1 cm error per joint location).

The invention discloses a three-dimensional Gaussian splash reconstruction method for an underwater scene, and belongs to the technical field of computer vision and three-dimensional reconstruction. Comprising the following steps: acquiring a monocular video frame sequence of a target underwater scene, a corresponding camera pose sequence, an initial sparse point cloud, an initial three-dimensional Gaussian point set and learnable physical parameters of an underwater imaging model; in the training process, performing weighted evaluation on a reconstruction error based on a multi-view consistency mechanism of opacity weighting, calculating an importance score of each Gaussian point, and performing densification operation on a three-dimensional Gaussian point set; adopting a staged freezing strategy to cooperatively optimize the three-dimensional Gaussian point set and underwater imaging model parameters; and performing rendering and underwater image synthesis on any new view angle camera pose based on the optimized three-dimensional Gaussian point set and underwater imaging model parameters, and outputting a new view angle synthesized image to represent a reconstruction result. According to the method, the geometric compactness, the visual fidelity and the physical interpretability of an underwater three-dimensional reconstruction result are improved.

The invention relates to the technical field of aviation target measurement, in particular to an aviation scene positioning and mapping method based on implicit neural rendering and optical flow assistance, and the method comprises the steps: extracting a key frame based on a monocular video stream, obtaining a dense depth value through the key frame in combination with TSDF model rendering, and estimating the posture of a camera; performing unsupervised dense scene measurement according to the adjacent key frames and the camera attitude, and performing training by adopting a loss function to obtain a depth map of the current key frame; obtaining a TSDF voxel grid through a TSDF model, fusing the depth map into the TSDF voxel grid, reconstructing a scene three-dimensional model, and forming a positioning and mapping model; an image data set is collected and preprocessed, a training data set is constructed, and a positioning and mapping model is trained; and carrying out aviation scene positioning and mapping based on the trained positioning and mapping model. Through positioning and dense mapping, the precision of positioning and reconstruction is improved, and then the perception capability of the aviation cockpit is improved.

The invention discloses a three-dimensional Gaussian sputtering-based animatable character reconstruction method, medium and equipment, and the method comprises the following specific steps: randomly initializing a group of three-dimensional gauss in a standard space, generating a skin weight based on a skin weight field of a voxel grid, mapping the three-dimensional gauss from the standard space to an observation space, and carrying out the reconstruction of the three-dimensional gauss in the observation space; calculating a reconstruction loss based on the rendered image and the target image, and optimizing the reconstruction loss through back propagation to obtain an animatable character in a standard space; constructing a Gaussian mixture model based on a skeleton in the observation space, and estimating a skin weight and identifying and correcting abnormal gauss according to the Gaussian mixture model; and mapping the corrected three-dimensional Gaussian back to the standard space as the three-dimensional Gaussian in the soft constraint optimization standard space. And a self-supervised cycle is established, continuous optimization is performed, and accurate role reconstruction is realized. According to the method, three-dimensional Gaussian is adopted as basic representation, and the animatable three-dimensional character can be generated according to the monocular video without depending on a template.

The invention discloses an abnormal gait assessment method and system based on a monocular video, and a computing device. The method comprises the following steps: acquiring a human skeleton point sequence of the monocular video; performing data correction on the obtained human skeleton point sequence; and inputting the corrected human skeleton point sequence into a pre-trained space-time diagram convolutional network model, and obtaining an output result of the space-time diagram convolutional network model, the output result including an evaluation result of the gait anomaly condition of the subject in the monocular video. The method can effectively solve the problems of left and right leg confusion, easy shielding of key points and the like of skeleton point extraction in the monocular video, can enhance capture and understanding of gait space-time dynamic features, realizes accurate extraction of human gait space-time features, improves the accuracy and robustness of monocular video gait analysis, and improves the accuracy and robustness of the monocular video gait analysis. Effective gait analysis can be carried out by using a common monocular camera (such as a smart phone), the cost is lower, and the application scene is wider.

The embodiment of the invention discloses a three-dimensional model sequence generation method and related equipment. The related equipment can comprise a three-dimensional model sequence generation device, electronic equipment, a computer program product and a computer readable storage medium. According to the embodiment of the invention, feature extraction is carried out on video frames in a monocular video to obtain image features, an initial noise sequence corresponding to a three-dimensional model of a target object is generated, denoising is carried out on the initial noise sequence according to the image features to obtain a feature sequence set, and based on the frame positions of the video frames and the time distance between the video frames, the target object is obtained. Screening at least one reference feature block associated with the feature block from the feature sequence set, denoising the feature block according to the reference feature block to obtain a target feature sequence of the video frame, and generating a three-dimensional model sequence of the target object based on the target feature sequence; according to the scheme, the reference feature blocks can be screened to perform block-level cross-frame information interaction, so that the generation quality of the three-dimensional model sequence can be improved.

The invention relates to the technical field of computer vision, in particular to a sheep abnormal behavior detection method based on video analysis, which comprises the following steps of: processing a multi-view video or monocular video data stream, performing posture recognition on each sheep in continuous video frames, establishing a skeleton movement time sequence record of each sheep in continuous time, and recording the skeleton movement time sequence record of each sheep; and generating a three-dimensional bone key point sequence of the sheep. According to the method, posture recognition is carried out on the skeleton structure of each sheep in the continuous video frames, the key point positions of the four limbs are positioned, and the motion trails in the continuous time period are established in cooperation with three-dimensional skeleton reconstruction, so that animal behaviors obtain the expression basis in the space dimension and the time dimension at the same time. And in combination with the key structure vertical displacement, step length and ground contact duration of the left and right limbs in the support phase, a parameter set with symmetry evaluation ability is constructed, and the abnormal gait state caused by small motion difference can be quantified.

Disclosed herein are vision-based three-dimensional (3D) pose estimation system and method for ergonomic risk assessment. An example system may comprise a computing device configured to obtain a monocular video capturing motions of a subject performing at least one working activity for a selected duration of time, perform a whole-body two dimensional (2D) pose estimation based at least on extracted frames of the monocular video, perform a whole-body 3D pose estimation based at least on the whole-body 2D pose estimation, calculate joint angles based at least on the whole-body 3D pose estimation, determine a posture score for each identified joint in each frame of the monocular video, and determine an ergonomic risk level of each identified joint based at least upon the posture score.

The embodiment of the invention discloses a video processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a three-dimensional video frame in a three-dimensional video stream received by a first terminal, and enabling the first terminal to generate the three-dimensional video frame based on a depth estimation result of a corresponding two-dimensional video frame in a two-dimensional video stream and a foreground matting result; separating foreground content from background content of the three-dimensional video frame, and updating a foreground texture patch and a background texture patch; and forming a current video frame based on the foreground texture patch and the determined background display area, and displaying the current video frame, the background display area being determined from the background texture patch based on the current pose information of the execution device. By using the method, special acquisition equipment is not needed, the effect of changing the playing visual angle of the video content along with the adjustment of the pose of the execution equipment can be realized in the visual effect only by processing the traditional monocular video frame, and the problems of cost, storage and transmission caused by the existing realization of the playing of the three-dimensional video / six-degree-of-freedom video are avoided.

The invention provides a vision-based auxiliary-mark-free port machinery equipment movement distance detection method and system, and relates to the field of computers.The method comprises the steps that monocular cameras are installed on lifting appliances on the two sides of a gantry crane, the gantry crane is controlled to move along a preset track, and a calibration image sequence containing ground linear features is obtained; a radial distortion parameter of the camera is calculated, a nonlinear mapping model from a pixel coordinate system to a world coordinate system is established, a ground unshielded area is selected as a dynamic monitoring area, and a texture richness thermodynamic diagram is generated; after receiving a measurement instruction of a scheduling system, initializing an optical flow accumulator, loading a current feature point set, synchronously collecting monocular video streams, and distributing the monocular video streams to at least three preprocessing threads to execute differential image enhancement; optical flow calculation is executed on all preprocessing results in parallel, a multi-mode optical flow vector field is generated, and three-level optical flow screening is implemented. The precision and the sensitivity of small-range movement measurement of the gantry crane are improved through an optical flow method, and the influence of environmental factors on a measurement result is reduced.

The invention relates to the technical field of computer vision, and particularly provides a virtual digital human generation method and electronic equipment, and the method can comprise the steps: obtaining three-dimensional human body modeling data corresponding to each frame of image in a monocular video sequence of a target object; the three-dimensional human body modeling data comprises body posture data and identity offset data; processing the three-dimensional human body modeling data by using a pre-constructed to-be-trained virtual model to generate a virtual object image of each frame of image; the to-be-trained virtual model comprises a to-be-optimized three-plane feature body and a plurality of multi-layer neural network MLP models; optimizing the to-be-trained virtual model through the loss value between the virtual object image and each corresponding frame image, and obtaining a three-dimensional digital human model corresponding to the target object; the three-dimensional digital human model is used to generate a virtual digital human based on any drive data. According to the embodiment of the invention, the generation and rendering quality of the virtual digital human can be improved.

The invention discloses a multi-layer clothing reconstruction and simulation method in a monocular video, and belongs to the technical field of artificial intelligence, and the method comprises the steps: S1, collecting the monocular video, carrying out the image segmentation of clothing in the video, and extracting the corresponding image label information; s2, initializing a standard digital human model through a neural network; s3, training a neural network so as to fit a digital human in the video, wherein each monocular video needs to train a neural network independently; s4, extracting a complete garment from each monocular video; s5, aligning and correcting the plurality of clothes extracted from the monocular video based on physical simulation; the invention provides a multilayer clothing reconstruction and simulation method in a monocular video. The method comprises the following steps: extracting a three-dimensional grid by using a neural network; according to the method, image segmentation labels are reversely projected to the vertexes of the grids to form a complete garment model, the interlayer penetration problem possibly occurring in the reconstruction process is solved through physical constraints, and the reality sense and usability of a final result are ensured.