42 results about "Human Modelling" patented technology

Disclosed in the present invention is a dynamic human modeling method based on three-dimensional Gaussians. In the present method, a multi-view human video is given firstly, and in order to correct and transform the dynamic human appearance, a latent code and a set of blend weights are added to each Gaussian; and the latent code and a target pose are inputted to an MLP to correct the Gaussian in a canonical space, so as to capture an appearance change under the target pose. The corrected Gaussian is transformed into the target pose by means of the blend weights thereof in a linear blend skinning (LBS) manner. Finally, a realistic human image at a new viewing angle or a new pose can be rendered in real time by means of Gaussian splatting. Compared with an existing method based on an implicit neural radiance field (NeRF), the three-dimensional Gaussian representation of the present invention can better capture high-frequency details while achieving optimal rendering performance.

A high-quality digital human avatar is quickly reconstructed from a monocular video and is rendered in real time, the method is widely applied to various fields, and an existing method based on a neural radiation field is high in training and rendering cost. According to the dynamic human body three-dimensional modeling method based on the three-dimensional Gaussian sputtering technology, three-dimensional Gaussian can be expanded to a dynamic human scene, and rapid training and real-time rendering can be achieved. Specifically, the three-dimensional Gaussian of the human body is initialized in a standard space, and the three-dimensional Gaussian is converted from the standard space to a posture space by utilizing linear hybrid skin (LBS) according to posture parameters, so that fine details of the human body are learned at very low calculation cost; a posture parameter refining module and an LBS weight optimization module are designed to improve the problem of inaccurate detection of posture parameters in a real data set, and multi-resolution hash coding is introduced to accelerate the model training speed.

The invention discloses a virtual garment customization method based on 3D modeling, and relates to the technical field of garment customization, and the method comprises the steps: collecting image video data corresponding to the body data of a user, and carrying out the preprocessing of the collected data, and obtaining the basic three-dimensional point cloud data of a human body; carrying out primary human body modeling according to the human body basic three-dimensional point cloud data, then adjusting the model shape based on key parameters input by a user, and carrying out body type matching to realize parameterized human body modeling; secondly, modeling the clothes, predicting clothes deformation by adopting machine learning, simulating cloth dynamics by adopting a physical engine, performing collaborative tracking and matching on clothes and human body dynamic characteristics according to a clothes template selected by a user, and realizing virtual try-on through an AR (Augmented Reality) technology; the customized order confirmation information of the user is obtained, the 2D garment pattern graph is generated and sent to the customized production server, it can be ensured that the user can obtain a high-quality 3D modeling result under different devices and environments, and the user experience can be enhanced through AR try-on.

The invention discloses a figure modeling and fitness action teaching method and system based on non-contact action capture, and the method carries out the real-time monitoring of a human body posture through a millimeter wave radar, carries out the 3D human body posture reconstruction and analysis through combining with a human body posture frame of Google image processing, and achieves the high-precision action capture. The method can be used for human body modeling, body building action scoring and action guidance, action evaluation and real-time feedback in body building teaching can be effectively supported, the millimeter wave radar is adopted for human body posture capture, external markers or wearable equipment are not needed, constraint on user activities is avoided, and use convenience and comfort are improved. The 3D posture of the human body can be accurately reconstructed in combination with a deep learning neural network, and real-time processing and analysis are carried out. The method not only can realize accurate motion capture, but also can be combined with fitness motion scoring and guidance, is widely applied to the fields of virtual fitness, exercise rehabilitation and the like, and has great application potential.

The invention discloses a digital human rendering method based on Gaussian splashing and multi-scale characteristic field distillation, and belongs to the field of three-dimensional human body digital reconstruction. According to the method, feature extraction is carried out through a Vision Transformer encoder, based on an SMPL model, through a cross-modal parameter estimation module and dynamic human body modeling of three-dimensional Gaussian splashing and semantic feature rendering, three-dimensional Gaussian is projected to a two-dimensional image plane to calculate a covariance matrix and color mixing, and after a rendered color image and an initial feature field image are output, a three-dimensional image is obtained. A student feature map is obtained through a convolution acceleration module, a teacher feature map is obtained after feature extraction is carried out through a two-dimensional basic model, the constructed model is trained, and optimization training is completed through comprehensive total loss function calculation; according to the method, the problems of fuzzy semantics, detail missing, low rendering efficiency and inaccurate human body-scene separation in the existing method are effectively solved, so that more efficient, fine and robust monocular or multi-view human body three-dimensional reconstruction is realized.

The invention relates to the technical field of virtual reality, and discloses a personalized clothing virtual try-on method and system, a terminal and a storage medium. The method comprises the following steps: S1, acquiring a three-dimensional body model of a user, and constructing a high-precision three-dimensional body model by analyzing a multi-angle picture uploaded by the user and utilizing a deep learning technology; s2, according to the three-dimensional body model and personal attributes of the user, performing personalized adjustment on the virtual clothes, including color, size and material; and S3, superposing the adjusted virtual costume on the three-dimensional body model of the user by using an augmented reality technology. Through combination of multi-angle image analysis and a deep learning three-dimensional reconstruction technology, a high-precision human body modeling effect is realized. The purpose that the user can quickly generate the three-dimensional body model only through the daily photos is achieved. Compared with a modeling method depending on a depth camera or somatosensory equipment in the prior art, the user operation process is remarkably simplified, and the problems that a traditional scheme is high in threshold and limited in application scene are solved.

The invention relates to the technical field of human body three-dimensional modeling, in particular to a human body measurement parameter set-based three-dimensional body surface shape generation method, which comprises the following steps of: S1, establishing a parameterized body surface model T; s2, determining a target anthropometric parameter group # imgabs0 #; S3, determining a transformation relation F based on the parameterized body surface model T and the target anthropometric parameter group # imgabs1 #; S4, constructing and verifying an optimized physical sign parameter beta *; and S5, substituting the optimized physical sign parameter beta * into any pose parameter theta, and changing the body surface geometric space pose of the optimized physical sign parameter beta *. By means of the method, finite element modeling of the human body in any form can be achieved, a statistical regression or condition generation model from one-dimensional measurement parameters to three-dimensional human body shapes does not need to be established, and various downstream human body modeling tasks oriented to sign diversity are achieved.

The invention relates to the technical field of computer vision, and particularly provides a virtual digital human generation method and electronic equipment, and the method can comprise the steps: obtaining three-dimensional human body modeling data corresponding to each frame of image in a monocular video sequence of a target object; the three-dimensional human body modeling data comprises body posture data and identity offset data; processing the three-dimensional human body modeling data by using a pre-constructed to-be-trained virtual model to generate a virtual object image of each frame of image; the to-be-trained virtual model comprises a to-be-optimized three-plane feature body and a plurality of multi-layer neural network MLP models; optimizing the to-be-trained virtual model through the loss value between the virtual object image and each corresponding frame image, and obtaining a three-dimensional digital human model corresponding to the target object; the three-dimensional digital human model is used to generate a virtual digital human based on any drive data. According to the embodiment of the invention, the generation and rendering quality of the virtual digital human can be improved.

The application discloses a kind of implicit surface human modeling methods based on weight mask strategy, first based on 2D human photo generation implicit expression, then the implicit expression of the space point generated is input into renderer, render human prediction image and predicted mask.Optimize the surface of human special part, according to loss function supervision and train model;Finally, according to the model obtained, the generation of human surface is carried out.The method of the application introduces adjustable weight mask in different areas of the model, so that the details of each part of the human body are more accurately described, especially in areas with large curvature changes, such as the face, joints, etc., can better capture the small contour and shape changes.Through the weight mask strategy, more detailed optimization can be performed on the complex areas of the model, thereby avoiding distortion, distortion or stretching phenomenon when the curved surface is converted.

The present invention discloses a virtual fitting method, including: making a three-dimensional model of clothing according to a two-dimensional image of clothing; fitting the three-dimensional clothing model to a three-dimensional standard human body model; obtaining a two-dimensional image of a target human body; inputting the obtained three-dimensional human body parameters into the standard human body model for fitting; matching the clothing three-dimensional model to the three-dimensional human body model; making the three-dimensional human body model and the target human body posture basically consistent. The present invention provides a method for obtaining accurate three-dimensional human body model parameters by analyzing a full-body photo of a human body through a deep neural network, and only an ordinary photo is needed to quickly model a human body; at the same time, the three-dimensional clothing model is obtained by processing a two-dimensional clothing picture in advance, and the user does not need to participate in these behind-the-scenes work, but only needs to select the clothing style that he wants to virtually fit, and the system will automatically match the corresponding clothing model, and maintain the authenticity and restoration through a series of methods.

The present invention discloses a three-dimensional human body modeling and motion acquisition method based on multi-view synchronous shooting. A tracking video of a person shot synchronously from multiple perspectives is obtained, and two-dimensional human body key points and two-dimensional hand key points are extracted from each frame; then, the arm parameters of the human body parametric model are optimized using the two-dimensional human body key points. Afterwards, the parameters of the hand model are optimized based on the two-dimensional hand key points; according to the wrist position, the arm position is corrected to obtain the arm parameters of the human body parametric model. By arranging the hand model point cloud and the human body parametric model point cloud, an index of points and triangular facets is established to obtain a fusion model, and the parameters of the human body parametric model and the hand model are imported into the fusion model to obtain the result of human body reconstruction. The present invention can use multi-view images to perform human body modeling and motion acquisition including hands, and does not require a sophisticated multi-camera configuration and motion capture clothing.

The invention provides a human body posture data set construction method based on a large model. A large language model is combined with a fine-grained attribute database to generate motion description, then the motion description is converted into 3D human skeleton motion data through a MoMask model, and quality control is carried out. And then view control and gait speed control are performed on the motion data to increase data diversity. And then radar data simulation is carried out, including human body modeling, RCS estimation, radar signal generation and processing, and micro-Doppler spectrogram generation. Meanwhile, a VQ-VAE model is adopted to generate background noise and add the background noise, shielding is simulated, and the data trueness is enhanced. And finally, constructing a data set containing a plurality of activities. The data set generated by the method has semantic and physical diversity, a real scene can be simulated with high fidelity, the performance of a human body posture recognition model is effectively improved, and powerful support is provided for application of related technologies in multiple fields.

The invention relates to the technical field of human body modeling, in particular to a cross-domain human body unified modeling method and device, equipment and a storage medium. The method comprises the following steps: representing the relative similarity between any two skeleton postures of a human body by using a relative posture similarity space, extracting the relative similarity between an unsampled skeleton posture and an anchor prompt from the relative posture similarity space, screening out a target posture from the unsampled skeleton postures on the basis of the relative similarity, and extracting the target posture from the unsampled skeleton postures. And updating the anchor prompt set by using the target attitude to obtain a final anchor prompt set. And finally, the anchor prompts and the soft prompts in the final anchor prompt set cooperate with each other to complete the reasoning tasks of the model, and the soft prompts are parameters generated when the model is trained based on each reasoning task, so that when different reasoning tasks are processed, only the soft prompts corresponding to the reasoning tasks need to be replaced, and the reasoning tasks can be processed. Therefore, the reasoning task can be completed in cooperation with anchor prompt without replacing the model. Therefore, cross-task unified modeling can be realized.

The application discloses a 3D human body modeling method and device based on contrast learning and a medium. The method comprises the following steps: acquiring 2D image information corresponding to a user and body state information corresponding to the user; extracting original features corresponding to the 2D image information through a shared encoder; performing mask prediction and determining clothing coverage rate according to the original features; performing feature fusion according to the original features, shape prior features corresponding to the body state information and the clothing coverage rate to obtain fused features; performing adaptive feature purification on the fused features according to the clothing coverage rate to obtain pure features; and performing human body parameter prediction according to the pure features.

The present invention relates to the technical field of human body modeling, and in particular to a cross-domain unified human body modeling method, apparatus, equipment and storage medium. The present invention uses a relative posture similarity space to represent the relative similarity between any two skeleton postures of the human body, and then extracts the relative similarity between the unsampled skeleton posture and the anchor prompt from the relative posture similarity space. Based on the relative similarity, the target posture is screened out from the unsampled skeleton posture, and the anchor prompt set is updated with the target posture to obtain a final set of anchor prompts. Finally, the anchor prompts and soft prompts in the final set of anchor prompts cooperate with each other to complete the reasoning task of the model. Since the soft prompts are based on the parameters generated when the model is trained for each reasoning task, when processing different reasoning tasks, it is only necessary to replace the soft prompts corresponding to the reasoning task to cooperate with the anchor prompts to complete the reasoning task without replacing the model. Therefore, the present invention can achieve unified modeling across tasks.

The invention discloses a convenient three-dimensional human body reconstruction method based on a monocular video, and the method comprises the steps: constructing a parameterized human body model low-dimensional shape space, and obtaining the approximate representation of a linear model. And extracting a multi-view key frame from the input human body rotation video. And extracting a feature vector of each view key frame by using an improved ResNet-50 network fused with an attention mechanism. And defining the multi-view feature vectors as graph nodes, and constructing a graph structure. And performing cross-view depth feature fusion through the graph convolutional network to obtain global features. And mapping the global features to a parameterized human body model low-dimensional shape space. And reconstructing a complete three-dimensional human body grid by using linear model approximate representation. The method has the advantages that multi-view feature extraction and a graph structure fusion mechanism are combined, the end-to-end process from consumption-level video input to high-precision three-dimensional human body reconstruction is achieved, certain generalization ability is achieved in a real scene, and a low-cost and high-precision three-dimensional human body modeling solution can be provided for virtual fitting, digital twinning and other applications.

The present invention relates to the field of computer vision technology, and provides a human body depth estimation method, device, electronic device, and storage medium. The method includes: obtaining human body images of a person to be estimated from multiple camera perspectives, and accordingly performing three-dimensional human body modeling to obtain three-dimensional human body models corresponding to the human body images from multiple camera perspectives; performing projection based on the three-dimensional human body models, and compressing and outputting the depth maps of the person to be estimated obtained by the projection from multiple camera perspectives, which solves the problem that the depth cannot be restored in the current weakly textured regions, resulting in holes in the depth map, and can effectively solve the depth information of the weakly textured regions, thereby ensuring the accuracy and integrity of human body depth estimation. By compressing and outputting the depth maps, not only the number of output files is reduced, the transmission speed is increased, the transmission space occupancy is reduced, but also the integrity and orderliness of the information are improved, realizing the effective simplification of the output results and facilitating access.

The invention discloses a 3D human body modeling method and device based on comparative learning and a medium. The method comprises the steps that 2D image information corresponding to a user and posture information corresponding to the user are acquired; extracting an original feature corresponding to the 2D image information through a shared encoder; according to the original features, mask prediction is carried out, and the clothing coverage rate is determined; performing feature fusion according to the original feature, a shape prior feature corresponding to the posture information and the clothing coverage rate to obtain a fused feature; according to the clothing coverage rate, performing adaptive feature purification on the fused feature to obtain a pure feature; and performing human body parameter prediction according to the pure features.

The invention relates to the technical field of three-dimensional human body modeling, and particularly discloses a modeling and rendering method and device based on layered three-dimensional Gaussian representation and a medium, and the method comprises the steps: obtaining an image sequence of a target, and initializing a three-dimensional Gaussian point in a standard space based on an SMPL human body template vertex; taking the initialized three-dimensional Gaussian point as the first layer of the innermost side, constructing a hierarchical Gaussian structure from coarse to fine, and establishing conditional dependence among the layers through a father-child node connection mode; setting a hierarchical motion condition, and predicting rigid body and non-rigid body deformation transformation parameters of each Gaussian point by using a rigid body transformation network and a non-rigid body transformation network; and mapping the layered Gaussian points to an observation space in a unified manner according to predicted transformation parameters, and performing image synthesis by using a three-dimensional Gaussian rendering technology. Compared with the prior art, the method can naturally adapt to and express deformation characteristics of different levels in human motion, and realizes high-quality reconstruction and real-time rendering of a complex dynamic scene.

The invention relates to the technical field of virtual fitting, in particular to an intelligent virtual fitting method based on three-dimensional human body modeling. The method comprises the following steps: accurately capturing various sizes and postures of a human body by collecting multi-angle image data of the human body and constructing a three-dimensional model of a human body structure, and providing accurate basic data for virtual fitting; fitting clothes are obtained, physical parameters of the clothes are collected, and therefore the appearance effect of the clothes is analyzed; performing posture simulation by using the human body three-dimensional model, and performing interactive dynamic response analysis of the human body and the clothes by combining physical characteristics of the clothes to obtain dynamic effect data of the clothes under different postures; the virtual fitting effect is further optimized by calculating the matching degree of the clothes and the human body; according to the method, intelligent virtual fitting is optimized, so that the virtual fitting matching degree is more perfect.

The invention discloses an intelligent fitting recommendation system based on 3D human body modeling and clothing parameter adaptation, and relates to the technical field of intelligent fitting recommendation. The system generates a high-precision dynamic body surface grid through a multi-modal data acquisition module; a human body-costume double-layer decoupling digital model construction module is utilized to realize efficient and realistic costume dynamic deformation simulation; through a dynamic pressure simulation and evaluation module, in combination with an improved Hertz contact theory and a fabric mechanical model, carrying out accurate simulation and multi-dimensional quantitative evaluation on dynamic contact pressure of clothes and a human body; and finally, the scene intelligent recommendation module provides personalized clothing recommendation according to the scene selected by the user and the dynamic evaluation result. According to the method, the problem that traditional virtual fitting lacks dynamic comfort evaluation is solved, the user experience is remarkably improved, the return rate of merchants is reduced, and the method has important market application value.

The invention discloses a shopping mall virtual fitting room implementation method and system based on the AR technology, and belongs to the technical field of e-commerce and computer vision, and the method comprises the steps: carrying out the real-time high-precision three-dimensional human body modeling based on a common camera; performing intelligent cloth physical simulation fused with a machine learning algorithm; multi-terminal cross-platform consistency rendering is supported; and carrying out personalized clothing recommendation in combination with user body type data. The virtual try-on effect of the user can be truly presented, multi-device terminal access is supported, the shopping experience is remarkably improved, and the refunding and changing rate is reduced.

The invention discloses an implicit surface human body modeling method based on a weight mask strategy, and the method comprises the steps: generating an implicit expression based on a 2D human body picture, inputting the generated implicit expression of a spatial point into a renderer, and rendering a human body prediction image and a prediction mask. Optimizing the surface of the special part of the human body, and supervising and training the model according to a loss function; and finally, generating a human body surface according to the obtained model. According to the method, adjustable weight masks are introduced into different areas of the model, so that details of all parts of the human body are described more accurately, and particularly, tiny contour and shape changes can be better captured in areas such as the face and joints with large curvature changes. Through the weight mask strategy, the complex area of the model can be optimized more meticulously, so that the distortion, distortion or stretching phenomenon during curved surface conversion is avoided.

The invention relates to the technical field of costume design, and discloses an intelligent interactive costume design system and method based on a data model. The system comprises an interaction module, a human body modeling module, a demand intelligent analysis module, an intelligent design generation module, a fabric matching module, a virtual fitting module and a scheme optimization module. The interaction module obtains PLY format human body data of a user and costume design requirements and receives modification feedback of the user, and the human body modeling module generates a 3D human body model, human body size parameters and posture feature tags. The demand intelligent analysis module converts a costume design demand into a structured demand document containing a style tag and a version constraint parameter in combination with a posture feature tag, the intelligent design generation module outputs an optimal costume design scheme through a VAE generation model, and the fabric matching module matches candidate fabrics based on the optimal costume design scheme. The virtual fitting module realizes dynamic fitting simulation, and the scheme optimization module iteratively optimizes the optimal costume design scheme to meet user requirements. The personalized costume design efficiency of the user is improved.

The present invention relates to the technical field of human body modeling, and provides a method, a system, a device and a storage medium for constructing a human body model. The method includes: when the user to be modeled enters the first preset position, invoking the first image sensor array to collect the first image acquisition result; performing clustering analysis on the user to be modeled to obtain the clustering result of the modeling area; traversing the clustering result of the modeling area and invoking the first image acquisition result to obtain the outer frame modeling result; reminding the user to be modeled to enter the second preset position, invoking the second image sensor array to collect the second image acquisition result; traversing the clustering result of the modeling area and invoking the second image acquisition result to obtain the inner skeleton modeling result; constructing a human body static simulation model according to the outer frame modeling result and the inner skeleton modeling result; uploading the basic parameters of the user, and initializing the human body static simulation model to generate a human body dynamic simulation model. The technical effect of improving the fitness between the human body model and the requirements of the metaverse human body model is achieved.

The invention discloses a three-dimensional human body modeling method and system based on binocular vision and a laser radar, solves the problem of poor reconstruction effect of a three-dimensional human body in a dynamic motion scene, and belongs to the technical field of computer vision, intelligent perception and three-dimensional modeling. A binocular camera and a laser radar are used for synchronously collecting a human body, and a binocular image and a laser radar point cloud are obtained; processing and fusing the obtained binocular image and the laser radar point cloud to generate a human body initial three-dimensional model; tracking and updating the parameterized human body model in real time, and outputting a dynamic three-dimensional human body model; in the real-time tracking and updating process, a parameterized human body model is constructed according to the human body initial three-dimensional model, the human body volume, the image contour and the point cloud distance are used as constraint terms, the human body elastic deformation is used as a physical regularization term to construct an optimized energy function, and the deformation of the parameterized human body model in motion is optimized by using the optimized energy function. According to the method, the geometric fidelity and the detail reduction degree of the three-dimensional model are effectively improved.

The invention discloses a human body posture estimation method and device, equipment and a storage medium, and belongs to the technical field of image processing. Training a three-dimensional human body modeling model by adopting the first data set; performing human body posture estimation based on a three-dimensional human body model output by the trained three-dimensional human body modeling model; wherein the three-dimensional human body modeling model comprises a ResNet feature extractor, an HMR regression device and an SMPL model which are connected in sequence, the input of the HMR regression device comprises image features of a first image and bounding box geometric features of a second human body, the output of the HMR regression device comprises joint point features, and the bounding box geometric features are used for converting the joint point features to a first coordinate system; the projection loss of the joint point features is calculated on a first coordinate system, and the first coordinate system is the coordinate system of the original camera. According to the method, human body posture estimation can be accurately carried out on the image with slight posture change.

The present application relates to the technical field of virtual fitting, and more particularly to an intelligent virtual fitting method based on three-dimensional human modeling. The method comprises the following steps: collecting multi-angle image data of a human body and constructing a three-dimensional model of the human body structure to accurately capture various dimensions and postures of the human body, thereby providing accurate basic data for virtual fitting; obtaining a garment and collecting garment physical parameters to analyze the appearance effect of the garment; using the three-dimensional model of the human body to simulate postures and combining the physical characteristics of the garment to analyze the interactive dynamic response of the human body and the garment, thereby obtaining dynamic effect data of the garment under different postures; and further optimizing the virtual fitting effect by calculating the matching degree of the garment and the human body; the present application optimizes intelligent virtual fitting to achieve more perfect virtual fitting matching degree.

A human posture estimation method, device, equipment and storage medium are disclosed, and belong to the technical field of image processing. The method comprises the following steps: acquiring a first data set; training a three-dimensional human modeling model by using the first data set; and performing human posture estimation based on a three-dimensional human model output by the trained three-dimensional human modeling model. The three-dimensional human modeling model comprises a ResNet feature extractor, an HMR regressor and an SMPL model connected in sequence. The input of the HMR regressor comprises image features of a first image and boundary box geometric features of a second human body. The output of the HMR regressor comprises joint point features. The boundary box geometric features are used to convert the joint point features to a first coordinate system, so as to calculate a projection loss of the joint point features in the first coordinate system. The first coordinate system is a coordinate system of an original camera. The method can accurately realize human posture estimation on an image with slight posture changes.

The application discloses a three-dimensional Gaussian dynamic human body modeling method based on compressed three planes, and comprises the following steps: obtaining a camera pose parameter sequence, human body SMPL shape information and a posture information sequence from a single-view video sequence, and obtaining an initialized three-dimensional Gaussian point cloud; obtaining high-dimensional features according to the three-dimensional Gaussian point cloud, and predicting information of the three-dimensional Gaussian in a standard space; distilling skin weights of each vertex of a standard SMPL model and smoothing; generating high-dimensional position coding of the three-dimensional Gaussian by splicing, combining SMPL posture information and three-plane high-dimensional features, predicting basic attribute transformation values, and correcting Gaussian attributes under different human body postures; calculating a transformation matrix by using the corrected skin weights and the SMPL posture information, and obtaining a transformed three-dimensional Gaussian point cloud by transformation; rendering the transformed three-dimensional Gaussian point cloud, combining an image loss function and attribute constraints of neighbor Gaussians, and jointly optimizing to obtain a drivable human body three-dimensional Gaussian model. The application also discloses a system for realizing the above method, and has wide application scenarios.