44 results about "3D pose estimation" patented technology

The invention discloses a dynamic Gaussian digital human image rendering method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a target multi-view image frame from a multi-view RGB video image sequence according to a preset attitude, and constructing a deformable parameterized model according to the target multi-view image frame; constructing an attitude space driving attitude corresponding to the multi-view RGB video image sequence based on a three-dimensional attitude estimation technology, and generating a two-dimensional position map according to the attitude space driving attitude and the deformable parameterized model; performing three-dimensional Gaussian binding on the deformable parameterized model to obtain a local attribute of the three-dimensional Gaussian; training is carried out according to the two-dimensional position map, and a target StyleUNet neural network is obtained; and predicting the new attitude through the target StyleUNet neural network to obtain a dynamic Gaussian digital human image. In this way, the high-fidelity drivable high-frequency detail digital human image can be automatically rendered.

Provided are a model training method and apparatus, a three-dimensional pose estimation method and apparatus, and an electronic device, relating to the field of computer vision. The method comprises: acquiring at least one training sample, and inputting each training sample into an initial three-dimensional pose estimation model for training to obtain a trained three-dimensional pose estimation model. Each round of training comprises: inputting a sample image of the training sample into a current three-dimensional pose estimation model to obtain the three-dimensional Gaussian mixture representation of each key point on a preset three-dimensional coordinate system; for each key point, obtaining the two-dimensional Gaussian mixture representations of the key point on three mutually perpendicular two-dimensional planes in the preset three-dimensional coordinate system; for each key point, determining the loss value of the key point on each two-dimensional plane; and adjusting model parameters of the current three-dimensional pose estimation model on the basis of the loss value of each key point on each two-dimensional plane. The embodiments of the present application effectively improve the accuracy of three-dimensional pose estimation.

The invention discloses a three-dimensional attitude estimation method and device, a storage medium and electronic equipment, and relates to the technical field of three-dimensional attitude estimation, and the method comprises the steps: carrying out the joint estimation of a multi-view image of a target object, and obtaining a joint feature map and a joint heat map of each view; performing cross-view-angle alignment analysis on the multi-view-angle joint heat map to obtain an initial three-dimensional joint coordinate of each joint point; carrying out deformable cross-attention feature aggregation on the multi-view joint feature map to obtain multi-view fusion features of the joints; and performing three-dimensional attitude prediction based on the initial three-dimensional joint coordinates of the joints and the multi-view fusion features to obtain target three-dimensional joint coordinates of the joints. According to the invention, the estimation accuracy of the three-dimensional coordinates of the joint points during three-dimensional attitude estimation can be effectively improved.

Disclosed herein are vision-based three-dimensional (3D) pose estimation system and method for ergonomic risk assessment. An example system may comprise a computing device configured to obtain a monocular video capturing motions of a subject performing at least one working activity for a selected duration of time, perform a whole-body two dimensional (2D) pose estimation based at least on extracted frames of the monocular video, perform a whole-body 3D pose estimation based at least on the whole-body 2D pose estimation, calculate joint angles based at least on the whole-body 3D pose estimation, determine a posture score for each identified joint in each frame of the monocular video, and determine an ergonomic risk level of each identified joint based at least upon the posture score.

The present teaching relates to estimating 3D pose of an ultrasound probe. An ultrasound image is acquired by an ultrasound probe deployed at a three-dimensional (3D) probe pose during a medical procedure directed to a target organ. A mask for each two-dimensional (2D) anatomical structure is identified and a corresponding label is estimated from the ultrasound image to generate an ASM / label pair. If a sequence of prior 3D probe poses does not exist, an ASM / label representation for the ultrasound image is generated based on the ASM / label pairs from the ultrasound image and used to estimate the 3D pose of the probe via an ASM-pose mapping model. If the sequence of prior 3D probe poses exists, the 3D probe pose is predicted based on the sequence of prior 3D probe poses and an ASM / label representation is generated based on a virtual ultrasound image created based on the predicted 3D probe pose.

The invention discloses a 3D human body posture estimation method based on a double-flow space-time diagram network. Collecting a plurality of human body posture maps and marking three-dimensional coordinates of key points, and further constructing a training set of three-dimensional human body posture estimation; establishing a three-dimensional human body posture detection network, and inputting the training set into the three-dimensional human body posture detection network for training to obtain a trained three-dimensional human body posture detection network; and obtaining a to-be-detected human body posture image, and inputting the to-be-detected human body posture image into the trained three-dimensional human body posture detection network for posture detection to obtain a three-dimensional posture estimation result of the human body. The method has a time form and a space form, the features of Transform and GCN can be aggregated in a simple and effective mode in the space-time dimension, and comprehensive modeling of local and global features of the human skeleton is achieved.

The application discloses a gesture recognition and posture estimation joint modeling method, application method and device, relates to the technical field of perception interaction, and the modeling method comprises the following steps: performing two-dimensional posture estimation on a historical gesture image region to obtain a two-dimensional key point sequence, and inputting the two-dimensional key point sequence and an initial gesture type label into a three-dimensional posture estimation network to obtain an initial three-dimensional key point sequence; based on a video frame sequence, a first predicted gesture type label is acquired; based on the initial three-dimensional key point sequence, a second predicted gesture type label is acquired; the two predicted gesture type labels are fused to obtain a next round gesture type label; the next round gesture type label is taken as the initial gesture type label, the initial three-dimensional key point sequence is taken as the two-dimensional key point sequence, and the initial three-dimensional key point sequence acquisition step is returned; when a set iteration number is reached or a convergence condition is met, a joint modeling model is obtained. The application can realize more efficient and more accurate gesture recognition and three-dimensional posture estimation.

The invention relates to a multi-view real-time human limb driving method and device and electronic equipment. The method comprises the following steps: acquiring a human body action multi-view image; performing two-dimensional human body posture estimation on the multi-view-angle image to obtain thermodynamic diagram coordinates and feature diagrams of two-dimensional human body key points of each view angle; obtaining the view angle weight of the view angle image where each key point is located by adopting an attention mechanism according to the feature map of each view angle; according to thermodynamic diagram coordinates and view angle weights of the two-dimensional human body key points, triangularizing the two-dimensional human body key points in each view angle image by using a deep learning model to obtain three-dimensional human body key point space coordinates; and driving the virtual human based on the three-dimensional human body key point space coordinates. The weight of each view angle can be dynamically optimized, view angle noise is suppressed, the accuracy of three-dimensional attitude estimation is effectively improved, and a real-time driving effect can be achieved.

The present application belongs to the field of 3D human posture estimation, and particularly relates to a multi-person 3D posture estimation method and system based on explicit limb representation, comprising: acquiring a multi-person scene image, inputting the multi-person scene image into a trained multi-person 3D posture estimation model to obtain a key point heat map, a limb orientation vector field, a limb relative depth map and a root key point absolute depth map, and post-processing the key point heat map, the limb orientation vector field, the limb relative depth map and the root key point absolute depth map to obtain an estimated multi-person 3D posture; the present application encodes the limb information of a person into a two-dimensional limb orientation vector field and a limb relative depth map to ensure that the matching between candidate key points can directly utilize the most intuitive limb information, so as to improve the phenomenon of incorrect matching of candidate key points of different human bodies and improve the quality of the matched posture.

The application discloses a kind of single-view animal 3D posture estimation method and system, comprising: backbone network receives 2D image data, the backbone network is trained, obtains trained backbone network;The output of backbone network is connected with 3D posture estimation network, constructs weakly supervised learning module, uses 3D labeled data to train weakly supervised learning module, forms initial model, uses initial model to predict 2D unlabeled data, obtains pseudo-labeled data set, and real 3D labeled data is merged to form training set, and weakly supervised learning module is trained;Real-time acquisition single-view animal image is input into trained weakly supervised learning module and carries out single-view animal 3D posture estimation;The application has the advantages that based on a large number of 2D unlabeled data, a small amount of 3D labeled data, under the premise of small dependence on labeled data, simultaneously not depending on artificial experience, effectively improve the performance upper limit on 3D posture estimation accuracy.

This invention relates to the field of pose estimation technology, specifically to a multi-view, multi-person 3D pose estimation method, apparatus, and device. The method includes: acquiring image information and depth information of the target using multi-view acquisition technology; segmenting the person in a single view based on the image information and extracting 2D semantic features; reconstructing the point cloud based on the depth information to obtain a 3D point cloud in a preset world coordinate system; segmenting the point cloud for different human figures based on the 3D point cloud and calculating the point cloud boundaries as human body boundaries; mapping the 2D semantic features to the 3D point cloud to obtain 3D features; and classifying different human figures and constructing 3D poses based on the 3D features and human body boundaries. This technical solution can segment the point cloud for different human figures based on the 3D point cloud, accurately define the joint positions of different figures, and achieve differentiation and matching of different figures. Furthermore, based on discrete point cloud data, it has better flexibility and fault tolerance.

Techniques related to 3D pose estimation from a 2D input image are discussed. Such techniques include incrementally adjusting an initial 3D pose generated by applying a lifting network to a detected 2D pose in the 2D input image by projecting each current 3D pose estimate to a 2D pose projection, applying a residual regressor to features based on the 2D pose projection and the detected 2D pose, and combining a 3D pose increment from the residual regressor to the current 3D pose estimate.

Techniques related to 3D pose estimation from a 2D input image are discussed. Such techniques include incrementally adjusting an initial 3D pose generated by applying a lifting network to a detected 2D pose in the 2D input image by projecting each current 3D pose estimate to a 2D pose projection, applying a residual regressor to features based on the 2D pose projection and the detected 2D pose, and combining a 3D pose increment from the residual regressor to the current 3D pose estimate.

This invention discloses a human pose estimation method based on adaptive multi-view feature fusion. Specifically, it involves: acquiring synchronized multi-view images and preprocessing them to form a dataset, which is then divided into a training set, a validation set, and a test set; designing a multi-view feature fusion network, including a 2D pose estimation backbone network, an appearance embedding network, a geometric embedding network, a weight prediction and learning network, and a multi-view feature weighted fusion network; training and validating the multi-view feature fusion network using the training and validation sets; inputting the test set into the trained multi-view feature fusion network for testing; and outputting a pose estimation image. This method can enhance the features of occluded views by borrowing features from visible views, while adaptively learning view weights to prevent low-quality views from contaminating global features, significantly improving the accuracy and robustness of 3D pose estimation.

The invention discloses a 3D human body posture estimation method and system based on meta-learning, and belongs to the technical field of computer vision and deep learning, and the method comprises the steps: constructing a dual-task framework of a 3D human body posture estimation main task and a self-supervised 2D posture reordering auxiliary task, and combining a mechanism of internal circulation fine tuning and external circulation optimization of meta-learning, and rapid adaptation of the model to new target domain data in a test stage is realized. According to the method, the problems of slow convergence and poor generalization ability of a traditional 3D attitude estimation model in a cross-domain scene are solved, and the 3D attitude estimation precision can be improved only by a small amount of iteration under the condition of no target domain labeling through the combination of the self-supervised auxiliary task and meta-learning.

A system and method for obtaining the 3D pose of an object using 2D images from a 2D camera and a learning-based neural network. The neural network extracts multiple features from the 2D image of the object and generates a generated heatmap for each of the extracted features, which uses color representation to identify the probability of the location of feature points on the object. The method provides a feature point image including each feature point from the respective heatmaps on the 2D image, and estimates the 3D pose of the object by comparing the feature point image with a 3D virtual CAD model of the object.

The invention discloses a zero-sample unknown object 3D pose estimation method based on two-stage RGB-D fusion. Comprising the following steps: giving a 3D model of an unknown object, and rendering the 3D model of the unknown object under different viewpoints to generate a series of RGB-D template images; based on a feature embedded network PoseFusion, texture features and geometric features are extracted from an RGB template image and a Depth template image through a modified ResNet50 network, and then a series of template scene features are generated through two-stage fusion of the texture features and the geometric features. Similarly, a real RGB-D image obtained by shooting an unknown object is also embedded into a real scene feature through PoseFusion. And finally, matching the real scene features with the series of template scene features by calculating occlusion local similarity, so that the 3D pose of the unknown object is determined as the matched template 3D pose.

This invention provides a multi-view 3D pose reconstruction method, comprising: constructing a virtual simulation environment; setting the parameters of the virtual simulation environment using a domain randomization strategy; acquiring multi-view images and extracting 2D joint coordinates and confidence scores; normalizing the 2D joint coordinates based on the parameters of each virtual camera to obtain normalized coordinates; concatenating the normalized coordinates with the confidence scores and inputting them into a shared-weight attention encoder; learning the joint topology within a single view through self-attention and outputting view features; concatenating the rotation matrix and translation vector of each view and mapping them to an explicit extrinsic semantic vector via a multilayer perceptron; inputting the view features and extrinsic semantic vector together into a global attention module; performing geometric consistency evaluation through an attention mechanism; and outputting 3D joint coordinates. This method solves the problems of weak cross-view generalization ability, large interference from differences in camera intrinsic parameters, and insufficient robustness against occlusion in existing technologies for multi-view 3D pose estimation.

Methods and systems are disclosed for estimating 3D poses and dimensions of vehicle occupants using neural networks. Images of the interior of a vehicle are captured and monocular depth maps are generated. Both the depth maps and the images are fed into a 3D pose estimation network as a combined four-channel RGBD input. The neural network can include an occlusion-aware masking layer that generates an occlusion score for keypoints associated with an occupant. The occlusion score helps the network adjust the weighting of the depth information such that keypoints with higher occlusion scores (indicating that they are more likely to be hidden or partially occluded) receive lower weights. A scaling function that estimates a scale factor integrates the depth information with the occlusion-aware mask to estimate the absolute depth position of the keypoints.

The present disclosure relates to the technical field of computers, and relates to a machine learning model training apparatus and method, and a three-dimensional pose estimation apparatus. The training apparatus comprises at least one processor, the at least one processor being configured to: acquire a plurality of two-dimensional images and a machine learning model to be trained, the plurality of two-dimensional images comprising a target, and the machine learning model comprising a first three-dimensional pose estimation module and a second three-dimensional pose estimation module; according to any one two-dimensional image, use the first three-dimensional pose estimation module to estimate a first three-dimensional pose of the target; according to at least two two-dimensional images, use the second three-dimensional pose estimation module to estimate a second three-dimensional pose of the target, the viewing angles of the at least two two-dimensional images being different from each other; and training at least one of the first three-dimensional pose estimation module or the second three-dimensional pose estimation module according to the difference between the first three-dimensional pose and the second three-dimensional pose.

The present disclosure provides a pose estimation method, which relates to the field of artificial intelligence. The method comprises: collecting N body images of a user at the same time from N perspectives, the N perspectives comprising an upper perspective, the collection position of the upper perspective being higher than the height of the user, and N being greater than or equal to 3; inputting the N body images into N two-dimensional pose estimation models respectively to obtain N two-dimensional pose data, wherein the N two-dimensional pose estimation models have a one-to-one mapping relationship with the N perspectives; and inputting the N two-dimensional pose data into a three-dimensional pose estimation model to obtain three-dimensional pose data of the user. The present disclosure also provides a pose estimation device, equipment, storage medium and program product.

The invention belongs to the technical field of robot control, and discloses a robot suction cup grabbing detection network and method based on multi-scale attention, and the network comprises a multi-scale attention module, a multi-scale fusion module and an improved suction cup grabbing evaluation module. The multi-scale attention module is based on a HarDNet-68 backbone network, introduces a frequency domain channel attention mechanism, combines multi-scale depth separable convolution and expansion convolution, and enhances the multi-scale feature extraction capability; the multi-scale fusion module adopts a two-stage progressive fusion strategy, effectively integrates shallow details and deep semantic features, and generates a capture quality map and an object center map; the improved evaluation module improves the precision of side grabbing detection and 3D attitude estimation through multi-level threshold and truth value supervision. According to the method, the multi-scale object detection and grabbing success rate is remarkably improved in a complex industrial scene, and high robustness and real-time performance are achieved.

The invention discloses a three-dimensional human body posture estimation method and device and a medium, and belongs to the technical field of computer vision. The method comprises the following steps: extracting two-dimensional human body posture key points based on an acquired video picture to obtain a two-dimensional human body posture key point sequence; projecting the two-dimensional key point sequence to a feature space through nonlinear high-dimensional mapping to obtain a high-dimensional feature space matrix; inputting a three-dimensional human body posture estimation model based on the high-dimensional feature matrix to obtain a three-dimensional human body posture key point sequence; based on the three-dimensional human body posture key point sequence, a three-dimensional human body posture estimation result is obtained through the three-dimensional coordinate point positions. According to the method, through the three-dimensional human body posture estimation model, the anti-interference capability of feature extraction is enhanced, and the robustness of three-dimensional posture estimation in a complex dynamic scene is remarkably improved.

The invention discloses an animal three-dimensional attitude estimation method based on feature screening, and relates to the field of computer vision and intelligent perception, and the method comprises the following steps: obtaining animal activity multi-view video frames, and generating a three-dimensional volume grid; the method comprises the following steps: constructing an improved 3D U-Net network with an encoder-decoder structure, and carrying out pre-training; inputting the three-dimensional volume grid into the trained network to generate an accurate three-dimensional coordinate of the animal posture; the improved network is specifically characterized in that a context-aware feature screening gate module is arranged on each jump connection path connecting encoder shallow features and decoder deep features; replacing a standard 3D convolution module in the 3D U-Net network encoder with a 3D aggregation transformation residual module; according to the method, the animal three-dimensional attitude estimation precision is improved, and the method can be widely applied to animal behavioristics, neuroscience, ecology and related experimental studies.

The invention provides a group behavior identification method using 2D and 3D attitude features, and relates to the technical field of video analysis, and the method comprises the steps: 1, obtaining a 3D attitude feature sequence with a spatial expression capability; on the basis of the 2D attitude sequence, estimating the depth and orientation of each individual and the 3D position on the plane through three-dimensional attitude estimation and spatial geometric constraint, mapping a local joint structure to a unified camera coordinate system, and obtaining 3D attitude representation reflecting the spatial relationship between the individuals; 2, constructing multi-modal group space-time modeling; a 2D attitude and a 3D attitude are fused by utilizing a cross attention mechanism, a multi-person behavior interaction process is expressed as a time-space diagram changing along with time, nodes correspond to individuals, fused attitude features serve as attributes of the individuals, the relation between the individuals is described, weights represent the intensity of action interaction between the individuals, and a multi-person behavior interaction process is obtained. And using a space-time diagram convolutional network to model interaction behaviors among group individuals on a time sequence, and using a feedforward neural network as a classifier to realize group behavior identification.

This application belongs to the field of computer vision technology and discloses a method and system for 3D pose estimation based on occlusion-aware vertex inference and semantic enhancement. The method includes: receiving a 2D pose sequence containing occluded joints; using an occlusion-aware vertex inference network to complete the position of the occluded joints; and generating adaptive confidence scores for each joint based on the joint motion trajectory and initial scores using a confidence score generator to obtain the completed 2D pose sequence and corresponding confidence scores; using a semantic feature enhancement network to denoise and optimize the 3D pose assumption sampled from a Gaussian distribution. During the training phase, this network extracts joint semantic features in parallel through a Transformer encoder and a Transformer-like convolutional network, and uses a semantic feature matching module to align the two types of features to enhance the semantic representation capability of the Transformer encoder; during the testing phase, only the trained Transformer encoder is used to iteratively optimize the 3D pose assumption, and finally outputs an accurate and complete 3D human pose.

A kind of 3D pose estimation method based on dynamic limb constraint Transformer network model, first, dynamic limb constraint Transformer network model is built, model includes image feature embedding module, Transformer encoder module and MLP module, each module contains the corresponding Layer norm layer of convolution level;Then select training set and test set, and set the training parameter of dynamic limb constraint Transformer network model;According to dynamic limb constraint Transformer network model and its training parameter, the dynamic limb constraint Transformer network model is trained with the minimum loss function as the goal;Finally, the 2D pose sequence to be processed is input into the 3D pose estimation model based on dynamic limb constraint Transformer network, and the corresponding 3D pose is output;The 3D pose obtained by the application has the advantages of high precision, fast calculation, low resource consumption and the like.

This invention discloses a monocular 3D human pose estimation method based on graph convolutional networks and the Mamba architecture, as well as a system for executing the method. The method first maps the input 2D human skeleton sequence to a high-dimensional feature space and superimposes positional encoding. Then, it constructs multiple parallel feature extraction paths, including a graph convolutional network path for explicit topological modeling based on the human skeleton's adjacency matrix, and a Mamba architecture path for implicit global sequence modeling using a selective scanning mechanism. The feature extraction paths encompass both space-to-time and time-to-space processing orders. Subsequently, an adaptive fusion module dynamically weights and integrates the multiple feature paths, finally mapping and outputting a 3D human pose sequence. This invention maintains linear computational complexity while enhancing the perception of the human skeleton's topological structure and the ability to capture long-distance spatiotemporal dependencies, significantly improving the accuracy and robustness of monocular 3D pose estimation.

The application discloses a risk prediction method based on video key frame flood personnel action recognition, including the following steps: extracting the key frame of the flood monitoring video; constructing a key frame image fast action recognition model; and performing risk level quantization based on disaster feature recognition. Through the integration of key frame extraction, three-dimensional pose estimation, action recognition and risk quantization and other technologies, efficient and accurate personnel behavior monitoring and risk assessment in a complex flood environment are realized; the method combines single-width flow information and multi-category behavior characteristics, significantly improves the accuracy and reliability of risk prediction, and ensures the timeliness and effectiveness of emergency response; the optimized deep learning model and efficient data processing process are adopted, the real-time performance and high performance in processing large-scale video data are ensured, and the monitoring efficiency and resource utilization in disaster management are greatly improved.

The present specification relates to a multi-view 2D pose matching method for real-time 3D pose estimation by a server, the method comprising the steps of: acquiring images from two or more terminals; acquiring extrinsic parameters of the terminals on the basis of the images; acquiring inertial measurement unit (IMU) data of the terminals on the basis of the images; and reflecting the IMU data in the extrinsic parameters.