Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

853 results about "Human motion" patented technology

Home abnormal state signal detection method and system based on multi-mode sensing

The invention provides a home abnormal state signal detection method and system based on multi-modal sensing, and relates to the technical field of detection, and the method comprises the steps: collecting human body motion, acoustics and environment parameters through a millimeter wave radar, an acoustic sensor and an environment sensor, inputting the parameters into a deep fusion network, a spatio-temporal attention mechanism and a multi-scale convolutional neural network are used to extract a spatio-temporal feature sequence, and joint probability features are formed in combination with an acoustic feature spectrum analysis result. And inputting the joint probability features into a time sequence knowledge graph, and obtaining scene adaptive features after processing by a graph attention network and a double-flow auto-encoder network. And finally, the multi-task learning network is combined with the risk propagation neural network and the recursive neural network to carry out anomaly prediction and risk level evaluation, and is combined with historical early warning information to output an early warning strategy. According to the invention, multi-modal data can be effectively fused, the accuracy and reliability of home abnormal state detection are improved, and a more accurate risk assessment and early warning strategy is provided.
Owner:DEXIAOBAO HEALTH TECHNOLOGY (CHANGZHOU) CO LTD

Human body motion posture evaluation method and system

The invention discloses a human body motion posture evaluation method and system. Low-cost and high-precision human body posture analysis is realized through common camera equipment. The method comprises the following steps: acquiring a human body motion video through camera equipment; detecting key points of a human body by using a deep learning model, and extracting two-dimensional or three-dimensional coordinates of the human body; calculating joint angle, speed and acceleration based on the key point data; and analyzing attitude stability, motion symmetry and fluency characteristics, and generating a comprehensive evaluation result. The system evaluates the motion quality of the user through comparison between dynamic feature extraction and a standard motion template, and generates personalized improvement suggestions based on specific indexes, including direction adjustment and amplitude range adjustment. The method can be widely applied to scenes such as exercise and fitness, rehabilitation medical treatment and competitive training, and the comprehensiveness, the accuracy and the guiding effect of action evaluation are improved. By means of the method, the user can obtain motion quality feedback in real time, the posture is effectively optimized, and the exercise performance or rehabilitation effect is improved.
Owner:HEBEI AKAI TECHNOLOGY CO LTD

Portable posture detection and gait analysis method and system

The invention relates to the technical field of human body posture detection and gait analysis, and discloses a portable posture detection and gait analysis method and system. The method comprises the steps that a hardware system is built, the hardware system comprises an embedded visual module, a data processing module and an interaction module, and human body motion video streams are collected through the embedded visual module; performing dynamic foreground extraction on the acquired video stream by adopting a background difference method and a self-adaptive illumination compensation algorithm, and outputting an illumination compensation image; according to the output illumination compensation image, the data processing module is used for increasing time sequence constraint loss on the basis of a YoLoV8 model, positioning key points of a human body and constructing a three-dimensional skeleton motion model; according to the method, a dynamic shielding processing engine and a multi-target decoupling mechanism are utilized, the constructed three-dimensional skeleton motion model is tracked based on a hierarchical association degree measurement function, human body motion data can be collected in real time, and accurate evaluation of postures and accurate extraction of gait features in multiple scenes are achieved.
Owner:BEIJING SCI & TECH PATENT OFFICE

Text-aligned human motion generation method and system

The invention discloses a human motion generation method and system based on text alignment, and belongs to the technical field of computer vision and deep learning. According to the method, a two-stage training mode is adopted, in the first stage, a quantized variational auto-encoder model for reconstructing human motion is trained at first, then parameter freezing is carried out on a decoder network in the quantized variational auto-encoder model and a pre-trained text-to-motion cross-modal retrieval model, the decoder network and a bidirectional mask Transform model form a training framework for training, and the quantized variational auto-encoder model and the pre-trained text-to-motion cross-modal retrieval model form a training framework for training; and after training is completed, reasoning tasks are performed. According to the method, by improving a text encoder, adding semantic alignment loss and adopting a bidirectional mask type Transform architecture, the overall motion perception ability is enhanced, semantic alignment of motion is generated, the generated human motion sequence better conforms to original data distribution, the generated human motion sequence and an input motion description text are more semantic-consistent, and the motion perception ability of a human body is improved. And the motion generation quality is improved.
Owner:ZHEJIANG UNIV

Lower limb exoskeleton gait track prediction method based on LSTM-KAN fusion model

The invention discloses a lower limb exoskeleton gait track prediction method based on an LSTM-KAN fusion model, and the method comprises the steps: collecting human motion data through a sensor assembly, and carrying out the filtering, missing value processing and normalization of the human motion data; then constructing an overall architecture of a prediction model based on an LSTM-KAN network, optimizing parameters of the prediction model by using a particle swarm optimization (PSO) algorithm, extracting key features in the preprocessed data as a training data set, and inputting the training data set into the prediction model for training; and finally, collecting current human body motion data, pre-processing the current human body motion data, inputting the pre-processed current human body motion data into the trained prediction model, and outputting future human body gaits and tracks by the prediction model. And the controller takes a future gait track generated by the prediction model as a reference track to generate a driving signal and sends the driving signal to the actuator so as to realize accurate control of the actuator. By adopting the gyroscope sensor, the acceleration sensor and the pressure sensor for gait estimation, exoskeleton motion control can be effectively improved, and man-machine interaction experience is effectively improved.
Owner:SHANGHAI UNIV OF ENG SCI

Motion state detection method and device based on multi-modal sensor

The invention relates to a motion state detection method and device based on a multi-modal sensor, and the method comprises the following steps: carrying out the inertial parameter extraction of an original acceleration signal in the motion process of a human body through a three-axis acceleration sensor, and obtaining a motion acceleration feature vector; the original pulse wave signals in the movement process are monitored in real time, and a cardiovascular physiological feature sequence is obtained; performing time-frequency domain conjoint analysis on the time sequence myoelectricity feature data set, the motion acceleration feature vector and the cardiovascular physiological feature sequence by adopting an adaptive wavelet transform technology to obtain a multi-dimensional feature fusion matrix; and performing dynamic segmentation on the multi-dimensional feature fusion matrix through a hierarchical clustering algorithm to obtain motion state feature subspaces, thereby solving the problem that a traditional motion monitoring method mainly depends on a single type of sensor, although the motion condition of a human body can be reflected to a certain degree, the motion state feature subspaces cannot be monitored. However, the technical problem of lack of comprehensive understanding of complex motion states is solved.
Owner:SHENZHEN TIANJIULONG TECH CO LTD

Video stream-based attitude feature recognition method

The invention discloses a posture feature recognition method based on a video stream, and the method comprises the steps: carrying out the preprocessing of a continuous video stream, obtaining video frame training data, extracting a key frame and an adjacent frame in each frame of image, constructing a feature extraction module for a human body region, and obtaining a global frame, performing local extraction on the human body area by using adjacent frames on the left side and the right side to obtain local frames, and constructing semantic association information for the global frame through time sequence continuity between the local adjacent frames and the current key frame; acquiring enhanced feature representation by adopting a conditional feature aggregation algorithm; obtaining attitude sequence data through the attitude detail features; the method comprises the following steps: establishing three-dimensional coordinates, adaptively extracting posture change data by adopting a human body motion decoupling model, predicting human body posture characteristics through a smooth optimization strategy, and introducing a cross attention mechanism to realize deep fusion of spatio-temporal characteristics, so that the understanding ability of the model to a complex action mode is enhanced; and the attitude expression capability of the model in a sheltered or fuzzy region is obviously improved.
Owner:北京汇畅数宇科技发展有限公司

Fall behavior judgment and early warning method and system based on multi-modal data

The invention discloses a tumble behavior judgment and early warning method and system based on multi-modal data, and belongs to the technical field of intelligent monitoring and health safety, and the method comprises the steps: processing a video stream obtained by a camera through a Jeson Nano processor, and obtaining a human body posture detection result; the millimeter-wave radar obtains human body motion state information, judges a motion state according to the sudden change value and obtains a human body motion state detection result; the blood oxygen bracelet collects human body physiological data and recognizes sudden physiological abnormalities to obtain physiological abnormality data; according to the posture detection result, the motion state detection result and the physiological anomaly data, sending different levels of signal states to a cloud platform; and the cloud platform receives and fuses the signal states, and based on the fused multi-dimensional information, adopts falling behavior grading early warning and triggers a grading alarm mechanism. The problems of low data utilization rate and single response means are solved, and the accuracy and adaptability of fall monitoring are improved.
Owner:XI'AN POLYTECHNIC UNIVERSITY

High-speed human motion trail motion mode recognition system

The invention relates to the technical field of computer vision, and discloses a high-speed human motion trail motion mode recognition system. Through multi-modal sensor fusion and an adaptive space-time attention network (ASTAN), the problems that a traditional optical system depends on mark points and IMU accumulative errors exist are effectively solved, and the high-speed movement track reconstruction precision and the environmental adaptability are remarkably improved; self-supervised data enhancement and dynamic model compression technologies are combined, so that the dependence on labeled data is greatly reduced, lightweight edge deployment is realized, and the real-time requirement is met; the system can synchronously output multi-mode feedback instructions, supports multi-scene application such as sports action correction, medical rehabilitation evaluation, security and protection anomaly detection and man-machine interaction control, and solves the core pain points of insufficient precision, real-time performance, robustness and generalization ability in the prior art. And an efficient, reliable and low-cost comprehensive solution is provided for high-speed motion analysis.
Owner:王高阳

Method and system for generating 3D (three-dimensional) human motion under text driving by using 2D (two-dimensional) video

The invention discloses a method and a system for generating 3D (three-dimensional) human motion under text driving by utilizing a 2D (two-dimensional) video. The method comprises the following steps of: acquiring the video and preprocessing to obtain a two-dimensional key point sequence and text description; the two-dimensional key point sequence passes through a spatiotemporal feature adapter to obtain a potential spatiotemporal feature sequence, a residual vector quantizer quantizes and outputs a three-dimensional SMP L parameter sequence, and meanwhile, potential spatiotemporal features and a discrete Token sequence are mapped; preprocessing a text to extract a semantic vector, partially covering a Token sequence of a basic quantization layer, reconstructing a prediction sequence through a predictor in combination with the semantic vector, and obtaining a complete sequence through a refiner; constructing a total loss function and a text-to-action loss function to train the module; and inputting the text description and the basic quantization layer Token to a trained module, outputting a three-dimensional SMPL parameter sequence, and rendering to generate a three-dimensional human body grid and animation. According to the method, the end-to-end generation from the text to the three-dimensional SMPL action is realized only by two-dimensional key points and text description.
Owner:ZHEJIANG UNIV

Human motion posture recognition method and system based on multi-modal data fusion

The invention discloses a human motion posture recognition method and system based on multi-modal data fusion, and relates to the technical field of posture recognition, and the method comprises the steps: firstly, synchronously obtaining a human motion posture video frame and an IMU segment, then carrying out the feature extraction of the two kinds of heterogeneous data in a shunt parallel mode, and carrying out the feature extraction of the two kinds of heterogeneous data; and respectively capturing visual space attitude information and dynamic inertial characteristics of the IMU. Afterwards, specific features of the modals are uniformly expressed through a mixed Token and embedding mechanism, and a space-inertia collaborative attention fusion mechanism is further introduced to realize dynamic association and deep fusion of cross-modal information; and finally, classifying the multi-modal fusion representation vector obtained by fusion so as to realize accurate recognition of the human motion posture. In this way, the defect that traditional fusion is insufficient in capturing subtle action differences can be overcome, and the accuracy and stability of action recognition in a complex scene are improved.
Owner:ZHEJIANG FUBAO INTELLIGENT TECH CO LTD

Robot joint control method and system based on motion capture equipment

The invention discloses a robot joint control method and system based on motion capture equipment, and belongs to the technical field of robot control. An existing robot joint control scheme lacks an effective man-machine cooperation and natural interaction mechanism, so that the flexibility of a robot is poor, and the intention of an operator cannot be naturally expressed. According to the robot joint control method based on the motion capture equipment, by constructing a motion capture module, a data processing module, a coordinate system transformation module and a solution optimization module, rotating posture data, collected by the motion capture equipment, of human body joints are converted into target joint control information needed by execution of a robot; the robot joint control based on the motion capture equipment is realized, so that the human motion can be efficiently and accurately mapped to the robot, the robot is good in flexibility, sudden motion or complex cooperative operation can be effectively processed, the intention of an operator can be naturally expressed, and man-machine cooperation and natural interaction are realized.
Owner:HANGZHOU YUSHU TECHNOLOGY CO LTD

Human body motion data processing method

The invention relates to a human motion data processing method, and belongs to the technical field of data processing. Comprising the following steps: controlling a microwave radar to emit a detection signal, and detecting a heart rate signal and a respiration signal of a human body; performing clustering analysis on the data according to the collected heart rate and respiratory rate, and extracting heart rate, respiratory rate change, duration and exercise intensity characteristics; according to the extracted heart rate change, duration, exercise intensity and other characteristics, the exercise stage is recognized, and according to the neural network model library, an energy consumption result is output in combination with data analysis. According to the human motion data processing method provided by the invention, the microwave radar is combined with the intelligent wearable device, so that the detection precision of heart rate and respiration signals is remarkably improved, multi-modal data fusion is realized, personalized heart rate prediction is performed, motion stage division is optimized in combination with acceleration data, and the accuracy of motion data processing is improved. The accuracy of motion mode recognition is improved, and personalized motion suggestions can be provided for the user.
Owner:NINGBO INST OF TECH ZHEJIANG UNIV ZHEJIANG

Three-dimensional digital human generation method and system capable of voice interaction

The invention belongs to the technical field of three-dimensional reconstruction, and discloses a three-dimensional digital human generation method and system capable of voice interaction. According to the invention, brand new speaking audios in different languages are automatically generated according to different languages of the input target text and the sampled human voice audios; the sequential stability and detail reduction capability of three-dimensional human motion are guaranteed by using multi-model joint estimation and a sequential loss function, and facial expression details and hand postures in the image can be accurately estimated. After the high-precision three-dimensional human body model is obtained through estimation, human body action and expression generation is carried out based on voice driving, accurate synchronization of actions and expressions generated through voice is achieved, and facial expression movement and body posture movement, namely a whole-body three-dimensional human body model, conforming to brand-new speaking audio are accurately generated; and finally, rendering the whole-body three-dimensional human body model into a real digital human capable of voice interaction by using a three-dimensional neural rendering model. According to the invention, the realization of single person picture input, high-precision three-dimensional digital person generation and voice interaction is facilitated.
Owner:NANJING UNIV OF SCI & TECH

Costume design assisting method and system based on artificial intelligence

The invention discloses a costume design assisting method and system based on artificial intelligence, and belongs to the field of costume intelligent design, and the method comprises the steps: extracting multi-dimensional style feature information from a pre-constructed costume image data set and a fashion trend database through employing a convolutional neural network, and constructing a style vector space model; vector similarity matching is carried out, and a costume design sketch is generated by adopting a graph neural network; based on the design sketch, generating a structured process data packet for proofing; the design sketch and the structural data are input into a three-dimensional human body modeling and simulation module, and wearing effect simulation of virtual clothes is realized based on human body motion capture and physical cloth simulation technologies; and constructing a human-computer interaction interface, and quickly responding to a modification request based on the generative adversarial network model. Structured information such as the garment pattern structure, the sewing sequence, the material specification and the process route is automatically generated through the reinforcement learning algorithm, and the garment proofing efficiency and accuracy are remarkably improved.
Owner:QUANZHOU NORMAL UNIV

Fine-grained video behavior recognition method based on double-text prompt

The invention belongs to the field of computer vision and image processing, and relates to fine-grained action classification of a picture sequence after video framing by adopting a deep convolutional neural network, in particular to a fine-grained video behavior recognition method based on double-text prompt. The method is simple in program and easy to implement, actions capable of recognizing the fine grit of the human body can be obtained, text description can be divided into different fine grits through a large language model for the fine grit actions of the human body, and then the generated text feature vectors and video features of different time scales are subjected to cross attention mechanism response, so that the recognition accuracy of the human body is improved. Unique details of human body motion in a video can be better found, so that fine-grained motion is more accurately reasoned.
Owner:DALIAN UNIV OF TECH

Scene-aware synthetic human motion generation using neural networks

A motion diffusion model may be pre-trained on motion data, and a scene-aware component (e.g., one or more layers of a neural network) may be connected and used to extract and inject a representation of scene information into the pre-trained motion diffusion model. For example, to predict orientations of joint waypoints along a path through a particular 3D scene, a scene-aware input channel that accepts a representation of the 3D structure of the scene may be added to a pre-trained motion diffusion model. To predict orientations of joint waypoints along a path that interacts with a 3D object in the 3D scene, a scene-aware input channel that accepts a representation of the 3D object and / or a surface thereof may be added to a pre-trained motion diffusion model. As such, the resulting scene-aware motion diffusion model(s) may be tuned on motion-scene data and used to generate human motion.
Owner:NVIDIA CORP

Grounded human motion generation with open vocabulary scene-and-text contexts

In an embodiment, a method for human motion generation with open vocabulary scene-and-text context is provided. The method involves receiving an input that includes a 3D point cloud of a scene containing a goal object with a natural language instruction related to the goal object. A text tokenizer is applied to the text to obtain tokenized text, and a text encoder from a pre-trained vision-language model generates text features. First scene features are generated by applying a pre-trained U-Net scene encoder to the 3D point cloud, which are down sampled to obtain second scene features. A conditional latent is obtained by fusing the second scene features with the text features. A conditional motion generator predicts motion parameters for a parametric human body model over a specific time duration. Finally, 3D human meshes for multiple motion frames are obtained based on the motion parameters and the parametric human body model.
Owner:FUJITSU LTD +1

Method and system for 3D scanning and dynamic posture capturing of figure model

The invention is suitable for the technical field of 3D scanning, and provides a figure model 3D scanning and dynamic attitude capturing method and system, which dynamically triggers 3D scanning by monitoring joint movement intensity in real time, continuously collects continuous movement tracks by using an inertial sensor in a non-scanning period, and reversely deduces space-time parameters of a shielded part through multi-light-source shadow boundary evolution. And constructing a spatio-temporal joint evaluation function to adaptively adjust the scanning frequency and the sampling rate, and finally performing spatio-temporal alignment and confidence weighted fusion on the discrete point cloud, the continuous trajectory and the reverse reckoning data to generate a dynamic biomechanical model. The scheme breaks through the bottleneck of resource waste and blind area shielding of traditional fixed frequency scanning, realizes space-time complementation and precision optimization of motion monitoring, and is suitable for dynamic scenes such as movie and television animation, medical rehabilitation, physical training and the like. According to the system, through multi-modal data fusion and self-adaptive resource scheduling, the data integrity, the real-time performance and the biomechanical analysis capability are remarkably improved, and an efficient solution is provided for human motion digitization.
Owner:DONGGUAN HENGCHUANGXIN CULTURAL CREATIVITY CO LTD

Intelligent wearable exoskeleton self-adaptive auxiliary regulation and control method and exoskeleton system

The invention discloses an intelligent wearable exoskeleton self-adaptive auxiliary regulation and control method and an intelligent wearable exoskeleton system.The intelligent wearable exoskeleton self-adaptive auxiliary regulation and control method comprises the steps that a motion data sample of human walking is obtained, and a human motion model is constructed based on a man-machine closed-chain motion model according to the motion data sample; collecting multi-modal data of the target user when the target user wears the exoskeleton in real time; calling a human body motion model and a man-machine closed-chain motion model, and generating an exoskeleton control instruction of man-machine motion cooperation according to the multi-modal data; and dynamically adjusting motion control parameters of the exoskeleton according to the control instruction. According to the method, the precise human body motion model is constructed based on the man-machine closed-chain motion model, the man-machine motion cooperative exoskeleton control instruction is generated in combination with the model, the exoskeleton motion control parameters are dynamically adjusted according to the instruction, the body type difference, the muscle strength level and the motion habit of different users can be precisely matched, and the wearing comfort and safety are effectively improved.
Owner:PEKING UNION MEDICAL COLLEGE HOSPITAL +1

Millimeter wave radar motion evaluation method, system and equipment based on human skeleton model

The invention relates to the technical field of millimeter-wave radars, in particular to a millimeter-wave radar action evaluation method, system and equipment based on a human skeleton model, and the method comprises the following steps: S1, millimeter-wave radar point cloud sampling: employing millimeter-wave radar equipment to collect three-dimensional point cloud data of a human body, and extracting target points related to human skeleton nodes; s2, skeleton node estimation: predicting three-dimensional coordinates of each joint of the human body by adjusting an output structure of a PointNet neural network model, so as to reconstruct and fit a skeleton structure of the human body; and S3, human body action evaluation: constructing the obtained three-dimensional coordinate data of the multiple frames of human body skeleton nodes into time sequence data, and analyzing a motion track by using an evaluation system based on an ST-GCN network. According to the invention, complex dynamic changes and behavior modes in human motion can be accurately captured, and high-precision real-time evaluation and feedback are provided for the fields of motion monitoring, rehabilitation training, health evaluation and the like.
Owner:UNIV OF SCI & TECH BEIJING

Human body posture recognition method based on millimeter wave radar sparse point cloud

The invention discloses a millimeter-wave radar sparse point cloud-based human body posture recognition method, which belongs to the technical field of human body posture recognition, and comprises the following steps of: acquiring three-dimensional point cloud data of human body actions through a millimeter-wave radar, and preprocessing the three-dimensional point cloud data; and training a lightweight neural network model by using the preprocessed point cloud data, wherein the model comprises an edge convolution module and a grouping sparse Transform encoder module. The edge convolution module extracts spatial geometric features and detects dynamics, static postures are directly classified, dynamic postures capture a time sequence dependency relationship through a grouping sparse Transform module, attention calculation is only carried out on frames with feature changes exceeding a threshold value, and mean pooling aggregation is carried out on other frames. And finally, classifying the human body postures based on the spatial geometric features and the time sequence dependency relationship to obtain a classification result. The device is simple in structure, accurate in recognition and suitable for efficient deployment of edge equipment.
Owner:LINYI UNIVERSITY

Dynamic millimeter wave radar point cloud human arm tracking system and method based on joint learning

The invention discloses a dynamic millimeter-wave radar point cloud human arm tracking system and method based on joint learning, and the method comprises the steps: obtaining a millimeter-wave point cloud generated by human motion through millimeter-wave radar equipment, and enabling the generated point cloud to serve as the input of an arm point cloud generation module based on heuristic and clustering; decoupling a dynamic millimeter wave point cloud generated by arm movement; taking the dynamic arm point cloud as the input of a spatial-temporal feature encoder, and carrying out feature extraction; and point cloud features are output as input of a multi-task joint learning module, gesture recognition is introduced as an auxiliary task, learning assistance is provided for an arm tracking task, and finally three-dimensional motion trajectory tracking of main joints of the arm is achieved. According to the method, the technical challenges of arm semantic information loss and dynamic change in millimeter wave point cloud are overcome, and the human arm motion robust tracking with privacy protection is realized.
Owner:SOUTHEAST UNIV

Human motion capture method based on mask perception graph convolution and skeleton prior

The invention belongs to the technical field of computer vision, and discloses a human motion capture method based on mask perceptual graph convolution and skeleton prior, and the method comprises the steps: obtaining an input image and a corresponding human mask; extracting features of the image and the mask and coding; constructing a mask perception graph convolutional network, constructing an adjacent matrix of the graph convolutional network by using mask information, and applying mask constraint loss to enhance the expression ability of the human body region features of the image; a skeleton prior decoupling network is constructed, skeleton and vertex information of an SMPL model is used as prior, a cross attention mechanism is combined, and multi-modal data enhancement of SMPL skeleton node features is guided through image features; and finally outputting a three-dimensional joint position coordinate and a shape grid vertex coordinate. According to the method, the local feature consistency is enhanced through mask perception image convolution, the geometric consistency is improved in combination with skeleton prior, and the precision of three-dimensional human body motion capture under monocular vision is effectively improved.
Owner:NANCHANG UNIV

Operator and occupant monitoring validation for autonomous and semi autonomous machines

In various examples, one or more validity checks that model one or more aspects of human physiology may be applied to frames of detected human features to detect and respond to the presence of faults. Example validity checks include human feature constraints derived from the kinematics of human motion, anatomical and spatial constraints, consistency across detection modalities, and / or others. The present techniques may be utilized to validate human features detected by various computer vision tasks, such as those involving pose estimation, facial detection, gesture recognition, and / or activity monitoring, to name a few examples.
Owner:NVIDIA CORP

Synthetic human motion generation using neural networks for scene awareness

The invention discloses synthetic human motion generation using a neural network for scene awareness. Motion diffusion models may be pre-trained on motion data, and scene awareness components (e.g., one or more layers of a neural network) may be connected and used to extract and inject representations of scene information into the pre-trained motion diffusion models. For example, in order to predict the orientation of joint waypoints along a path in a particular 3D scene, scene-aware input channels that accept a representation of a 3D structure of the scene may be added to a pre-trained motion diffusion model. To predict the orientation of joint path points along a path interacting with a 3D object in the 3D scene, a scene-aware input channel accepting a representation of the 3D object and / or a surface thereof may be added to a pre-trained motion diffusion model. Therefore, the obtained scene perception motion diffusion model can be adjusted on motion-scene data and is used for generating human motion.
Owner:NVIDIA CORP

Human body action sequence generation method and device, equipment and storage medium

The invention relates to the technical field of human body action generation. The invention discloses a human body action sequence generation method and device, equipment and a storage medium. The generation quality of human body actions can be improved, and the generalization ability of a model can be improved. The human body action sequence generation method comprises the following steps: acquiring text information and a scene image; and inputting the text information and the scene image into an action planning model for task decomposition and action generation processing, and outputting a human body action sequence.
Owner:PEKING UNIV SHENZHEN GRADUATE SCHOOL

Reconstruction and driving method based on multi-view motion video of clothed human body

Provided in the present invention is a reconstruction and driving method based on a multi-view motion video of a clothed human body. The method comprises: using a two-layer three-dimensional Gaussian representation to model a human body and corresponding clothes; defining geometric attributes of three-dimensional Gaussian ellipsoids for any face on coarse three-dimensional meshes of the human body and the clothes; at a reconstruction stage, transforming coarse triangular meshes of the clothes and the body from a canonical space to an observation space, and updating Gaussian geometric attributes; and on the basis of a camera pose and parameters of an input video, rasterizing a three-dimensional Gaussian representation of the observation space to an image space, and rendering a picture on the basis of a Gaussian splatting formula. Self-supervised training can be performed on a model by means of calculating the color difference between a rendered picture and a real picture and using same as an optimization objective; when reconstruction is completed and driving is performed, a geometric result of a current frame is used to calculate the position of the coarse triangular mesh of the clothes in the next frame; and the reconstruction-stage method and related parameters are used to render a driving picture in a new posture.
Owner:UNIV OF SCI & TECH OF CHINA

Human motion understanding using state space models

Various implementations disclosed herein include devices, systems, and methods that generate 3-dimensional (3D) information related to a user from a continuous time light signal. For example, a process may obtain two-dimensional (2D) information corresponding to a continuous time light signal providing information about a user in a 3D environment. The 2D information may be based on frames comprising images capturing the continuous time light signal at one or more frame rates. The process may further obtain discretization information corresponding to the one or more frame rates. The process may further determine 3D information about the user by inputting the 2D information and the discretization information into a state space model. The state space model may be a continuous time learnable framework for mapping between continuous time 2D scalar inputs and continuous time scalar 3D outputs.
Owner:APPLE INC

Generative prediction method for pedestrian motion trail guided by double-pendulum dynamics

The invention provides a double-pendulum dynamics guided pedestrian motion track generation type prediction method, and belongs to the technical field of automatic driving scene prediction. Comprising the following steps: establishing a pedestrian motion nonlinear dynamic model based on double-pendulum dynamics, and defining a mapping relation from an attitude space to a double-pendulum state space; establishing a high-degree-of-freedom pedestrian motion nonlinear dynamic state prediction model for predicting the motion state of the pedestrian at the future moment in the double-pendulum state space; establishing a conditional diffusion model for reconstructing the human body posture of the pedestrian at the future moment in the double-pendulum state space; the past state track of the pedestrian, the reconstructed human body posture of the pedestrian at the future moment, the past track of the adjacent pedestrian and semantic map information serve as guiding conditions of a conditional diffusion model, and generation of a future motion track of the pedestrian is guided. The motion state space of the double-pendulum system represents the non-linear dynamic change space of human body motion, and accurate prediction of the pedestrian motion track is realized by utilizing the generation of the conditional diffusion model.
Owner:NINGXIA UNIVERSITY