994 results about "Motion capture" patented technology

The invention provides a rehabilitation training action evaluation method and device based on multi-view vision. According to the method, a multi-view camera is adopted to synchronously collect a rehabilitation training image sequence, two-dimensional coordinates of key points of a human body are extracted by improving an HRNet deep learning model, three-dimensional reconstruction is performed in combination with a Gaussian process, motion feature data are extracted, and rehabilitation training quality is evaluated. Accurate motion capture without wearing mark points by the patient is realized, the training constraint feeling of the patient is effectively reduced, the rehabilitation evaluation accuracy is improved, and a personalized rehabilitation scheme can be generated.

The invention provides a high-precision motion capture real-time calibration method based on multi-sensor fusion, and the method comprises the steps: obtaining motion data and environmental parameters collected by multiple sensors, the multiple sensors comprising an inertial measurement unit and an optical sensor; constructing a dynamic calibration model based on the action data and the environmental parameters, and performing real-time error compensation and fusion processing on the sensor data; and establishing a time-varying error dynamic model, and performing real-time compensation on zero offset drift, vibration noise and electromagnetic interference of the inertial measurement unit based on the environmental parameters. According to the invention, the IMU and the optical sensor are fused, respective advantages are exerted, and motion data are comprehensively collected. And meanwhile, the time-varying error dynamic model compensates errors generated by environmental interference in real time, so that real actions can be accurately restored through action capture.

Gait characterization and analysis based on a monocular video of a walking subject are achieved, in various embodiments, by processing the video using one or more machine-learning models to predict three-dimensional (3D) keypoint coordinates for a set of anatomical keypoints of the subject, along with joint angles, body-segment rotations, and / or gait event classifications, for multiple video frames. The machine-learning models may be trained using ground-truth data acquired with a marker-based motion capture system and / or pressure-sensitive walkway.

The invention relates to the technical field of evaluation and management, in particular to a classroom learning dynamic evaluation and management system, which comprises a multi-source heterogeneous data acquisition module, an acoustic sensor array, an optical motion capture unit, an intelligent writing track acquisition device and a brain wave monitoring terminal, the data preprocessing and feature fusion module is used for eliminating environment interference signals and generating preprocessed data; the dynamic learning state evaluation module is used for extracting three-dimensional features and outputting a dynamic evaluation matrix; the group learning portrait generation module is used for generating a multi-dimensional learning portrait in combination with a preset knowledge graph topological structure; the self-adaptive management strategy generation module is used for dynamically generating a management strategy set; and the intelligent feedback execution module is used for driving the self-adaptive illumination system, the wearable vibration reminding device and the AR enhancement projection equipment to execute physical environment regulation and control. According to the invention, personalized strategy dynamic generation is realized, and adaptive regulation and control of the classroom environment and the teaching rhythm are realized in combination with the intelligent Internet of Things equipment, so that an intelligent collaborative feedback closed loop is formed.

The invention discloses a 500kV standardized switching operation method based on a VR virtual reality technology, and aims to solve the problems of unreal environment, incomplete evaluation and low efficiency in the switching operation training of an existing 500kV transformer substation. The method comprises the following steps: constructing a full-immersive virtual reality scene, collecting actual environment data of a transformer substation, and constructing and optimizing a high-precision model; designing multi-modal interaction of the standardized switching operation process, wherein the multi-modal interaction comprises visual guidance, a multi-modal interaction interface and an intelligent sensing function; dynamic action scoring and intelligent behavior evaluation are realized, actions are captured in real time by using high-precision action capture equipment, an instant score is generated by comparing with a standard, and dynamic feedback is provided; and real-time optimization and learning migration of the virtual character model are realized, user operation habits are captured, virtual character actions are optimized, and multi-scene migration and adaptation are realized. The method is suitable for VR-based 500kV standardized switching operation work, and the training authenticity and efficiency are improved.

The invention discloses an inspection robot collaborative planning method and system based on digital twinning, and the method comprises the steps: constructing a virtual simulation environment of a large-scale component through a point cloud modeling technology, and integrating a motion capture system to achieve the real-time mapping of a physical environment and a virtual model; defining a cooperative mechanism of the scanning robot and the positioning robot, providing global coordinates and path guidance by the positioning robot, and correcting position errors in real time by the motion capture system; simulating a dual-machine motion track in the digital twin model, and identifying a potential collision risk through a conflict prediction model; designing a collaborative planning algorithm under coupling constraint, and performing optimization by taking path optimization and scanning visual angle coverage as a joint target; based on a twin intelligent framework with a body, the path and the visual angle of the robot are dynamically adjusted by combining reinforcement learning with real-time feedback, and self-adaptive collaborative planning is achieved. Through double-machine division cooperation and real-time correction, the scanning efficiency and precision of the large-size component can be remarkably improved, and the collision risk is reduced.

The invention relates to the technical field of physical education, in particular to a physical education method and system based on multimedia and holographic AR fusion, and the method comprises the following steps: collecting the motion data of a trainee based on a multi-source sensor array, constructing a motion parameterized model, and generating a holographic projection generation module through the cooperation with a standard motion database; a holographic AR teaching image is generated in real time, an environment perception module is combined to carry out adaptive projection on illumination and spatial layout, a spatial positioning module is used to establish coordinate mapping between a trainee and a virtual image, and a multi-channel feedback device is started to output visual, tactile and auditory signals. And finally, comparing the action parameters with the standard action data to automatically generate a training report containing wrong action labels and correction suggestions. According to the method, the motion capture precision and the interactive immersion can be effectively improved, and efficient and extensible technical support is provided for modern physical education and training.

The invention relates to the technical field of motion capture data processing, and discloses a robot reinforcement learning model construction method based on optical inertial fusion whole body motion capture, and the method comprises the steps: measuring and recording a plurality of key parameters of a target human body; carrying out primary calibration and recording on all inertial sensors worn on a target human body, and calibrating all cameras arranged around at the same time; in the process that the target human body executes the specified action, synchronously acquiring action sampling data of the target human body; the action sampling data comprises motion data corresponding to the inertial sensor and camera image data corresponding to the camera; the motion data and the camera image data are processed, fused and optimized, and optimized joint posture quaternary data and human body motion sequence data are obtained; and inputting the human body action sequence data into the reinforcement learning model of the humanoid robot, and configuring the preset reward function to perform reinforcement learning training, so that the learning efficiency of the robot and the accuracy of action execution can be improved.

The invention relates to a multi-person motion capture real-time digital human driving method based on a skeleton-muscle interaction model, and belongs to the field of artificial intelligence and ubiquitous universe. According to the method, the key points of the human body are detected through YOLO-Pose, the positions of the shielding points are obtained through a double-stage dynamic shielding point compensation method, and the accuracy of the key points is guaranteed. And outputting the three-dimensional coordinates of the key points of the human body by adopting a three-dimensional key point depth extractor based on multi-dimensional exclusive-shared feature reasoning in combination with an MT ransPose model. A real person-skeleton model-digital person real-time mapping method is provided, three-dimensional key points are mapped to a standard human body skeleton model, and a digital person role and the skeleton model are bound through a weight mapping algorithm. Based on a skeleton-muscle interaction model, the digital person accurately reflects a real motion posture by introducing the mechanical action of a skeleton rigid body dynamic model and a muscle force line model, and the process of driving the digital person in real time through multi-person motion capture is achieved.

The invention provides an immersive concert experience system based on virtual reality, and belongs to the technical field of virtual reality, and the system comprises a dynamic adjustment module which is used for predicting the preference of a user for stage arrangement, light effects and music types through a machine learning model according to the historical preference data of the user, and outputting the preference to the stage arrangement, light effects and music types; dynamically adjusting visual elements and sound field parameters of the virtual scene and a refresh rate and a field angle of the virtual scene in combination with the user physiological indexes and the user head motion trail which are acquired in real time; the action capture and feedback module is used for capturing an interaction instruction sent to the dynamically adjusted virtual scene by the limb action of the user in real time; and the interaction feedback module is used for enabling the virtual performer to respond to the interaction instruction of the user in real time based on the expression recognition and voice synthesis technology, and generating personalized interaction feedback content. A user constructs an immersive music interaction field domain, and experience logic of a virtual music meeting is completely reconstructed.

The invention discloses a robot joint control method and system based on motion capture equipment, and belongs to the technical field of robot control. An existing robot joint control scheme lacks an effective man-machine cooperation and natural interaction mechanism, so that the flexibility of a robot is poor, and the intention of an operator cannot be naturally expressed. According to the robot joint control method based on the motion capture equipment, by constructing a motion capture module, a data processing module, a coordinate system transformation module and a solution optimization module, rotating posture data, collected by the motion capture equipment, of human body joints are converted into target joint control information needed by execution of a robot; the robot joint control based on the motion capture equipment is realized, so that the human motion can be efficiently and accurately mapped to the robot, the robot is good in flexibility, sudden motion or complex cooperative operation can be effectively processed, the intention of an operator can be naturally expressed, and man-machine cooperation and natural interaction are realized.

The invention discloses a scene interactive limb active and passive rehabilitation training system based on a virtual three-dimensional game in the field of rehabilitation medical treatment. The scene interactive limb active and passive rehabilitation training system comprises a virtual scene generation module, a multi-mode motion capture module, an active and passive cooperative control module, a real-time feedback and excitation module and a data statistics module. And the virtual scene generation module is used for constructing various three-dimensional game scenes, so that scene tasks are strongly associated with actions of upper limbs, lower limbs or upper and lower limbs, and scene difficulty is dynamically adjusted. Compared with the prior art, the interactive guidance of the immersive virtual three-dimensional game scene is adopted, the boring rehabilitation training is converted into interesting game experience, and the participation enthusiasm and compliance of the patient are greatly improved. Compared with traditional two-dimensional animation game training, the patient is more willing to actively participate in training, and the conflict emotion caused by boring training is reduced.

The invention provides a motion capture and feedback adjustment method and system for smart motion of meta universe, and relates to the technical field of meta universe, and the method comprises the steps: collecting actual motion data of a user through a plurality of inertial measurement units, establishing a mapping relation with skeleton nodes, evaluating and optimizing the motion based on a multi-dimensional scoring matrix, and constructing a user virtual image. And performing time sequence analysis on the virtual motion data to obtain local and global motion features, comparing the local and global motion features with standard motion features to generate an adjustment amount, and converting the adjustment amount into a motion correction instruction to guide a user to adjust a posture, so that accurate capture and real-time feedback can be realized, and a motion learning effect is improved.

The invention relates to a motion capture and evaluation method and device, equipment and a medium. The method comprises the following steps: acquiring motion capture data; repairing the motion capture data to obtain standardized motion data; obtaining a virtual character animation file based on the standardized action data and a virtual character skeleton model; performing multi-dimensional evaluation on the virtual character animation file to generate a comprehensive evaluation report; and adjusting the virtual character animation file based on the comprehensive evaluation report to obtain a final animation file. By adopting the method, the motion capture animation can be automatically evaluated and corrected, and the effect of improving the working efficiency is achieved.

A user workload state processing method and apparatus based on multimodal data, and a device. The method comprises: acquiring state information of a first user; determining workload information of the first user on the basis of the acquired state information, wherein the state information comprises at least two of the following information: physiological information, eye movement information, electroencephalogram information, brain function imaging information, motion capture information, spatiotemporal acquisition information, behavior acquisition information, and facial expression and state information; and feeding back the workload information to a first management account, and formulating a training scheme matched with the workload information. According to the embodiments of the present application, workload of users can be identified on the basis of multimodal data of users.

The invention discloses an intelligent motion capture system and method of a multi-mode sensor, and the method comprises the steps: creating and installing links among an optical module, an inertia module, a touch module, a synchronization core module and a to-be-detected object; sending a synchronous starting pulse signal based on the synchronous core module, setting the synchronous starting pulse signal as a clock reference starting point, and starting the optical module, the inertia module and the touch module based on the clock reference starting point; performing data acquisition on a to-be-measured object based on the optical module, the inertia module and the touch module to obtain an object acquisition information data set; and creating an acquisition original timestamp based on the synchronous start pulse signal, creating a sensor network delay model to dynamically adjust the acquisition original timestamp to obtain a calibration timestamp, and analyzing and correcting the time offset of the object data acquisition information. The method and the device have the effect of improving the accuracy of intelligent motion capture of the multi-mode sensor.

The invention discloses a motion capture system, a motion capture method and a readable storage medium, and belongs to the technical field of motion capture. The motion capture system comprises a motion capture sensor group and a motion capture processing unit, the motion capture sensor group comprises at least one invisible light image sensor and at least one visible light image sensor, the invisible light image sensor is used for sensing a detected target in a near-infrared or infrared band, and the visible light image sensor is used for sensing the detected target in a near-infrared or infrared band. The visible light image sensor is used for sensing a detected target in a visible light wave band and outputting a visible light image; and the motion capture processing unit is connected with the motion capture sensor group and is used for determining motion posture data of the detected target based on the visible light image and the near-infrared or infrared band image. According to the invention, the motion attitude data of the measured target is obtained based on the sensing data of different modes, and the precision and robustness of motion capture of the measured target by the motion capture system are improved.

The invention relates to the technical field of image recognition, in particular to an AI-based park intrusion detection and response system, which comprises a target trajectory acquisition module, an event density analysis module, a disturbance behavior recognition module and a response joint control execution module. According to the invention, through fusion of the spatial-temporal characteristics of the target trajectory and motion vector pattern recognition, intrusion behavior dynamic risk assessment and early warning collaborative optimization is realized, and through combination of multi-dimensional data association calculation and event density increment monitoring, abnormal situation sensing sensitivity and discrimination accuracy in a complex scene are improved. Motion vector time sequence modeling and disturbance feature extraction are utilized, the capacity of capturing camouflage behaviors and irregular motion is enhanced, an environment perception and response execution closed-loop control link is constructed by means of intelligent matching and parameter self-adaptive adjustment of multi-source sensing equipment, the dependency degree of manual review is remarkably reduced, the delay from recognition to treatment response is shortened, and the recognition efficiency is improved. And an early warning research and judgment and linkage disposal full-process automation system is formed.

The invention provides a training and examination robot system based on a multi-modal large model, equipment and a computer program product, and aims to accelerate training transformation and upgrading and improve coping ability as basic appeals, and based on regular training work of teams, training and examination robot, sensor acquisition equipment and big data platform three-terminal equipment. A multi-modal data acquisition layer for realizing real-time attitude detection and motion capture by using an edge calculation algorithm, a data processing and analysis layer for finishing real-time image processing through localized edge calculation in a network-free or power-free environment, and system control functions for finishing intelligent decision making, evaluation, error correction and the like are provided; an intelligent decision-making and control layer of a training subject rewriting decision-making mechanism is established, cross-modal collaborative learning middleware is utilized to realize modal representation consistency while training subject transposition evaluation is realized, training and assessment operations of the same subject of multiple persons or different subjects of multiple persons are supported, and the training and assessment efficiency is remarkably improved.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

The invention belongs to the technical field of computer vision, and discloses a human motion capture method based on mask perceptual graph convolution and skeleton prior, and the method comprises the steps: obtaining an input image and a corresponding human mask; extracting features of the image and the mask and coding; constructing a mask perception graph convolutional network, constructing an adjacent matrix of the graph convolutional network by using mask information, and applying mask constraint loss to enhance the expression ability of the human body region features of the image; a skeleton prior decoupling network is constructed, skeleton and vertex information of an SMPL model is used as prior, a cross attention mechanism is combined, and multi-modal data enhancement of SMPL skeleton node features is guided through image features; and finally outputting a three-dimensional joint position coordinate and a shape grid vertex coordinate. According to the method, the local feature consistency is enhanced through mask perception image convolution, the geometric consistency is improved in combination with skeleton prior, and the precision of three-dimensional human body motion capture under monocular vision is effectively improved.

The invention relates to an all-humanoid robot motion redirection method and device based on motion capture equipment, and the method comprises the steps: converting a human body posture quaternion collected by the motion capture equipment into a rotation matrix, and achieving the precise mapping from a Cartesian space to a robot joint space in combination with an Euler angle decomposition strategy; a quaternion-disassembled joint angle is used as an initial solution, various physical constraints are introduced in real time according to the actual supporting state of the robot, and the lower limb joint angle is corrected online by constructing and solving an inverse kinematics problem with various constraints, so that the robot reproduces the action of an operator, and the joint angle of the lower limb can be corrected online. And the requirements of joint limiting, self-collision avoidance, dynamic balance and the like are strictly met, so that the accuracy of motion simulation and the stability of the system are effectively unified.

The invention relates to the technical field of wing-mounted flight simulation, in particular to an immersive active control wing-mounted flight simulation system. The system comprises a physical simulation platform used for bearing an experiencer and providing immersive sensory feedback; the motion capture subsystem is used for collecting motion data of an experiencer in real time; the wing-mounted flight aerodynamic modeling subsystem is used for calculating aerodynamic characteristics in the wing-mounted flight process; the dynamic wind feeling simulation subsystem is used for dynamically adjusting the intensity and direction of a wind field and simulating wind pressure distribution in different flight states; the dynamic stress control subsystem is used for applying dynamic force feedback to the experiencer and simulating stress change in the flight process; the three-dimensional visual subsystem is used for simulating a visual picture in real time; and the sound subsystem is used for comprehensively simulating the environment sound effect in the flight process. According to the invention, the immersion and reality of wing-mounted flight simulation are greatly improved, and the safety and reliability are ensured.

The invention discloses a figure modeling and fitness action teaching method and system based on non-contact action capture, and the method carries out the real-time monitoring of a human body posture through a millimeter wave radar, carries out the 3D human body posture reconstruction and analysis through combining with a human body posture frame of Google image processing, and achieves the high-precision action capture. The method can be used for human body modeling, body building action scoring and action guidance, action evaluation and real-time feedback in body building teaching can be effectively supported, the millimeter wave radar is adopted for human body posture capture, external markers or wearable equipment are not needed, constraint on user activities is avoided, and use convenience and comfort are improved. The 3D posture of the human body can be accurately reconstructed in combination with a deep learning neural network, and real-time processing and analysis are carried out. The method not only can realize accurate motion capture, but also can be combined with fitness motion scoring and guidance, is widely applied to the fields of virtual fitness, exercise rehabilitation and the like, and has great application potential.

The invention relates to a multi-source perception-based mechanical arm operation data acquisition system and method, the system comprises an isomorphic mechanical arm operation module, a multi-view visual image acquisition module, an optical motion capture module and a shielding detection and data fusion module, vision, pose and force sense data are cooperatively acquired through multiple sensors, and time-space consistency is synchronously kept by adopting hardware. The system has a multi-dimensional occlusion detection mechanism and can dynamically switch data processing strategies according to the occlusion degree: visual data is preferentially adopted when no occlusion exists, multi-sensor information is fused when partial occlusion exists, and force feedback compensation is predicted and introduced based on a dynamic model when complete occlusion exists. Physical feedback and visual perception are creatively combined, the problem of data missing of a traditional single sensor data acquisition system under the shielding condition is solved, more comprehensive and more reliable data support is provided for mechanical arm algorithm training and performance optimization, and the method is particularly suitable for complex operation scenes with shielding.

A method for generating augmented reality (AR) instructional content is disclosed. The method advantageously provides an AR graphical user interface for generating AR instructional content for performing a task from user-input text descriptions of the task. The method advantageously leverages generative artificial intelligence to enable a code-free and motion-capture-free experience for authoring the AR instructional content, including virtual avatar animations demonstrating performance of the steps of the task. Additionally, the method advantageously overcomes the contextual barrier by enabling the user to author context-aware AR instructions that understand the context and blend physical reality with virtual components.

The invention relates to the technical field of virtual reality, in particular to an immersive virtual character interaction method based on virtual reality, which comprises the following steps: acquiring voice information input by a user and action information acquired by action capture equipment, processing and analyzing the voice information and the action information, and on the basis of the obtained initial performance state information, acquiring a virtual character interaction result. Adjusting the initial expression state information under the condition that the association degree of the initial text information and the initial expression state information meets a preset association degree standard, and adjusting the initial text information of the user according to the initial expression state information of the user to obtain target pronunciation information; therefore, the interactive voice and action represented by the virtual character better meet the requirements of the user, and the immersion of the user is improved. According to the method, the problem that the immersive experience of a user is poor due to the fact that the traditional virtual interaction method lacks recognition of postures of legs, feet and the like and the actions made by the user cannot be reflected in a virtual scene is solved.

The invention discloses an underground combined positioning method based on UWB and IMU, and the method comprises the following steps: arranging a UWB base station and a motion capture system in a positioning area, obtaining the distance between the base station and a label through UWB distance measurement, collecting IMU speed data through the movement of a person, recording the position of the label on the person through the motion capture system, and constructing a data set; constructing a positioning model based on the data set and the bidirectional long-short-term memory neural network; positioning the personnel through the positioning model to obtain a predicted positioning result; and fusing the predicted positioning result and IMU data through extended Kalman filtering to obtain a final positioning result. According to the invention, the accurate position of the underground coal mine personnel can be accurately positioned.

The invention discloses a dance movement digital acquisition method based on motion capture and a teaching system, and solves the problems of subjective evaluation of traditional teaching, neglect of posture difference in the prior art and the like. The motion capture module collects posture and motion data of a dance actor, decomposes the data into basic motions and extracts features to form a standard sequence, generates a personalized teaching evaluation sequence in combination with a student posture feature difference, collects student motions in real time and compares the student motions with the standard, and generates a targeted teaching strategy according to the difference. The system comprises a cloud data server, a motion capture module and the like, and can store standard data, analyze motion characteristics and perform interactive feedback. According to the invention, through accurate capture, personalized evaluation and real-time feedback, the teaching accuracy and efficiency are improved; and data support is provided for dance innovation.

There is provided an information processing device, an information processing method, and a program that improve detection accuracy in detecting a motion or posture of a person or an object using an inertial sensor. A posture of a measurement target is estimated based on a position and orientation of each of a plurality of measurement devices arranged at a plurality of parts of the measurement target and based on a connection relationship between the measurement devices that is set based on positions and orientations of the measurement devices, each measurement device including an inertial sensor. The present technology can be applied to a motion capture system.