1197 results about "Motion capture" patented technology

The invention provides a rehabilitation training action evaluation method and device based on multi-view vision. According to the method, a multi-view camera is adopted to synchronously collect a rehabilitation training image sequence, two-dimensional coordinates of key points of a human body are extracted by improving an HRNet deep learning model, three-dimensional reconstruction is performed in combination with a Gaussian process, motion feature data are extracted, and rehabilitation training quality is evaluated. Accurate motion capture without wearing mark points by the patient is realized, the training constraint feeling of the patient is effectively reduced, the rehabilitation evaluation accuracy is improved, and a personalized rehabilitation scheme can be generated.

The invention relates to the technical field of power system operation training, and discloses a virtual reality-based power grid fault simulation training system and method, and the system comprises a virtual scene modeling module which is used for constructing a three-dimensional model of power grid equipment and an operation environment thereof, and providing a vivid training environment for subsequent modules; and the interaction and operation module is connected with the virtual scene modeling module and realizes interaction between the trainee and the virtual environment by using a VR head-mounted display, a handle and a motion capture device. By adopting a high-precision three-dimensional modeling technology, PBR material rendering and an LOD optimization strategy, high simulation of power grid equipment and an operation environment is realized, students can experience a vivid visual effect and immersion in a virtual reality scene, and meanwhile, by combining a gesture recognition technology realized by motion capture equipment and a convolutional neural network, the real-time simulation of the power grid equipment and the operation environment is realized. Operation actions of trainees can be accurately captured, a natural and visual interaction mode is provided, and the authenticity and operation experience of the training system are greatly improved.

The invention provides a high-precision motion capture real-time calibration method based on multi-sensor fusion, and the method comprises the steps: obtaining motion data and environmental parameters collected by multiple sensors, the multiple sensors comprising an inertial measurement unit and an optical sensor; constructing a dynamic calibration model based on the action data and the environmental parameters, and performing real-time error compensation and fusion processing on the sensor data; and establishing a time-varying error dynamic model, and performing real-time compensation on zero offset drift, vibration noise and electromagnetic interference of the inertial measurement unit based on the environmental parameters. According to the invention, the IMU and the optical sensor are fused, respective advantages are exerted, and motion data are comprehensively collected. And meanwhile, the time-varying error dynamic model compensates errors generated by environmental interference in real time, so that real actions can be accurately restored through action capture.

Gait characterization and analysis based on a monocular video of a walking subject are achieved, in various embodiments, by processing the video using one or more machine-learning models to predict three-dimensional (3D) keypoint coordinates for a set of anatomical keypoints of the subject, along with joint angles, body-segment rotations, and / or gait event classifications, for multiple video frames. The machine-learning models may be trained using ground-truth data acquired with a marker-based motion capture system and / or pressure-sensitive walkway.

The invention relates to a light control method and system based on a holographic interactive projection technology, and the method comprises the steps: projection region recognition, user interaction detection, light parameter calculation and light real-time regulation and control, the projection region recognition, scanning of a target region through a holographic projection device and a depth induction sensor, and generation of a three-dimensional space model of an environment; a motion capture device and a deep learning algorithm are used for carrying out real-time recognition on gestures or motions of a user, interaction intentions and priority ranking are generated, a holographic projection device is used for generating a three-dimensional environment model, the motion capture device recognizes the intentions of the user, and an illumination optimization model is used for calculating regulation and control parameters; and the output of the light equipment is dynamically adjusted, so that high-precision and intelligent light control is realized, and the problems that a traditional light control method is insufficient in environment perception capability, lacks a user interaction mode, is limited in flexibility and adaptability of light regulation and control, and lacks a multi-module cooperative working mechanism are solved.

The invention relates to the technical field of evaluation and management, in particular to a classroom learning dynamic evaluation and management system, which comprises a multi-source heterogeneous data acquisition module, an acoustic sensor array, an optical motion capture unit, an intelligent writing track acquisition device and a brain wave monitoring terminal, the data preprocessing and feature fusion module is used for eliminating environment interference signals and generating preprocessed data; the dynamic learning state evaluation module is used for extracting three-dimensional features and outputting a dynamic evaluation matrix; the group learning portrait generation module is used for generating a multi-dimensional learning portrait in combination with a preset knowledge graph topological structure; the self-adaptive management strategy generation module is used for dynamically generating a management strategy set; and the intelligent feedback execution module is used for driving the self-adaptive illumination system, the wearable vibration reminding device and the AR enhancement projection equipment to execute physical environment regulation and control. According to the invention, personalized strategy dynamic generation is realized, and adaptive regulation and control of the classroom environment and the teaching rhythm are realized in combination with the intelligent Internet of Things equipment, so that an intelligent collaborative feedback closed loop is formed.

The invention discloses a 500kV standardized switching operation method based on a VR virtual reality technology, and aims to solve the problems of unreal environment, incomplete evaluation and low efficiency in the switching operation training of an existing 500kV transformer substation. The method comprises the following steps: constructing a full-immersive virtual reality scene, collecting actual environment data of a transformer substation, and constructing and optimizing a high-precision model; designing multi-modal interaction of the standardized switching operation process, wherein the multi-modal interaction comprises visual guidance, a multi-modal interaction interface and an intelligent sensing function; dynamic action scoring and intelligent behavior evaluation are realized, actions are captured in real time by using high-precision action capture equipment, an instant score is generated by comparing with a standard, and dynamic feedback is provided; and real-time optimization and learning migration of the virtual character model are realized, user operation habits are captured, virtual character actions are optimized, and multi-scene migration and adaptation are realized. The method is suitable for VR-based 500kV standardized switching operation work, and the training authenticity and efficiency are improved.

The invention discloses an inspection robot collaborative planning method and system based on digital twinning, and the method comprises the steps: constructing a virtual simulation environment of a large-scale component through a point cloud modeling technology, and integrating a motion capture system to achieve the real-time mapping of a physical environment and a virtual model; defining a cooperative mechanism of the scanning robot and the positioning robot, providing global coordinates and path guidance by the positioning robot, and correcting position errors in real time by the motion capture system; simulating a dual-machine motion track in the digital twin model, and identifying a potential collision risk through a conflict prediction model; designing a collaborative planning algorithm under coupling constraint, and performing optimization by taking path optimization and scanning visual angle coverage as a joint target; based on a twin intelligent framework with a body, the path and the visual angle of the robot are dynamically adjusted by combining reinforcement learning with real-time feedback, and self-adaptive collaborative planning is achieved. Through double-machine division cooperation and real-time correction, the scanning efficiency and precision of the large-size component can be remarkably improved, and the collision risk is reduced.

The invention relates to the technical field of physical education, in particular to a physical education method and system based on multimedia and holographic AR fusion, and the method comprises the following steps: collecting the motion data of a trainee based on a multi-source sensor array, constructing a motion parameterized model, and generating a holographic projection generation module through the cooperation with a standard motion database; a holographic AR teaching image is generated in real time, an environment perception module is combined to carry out adaptive projection on illumination and spatial layout, a spatial positioning module is used to establish coordinate mapping between a trainee and a virtual image, and a multi-channel feedback device is started to output visual, tactile and auditory signals. And finally, comparing the action parameters with the standard action data to automatically generate a training report containing wrong action labels and correction suggestions. According to the method, the motion capture precision and the interactive immersion can be effectively improved, and efficient and extensible technical support is provided for modern physical education and training.

The invention discloses a reference camera device and system based on a motion capture system, and the device comprises a camera connection module which is used for building the signal connection between a reference camera and the motion capture system, and outputting a successful connection result of the reference camera after the successful connection; the synchronous calibration module is used for synchronously calibrating the reference camera and the motion capture camera and outputting a calibration success result after calibration succeeds; the projection module is used for projecting the to-be-measured object in the three-dimensional space to the reference camera for two-dimensional display to obtain virtual scene information; the virtual-real superposition module is used for superposing the virtual scene information and an image actually shot by the reference camera to judge whether the motion capture effect is correct or not, and if the motion capture effect is correct, outputting a motion capture effect correct result; and the data recording module is used for judging whether a recording button in the motion capture system is clicked or not, and if the recording button is clicked, recording to obtain recorded video data information and caching the recorded video data information. The method has the effect of improving the accuracy of virtual and real scene comparison during shooting of the reference camera based on the motion capture system.

Methods and systems for human gesture recognition and activity detection using augmented data. A computer-implemented method includes receiving motion capture data of a target from a camera, generating a first set of motion trajectories from the motion capture data, generating a first set of augmented motion trajectories using a set of data augmentation functions on the first set of motion trajectories, generating a radar cross-section of the target using the motion capture data to perform at least one of gesture recognition or activity detection, generating one or more synthetic electromagnetic (EM) signatures of one or more activities of the target using the first set of augmented motion trajectories and the radar cross-section, and training a machine learning model configured for EM signature-based gesture recognition or activity detection with a domain adaptation process using the one or more synthetic EM signatures.

The invention relates to the technical field of motion capture data processing, and discloses a robot reinforcement learning model construction method based on optical inertial fusion whole body motion capture, and the method comprises the steps: measuring and recording a plurality of key parameters of a target human body; carrying out primary calibration and recording on all inertial sensors worn on a target human body, and calibrating all cameras arranged around at the same time; in the process that the target human body executes the specified action, synchronously acquiring action sampling data of the target human body; the action sampling data comprises motion data corresponding to the inertial sensor and camera image data corresponding to the camera; the motion data and the camera image data are processed, fused and optimized, and optimized joint posture quaternary data and human body motion sequence data are obtained; and inputting the human body action sequence data into the reinforcement learning model of the humanoid robot, and configuring the preset reward function to perform reinforcement learning training, so that the learning efficiency of the robot and the accuracy of action execution can be improved.

The invention relates to a multi-person motion capture real-time digital human driving method based on a skeleton-muscle interaction model, and belongs to the field of artificial intelligence and ubiquitous universe. According to the method, the key points of the human body are detected through YOLO-Pose, the positions of the shielding points are obtained through a double-stage dynamic shielding point compensation method, and the accuracy of the key points is guaranteed. And outputting the three-dimensional coordinates of the key points of the human body by adopting a three-dimensional key point depth extractor based on multi-dimensional exclusive-shared feature reasoning in combination with an MT ransPose model. A real person-skeleton model-digital person real-time mapping method is provided, three-dimensional key points are mapped to a standard human body skeleton model, and a digital person role and the skeleton model are bound through a weight mapping algorithm. Based on a skeleton-muscle interaction model, the digital person accurately reflects a real motion posture by introducing the mechanical action of a skeleton rigid body dynamic model and a muscle force line model, and the process of driving the digital person in real time through multi-person motion capture is achieved.

The invention provides an immersive concert experience system based on virtual reality, and belongs to the technical field of virtual reality, and the system comprises a dynamic adjustment module which is used for predicting the preference of a user for stage arrangement, light effects and music types through a machine learning model according to the historical preference data of the user, and outputting the preference to the stage arrangement, light effects and music types; dynamically adjusting visual elements and sound field parameters of the virtual scene and a refresh rate and a field angle of the virtual scene in combination with the user physiological indexes and the user head motion trail which are acquired in real time; the action capture and feedback module is used for capturing an interaction instruction sent to the dynamically adjusted virtual scene by the limb action of the user in real time; and the interaction feedback module is used for enabling the virtual performer to respond to the interaction instruction of the user in real time based on the expression recognition and voice synthesis technology, and generating personalized interaction feedback content. A user constructs an immersive music interaction field domain, and experience logic of a virtual music meeting is completely reconstructed.

The invention discloses a robot joint control method and system based on motion capture equipment, and belongs to the technical field of robot control. An existing robot joint control scheme lacks an effective man-machine cooperation and natural interaction mechanism, so that the flexibility of a robot is poor, and the intention of an operator cannot be naturally expressed. According to the robot joint control method based on the motion capture equipment, by constructing a motion capture module, a data processing module, a coordinate system transformation module and a solution optimization module, rotating posture data, collected by the motion capture equipment, of human body joints are converted into target joint control information needed by execution of a robot; the robot joint control based on the motion capture equipment is realized, so that the human motion can be efficiently and accurately mapped to the robot, the robot is good in flexibility, sudden motion or complex cooperative operation can be effectively processed, the intention of an operator can be naturally expressed, and man-machine cooperation and natural interaction are realized.

The invention discloses a scene interactive limb active and passive rehabilitation training system based on a virtual three-dimensional game in the field of rehabilitation medical treatment. The scene interactive limb active and passive rehabilitation training system comprises a virtual scene generation module, a multi-mode motion capture module, an active and passive cooperative control module, a real-time feedback and excitation module and a data statistics module. And the virtual scene generation module is used for constructing various three-dimensional game scenes, so that scene tasks are strongly associated with actions of upper limbs, lower limbs or upper and lower limbs, and scene difficulty is dynamically adjusted. Compared with the prior art, the interactive guidance of the immersive virtual three-dimensional game scene is adopted, the boring rehabilitation training is converted into interesting game experience, and the participation enthusiasm and compliance of the patient are greatly improved. Compared with traditional two-dimensional animation game training, the patient is more willing to actively participate in training, and the conflict emotion caused by boring training is reduced.

The invention provides a motion capture and feedback adjustment method and system for smart motion of meta universe, and relates to the technical field of meta universe, and the method comprises the steps: collecting actual motion data of a user through a plurality of inertial measurement units, establishing a mapping relation with skeleton nodes, evaluating and optimizing the motion based on a multi-dimensional scoring matrix, and constructing a user virtual image. And performing time sequence analysis on the virtual motion data to obtain local and global motion features, comparing the local and global motion features with standard motion features to generate an adjustment amount, and converting the adjustment amount into a motion correction instruction to guide a user to adjust a posture, so that accurate capture and real-time feedback can be realized, and a motion learning effect is improved.

The invention relates to a motion capture and evaluation method and device, equipment and a medium. The method comprises the following steps: acquiring motion capture data; repairing the motion capture data to obtain standardized motion data; obtaining a virtual character animation file based on the standardized action data and a virtual character skeleton model; performing multi-dimensional evaluation on the virtual character animation file to generate a comprehensive evaluation report; and adjusting the virtual character animation file based on the comprehensive evaluation report to obtain a final animation file. By adopting the method, the motion capture animation can be automatically evaluated and corrected, and the effect of improving the working efficiency is achieved.

A user workload state processing method and apparatus based on multimodal data, and a device. The method comprises: acquiring state information of a first user; determining workload information of the first user on the basis of the acquired state information, wherein the state information comprises at least two of the following information: physiological information, eye movement information, electroencephalogram information, brain function imaging information, motion capture information, spatiotemporal acquisition information, behavior acquisition information, and facial expression and state information; and feeding back the workload information to a first management account, and formulating a training scheme matched with the workload information. According to the embodiments of the present application, workload of users can be identified on the basis of multimodal data of users.

The invention discloses an intelligent motion capture system and method of a multi-mode sensor, and the method comprises the steps: creating and installing links among an optical module, an inertia module, a touch module, a synchronization core module and a to-be-detected object; sending a synchronous starting pulse signal based on the synchronous core module, setting the synchronous starting pulse signal as a clock reference starting point, and starting the optical module, the inertia module and the touch module based on the clock reference starting point; performing data acquisition on a to-be-measured object based on the optical module, the inertia module and the touch module to obtain an object acquisition information data set; and creating an acquisition original timestamp based on the synchronous start pulse signal, creating a sensor network delay model to dynamically adjust the acquisition original timestamp to obtain a calibration timestamp, and analyzing and correcting the time offset of the object data acquisition information. The method and the device have the effect of improving the accuracy of intelligent motion capture of the multi-mode sensor.

The invention discloses a motion capture system, a motion capture method and a readable storage medium, and belongs to the technical field of motion capture. The motion capture system comprises a motion capture sensor group and a motion capture processing unit, the motion capture sensor group comprises at least one invisible light image sensor and at least one visible light image sensor, the invisible light image sensor is used for sensing a detected target in a near-infrared or infrared band, and the visible light image sensor is used for sensing the detected target in a near-infrared or infrared band. The visible light image sensor is used for sensing a detected target in a visible light wave band and outputting a visible light image; and the motion capture processing unit is connected with the motion capture sensor group and is used for determining motion posture data of the detected target based on the visible light image and the near-infrared or infrared band image. According to the invention, the motion attitude data of the measured target is obtained based on the sensing data of different modes, and the precision and robustness of motion capture of the measured target by the motion capture system are improved.

The invention relates to the technical field of image recognition, in particular to an AI-based park intrusion detection and response system, which comprises a target trajectory acquisition module, an event density analysis module, a disturbance behavior recognition module and a response joint control execution module. According to the invention, through fusion of the spatial-temporal characteristics of the target trajectory and motion vector pattern recognition, intrusion behavior dynamic risk assessment and early warning collaborative optimization is realized, and through combination of multi-dimensional data association calculation and event density increment monitoring, abnormal situation sensing sensitivity and discrimination accuracy in a complex scene are improved. Motion vector time sequence modeling and disturbance feature extraction are utilized, the capacity of capturing camouflage behaviors and irregular motion is enhanced, an environment perception and response execution closed-loop control link is constructed by means of intelligent matching and parameter self-adaptive adjustment of multi-source sensing equipment, the dependency degree of manual review is remarkably reduced, the delay from recognition to treatment response is shortened, and the recognition efficiency is improved. And an early warning research and judgment and linkage disposal full-process automation system is formed.

The invention provides a training and examination robot system based on a multi-modal large model, equipment and a computer program product, and aims to accelerate training transformation and upgrading and improve coping ability as basic appeals, and based on regular training work of teams, training and examination robot, sensor acquisition equipment and big data platform three-terminal equipment. A multi-modal data acquisition layer for realizing real-time attitude detection and motion capture by using an edge calculation algorithm, a data processing and analysis layer for finishing real-time image processing through localized edge calculation in a network-free or power-free environment, and system control functions for finishing intelligent decision making, evaluation, error correction and the like are provided; an intelligent decision-making and control layer of a training subject rewriting decision-making mechanism is established, cross-modal collaborative learning middleware is utilized to realize modal representation consistency while training subject transposition evaluation is realized, training and assessment operations of the same subject of multiple persons or different subjects of multiple persons are supported, and the training and assessment efficiency is remarkably improved.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

The invention provides a calibration method and device of motion capture equipment, equipment and a medium, the motion capture equipment is worn on a limb of a human body, the method is applied to head-mounted display equipment, the head-mounted display equipment comprises a camera, and the method comprises the following steps: collecting an image of the motion capture equipment through the camera; according to the motion capture equipment image, determining target attitude information of an equipment coordinate system relative to a world coordinate system; and calibrating the motion capture equipment according to the target attitude information. The motion capture equipment is calibrated by using the image of the motion capture equipment, so that the calibration difficulty of the motion capture equipment can be effectively reduced, the calibration time can be shortened, the calibration error can be reduced, and the calibration precision of the motion capture equipment can be improved.

The invention belongs to the technical field of computer vision, and discloses a human motion capture method based on mask perceptual graph convolution and skeleton prior, and the method comprises the steps: obtaining an input image and a corresponding human mask; extracting features of the image and the mask and coding; constructing a mask perception graph convolutional network, constructing an adjacent matrix of the graph convolutional network by using mask information, and applying mask constraint loss to enhance the expression ability of the human body region features of the image; a skeleton prior decoupling network is constructed, skeleton and vertex information of an SMPL model is used as prior, a cross attention mechanism is combined, and multi-modal data enhancement of SMPL skeleton node features is guided through image features; and finally outputting a three-dimensional joint position coordinate and a shape grid vertex coordinate. According to the method, the local feature consistency is enhanced through mask perception image convolution, the geometric consistency is improved in combination with skeleton prior, and the precision of three-dimensional human body motion capture under monocular vision is effectively improved.

The invention discloses a human motion optimization method and system for a multi-lens video, a terminal and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: performing shot switching point detection and multi-level segmentation on a multi-shot video frame sequence to obtain a target video frame sequence; calculating a target camera track and a human body motion parameter according to the target video frame sequence; and obtaining a target human body motion parameter of the multi-shot video frame sequence based on the target camera track and the human body motion parameter. The method comprises the following steps: firstly, segmenting a multi-shot video frame sequence into a plurality of target video frame sequences without shot switching points according to the shot switching points; and calculating a target camera track and a human body motion parameter according to each target video frame sequence. And finally, the human body motion state in the video is analyzed by combining the target camera track and the human body motion parameters, so that target human body motion parameters with high precision, continuity and physical consistency are obtained. The method is suitable for various application scenes such as motion capture, virtual reality and sports analysis.

The invention discloses a human-machine motion mapping control system and method based on a hybrid LM-GA algorithm, and relates to the field of human-machine motion mapping control. According to arm joint position information obtained by an optical motion capture system, a corresponding arm joint rotation angle is obtained by using a geometric solution in inverse kinematics; secondly, joint angle data of the mechanical arm are obtained through an inverse kinematics iterative algorithm, and the joint angle data are verified in the virtual environment; then track data of the tail end of the mechanical arm is obtained through an optical motion capture system, the motion reproduction precision of the mechanical arm can be evaluated by comparing the track data of the tail end of the human arm and the track data of the tail end of the mechanical arm, and the accuracy and reliability of a human-machine motion mapping system are ensured; by continuously approaching the optimal solution, the iterative algorithm can gradually adjust the angle of the robotic arm joint until a predetermined end effector position is reached.

The invention relates to the technical field of image recognition analysis, in particular to an indoor safety monitoring system and method based on image recognition, and the system comprises a plurality of modules and a lower setting unit, an image collection module selects a high-frame-rate and high-resolution camera, and is connected with a rear-end processing device through a USB, Ethernet or Wi-Fi communication interface; according to the scheme, after an owner receives an alarm and checks monitoring content, the owner can manually cancel the alarm through visual judgment of a door condition, after the alarm is cancelled, the system automatically enters the dangerous action capture module, and the module focuses on danger assessment of personnel actions and temporarily ignores other processes such as personnel identity recognition within a certain time after being started, so that the system is more convenient to use. During the period, if a new person enters the monitoring area, the system only carries out dangerous action capture, such setting can filter a large amount of alarm information generated due to frequent person contact to a certain extent, reduce interference to owners, and ensure that potential dangerous actions are kept alert at the same time.

The invention relates to an all-humanoid robot motion redirection method and device based on motion capture equipment, and the method comprises the steps: converting a human body posture quaternion collected by the motion capture equipment into a rotation matrix, and achieving the precise mapping from a Cartesian space to a robot joint space in combination with an Euler angle decomposition strategy; a quaternion-disassembled joint angle is used as an initial solution, various physical constraints are introduced in real time according to the actual supporting state of the robot, and the lower limb joint angle is corrected online by constructing and solving an inverse kinematics problem with various constraints, so that the robot reproduces the action of an operator, and the joint angle of the lower limb can be corrected online. And the requirements of joint limiting, self-collision avoidance, dynamic balance and the like are strictly met, so that the accuracy of motion simulation and the stability of the system are effectively unified.