24 results about "Motion reconstruction" patented technology

The invention relates to the technical field of multi-modal large language model training, in particular to a prompt construction method and system for a multi-modal large language model, computer equipment and a medium. The method comprises the following steps: extracting a key frame set from an input video stream; executing a motion reconstruction process on the video stream to generate motion track information; and performing visualization processing on the motion track information to generate a track visualization graph. Performing space-time correlation coding on the key frame set and the motion track information to generate an enhanced key frame; a multi-modal prompt is constructed in a mode of integrating visual input and text input, and the multi-modal prompt is input into a preset multi-modal large language model for spatial reasoning. Through the mode, the technical problem that an existing prompting method is difficult to give consideration to the spatial reasoning precision and the calculation efficiency is solved, efficient and accurate spatial reasoning of the multi-modal large language model is achieved, and the calculation efficiency, the reasoning precision and the environmental adaptability of the model are improved.

The invention belongs to the technical field of automatic driving automobile testing, and particularly relates to an automatic driving collision danger scene generation method based on reverse motion reconstruction. The method comprises the following steps: firstly, modeling a vehicle running state, and modeling a vehicle reverse motion process into a mathematical expression form; a starting point stable driving state set is established, and stable state constraints are defined; 3, determining sampling process constraints, introducing vehicle dynamics and a road model, and preventing vehicle motion from not conforming to physical constraints; and 4, converting the generated interaction track into a scene description form. The result of the invention can help an enterprise to establish a set of efficient collision dangerous scene generation system, improve the dangerous scene generation efficiency, reduce the computing power waste, finally establish a complete dangerous scene database, and accelerate the test verification of the autonomous vehicle.

This specification discloses a method and system for correcting scanned data. The method includes: acquiring multiple motion vector fields corresponding to multiple motion time points of a target object based on reference time points; acquiring image motion deviations corresponding to each motion time point based on the motion vector fields corresponding to each motion time point and the motion reconstruction images corresponding to each motion time point; acquiring raw data deviations corresponding to each motion time point based on each image motion deviation; and correcting the scanned data based on the raw data deviations corresponding to at least one of the motion time points to obtain corrected data.

This invention discloses a LiDAR human motion capture method based on Bézier curve degradation modeling, comprising: acquiring a continuous multi-frame LiDAR point cloud sequence; constructing a trajectory-aware Bézier motion degradation module to fit the original joint trajectory and gradually reduce control points through a trajectory preservation strategy to generate a multi-level motion representation from coarse to fine; designing a progressive motion reconstruction module, using a multi-timescale motion transformer (TMT) to predict Bézier motion curves at multiple time scales based on point cloud features, and using a multi-level motion aggregator (MMA) to adaptively fuse the multi-scale curves to reconstruct a detailed and temporally coherent 3D human posture sequence. This invention effectively alleviates the posture jitter or failure problem caused by occlusion, noise, and sparse point clouds, significantly improving the accuracy and temporal continuity of motion capture, and is suitable for complex open scenarios such as autonomous driving and robotics.

The invention discloses a character animation key frame simplification method and system based on visual saliency, and relates to the technical field of image processing, and the method comprises the steps: obtaining a character animation sequence, and extracting a position path and a rotation path of each skeleton node in the character animation sequence in a time dimension; calculating a first motion feature sequence based on the position path of each skeleton node; calculating a second motion feature sequence based on the rotation path of each skeleton node; combining the first motion feature sequence and the second motion feature sequence of each skeleton node into a third motion feature sequence; and selecting a target key frame sequence from the original time points based on the third motion feature sequences of all skeleton nodes by taking minimization of a motion reconstruction error of the role animation sequence as an optimization target, and outputting the target key frame sequence. Through feature extraction and fusion combination of the position path and the rotation path, accurate screening of animation key frames is realized, redundant frames are reduced, and data compression efficiency and animation coherence are improved.

The invention provides a low-complexity human body motion reconstruction method based on a sparse inertial measurement unit, and belongs to the technical field of virtual reality and three-dimensional human body motion reconstruction, and the method comprises the following steps: obtaining a data set required for reconstruction training based on human body postures, carrying out time modeling through a time sequence encoder, updating joint features on a human body skeleton graph, and obtaining a reconstruction result; a skeleton is divided into a trunk and four limbs according to a human anatomical structure, global rotation of a root joint and local rotation of each joint are respectively predicted by a partition kinematics regression head, low-rank decomposition is introduced into a large-scale linear layer to compress model parameters, and forward kinematics is utilized to recover three-dimensional joint positions of the whole body. In the training process, a two-stage teacher-student distillation model framework is adopted, a teacher network is trained through real labels, and then joint rotation and joint positions output by a teacher are used as soft targets to jointly restrain a student network through rotary distillation and position distillation. While the parameter quantity is reduced, the reconstruction precision and the motion smoothness of the whole body are improved.

The application discloses a role animation key frame simplification method and system based on visual saliency, and relates to the technical field of image processing, comprising: acquiring a role animation sequence, extracting the position path and rotation path of each bone node in the role animation sequence in the time dimension; calculating a first motion feature sequence based on the position path of each bone node; calculating a second motion feature sequence based on the rotation path of each bone node; combining the first motion feature sequence and the second motion feature sequence of each bone node into a third motion feature sequence; and selecting a target key frame sequence from the original time points based on the third motion feature sequence of all bone nodes, with the optimization target being to minimize the motion reconstruction error of the role animation sequence, and outputting the target key frame sequence. Through the feature extraction and fusion combination of the position path and the rotation path, the application realizes accurate screening of animation key frames, reduces redundant frames, and improves data compression efficiency and animation coherence.

The application discloses a speech-driven 3D face animation method based on multi-modal synchronous alignment, relates to the technical field of computer graphics and deep learning, and comprises a priori guide-based emotion-content decoupling strategy, content features and emotion features of an audio signal are acquired, a grid refinement module based on a Transformer architecture is used to generate a time-sequentially coherent fusion deformation coefficient sequence by using multi-modal features, the grid refinement module based on the Transformer architecture is used to directly predict vertex positions of a face grid by using the fusion deformation coefficient, a grid sequence is obtained, and a lip reading module is used to reconstruct content features from mouth region motion, and a 3D face animation is generated by using cross-modal consistency constraints and hierarchical perception reconstruction constraints. Therefore, the speech-driven 3D face animation method based on multi-modal synchronous alignment can realize high-quality face animation which is expressive, semantically consistent and visually coherent.

The invention provides a human body motion capture system based on posture coordinates, and belongs to the technical field of human body motion capture, the system collects human body motion data through a sensor collection module, and carries out data denoising, error correction, gravitational acceleration removal and other processing on the collected data through a data processing module; the method comprises the following steps: optimizing a human body motion data acquisition effect, obtaining human body motion data close to reality, extracting human body motion characteristics based on the obtained human body motion data close to reality, carrying out classification identification on motions in combination with a support vector machine, realizing human body motion capture, and then carrying out reproduction through a motion 3D reproduction module. Therefore, the human body motion data can be preprocessed in time, the capturing effect is better, key information is prevented from being lost, and the key information can be visually reflected.

The invention relates to a heart motion estimation method and device based on sparse key point driving and a medium, and the method comprises the steps: respectively inputting a moving image and a fixed image into a key point detection network with time-space consistency, and carrying out the extraction to obtain a key point pair; wherein in the key point detection process, a local correlation graph is generated according to normalized cross-correlation of key points in a fixed image and key points in a moving image, and after the local correlation graph is converted into a probability distribution graph, the positions of the key points in the moving image are corrected; and constructing a sparse basic motion vector according to the relative motion between each pair of key points to obtain a sparse motion vector expressed by heart motion, and performing dense motion reconstruction on the moving image by using a deep neural network according to the sparse motion vector to generate a dense motion field. Compared with the prior art, the method has the advantages of being high in structure retentivity, high in individual generalization and high in heart motion detail modeling capacity.

The invention discloses an engineering field three-dimensional reconstruction physical scale estimation method with reference to a standard component, and belongs to the technical field of building engineering digitization, three-dimensional reconstruction and intelligent construction, and the method comprises the steps: constructing a three-dimensional reconstruction physical scale estimation platform; a multi-view image sequence containing a standard component is collected, a scale-free three-dimensional point cloud and a camera pose are obtained through structure self-motion reconstruction SfM and multi-view stereo matching MVS reconstruction, a point cloud subset of the standard component is detected and positioned, a reconstruction scale size is obtained through geometric fitting, a scale factor is calculated in combination with a real physical size, and a three-dimensional point cloud is obtained. And carrying out global scaling on the scale-free model to finally obtain a real physical scale three-dimensional model. The method is suitable for various image acquisition devices and construction scenes, the scale recovery precision reaches the millimeter-to-centimeter level, the robustness is high, and the method can be widely applied to engineering measurement, BIM model alignment, digital twinning construction and other scenes and has remarkable engineering application value.

This application provides a method and apparatus for full-body motion capture based on monocular half-body video. The method first acquires a monocular half-body video to be processed, which includes the visible upper body area of ​​the target object. Then, the monocular half-body video is serialized to obtain a sequence of data. Finally, the sequence of data is input into a full-body motion capture model to obtain the full-body motion posture data of the target object. The full-body motion capture model is determined based on multiple monocular half-body videos and the corresponding labeled full-body motion posture data for each monocular half-body video. The method of this application accurately captures the half-body movements of the target object in a monocular half-body video scenario using a full-body motion capture model, and achieves high-precision full-body motion reconstruction.

This invention discloses a multi-axis cooperative motion control system and method for industrial robots, relating to the field of robot control technology. The system includes a data acquisition module, a machining constraint processing module, a constraint matching module, a conflict relationship generation module, a constraint arbitration module, and a motion reconstruction module. The data acquisition module acquires the machining trajectory, tool status, axis motion status of each joint axis, and current machining segment number in the surface grinding machining program. The machining constraint processing module generates a set of machining constraints including maintenance-type machining constraints, amplitude-limiting deviation-type machining constraints, substitution adjustment-type machining constraints, and axis motion boundary constraints. The constraint matching module calculates the execution requirements and matches them with the axis motion boundary constraints. The conflict relationship generation module generates a machining constraint conflict relationship table. The constraint arbitration module generates machining constraint arbitration results. The motion reconstruction module generates local multi-axis motion reconstruction instructions based on these results. The system can transform machining constraint conflicts in surface grinding into executable multi-axis motion control results.

The application relates to the technical field of motion capture, and discloses a motion capture anti-reflection identification method based on a ring-shaped sticker mark in an ice-water mixed environment. The method first arranges a ring-shaped sticker mark with a three-layer concentric ring structure on the surface of measured floating ice, and arranges multiple high-speed cameras in a high-position overhead manner and synchronously collects images; after pre-processing the collected images, candidate targets are obtained through area and roundness double-threshold screening; the candidate targets are verified based on the fixed interval geometric constraint of double mark points on the floating ice, and interference signals are removed through time sequence continuity filtering, so that real mark points are finally obtained for three-dimensional motion reconstruction. The application can effectively suppress mirror surface highlight interference in the ice-water mixed environment, improve the mark point identification accuracy and anti-interference ability, and is suitable for high-precision motion measurement in polar scientific investigation, ice area ship testing and other scenes.

This application relates to the field of motor reconstruction technology, specifically disclosing a closed-loop motor reconstruction system and method. The system includes an implantable signal acquisition component, an implantable controller, a non-implantable controller, and an implantable neurostimulation component. The implantable signal acquisition component and the implantable neurostimulation component are electrically connected to the implantable controller, which is also electrically connected to the non-implantable controller. The implantable signal acquisition component is configured to acquire electroencephalogram (EEG) signals. The implantable controller is configured to transmit the acquired EEG signals to the non-implantable controller and receive stimulation commands generated by the non-implantable controller based on the EEG signals, then transmit the stimulation commands to the implantable neurostimulation component. The implantable neurostimulation component is configured to parse the stimulation commands and perform electrical stimulation on the nerve roots at the target location. It requires no manual or audio control from the patient, is relatively simple to use, and is real-time, reducing computational complexity while ensuring accuracy.

The embodiment of the application discloses a two-order end memory and a video anomaly detection device for video anomaly detection, which comprises a first-order network and a second-order network; the first-order network comprises a first-order encoder, a motion memory and a first-order decoder connected in sequence, is used for motion reconstruction on a motion part, and obtains a motion prototype; the second-order network comprises a forward process network and a reverse process network connected in series; the forward process network is used for realizing a forward process, and the forward process network comprises a second-order encoder and a second-order decoder, which are used for predicting a predicted future frame of an input video; the reverse process network is cascaded with the forward process network, realizes a reverse process, and reconstructs a predicted initial frame of the input video; the predicted future frame and the predicted initial frame are used for unsupervised anomaly detection, and an anomaly detection result of the input video is output. The two-order end memory is complete in motion representation, fully utilizes the bidirectional consistency of the video, and improves the accuracy of anomaly detection.

The invention relates to a monocular video multi-person 3D human body motion reconstruction method based on multi-module fusion, and solves the defect that discontinuous or unstable reconstruction is generated when a target appears again due to the fact that tracking is lost when the target is partially shielded or moves rapidly compared with the prior art. The method comprises the following steps: acquiring a single-frame or multi-frame motion video sequence; constructing a motion perception semantic tracking module; generating a coherent human body grid sequence; predicting future attitude features in the occlusion scene; and outputting the 3D motion state. According to the invention, 3D motion reconstruction with robust shielding, stable identity and consistent time sequence is realized through collaborative design of motion perception tracking, time sequence enhancement reconstruction, motion prediction and adaptive fusion.

The invention belongs to the technical field of digital cultural heritage protection, and discloses a deep learning-based non-abandoned skill motion capture and digital reproduction method, which comprises the following steps of: obtaining original motion data and carrying out multi-modal fusion on the original motion data to obtain fused motion data; constructing a three-level action characteristic spectrum based on the fused action data, and constructing a fine action key frame sequence for a characteristic sequence in a fine operation layer in the three-level action characteristic spectrum; the key frame sequence is optimized in combination with time continuity and space coordination constraint, and enhanced action feature representation is obtained; semantic annotation and digital action reconstruction are carried out on the enhanced action feature representation; according to the method, the fidelity of skill details and the interpretability of cultural semantics are remarkably improved, and high-quality technical support is provided for non-abandoned inheritance and digital protection.

The present application belongs to the technical field of automatic driving car testing, and in particular to an automatic driving collision danger scene generation method based on reverse motion reconstruction. First, the vehicle operating state is modeled, and the reverse motion process of the vehicle is modeled into a mathematical expression form. Then, a starting point stable driving state set is established, and the smooth state constraint is clarified. The third step is to clarify the sampling process constraint, introduce the vehicle dynamics and road model, and avoid the vehicle motion not conforming to the physical constraint. The fourth step is to convert the generated interaction trajectory into a scene description form. The present application can help enterprises establish an efficient collision danger scene generation system, improve the efficiency of danger scene generation, reduce the waste of computing power, and ultimately establish a complete danger scene database to accelerate the testing and verification of automatic driving cars.

The present application relates to the technical field of flexible riser motion analysis, and particularly relates to a marine dynamic flexible riser motion response reconstruction method based on monitoring data, comprising: using a structural embedded layout method or an external attached layout method to layout sensors in a flexible riser system, and optimizing the sensor layout mode; setting a data acquisition and transmission system on a marine platform to obtain flexible riser motion response data; converting the flexible riser motion response data into flexible riser displacement data based on a Timoshenko beam model and Tikhonov regularization; carrying out noise reduction processing on the flexible riser displacement data based on a wavelet threshold noise reduction method to obtain flexible riser displacement noise reduction data; and carrying out motion response reconstruction on the flexible riser displacement noise reduction data based on variational mode decomposition VMD. The present application can effectively improve the data interpretation, data reconstruction accuracy and real-time performance of motion reconstruction of the flexible riser.

The application relates to a kind of manta ray submarine gliding propulsion kinematics modeling and numerical calculation method;Belong to underwater bionic robot technical field.This method first carries out feature point marking and three-dimensional motion reconstruction to real manta ray biology, extracts its kinematics parameter, and then establishes a kind of unified kinematics equation which strictly satisfies the constraint of body length invariable, can accurately couple the deformation of pectoral fin spanwise and chordwise;By adjusting specific parameters, the mathematical description of multiple real motion modes such as flapping, gliding, gliding and compound, and maneuvering turn can be realized.Secondly, the high-fidelity kinematics model dynamic boundary input condition is combined with the immersed boundary method and the computational fluid dynamics solver to efficiently and stably calculate the hydrodynamic performance and flow field structure of the manta ray submarine in the complex motion process.The application effectively solves the problem that the simulation result deviates from the actual situation and it is difficult to capture the key vortex dynamics structure caused by the distortion of the kinematics model of the prior art.

This invention discloses a method for generating 3D human motion based on a parallel multi-granularity Transformer, comprising: acquiring a 3D human motion dataset; constructing an initial model, which includes a parallel multi-granularity motion generation module, a multi-granularity motion fusion module, and a motion reconstruction and output module; and training the initial model using the 3D human motion dataset to obtain an image generation model for generating high-fidelity 3D human motion sequences. This invention also provides a 3D human motion generation device. The method provided by this invention can generate high-fidelity, semantically coherent, and detail-rich 3D human motion sequences.

The application provides a flexible wing deformation detection and motion reconstruction method and device based on binocular vision, which comprises a binocular vision detection method, an adaptive KCF tracking algorithm, a binocular camera synchronous correction method and an experimental device. The binocular vision detection method can reconstruct a motion model of the flexible wing and detect the deformation of the wing. The adaptive KCF tracking algorithm can stably track and locate the spatial position of the whole and local area of the moving wing. The synchronous error correction method can correct the time difference of the asynchronously triggered binocular cameras and realize the synchronization of the images shot by the binocular cameras. The experimental device comprises binocular cameras, a flexible wing, markers and a binocular synchronous correction device. The application improves the tracking accuracy of the high-speed moving target through the adaptive KCF tracking algorithm and the synchronous correction method, reduces the binocular camera synchronous error, improves the flexible wing deformation detection accuracy and motion model reconstruction accuracy, and reduces the cost of the measuring equipment.

This invention discloses a method for reconstructing the motion of a virtual sign language human based on prior decoding and posture smoothing. The process is as follows: First, a monocular sign language video sequence is acquired; then, the sign language video sequence is processed by a component-based perceptual encoder to obtain hand and facial feature maps, as well as body posture parameters, body shape parameters, and camera parameters for each frame; next, the hand and facial feature maps are processed by a prior decoder to obtain left-hand, right-hand, and facial posture parameters; subsequently, an initial whole-body posture parameter sequence is constructed using the body posture parameters, left-hand posture parameters, and right-hand posture parameters, and smoothed using a posture smoothing filter to obtain a smoothed whole-body posture parameter sequence; finally, the virtual sign language human is reconstructed based on the smoothed whole-body posture parameter sequence, body shape, camera parameters, and facial parameters. This invention can improve the stability and practicality of motion reconstruction of virtual sign language humans under monocular video conditions.