32 results about "Motion reconstruction" patented technology

The invention relates to the technical field of multi-modal large language model training, in particular to a prompt construction method and system for a multi-modal large language model, computer equipment and a medium. The method comprises the following steps: extracting a key frame set from an input video stream; executing a motion reconstruction process on the video stream to generate motion track information; and performing visualization processing on the motion track information to generate a track visualization graph. Performing space-time correlation coding on the key frame set and the motion track information to generate an enhanced key frame; a multi-modal prompt is constructed in a mode of integrating visual input and text input, and the multi-modal prompt is input into a preset multi-modal large language model for spatial reasoning. Through the mode, the technical problem that an existing prompting method is difficult to give consideration to the spatial reasoning precision and the calculation efficiency is solved, efficient and accurate spatial reasoning of the multi-modal large language model is achieved, and the calculation efficiency, the reasoning precision and the environmental adaptability of the model are improved.

The invention relates to the technical field of flexible riser motion analysis, in particular to a marine dynamic flexible riser motion response reconstruction method based on monitoring data, which comprises the following steps of: arranging sensors in a flexible riser system by adopting a structure embedded type arrangement method or an external attachment type arrangement method, and optimizing a sensor arrangement mode; a data acquisition and transmission system is arranged on an ocean platform to obtain flexible riser motion response data, the flexible riser motion response data is converted into flexible riser displacement data based on a Timoshenko beam model and Tikhonov regularization, the flexible riser displacement data is subjected to noise reduction processing based on a wavelet threshold noise reduction method to obtain flexible riser displacement noise reduction data, and the flexible riser displacement noise reduction data is subjected to flexible riser displacement noise reduction. And motion response reconstruction is performed on the flexible riser displacement noise reduction data based on variational mode decomposition (VMD). According to the method, the data interpretability, the data reconstruction precision and the real-time performance of motion reconstruction of the flexible riser can be effectively improved.

This invention relates to the field of intelligent reconstruction technology for 3D models, specifically a method and system for intelligent reconstruction of 3D building models based on deep learning. The method includes acquiring multi-source data of the target ancient building; generating a dense point cloud through motion reconstruction and multi-view stereo matching; identifying structural and non-structural components through semantic segmentation and reconstructing an initial mesh model with semantically labeled information; using a graph neural network to analyze the spatial topological relationships of structural components, and combining this with a historical construction knowledge base to generate hierarchical combination methods and connection constraint rules; adaptively adjusting the pose of mesh nodes according to the connection constraint rules to drive the precise assembly of parameterized components, and fusing them to obtain a structured 3D model of the target ancient building. This invention can restore the structural component connection relationships in hidden areas of ancient buildings under non-contact acquisition conditions, significantly improving the structural rationality and geometric accuracy of the 3D model.

This application discloses a real-time image stitching method and apparatus. The method includes: receiving images uploaded to the cloud by a camera device in real time; extracting feature points from the images and matching feature points between the current frame and the previous frame; calculating the ratio of the homography matrix score to the sum of the homography matrix score and the fundamental matrix score to determine whether to use the homography matrix or the fundamental matrix to recover the image's pose parameters and map points, and initializing the map; estimating the pose of the current frame and optimizing the pose of the current frame using candidate frames; converting the coordinates of the map points to two-dimensional coordinates and performing plane fitting; converting the coordinates of the image corner points in the camera coordinate system to the object coordinate system, calculating perspective transformation parameters and performing geometric transformation on the image; and performing Gaussian pyramid fusion on the generated tiles. This method solves the problems of low efficiency in scene reconstruction using motion reconstruction algorithms and the inability to process real-time photogrammetric data.

The invention belongs to the technical field of automatic driving automobile testing, and particularly relates to an automatic driving collision danger scene generation method based on reverse motion reconstruction. The method comprises the following steps: firstly, modeling a vehicle running state, and modeling a vehicle reverse motion process into a mathematical expression form; a starting point stable driving state set is established, and stable state constraints are defined; 3, determining sampling process constraints, introducing vehicle dynamics and a road model, and preventing vehicle motion from not conforming to physical constraints; and 4, converting the generated interaction track into a scene description form. The result of the invention can help an enterprise to establish a set of efficient collision dangerous scene generation system, improve the dangerous scene generation efficiency, reduce the computing power waste, finally establish a complete dangerous scene database, and accelerate the test verification of the autonomous vehicle.

This specification discloses a method and system for correcting scanned data. The method includes: acquiring multiple motion vector fields corresponding to multiple motion time points of a target object based on reference time points; acquiring image motion deviations corresponding to each motion time point based on the motion vector fields corresponding to each motion time point and the motion reconstruction images corresponding to each motion time point; acquiring raw data deviations corresponding to each motion time point based on each image motion deviation; and correcting the scanned data based on the raw data deviations corresponding to at least one of the motion time points to obtain corrected data.

This invention discloses a LiDAR human motion capture method based on Bézier curve degradation modeling, comprising: acquiring a continuous multi-frame LiDAR point cloud sequence; constructing a trajectory-aware Bézier motion degradation module to fit the original joint trajectory and gradually reduce control points through a trajectory preservation strategy to generate a multi-level motion representation from coarse to fine; designing a progressive motion reconstruction module, using a multi-timescale motion transformer (TMT) to predict Bézier motion curves at multiple time scales based on point cloud features, and using a multi-level motion aggregator (MMA) to adaptively fuse the multi-scale curves to reconstruct a detailed and temporally coherent 3D human posture sequence. This invention effectively alleviates the posture jitter or failure problem caused by occlusion, noise, and sparse point clouds, significantly improving the accuracy and temporal continuity of motion capture, and is suitable for complex open scenarios such as autonomous driving and robotics.

The invention discloses a character animation key frame simplification method and system based on visual saliency, and relates to the technical field of image processing, and the method comprises the steps: obtaining a character animation sequence, and extracting a position path and a rotation path of each skeleton node in the character animation sequence in a time dimension; calculating a first motion feature sequence based on the position path of each skeleton node; calculating a second motion feature sequence based on the rotation path of each skeleton node; combining the first motion feature sequence and the second motion feature sequence of each skeleton node into a third motion feature sequence; and selecting a target key frame sequence from the original time points based on the third motion feature sequences of all skeleton nodes by taking minimization of a motion reconstruction error of the role animation sequence as an optimization target, and outputting the target key frame sequence. Through feature extraction and fusion combination of the position path and the rotation path, accurate screening of animation key frames is realized, redundant frames are reduced, and data compression efficiency and animation coherence are improved.

The invention provides a low-complexity human body motion reconstruction method based on a sparse inertial measurement unit, and belongs to the technical field of virtual reality and three-dimensional human body motion reconstruction, and the method comprises the following steps: obtaining a data set required for reconstruction training based on human body postures, carrying out time modeling through a time sequence encoder, updating joint features on a human body skeleton graph, and obtaining a reconstruction result; a skeleton is divided into a trunk and four limbs according to a human anatomical structure, global rotation of a root joint and local rotation of each joint are respectively predicted by a partition kinematics regression head, low-rank decomposition is introduced into a large-scale linear layer to compress model parameters, and forward kinematics is utilized to recover three-dimensional joint positions of the whole body. In the training process, a two-stage teacher-student distillation model framework is adopted, a teacher network is trained through real labels, and then joint rotation and joint positions output by a teacher are used as soft targets to jointly restrain a student network through rotary distillation and position distillation. While the parameter quantity is reduced, the reconstruction precision and the motion smoothness of the whole body are improved.

The invention discloses a voice-driven 3D facial animation method based on multi-modal synchronous alignment, and relates to the technical field of computer graphics and deep learning, and the method comprises the steps: obtaining the content features and emotion features of an audio signal based on a priori guidance emotion-content decoupling strategy; generating a fusion deformation coefficient sequence coherent in time sequence through a mesh refinement module based on a Transform system structure by using a multi-modal feature; a mesh refining module based on a Transform architecture directly predicts the vertex position of a face mesh by using a fusion deformation coefficient to obtain a mesh sequence, and then a lip reading module moves from a lip region to reconstruct content features; and generating a 3D facial animation through a cross-modal consistency constraint and a hierarchical perception reconstruction constraint. Therefore, by adopting the voice-driven 3D facial animation method based on multi-modal synchronous alignment, the high-quality facial animation which is rich in expressive force, consistent in semantics and coherent in vision can be realized.

The application discloses a role animation key frame simplification method and system based on visual saliency, and relates to the technical field of image processing, comprising: acquiring a role animation sequence, extracting the position path and rotation path of each bone node in the role animation sequence in the time dimension; calculating a first motion feature sequence based on the position path of each bone node; calculating a second motion feature sequence based on the rotation path of each bone node; combining the first motion feature sequence and the second motion feature sequence of each bone node into a third motion feature sequence; and selecting a target key frame sequence from the original time points based on the third motion feature sequence of all bone nodes, with the optimization target being to minimize the motion reconstruction error of the role animation sequence, and outputting the target key frame sequence. Through the feature extraction and fusion combination of the position path and the rotation path, the application realizes accurate screening of animation key frames, reduces redundant frames, and improves data compression efficiency and animation coherence.

The application discloses a speech-driven 3D face animation method based on multi-modal synchronous alignment, relates to the technical field of computer graphics and deep learning, and comprises a priori guide-based emotion-content decoupling strategy, content features and emotion features of an audio signal are acquired, a grid refinement module based on a Transformer architecture is used to generate a time-sequentially coherent fusion deformation coefficient sequence by using multi-modal features, the grid refinement module based on the Transformer architecture is used to directly predict vertex positions of a face grid by using the fusion deformation coefficient, a grid sequence is obtained, and a lip reading module is used to reconstruct content features from mouth region motion, and a 3D face animation is generated by using cross-modal consistency constraints and hierarchical perception reconstruction constraints. Therefore, the speech-driven 3D face animation method based on multi-modal synchronous alignment can realize high-quality face animation which is expressive, semantically consistent and visually coherent.

The invention provides a human body motion capture system based on posture coordinates, and belongs to the technical field of human body motion capture, the system collects human body motion data through a sensor collection module, and carries out data denoising, error correction, gravitational acceleration removal and other processing on the collected data through a data processing module; the method comprises the following steps: optimizing a human body motion data acquisition effect, obtaining human body motion data close to reality, extracting human body motion characteristics based on the obtained human body motion data close to reality, carrying out classification identification on motions in combination with a support vector machine, realizing human body motion capture, and then carrying out reproduction through a motion 3D reproduction module. Therefore, the human body motion data can be preprocessed in time, the capturing effect is better, key information is prevented from being lost, and the key information can be visually reflected.

The invention relates to a heart motion estimation method and device based on sparse key point driving and a medium, and the method comprises the steps: respectively inputting a moving image and a fixed image into a key point detection network with time-space consistency, and carrying out the extraction to obtain a key point pair; wherein in the key point detection process, a local correlation graph is generated according to normalized cross-correlation of key points in a fixed image and key points in a moving image, and after the local correlation graph is converted into a probability distribution graph, the positions of the key points in the moving image are corrected; and constructing a sparse basic motion vector according to the relative motion between each pair of key points to obtain a sparse motion vector expressed by heart motion, and performing dense motion reconstruction on the moving image by using a deep neural network according to the sparse motion vector to generate a dense motion field. Compared with the prior art, the method has the advantages of being high in structure retentivity, high in individual generalization and high in heart motion detail modeling capacity.

The invention discloses an engineering field three-dimensional reconstruction physical scale estimation method with reference to a standard component, and belongs to the technical field of building engineering digitization, three-dimensional reconstruction and intelligent construction, and the method comprises the steps: constructing a three-dimensional reconstruction physical scale estimation platform; a multi-view image sequence containing a standard component is collected, a scale-free three-dimensional point cloud and a camera pose are obtained through structure self-motion reconstruction SfM and multi-view stereo matching MVS reconstruction, a point cloud subset of the standard component is detected and positioned, a reconstruction scale size is obtained through geometric fitting, a scale factor is calculated in combination with a real physical size, and a three-dimensional point cloud is obtained. And carrying out global scaling on the scale-free model to finally obtain a real physical scale three-dimensional model. The method is suitable for various image acquisition devices and construction scenes, the scale recovery precision reaches the millimeter-to-centimeter level, the robustness is high, and the method can be widely applied to engineering measurement, BIM model alignment, digital twinning construction and other scenes and has remarkable engineering application value.

To provide an operation reconfiguration processing system capable of making a robot imitate both a steady state and a transient state of operation by performing processing by a learnable model using a latent expression capable of absorbing difference between human operation and robot operation.SOLUTION: In an operation reconfiguration processing system 1000, a latent representation space, in which phase data capable of representing a response in a steady state and amplitude data capable of representing a response in a transient state are latent variables, is defined, processing is performed by using the data (latent variables) in the latent representation space, and further, processing in which correction by mutual feedback from a control target is reflected in the data in the latent representation space is performed, so that both the steady state and the transient state of a human operation (operation to be imitated) can be imitated by the control target.SELECTED DRAWING: Figure 1

This application provides a method and apparatus for full-body motion capture based on monocular half-body video. The method first acquires a monocular half-body video to be processed, which includes the visible upper body area of ​​the target object. Then, the monocular half-body video is serialized to obtain a sequence of data. Finally, the sequence of data is input into a full-body motion capture model to obtain the full-body motion posture data of the target object. The full-body motion capture model is determined based on multiple monocular half-body videos and the corresponding labeled full-body motion posture data for each monocular half-body video. The method of this application accurately captures the half-body movements of the target object in a monocular half-body video scenario using a full-body motion capture model, and achieves high-precision full-body motion reconstruction.

This invention discloses a multi-axis cooperative motion control system and method for industrial robots, relating to the field of robot control technology. The system includes a data acquisition module, a machining constraint processing module, a constraint matching module, a conflict relationship generation module, a constraint arbitration module, and a motion reconstruction module. The data acquisition module acquires the machining trajectory, tool status, axis motion status of each joint axis, and current machining segment number in the surface grinding machining program. The machining constraint processing module generates a set of machining constraints including maintenance-type machining constraints, amplitude-limiting deviation-type machining constraints, substitution adjustment-type machining constraints, and axis motion boundary constraints. The constraint matching module calculates the execution requirements and matches them with the axis motion boundary constraints. The conflict relationship generation module generates a machining constraint conflict relationship table. The constraint arbitration module generates machining constraint arbitration results. The motion reconstruction module generates local multi-axis motion reconstruction instructions based on these results. The system can transform machining constraint conflicts in surface grinding into executable multi-axis motion control results.

The application relates to the technical field of motion capture, and discloses a motion capture anti-reflection identification method based on a ring-shaped sticker mark in an ice-water mixed environment. The method first arranges a ring-shaped sticker mark with a three-layer concentric ring structure on the surface of measured floating ice, and arranges multiple high-speed cameras in a high-position overhead manner and synchronously collects images; after pre-processing the collected images, candidate targets are obtained through area and roundness double-threshold screening; the candidate targets are verified based on the fixed interval geometric constraint of double mark points on the floating ice, and interference signals are removed through time sequence continuity filtering, so that real mark points are finally obtained for three-dimensional motion reconstruction. The application can effectively suppress mirror surface highlight interference in the ice-water mixed environment, improve the mark point identification accuracy and anti-interference ability, and is suitable for high-precision motion measurement in polar scientific investigation, ice area ship testing and other scenes.

The application discloses a metamorphic operation platform based on a face-symmetrical Bricard mechanism and belongs to the technical field of machinery. The whole metamorphic operation platform comprises a fixed platform and a moving platform, and the two platforms are connected through left and right moving branch chains. For each moving branch chain, a face-symmetrical Bricard mechanism with equivalent rotation axes in a Bricard mechanism family is used, each moving branch chain has two face-symmetrical Bricard mechanisms, and finally, the metamorphic operation platform with two different motion modes of translation and rotation is obtained through connection design between parts. Therefore, the metamorphic operation platform has changeable operation modes, and has a structure form and a motion reconstruction mechanism obviously different from a metamorphic mechanism obtained by combining connection of conventional typical mechanical joints such as rotating pairs.

This application relates to the field of motor reconstruction technology, specifically disclosing a closed-loop motor reconstruction system and method. The system includes an implantable signal acquisition component, an implantable controller, a non-implantable controller, and an implantable neurostimulation component. The implantable signal acquisition component and the implantable neurostimulation component are electrically connected to the implantable controller, which is also electrically connected to the non-implantable controller. The implantable signal acquisition component is configured to acquire electroencephalogram (EEG) signals. The implantable controller is configured to transmit the acquired EEG signals to the non-implantable controller and receive stimulation commands generated by the non-implantable controller based on the EEG signals, then transmit the stimulation commands to the implantable neurostimulation component. The implantable neurostimulation component is configured to parse the stimulation commands and perform electrical stimulation on the nerve roots at the target location. It requires no manual or audio control from the patient, is relatively simple to use, and is real-time, reducing computational complexity while ensuring accuracy.

The embodiment of the application discloses a two-order end memory and a video anomaly detection device for video anomaly detection, which comprises a first-order network and a second-order network; the first-order network comprises a first-order encoder, a motion memory and a first-order decoder connected in sequence, is used for motion reconstruction on a motion part, and obtains a motion prototype; the second-order network comprises a forward process network and a reverse process network connected in series; the forward process network is used for realizing a forward process, and the forward process network comprises a second-order encoder and a second-order decoder, which are used for predicting a predicted future frame of an input video; the reverse process network is cascaded with the forward process network, realizes a reverse process, and reconstructs a predicted initial frame of the input video; the predicted future frame and the predicted initial frame are used for unsupervised anomaly detection, and an anomaly detection result of the input video is output. The two-order end memory is complete in motion representation, fully utilizes the bidirectional consistency of the video, and improves the accuracy of anomaly detection.

The invention relates to a monocular video multi-person 3D human body motion reconstruction method based on multi-module fusion, and solves the defect that discontinuous or unstable reconstruction is generated when a target appears again due to the fact that tracking is lost when the target is partially shielded or moves rapidly compared with the prior art. The method comprises the following steps: acquiring a single-frame or multi-frame motion video sequence; constructing a motion perception semantic tracking module; generating a coherent human body grid sequence; predicting future attitude features in the occlusion scene; and outputting the 3D motion state. According to the invention, 3D motion reconstruction with robust shielding, stable identity and consistent time sequence is realized through collaborative design of motion perception tracking, time sequence enhancement reconstruction, motion prediction and adaptive fusion.

The invention belongs to the technical field of digital cultural heritage protection, and discloses a deep learning-based non-abandoned skill motion capture and digital reproduction method, which comprises the following steps of: obtaining original motion data and carrying out multi-modal fusion on the original motion data to obtain fused motion data; constructing a three-level action characteristic spectrum based on the fused action data, and constructing a fine action key frame sequence for a characteristic sequence in a fine operation layer in the three-level action characteristic spectrum; the key frame sequence is optimized in combination with time continuity and space coordination constraint, and enhanced action feature representation is obtained; semantic annotation and digital action reconstruction are carried out on the enhanced action feature representation; according to the method, the fidelity of skill details and the interpretability of cultural semantics are remarkably improved, and high-quality technical support is provided for non-abandoned inheritance and digital protection.

The present application belongs to the technical field of automatic driving car testing, and in particular to an automatic driving collision danger scene generation method based on reverse motion reconstruction. First, the vehicle operating state is modeled, and the reverse motion process of the vehicle is modeled into a mathematical expression form. Then, a starting point stable driving state set is established, and the smooth state constraint is clarified. The third step is to clarify the sampling process constraint, introduce the vehicle dynamics and road model, and avoid the vehicle motion not conforming to the physical constraint. The fourth step is to convert the generated interaction trajectory into a scene description form. The present application can help enterprises establish an efficient collision danger scene generation system, improve the efficiency of danger scene generation, reduce the waste of computing power, and ultimately establish a complete danger scene database to accelerate the testing and verification of automatic driving cars.

The present application relates to the technical field of flexible riser motion analysis, and particularly relates to a marine dynamic flexible riser motion response reconstruction method based on monitoring data, comprising: using a structural embedded layout method or an external attached layout method to layout sensors in a flexible riser system, and optimizing the sensor layout mode; setting a data acquisition and transmission system on a marine platform to obtain flexible riser motion response data; converting the flexible riser motion response data into flexible riser displacement data based on a Timoshenko beam model and Tikhonov regularization; carrying out noise reduction processing on the flexible riser displacement data based on a wavelet threshold noise reduction method to obtain flexible riser displacement noise reduction data; and carrying out motion response reconstruction on the flexible riser displacement noise reduction data based on variational mode decomposition VMD. The present application can effectively improve the data interpretation, data reconstruction accuracy and real-time performance of motion reconstruction of the flexible riser.

The application relates to a kind of manta ray submarine gliding propulsion kinematics modeling and numerical calculation method;Belong to underwater bionic robot technical field.This method first carries out feature point marking and three-dimensional motion reconstruction to real manta ray biology, extracts its kinematics parameter, and then establishes a kind of unified kinematics equation which strictly satisfies the constraint of body length invariable, can accurately couple the deformation of pectoral fin spanwise and chordwise;By adjusting specific parameters, the mathematical description of multiple real motion modes such as flapping, gliding, gliding and compound, and maneuvering turn can be realized.Secondly, the high-fidelity kinematics model dynamic boundary input condition is combined with the immersed boundary method and the computational fluid dynamics solver to efficiently and stably calculate the hydrodynamic performance and flow field structure of the manta ray submarine in the complex motion process.The application effectively solves the problem that the simulation result deviates from the actual situation and it is difficult to capture the key vortex dynamics structure caused by the distortion of the kinematics model of the prior art.

This invention discloses a method for generating 3D human motion based on a parallel multi-granularity Transformer, comprising: acquiring a 3D human motion dataset; constructing an initial model, which includes a parallel multi-granularity motion generation module, a multi-granularity motion fusion module, and a motion reconstruction and output module; and training the initial model using the 3D human motion dataset to obtain an image generation model for generating high-fidelity 3D human motion sequences. This invention also provides a 3D human motion generation device. The method provided by this invention can generate high-fidelity, semantically coherent, and detail-rich 3D human motion sequences.

The application discloses a whole body posture estimation method based on a sparse inertial sensor. The application comprises the following steps: 1: synthesizing acceleration data, direction data and angular velocity data through a finite difference method and a forward kinematics algorithm; 2: performing alignment processing on original acceleration data and original direction data collected by three inertial sensors worn on the user's head joint, left wrist joint and right wrist joint through a rotation transformation matrix; 3: extracting deep features of inertial motion data samples by combining a Transformer Encoder and a bidirectional long short-term memory network encoder, designing a decoder based on a multilayer perceptron to calculate the positions and relative rotations of joints, and combining a human body kinematics model to estimate the relative rotations of joints in different parts in multiple stages to realize whole body posture estimation. The application can effectively overcome the defects of using sparse inertial input for whole body motion reconstruction in the existing posture estimation method, and effectively improve the robustness and prediction accuracy of the system.

The application provides a flexible wing deformation detection and motion reconstruction method and device based on binocular vision, which comprises a binocular vision detection method, an adaptive KCF tracking algorithm, a binocular camera synchronous correction method and an experimental device. The binocular vision detection method can reconstruct a motion model of the flexible wing and detect the deformation of the wing. The adaptive KCF tracking algorithm can stably track and locate the spatial position of the whole and local area of the moving wing. The synchronous error correction method can correct the time difference of the asynchronously triggered binocular cameras and realize the synchronization of the images shot by the binocular cameras. The experimental device comprises binocular cameras, a flexible wing, markers and a binocular synchronous correction device. The application improves the tracking accuracy of the high-speed moving target through the adaptive KCF tracking algorithm and the synchronous correction method, reduces the binocular camera synchronous error, improves the flexible wing deformation detection accuracy and motion model reconstruction accuracy, and reduces the cost of the measuring equipment.