254 results about "Motion generation" patented technology

The invention relates to the technical field of artificial intelligence, can be applied to the field of medical health and the field of financial transaction, discloses an action generation method and device based on multi-modal pre-training, a robot and a medium, and is applied to a high-frequency action generation scene of an intelligent surgical robot or an intelligent customer service and wealth management scene. The method comprises the following steps: acquiring a language instruction, a visual image and robot body sensing data; performing multi-modal feature alignment and cross-modal feature fusion on the language instruction, the visual image and the robot body sensing data through a pre-trained visual language model to generate a fused joint feature vector; generating a target continuous control instruction based on the fused joint features through a pre-trained target action model by adopting a flow matching technology; and generating continuous actions of the robot based on the target continuous control instruction. According to the method, the robot action generation efficiency and the cross-platform adaptability are improved.

The invention provides an unmanned aerial vehicle obstacle avoidance method and system based on inertial navigation, and belongs to the technical field of unmanned aerial vehicle obstacle avoidance systems. The problem that an existing unmanned aerial vehicle is insufficient in obstacle avoidance capacity under the environment that GPS signals are weak, vision is limited or a sensor detection blind area exists is solved. Acquiring broadband high-frequency dynamic response data output by the inertial navigation system of the unmanned aerial vehicle in real time; establishing or acquiring a reference high-frequency dynamic response model when the unmanned aerial vehicle is in a specific flight state and a power system working mode and is not influenced by an external obstacle; the signal is dynamically responded according to high frequency; identifying whether a specific air medium disturbance echo feature formed by disturbance generated by motion of the unmanned aerial vehicle through an external obstacle exists or not; and finally, generating and executing an obstacle avoidance flight instruction according to the estimation information. The method does not depend on a traditional external environment sensor for initial detection, physical interaction is sensed through the INS, and the autonomous obstacle avoidance capacity and safety of the unmanned aerial vehicle in the complex and challenging environment are improved.

The invention discloses a flight vision obstacle avoidance method for a low-altitude inspection unmanned aerial vehicle, particularly relates to the technical field of image processing and computer vision, and is used for solving the problem of how to stably detect and reliably estimate the distance and contact time of such obstacles on the premise of only depending on airborne vision and being limited in calculation power and time delay. A space-time skeleton of a linear obstacle is constructed and continuously tracked in the strip-shaped candidates, and a grading distance and contact time are given in combination with an imaging time sequence difference and projection drift; uncertainties generated by the detection side and the self-motion are unified to the same coordinate and time mark, and risk density and safety envelope are formed through recursion; and then a path is updated according to a fixed priority of'deceleration-bypassing-still hovering 'by taking the risk as a hard constraint, so that high recall and stable distance evaluation and executable real-time avoidance can still be realized for a slender and swinging real obstacle under the conditions of only dependence on airborne vision and limited computing power and time delay. And the near loss event occurrence rate and the meaningless braking ratio are obviously reduced.

The invention discloses a linkage differential measurement system and method suitable for micro thrust calibration. The system comprises a linkage differential measurement rack, a liquid metal radio frequency coaxial cable connector arranged on the rack, and a mass center precise regulation and control device arranged on the rack. The linkage differential measurement system provides micro-Newton-level micro thrust except environment common-mode noise; the method comprises the following steps: a torsional pendulum inclination angle control module controls a torsional pendulum inclination angle # imgabs0 #; the centroid position deviation calculation module is used for calculating centroid position deviation; judging whether the offset s of the current centroid position is less than 10 [mu] m or not; the mass center position adjusting module is used for adjusting the mass center position; the gravitational wave micro-thrust calibration torsional pendulum solves the problems that the gravitational wave thruster needs to calibrate thrust noise besides calibrating thrust, high rigidity of a coaxial cable can generate interference on motion of the gravitational wave micro-thrust calibration torsional pendulum, and the gravitational wave micro-thrust calibration torsional pendulum has extremely high requirements on sensing precision of mass center regulation and control.

The invention discloses an axial-radial integrated loading rotating shaft bush high-temperature swing friction-wear testing machine, and belongs to the technical field of friction-wear testing, the axial-radial integrated loading rotating shaft bush high-temperature swing friction-wear testing machine comprises a bottom plate, a loading module is arranged on the bottom plate, one side of the loading module is connected with a high-temperature clamp module, one side of the high-temperature clamp module is connected with a motion generation module, and the motion generation module is connected with the loading module. One side of the motion generation module is connected with an electric control box body, and one side of the electric control box body is connected with a control computer. According to the axial-radial integrated loading rotating shaft bush high-temperature swing friction wear testing machine disclosed by the invention, synchronous application of an axial load and a radial load is realized through cooperative control of double stepping motors in the loading module; meanwhile, the influence of the coupling on torque measurement is effectively eliminated through the self-designed coupling, and the accuracy of synchronous application is improved.

A method for training a model for generating a representation of long-term motion from a text input comprises: training a motion encoder of an autoencoder to compress and map an input motion into a latent representation comprising a sequence of latent vectors in a discrete latent space, each latent vector representing a fixed length of motion; training a quantization module to quantize the latent vectors to a sequence of quantized latent vectors in quantized latent space; and training a motion decoder to reconstruct the quantized sequence as a sequence of single-frame pose representations. A text encoder is trained to predict a latent sequence conditioned on a text input and a duration using the mapped latent representation as a target.

The invention discloses a motion planning method for a heterogeneous robot, and relates to the technical field of robots. The method comprises the following steps: step 1, subspace definition and differential homeomorphic mapping establishment; step 2, dynamic situation field construction; step 3, geometric motion generation; and 4, controlling system energy. According to the method, universal adaptation of robot systems with different configurations can be realized, the algorithm reuse rate is greatly improved, the development cost is remarkably reduced, efficient obstacle avoidance can be realized in a dynamic environment, and the method has excellent real-time performance and wide applicability and robustness.

A multi-modal human body action generation method based on a large language model comprises the steps that structural features are extracted based on 3D action data, body part-level atomic semantic description is generated in combination with the large language model, and a multi-modal alignment data set containing texts, voices and music is constructed; whole-body actions are decoupled according to parts, an independent vector quantization encoder is adopted to perform residual quantization, and an atomic action token strongly associated with a fine-grained text is generated; splicing the text description and the action token into a mixed action sentence containing a special mark according to a body structure; multi-modal input joint modeling is realized through a large language model, a fine-grained text and an action token sequence are synchronously generated, 3D actions conforming to semantics are output through decoding, and zero sample generation and part-level accurate control are supported. According to the method, the limitation of coarse-grained alignment of a traditional method is broken through, and the semantic consistency, multi-modal adaptability and local controllability of action generation are remarkably improved through fine-grained semantic mapping, decoupling action coding and unified sequence modeling.

The embodiment of the invention discloses an action generation method and device, a model training method and device, equipment, a storage medium and a product. The action generation method comprises the following steps: acquiring to-be-processed audio and noise features, separating voice component features, emotion component features and style component features from the to-be-processed audio, and performing feature fusion processing on the noise features, the emotion component features, the voice component features and the style component features through an action generation model to obtain an action result. And performing iterative denoising processing on the fusion features through the action generation model to obtain an action sequence corresponding to the audio to be processed. Therefore, in the action sequence generation process, the to-be-processed audio is separated into the component features of multiple dimensions, so that the information contained in the to-be-processed audio can be mined more finely, and the quality of the action video is improved.

The invention provides a road detection vehicle pavement disease detection method and system based on multi-sensor fusion, and relates to the technical field of road engineering detection, and the method comprises the steps: 1, configuring at least one two-dimensional image collection unit and at least one three-dimensional image collection unit on a detection vehicle to obtain a configuration result; 2, according to the configuration result, in the driving process of the detection vehicle, a geometric reference area used for multi-source data space alignment is established based on lane marking lines in a to-be-detected road surface area; and step 3, performing structured division on the geometric reference area to obtain regularly distributed detection areas, and calculating spatial geometric feature changes generated by vehicle composite motion in the detection areas based on distribution features of the detection areas in combination with translational motion and rotary composite motion states existing in the driving process of the detection vehicle so as to obtain the spatial geometric feature changes of the detection areas. And obtaining a rigid body space composite kinematics characteristic resolving result of the detection area. According to the invention, efficient, accurate and full-dimensional detection of pavement diseases is realized.

The invention discloses a video-based digital human motion capture and motion generation method and device and a medium, and relates to the technical field of machine vision, and the method comprises the steps: obtaining a video containing human motion; extracting a body area image frame and a hand area image frame corresponding to the figure based on the video; performing human body reconstruction based on the body area image frame and the hand area image frame to obtain a 3D human body model; and acquiring human skeleton information based on the 3D human body model, and generating a skeleton action file based on the human skeleton information. According to the scheme disclosed by the invention, the digital human body motion capture cost is reduced, and the problem that animation needs to be made again due to skeleton adjustment is solved.

The invention provides a robot action generation method and a robot control system, and the method comprises the steps: obtaining multi-modal observation data at a current moment; the modal weight and the initial noise trajectory of each modal are acquired; and according to the multi-modal observation data, the modal weight of each modal and the noise estimation model corresponding to each modal, performing iterative exploration on the initial noise trajectory according to a time inverse sequence, and generating an action trajectory sequence of the robot. According to the method, noise estimation of the pre-trained single-mode diffusion strategy can be combined during reasoning, and information of different modes can be efficiently fused, so that on the premise of avoiding complexity of multi-mode data collection and joint training, multi-mode strategy fusion is realized, and meanwhile, the multi-mode data collection and joint training efficiency is improved. And the flexibility, the real-time adaptability, the robustness and the expandability of the action generation process are enhanced. Moreover, by avoiding multi-modal joint training, the training cost and the computing resource demand are greatly reduced.

The present invention provides a controllable music-driven three-dimensional dance motion generation model training method, generation method and device, including: constructing a training set containing music-three-dimensional dance motion data and two-dimensional key point data; constructing an initial generation model, including a backbone network and a control network; the backbone network takes music and three-dimensional dance motion data as inputs and outputs predicted motion data; the control network takes music, three-dimensional dance motion data and two-dimensional key point data as inputs and outputs motion features; splicing the motion features with the intermediate features extracted by the backbone network to control the predicted motion data; training the initial generation model in two stages using the training set to finally obtain a three-dimensional dance motion generation model. The three-dimensional dance motion generation model obtained by training in the present invention can be used to generate three-dimensional dance motion data matching the music and has the ability of controllable generation.

The invention discloses a text-to-action generation method and system based on body part fine-grained representation, and belongs to the field of human body action generation. Body part-level fine-grained discretization modeling is performed on human body actions to obtain an action encoder and an action decoder which can stably reconstruct a continuous action sequence; the method comprises the following steps: taking a part fine-grained discrete action space as a producible space, introducing video potential representation as space-time dynamic priori, establishing an alignment modeling mechanism of the video potential representation and the action discrete space, and training to obtain a conditional generation network, so as to realize stable prediction and iterative completion of an action discrete index sequence in a reasoning stage; and a continuous human body action sequence is reconstructed through an action decoder, and finally a human body action sequence which is consistent with text semantics, richer in details and more coherent in time sequence is generated. According to the method, the technical threshold and data dependence of action content production can be reduced, and the controllability and generalization ability of action generation are improved.

The invention relates to the technical field of chemical stirring, in particular to an automatic reagent stirring device for chemical experiments and a stirring method thereof. Comprising a reaction tank, a titration frame and a central control unit, a displaceable liquid storage titration assembly is arranged on the titration frame, a double-spectrum sensor is mounted on the titration frame, the titration device further comprises a liftable protective frame and a lifting frame, two sealing covers with an adjustable distance are mounted on the protective frame, an online monitoring system for monitoring chemical liquid reaction data is arranged between the two sealing covers, and the central control unit is connected with the lifting frame. Arc clamping plates are installed on the two sealing covers, a motion generation system is installed on the lifting frame, and a motion frame capable of vertically moving in a reciprocating mode and a forward stirring shaft capable of alternately rotating forwards and backwards are connected to the motion generation system. Through cooperative linkage of the liquid storage titration assembly, the double-spectrum sensor and the central control unit, the problem of low titration precision of a traditional artificial chemical titration mode is solved.

A motion generation system includes a tracking model, executed by a processor, configured to track at least one kinematic reference motion of a robotic device; a reward surrogate model, executed by the processor, that evaluates a performance of the tracking model with respect to the at least one kinematic reference motion and estimates at least one reward for the tracking model based on the performance; and a generative model, executed by the processor, configured to generate a motion for the robotic device based on a contextual input and the estimated at least one reward, wherein the generative model is trained with a pre-training operation and a refinement operation separate from the pre-training operation.

The invention discloses a listener video generation method based on multi-modal unified embedding space guidance, and the method comprises the steps: obtaining a first real video of a speaker, and extracting corresponding video features and audio features as initiating user features; performing emotion prediction according to the characteristics of the initiating user through a pre-trained emotion prediction model to obtain receiving emotion characteristics of the listener; fusing the initiating user features and the receiving emotion features through a modal self-adaptive hybrid expert encoder to obtain fused features; generating motion data corresponding to the listener according to the fusion features based on the cross attention layer through a pre-trained motion generation model; and rendering according to the action data to generate a head video corresponding to the listener. The method focuses on generating core action data instead of directly generating an original pixel video, and subsequent rendering steps are relatively independent, so that the calculation complexity of real-time generation is reduced, and the real-time potential of response is improved.

Disclosed is a motion generation device for generating motion based on input information including a text and an operation method thereof, and the motion generation device may include a memory configured to store at least one instruction; and at least one processor configured to execute the at least one instruction stored in the memory, wherein the at least one processor is configured to: obtain the input information including the text representing a movement of a character, extract feature information from the obtained input information, generate motion data for the movement of the character using the extracted feature information, obtain correction motion data by post-processing the generated motion data, and generate motion animation of the character based on the obtained correction motion data.

The invention provides an intelligent motion generation method, an intelligent motion generation system, electronic equipment and a computer readable storage medium, in the intelligent motion generation method, motion feature data are mapped and aligned to a virtual human skeleton structure, a preliminary motion sequence is obtained, and in addition, the motion feature data are mapped and aligned to the virtual human skeleton structure; and according to the virtual human skeleton structure and the preliminary action sequence, optimizing the action of the virtual human skeleton structure, and outputting an optimized action sequence, so that after the preliminary action sequence is obtained according to a redirection strategy based on the frame sequence of the video of the human body motion, the motion of the virtual human skeleton structure is optimized. And an optimized action sequence obtained by optimizing the preliminary action sequence can be more matched with the body shape of the virtual human.

The invention discloses an unmanned aerial vehicle multi-mode fusion radio positioning and signal identification method and system, and relates to the technical field of radio positioning and detection.The method comprises the steps that an airspace radio environment is continuously monitored in a monitoring area through all receiving stations, a power spectrum sequence and an unmanned aerial vehicle communication signal are obtained, and spectrum features are extracted from the power spectrum sequence; for the same launch event, acquiring a three-dimensional motion track of the unmanned aerial vehicle according to the arrival time of each receiving station; echo signals generated by unmanned aerial vehicle rotor wing and vehicle body movement are obtained through a radar, and micro-Doppler features are extracted from the echo signals; acquiring protocol related modulation features, cyclic spectrum features and protocol features according to the unmanned aerial vehicle communication signals; performing multi-modal feature fusion on the spectrum feature, the micro-Doppler feature, the modulation feature, the cyclic spectrum feature and the protocol feature to obtain a deep joint feature; identifying the type of the unmanned aerial vehicle according to the deep joint feature, and obtaining a type label; and constructing an unmanned aerial vehicle feature signature by using the category label, the three-dimensional motion trail and the depth joint feature. According to the invention, the accuracy, robustness and supervision capability of unmanned aerial vehicle positioning and signal detection in a complex environment can be improved.

The present invention provides a new control technique for a robot without using inverse kinematics. A disclosed motion retargeting device for motion generation of a robot is configured to provide first motion data of a person to a neural network that has undergone machine learning and generate second motion data for operating the robot by using the neural network. The neural network comprises: an encoder that encodes the first motion data into latent motion data on a latent space represented by a primitive skeleton that is a common skeleton structure between the person and the robot; and a decoder that generates the second motion data of the robot on the basis of the latent motion data and the skeleton information of the robot.

A device may convert a motion dataset to a compatible representation for use in a physics engine, wherein the physics engine includes a physics simulator and inverse dynamics network. The device may further downsample the motion dataset to obtain keyframes for motion generation and forming a downsampled motion dataset. The device may further execute a deep generative model based on the downsampled motion dataset to generate a first generated motion. The device may further execute the physics engine by feeding pairs of consecutive keyframes into the physics simulator and the inverse dynamics network to generate a second generated motion. The device may further combine the first generated motion and the second generated motion to form a combined generated motion, wherein the combined generated motion is generated by executing the physics engine with the first generated motion. The device may further generate a simulated motion video from the combined generated motion.

The invention provides an infrared gesture recognition method based on a Mean Shift algorithm, and the method comprises the steps: capturing a heat radiation signal generated by the movement of a hand of a human body through an infrared sensor, and generating a hand heat map based on the heat radiation signal; analog-to-digital conversion, signal enhancement, filtering and noise reduction processing are carried out on the hand heat map through the front-end data processing unit, and a high-quality digital hand heat map is generated; adopting a Mean Shift algorithm to carry out gesture track positioning and tracking analysis on the digital hand heat map, calculating a vector of a window moving to a high-density area and iteratively updating the position of the window, and extracting hand feature points and a motion track thereof; comparing the extracted hand features with a preset gesture model library, and recognizing the gesture action of the user according to a comparison result; and generating an interaction instruction according to the recognized gesture action, and transmitting the interaction instruction to external equipment to realize non-contact man-machine interaction. According to the invention, the hand infrared signal is collected, and the Mean Shift algorithm is used for positioning and tracking analysis of the gesture track, so that the gesture action of the user is recognized.

The invention provides a robot control method and a robot control system.The method comprises the steps that electroencephalogram signals of a user are collected in real time; determining a motion intention keyword corresponding to the electroencephalogram signal according to the electroencephalogram signal and a motion intention decoding model obtained by pre-training; determining a reference trajectory sequence of the robot according to the motion intention keyword and a pre-trained motion conversion model; and controlling the robot to move according to the reference track sequence. According to the method, the signal quality and the motion generation can be decoupled, the electroencephalogram signal is prevented from being directly regressed into a space trajectory, so that the motion process of the robot is not limited to the signal quality of the electroencephalogram signal any more, the motion stability of the robot is remarkably improved, and the physical feasibility of the generated trajectory is ensured. In addition, the practicability and reliability of the control process are greatly improved. And the real-time calculation load is reduced, and the calculation efficiency and the expandability are improved.

A system is provided that stabilizes an applicator in response to the motion caused by a user. The system includes a motion stabilizer; and an adapter that holds a cosmetic applicator. The motion stabilizer includes a receiver configured to be coupled with the adapter, at least one sensor configured to detect a movement caused by a user, circuitry configured to determine a compensation movement to offset the detected movement; and at least one motion generation device embedded in the receiver configured to control a motion of the cosmetic applicator according to the determined compensation movement, wherein the adapter includes circuitry that is configured to transmit an identifier to the receiver, the identifier indicating a type of the cosmetic applicator, and the motion generation device is configured to set at least one parameter for controlling the motion of the cosmetic applicator according to the type of the cosmetic applicator.

The invention provides an intelligent motion generation method and system of a human body, and electronic equipment, and the method comprises the steps: obtaining motion capture data based on video data, obtaining target motion information and an initial human body skeleton key point sequence based on the motion capture data, obtaining source skeleton motion data based on a human body structure prior key point sequence of the human body skeleton, and obtaining the target motion information and the initial human body skeleton key point sequence; obtaining a virtual human image based on the target motion information, obtaining a virtual human skeleton structure based on the virtual human image, obtaining redirected skeleton data, obtaining an optical flow graph and a mask graph based on the virtual human image and the redirected skeleton data, and obtaining an initial virtual human motion image sequence based on the optical flow graph and the mask graph; and acquiring a virtual human motion image sequence based on the initial virtual human motion image sequence, and acquiring motion output of the living agent based on the virtual human motion image sequence. Therefore, coherent and natural action generation is realized.

The invention provides a long human motion sequence generation method based on a diffusion model, and belongs to the technical field of computer vision and computer graphics. Comprising the following steps: constructing a long-time human body action data set; constructing a long text-to-long action generation network, wherein the long text-to-long action generation network comprises a text-to-short action sequence generation model and a motion sequence splicing model; the text-to-short action sequence generation model adopts a variational auto-encoder and diffusion model combined frame to generate a short-time human action sequence based on text conditions; the motion sequence splicing model realizes seamless splicing of motion sequences; training a long text-to-long action generation network by using the long-time human body action data set; and inputting the long text description into the trained long text-to-long action generation network to obtain a long human motion sequence matched with the text semantics. According to the invention, the problem of fragment transition is effectively solved, so that long-time human body action generation is realized.

The invention discloses a method for simulating particle transport and related equipment, and relates to the technical field of radiotherapy, and the method comprises the steps: constructing an equivalent density model along a particle transport direction based on the structure parameters of an energy regulator combination, and building a matched boundary information table, the boundary information table at least records the position information of the downstream boundary of each energy regulator piece in the energy regulator combination, the equivalent water thickness accumulated from the incident end to the boundary, and the air gap length between the boundary and the upstream boundary of the next energy regulator piece; obtaining a particle current position and boundary information table; the medium area where the current free path of the particles is located is judged, and differential calculation is carried out on offset generated by movement of the particles in different areas according to the medium area until the particles pass through the energy regulator combination to be transported; according to the method, rapid and accurate judgment and differential offset calculation of the boundary are realized, so that the boundary processing complexity is remarkably reduced and the calculation efficiency is improved on the premise of ensuring the physical simulation precision when the particles pass through the energy regulator.

The embodiment of the utility model relates to a running board assembly and a running machine. Comprising a first running board; the end parts, close to each other, of the second running plate and the first running plate are rotationally connected; the first metal piece is arranged on the upper surface of the first running plate and the end face, close to the second running plate, of the first running plate; the second metal piece is arranged on the upper surface of the second running plate and the end face, close to the first running plate, of the second running plate. According to the running board, the first metal piece and the second metal piece between the first running board and the second running board are in contact, so that noise generated at the joint between the first running board and the second running board due to movement of a user can be avoided. Besides, through a first metal piece located on the upper surface of the first running plate and a second metal piece located on the upper surface of the second running plate, the smoothness between the running belt and the first running plate and the smoothness between the running belt and the second running plate can be improved, friction between the running belt and the first running plate and friction between the running belt and the second running plate are reduced, and therefore the experience feeling of a user is further improved.

The utility model discloses an electrostatic induction pendulum for earthquake monitoring and early warning based on electrostatic induction. The device comprises a pendulum and an electrostatic induction sensor, and tiny potential changes can be detected in the swinging process. As a core component, the electrostatic induction sensor consists of the following key parts: a weight module, a connecting layer I, an electrode layer, a connecting layer II and an electrostatic interaction layer. Wherein the weight module provides necessary mass so as to ensure the dynamic characteristics of the pendulum; the first connecting layer is used for fixing the connection between the weight module and the electrode layer; the electrode layer is responsible for detecting electrostatic induction signals generated by pendulum motion and converting the signals into measurable potential change; the second connecting layer is used for fixing the electrode layer and the electrostatic interaction layer; the material of the electrostatic interaction layer carries charges, and the electrostatic induction signal of the electrode layer is enhanced in the earthquake monitoring and early warning process. Through the design, the device can realize high-sensitivity monitoring of earthquake precursor, and can capture fine ground movement caused by earthquake waves. In combination with a data acquisition system and a machine learning algorithm, the device can analyze the signals in real time and automatically send early warning information to related departments and individuals when an abnormal vibration mode is detected, so that precious time is won for taking preventive measures.