287 results about "Motion generation" patented technology

The invention relates to the technical field of artificial intelligence, can be applied to the field of medical health and the field of financial transaction, discloses an action generation method and device based on multi-modal pre-training, a robot and a medium, and is applied to a high-frequency action generation scene of an intelligent surgical robot or an intelligent customer service and wealth management scene. The method comprises the following steps: acquiring a language instruction, a visual image and robot body sensing data; performing multi-modal feature alignment and cross-modal feature fusion on the language instruction, the visual image and the robot body sensing data through a pre-trained visual language model to generate a fused joint feature vector; generating a target continuous control instruction based on the fused joint features through a pre-trained target action model by adopting a flow matching technology; and generating continuous actions of the robot based on the target continuous control instruction. According to the method, the robot action generation efficiency and the cross-platform adaptability are improved.

The invention discloses a human motion generation method and system based on text alignment, and belongs to the technical field of computer vision and deep learning. According to the method, a two-stage training mode is adopted, in the first stage, a quantized variational auto-encoder model for reconstructing human motion is trained at first, then parameter freezing is carried out on a decoder network in the quantized variational auto-encoder model and a pre-trained text-to-motion cross-modal retrieval model, the decoder network and a bidirectional mask Transform model form a training framework for training, and the quantized variational auto-encoder model and the pre-trained text-to-motion cross-modal retrieval model form a training framework for training; and after training is completed, reasoning tasks are performed. According to the method, by improving a text encoder, adding semantic alignment loss and adopting a bidirectional mask type Transform architecture, the overall motion perception ability is enhanced, semantic alignment of motion is generated, the generated human motion sequence better conforms to original data distribution, the generated human motion sequence and an input motion description text are more semantic-consistent, and the motion perception ability of a human body is improved. And the motion generation quality is improved.

The invention provides an unmanned aerial vehicle obstacle avoidance method and system based on inertial navigation, and belongs to the technical field of unmanned aerial vehicle obstacle avoidance systems. The problem that an existing unmanned aerial vehicle is insufficient in obstacle avoidance capacity under the environment that GPS signals are weak, vision is limited or a sensor detection blind area exists is solved. Acquiring broadband high-frequency dynamic response data output by the inertial navigation system of the unmanned aerial vehicle in real time; establishing or acquiring a reference high-frequency dynamic response model when the unmanned aerial vehicle is in a specific flight state and a power system working mode and is not influenced by an external obstacle; the signal is dynamically responded according to high frequency; identifying whether a specific air medium disturbance echo feature formed by disturbance generated by motion of the unmanned aerial vehicle through an external obstacle exists or not; and finally, generating and executing an obstacle avoidance flight instruction according to the estimation information. The method does not depend on a traditional external environment sensor for initial detection, physical interaction is sensed through the INS, and the autonomous obstacle avoidance capacity and safety of the unmanned aerial vehicle in the complex and challenging environment are improved.

The invention discloses a flight vision obstacle avoidance method for a low-altitude inspection unmanned aerial vehicle, particularly relates to the technical field of image processing and computer vision, and is used for solving the problem of how to stably detect and reliably estimate the distance and contact time of such obstacles on the premise of only depending on airborne vision and being limited in calculation power and time delay. A space-time skeleton of a linear obstacle is constructed and continuously tracked in the strip-shaped candidates, and a grading distance and contact time are given in combination with an imaging time sequence difference and projection drift; uncertainties generated by the detection side and the self-motion are unified to the same coordinate and time mark, and risk density and safety envelope are formed through recursion; and then a path is updated according to a fixed priority of'deceleration-bypassing-still hovering 'by taking the risk as a hard constraint, so that high recall and stable distance evaluation and executable real-time avoidance can still be realized for a slender and swinging real obstacle under the conditions of only dependence on airborne vision and limited computing power and time delay. And the near loss event occurrence rate and the meaningless braking ratio are obviously reduced.

The invention discloses a linkage differential measurement system and method suitable for micro thrust calibration. The system comprises a linkage differential measurement rack, a liquid metal radio frequency coaxial cable connector arranged on the rack, and a mass center precise regulation and control device arranged on the rack. The linkage differential measurement system provides micro-Newton-level micro thrust except environment common-mode noise; the method comprises the following steps: a torsional pendulum inclination angle control module controls a torsional pendulum inclination angle # imgabs0 #; the centroid position deviation calculation module is used for calculating centroid position deviation; judging whether the offset s of the current centroid position is less than 10 [mu] m or not; the mass center position adjusting module is used for adjusting the mass center position; the gravitational wave micro-thrust calibration torsional pendulum solves the problems that the gravitational wave thruster needs to calibrate thrust noise besides calibrating thrust, high rigidity of a coaxial cable can generate interference on motion of the gravitational wave micro-thrust calibration torsional pendulum, and the gravitational wave micro-thrust calibration torsional pendulum has extremely high requirements on sensing precision of mass center regulation and control.

To provide a vehicle control device and a vehicle control method that take into consideration individual differences and the effect of vehicle motion, not only in the horizontal direction but also in the front-rear direction, on the head movement of a passenger, and further reduce the head movement of the passenger through vehicle motion that coordinates the front-rear and horizontal directions. The vehicle control device includes: a head movement prediction model that finds, relative to the head movement of a passenger in a first direction caused by vehicle motion in the first direction, the change in the head movement of the passenger in the first direction caused by the vehicle motion in a second direction different from the first direction; and a vehicle motion generation unit that generates the vehicle motion on the basis of the head movement prediction model.

The invention discloses an axial-radial integrated loading rotating shaft bush high-temperature swing friction-wear testing machine, and belongs to the technical field of friction-wear testing, the axial-radial integrated loading rotating shaft bush high-temperature swing friction-wear testing machine comprises a bottom plate, a loading module is arranged on the bottom plate, one side of the loading module is connected with a high-temperature clamp module, one side of the high-temperature clamp module is connected with a motion generation module, and the motion generation module is connected with the loading module. One side of the motion generation module is connected with an electric control box body, and one side of the electric control box body is connected with a control computer. According to the axial-radial integrated loading rotating shaft bush high-temperature swing friction wear testing machine disclosed by the invention, synchronous application of an axial load and a radial load is realized through cooperative control of double stepping motors in the loading module; meanwhile, the influence of the coupling on torque measurement is effectively eliminated through the self-designed coupling, and the accuracy of synchronous application is improved.

A method for training a model for generating a representation of long-term motion from a text input comprises: training a motion encoder of an autoencoder to compress and map an input motion into a latent representation comprising a sequence of latent vectors in a discrete latent space, each latent vector representing a fixed length of motion; training a quantization module to quantize the latent vectors to a sequence of quantized latent vectors in quantized latent space; and training a motion decoder to reconstruct the quantized sequence as a sequence of single-frame pose representations. A text encoder is trained to predict a latent sequence conditioned on a text input and a duration using the mapped latent representation as a target.

The invention discloses a motion planning method for a heterogeneous robot, and relates to the technical field of robots. The method comprises the following steps: step 1, subspace definition and differential homeomorphic mapping establishment; step 2, dynamic situation field construction; step 3, geometric motion generation; and 4, controlling system energy. According to the method, universal adaptation of robot systems with different configurations can be realized, the algorithm reuse rate is greatly improved, the development cost is remarkably reduced, efficient obstacle avoidance can be realized in a dynamic environment, and the method has excellent real-time performance and wide applicability and robustness.

To convert an abstract instruction by voice or text into a concrete motion, and generate an original character.SOLUTION: A system according to an embodiment includes a reception unit, a motion generation unit, a character generation unit, and an application unit. The reception unit receives an instruction by a voice of a user or a text. The motion generation unit analyzes the instruction received by the reception unit and generates a concrete motion. The character generation unit generates an original character on the basis of features input by the user. The application unit applies the motion generated by the motion generation unit to the character generated by the character generation unit.SELECTED DRAWING: Figure 1

A multi-modal human body action generation method based on a large language model comprises the steps that structural features are extracted based on 3D action data, body part-level atomic semantic description is generated in combination with the large language model, and a multi-modal alignment data set containing texts, voices and music is constructed; whole-body actions are decoupled according to parts, an independent vector quantization encoder is adopted to perform residual quantization, and an atomic action token strongly associated with a fine-grained text is generated; splicing the text description and the action token into a mixed action sentence containing a special mark according to a body structure; multi-modal input joint modeling is realized through a large language model, a fine-grained text and an action token sequence are synchronously generated, 3D actions conforming to semantics are output through decoding, and zero sample generation and part-level accurate control are supported. According to the method, the limitation of coarse-grained alignment of a traditional method is broken through, and the semantic consistency, multi-modal adaptability and local controllability of action generation are remarkably improved through fine-grained semantic mapping, decoupling action coding and unified sequence modeling.

The invention discloses an enhanced electromagnetic track finite element simulation method, device and system and a storage medium, and the method comprises the steps: S1, generating an electromagnetic force through an electromagnetic module when a projectile is located at an initial position and current is introduced; wherein the performance of the rail is enhanced through circular flow of current in the enhanced rail; simulation of the contact resistance is realized through a weak contribution form of the contact resistance; s2, through a mechanical module, the armature is installed in a track through interference assembly, and electromagnetic force enables the armature to obtain acceleration and speed; wherein the armature moves to generate displacement so as to cause grid deformation; and S3, the electromagnetic module realizes control of armature displacement through moving the grid. By adopting the technical scheme of the invention, a contact layer does not need to be established, so that the calculation complexity is greatly reduced; and meanwhile, the enhancement effect of the parallel enhanced track can be quantitatively evaluated.

A multi-modal robot that is configured to operate with a bipedal locomotion that may be augmented with aerial locomotion. Many embodiments of a robot may incorporate a robot with a main body portion that houses the various control systems and mechanical controls of the robot. The body of the robot can have a number of different propellers connected to an upper portion of the body and configured to generate lift and / or stability for the body of the robot. Additionally, many embodiments have at least two leg elements connected to a bottom portion of the body by way of a servo mechanism. The legs are configured to provide support for the body of the robot as well as generate a walking locomotion through the movement of the legs.

The embodiment of the invention discloses an action generation method and device, a model training method and device, equipment, a storage medium and a product. The action generation method comprises the following steps: acquiring to-be-processed audio and noise features, separating voice component features, emotion component features and style component features from the to-be-processed audio, and performing feature fusion processing on the noise features, the emotion component features, the voice component features and the style component features through an action generation model to obtain an action result. And performing iterative denoising processing on the fusion features through the action generation model to obtain an action sequence corresponding to the audio to be processed. Therefore, in the action sequence generation process, the to-be-processed audio is separated into the component features of multiple dimensions, so that the information contained in the to-be-processed audio can be mined more finely, and the quality of the action video is improved.

The invention provides a road detection vehicle pavement disease detection method and system based on multi-sensor fusion, and relates to the technical field of road engineering detection, and the method comprises the steps: 1, configuring at least one two-dimensional image collection unit and at least one three-dimensional image collection unit on a detection vehicle to obtain a configuration result; 2, according to the configuration result, in the driving process of the detection vehicle, a geometric reference area used for multi-source data space alignment is established based on lane marking lines in a to-be-detected road surface area; and step 3, performing structured division on the geometric reference area to obtain regularly distributed detection areas, and calculating spatial geometric feature changes generated by vehicle composite motion in the detection areas based on distribution features of the detection areas in combination with translational motion and rotary composite motion states existing in the driving process of the detection vehicle so as to obtain the spatial geometric feature changes of the detection areas. And obtaining a rigid body space composite kinematics characteristic resolving result of the detection area. According to the invention, efficient, accurate and full-dimensional detection of pavement diseases is realized.

The invention discloses a video-based digital human motion capture and motion generation method and device and a medium, and relates to the technical field of machine vision, and the method comprises the steps: obtaining a video containing human motion; extracting a body area image frame and a hand area image frame corresponding to the figure based on the video; performing human body reconstruction based on the body area image frame and the hand area image frame to obtain a 3D human body model; and acquiring human skeleton information based on the 3D human body model, and generating a skeleton action file based on the human skeleton information. According to the scheme disclosed by the invention, the digital human body motion capture cost is reduced, and the problem that animation needs to be made again due to skeleton adjustment is solved.

The invention provides a robot action generation method and a robot control system, and the method comprises the steps: obtaining multi-modal observation data at a current moment; the modal weight and the initial noise trajectory of each modal are acquired; and according to the multi-modal observation data, the modal weight of each modal and the noise estimation model corresponding to each modal, performing iterative exploration on the initial noise trajectory according to a time inverse sequence, and generating an action trajectory sequence of the robot. According to the method, noise estimation of the pre-trained single-mode diffusion strategy can be combined during reasoning, and information of different modes can be efficiently fused, so that on the premise of avoiding complexity of multi-mode data collection and joint training, multi-mode strategy fusion is realized, and meanwhile, the multi-mode data collection and joint training efficiency is improved. And the flexibility, the real-time adaptability, the robustness and the expandability of the action generation process are enhanced. Moreover, by avoiding multi-modal joint training, the training cost and the computing resource demand are greatly reduced.

The present invention provides a controllable music-driven three-dimensional dance motion generation model training method, generation method and device, including: constructing a training set containing music-three-dimensional dance motion data and two-dimensional key point data; constructing an initial generation model, including a backbone network and a control network; the backbone network takes music and three-dimensional dance motion data as inputs and outputs predicted motion data; the control network takes music, three-dimensional dance motion data and two-dimensional key point data as inputs and outputs motion features; splicing the motion features with the intermediate features extracted by the backbone network to control the predicted motion data; training the initial generation model in two stages using the training set to finally obtain a three-dimensional dance motion generation model. The three-dimensional dance motion generation model obtained by training in the present invention can be used to generate three-dimensional dance motion data matching the music and has the ability of controllable generation.

The invention discloses a text-to-action generation method and system based on body part fine-grained representation, and belongs to the field of human body action generation. Body part-level fine-grained discretization modeling is performed on human body actions to obtain an action encoder and an action decoder which can stably reconstruct a continuous action sequence; the method comprises the following steps: taking a part fine-grained discrete action space as a producible space, introducing video potential representation as space-time dynamic priori, establishing an alignment modeling mechanism of the video potential representation and the action discrete space, and training to obtain a conditional generation network, so as to realize stable prediction and iterative completion of an action discrete index sequence in a reasoning stage; and a continuous human body action sequence is reconstructed through an action decoder, and finally a human body action sequence which is consistent with text semantics, richer in details and more coherent in time sequence is generated. According to the method, the technical threshold and data dependence of action content production can be reduced, and the controllability and generalization ability of action generation are improved.

The invention relates to the technical field of chemical stirring, in particular to an automatic reagent stirring device for chemical experiments and a stirring method thereof. Comprising a reaction tank, a titration frame and a central control unit, a displaceable liquid storage titration assembly is arranged on the titration frame, a double-spectrum sensor is mounted on the titration frame, the titration device further comprises a liftable protective frame and a lifting frame, two sealing covers with an adjustable distance are mounted on the protective frame, an online monitoring system for monitoring chemical liquid reaction data is arranged between the two sealing covers, and the central control unit is connected with the lifting frame. Arc clamping plates are installed on the two sealing covers, a motion generation system is installed on the lifting frame, and a motion frame capable of vertically moving in a reciprocating mode and a forward stirring shaft capable of alternately rotating forwards and backwards are connected to the motion generation system. Through cooperative linkage of the liquid storage titration assembly, the double-spectrum sensor and the central control unit, the problem of low titration precision of a traditional artificial chemical titration mode is solved.

A mechanical arm (5) motion generation method and apparatus, a computer-readable storage medium, and a mechanical arm (5). The mechanical arm (5) motion generation method comprises: decomposing a mechanical arm (5) grabbing task oriented to a goods shelf environment into a plurality of sub-tasks; on the basis of a geometric dynamical system, separately determining a Riemannian motion policy of each sub-task; and, on the basis of graph calculation processes of the Riemannian motion policies, integrating the Riemannian motion policy of each sub-task to obtain a global motion policy of a mechanical arm (5).

A motion generation system includes a tracking model, executed by a processor, configured to track at least one kinematic reference motion of a robotic device; a reward surrogate model, executed by the processor, that evaluates a performance of the tracking model with respect to the at least one kinematic reference motion and estimates at least one reward for the tracking model based on the performance; and a generative model, executed by the processor, configured to generate a motion for the robotic device based on a contextual input and the estimated at least one reward, wherein the generative model is trained with a pre-training operation and a refinement operation separate from the pre-training operation.

The invention discloses a listener video generation method based on multi-modal unified embedding space guidance, and the method comprises the steps: obtaining a first real video of a speaker, and extracting corresponding video features and audio features as initiating user features; performing emotion prediction according to the characteristics of the initiating user through a pre-trained emotion prediction model to obtain receiving emotion characteristics of the listener; fusing the initiating user features and the receiving emotion features through a modal self-adaptive hybrid expert encoder to obtain fused features; generating motion data corresponding to the listener according to the fusion features based on the cross attention layer through a pre-trained motion generation model; and rendering according to the action data to generate a head video corresponding to the listener. The method focuses on generating core action data instead of directly generating an original pixel video, and subsequent rendering steps are relatively independent, so that the calculation complexity of real-time generation is reduced, and the real-time potential of response is improved.

Disclosed is a motion generation device for generating motion based on input information including a text and an operation method thereof, and the motion generation device may include a memory configured to store at least one instruction; and at least one processor configured to execute the at least one instruction stored in the memory, wherein the at least one processor is configured to: obtain the input information including the text representing a movement of a character, extract feature information from the obtained input information, generate motion data for the movement of the character using the extracted feature information, obtain correction motion data by post-processing the generated motion data, and generate motion animation of the character based on the obtained correction motion data.

The invention provides an intelligent motion generation method, an intelligent motion generation system, electronic equipment and a computer readable storage medium, in the intelligent motion generation method, motion feature data are mapped and aligned to a virtual human skeleton structure, a preliminary motion sequence is obtained, and in addition, the motion feature data are mapped and aligned to the virtual human skeleton structure; and according to the virtual human skeleton structure and the preliminary action sequence, optimizing the action of the virtual human skeleton structure, and outputting an optimized action sequence, so that after the preliminary action sequence is obtained according to a redirection strategy based on the frame sequence of the video of the human body motion, the motion of the virtual human skeleton structure is optimized. And an optimized action sequence obtained by optimizing the preliminary action sequence can be more matched with the body shape of the virtual human.

The invention discloses an unmanned aerial vehicle multi-mode fusion radio positioning and signal identification method and system, and relates to the technical field of radio positioning and detection.The method comprises the steps that an airspace radio environment is continuously monitored in a monitoring area through all receiving stations, a power spectrum sequence and an unmanned aerial vehicle communication signal are obtained, and spectrum features are extracted from the power spectrum sequence; for the same launch event, acquiring a three-dimensional motion track of the unmanned aerial vehicle according to the arrival time of each receiving station; echo signals generated by unmanned aerial vehicle rotor wing and vehicle body movement are obtained through a radar, and micro-Doppler features are extracted from the echo signals; acquiring protocol related modulation features, cyclic spectrum features and protocol features according to the unmanned aerial vehicle communication signals; performing multi-modal feature fusion on the spectrum feature, the micro-Doppler feature, the modulation feature, the cyclic spectrum feature and the protocol feature to obtain a deep joint feature; identifying the type of the unmanned aerial vehicle according to the deep joint feature, and obtaining a type label; and constructing an unmanned aerial vehicle feature signature by using the category label, the three-dimensional motion trail and the depth joint feature. According to the invention, the accuracy, robustness and supervision capability of unmanned aerial vehicle positioning and signal detection in a complex environment can be improved.

The invention provides a motion video generation method and device, computer equipment and a storage medium. Based on the motion video generation method in the embodiment, a target character image, a target background and a motion-driven video are firstly obtained, then two-dimensional and three-dimensional attitude features are extracted from the motion-driven video, and the two features are fused to obtain a fused attitude feature. And finally, inputting the target character image, the target background and the fused posture features into a video generation model to obtain a target motion video. According to the scheme, the three-dimensional motion information and the two-dimensional motion information are simultaneously utilized to guide the motion, the ambiguous motion generation probability is reduced, and the video generation quality is greatly improved.

The invention discloses a displacement sensor detection device with a multi-position synchronous test function, and relates to the technical field of displacement sensor detection, the displacement sensor detection device comprises a workbench, an H-shaped plate, an unfolding device, an X-axis moving device, a guide column and a Z-axis moving device, the unfolding device can realize simultaneous detection of various displacement sensors, and the Z-axis moving device can realize simultaneous detection of the X-axis moving device and the Z-axis moving device; a large amount of time and labor cost are saved, the X-axis moving device can change the horizontal position of the magnetic steel and detect the plane displacement precision of the displacement sensor, the Z-axis moving device can clamp the displacement sensors of different sizes at the same time, the sensors can be stably fixed in the clamping process, too large clamping force cannot be applied, and the clamping precision is improved. Meanwhile, the displacement sensor is lifted, the space displacement precision of the displacement sensor is detected, the balance state of the displacement sensor is automatically maintained in the moving process of the displacement sensor, vibration generated by external interference or self movement is effectively restrained, and therefore measurement errors caused by vibration are avoided.

The present invention provides a new control technique for a robot without using inverse kinematics. A disclosed motion retargeting device for motion generation of a robot is configured to provide first motion data of a person to a neural network that has undergone machine learning and generate second motion data for operating the robot by using the neural network. The neural network comprises: an encoder that encodes the first motion data into latent motion data on a latent space represented by a primitive skeleton that is a common skeleton structure between the person and the robot; and a decoder that generates the second motion data of the robot on the basis of the latent motion data and the skeleton information of the robot.

A device may convert a motion dataset to a compatible representation for use in a physics engine, wherein the physics engine includes a physics simulator and inverse dynamics network. The device may further downsample the motion dataset to obtain keyframes for motion generation and forming a downsampled motion dataset. The device may further execute a deep generative model based on the downsampled motion dataset to generate a first generated motion. The device may further execute the physics engine by feeding pairs of consecutive keyframes into the physics simulator and the inverse dynamics network to generate a second generated motion. The device may further combine the first generated motion and the second generated motion to form a combined generated motion, wherein the combined generated motion is generated by executing the physics engine with the first generated motion. The device may further generate a simulated motion video from the combined generated motion.