VLA model-based intelligent motion planning method for body

Abstract

The invention discloses a VLA model-based intelligent motion planning method for a human body. The VLA model comprises a multi-mode sub-model and an action generation sub-model. The method comprises the following steps: acquiring multi-modal data of an intelligent agent with a body; inputting an instruction and an image into the multi-modal sub-model to obtain a multi-modal fusion vector; based on the multi-modal fusion vector and the state vector, obtaining an embedded vector; inputting the embedded vector into the action generation sub-model to obtain an action sequence; mapping the action sequence to a vector space of a controller with an intelligent agent, and executing the action; by adopting the technical scheme, the problems of inaccurate task instruction understanding, poor environment adaptability and insufficient ontology state and action collaboration in a traditional motion planning method are effectively solved, and the task execution precision, action coherence and environment robustness of the agent with the body in an unstructured scene are remarkably improved.

Description

Technical Field

[0001] This application relates to the fields of robot control and artificial intelligence technology, specifically to an embodied intelligent motion planning method based on the VLA model. Background Technology

[0002] With the development of artificial intelligence and robotics, embodied agents are playing an increasingly important role in applications such as human-computer interaction, automated manufacturing, and home services. Most existing motion planning methods for embodied agents rely on a single perception mode based on visual or linguistic monomodal input, lacking a deep understanding of complex multimodal contexts. This results in limited robustness and generalization ability of motion planning when executing diverse tasks under natural language instructions. Summary of the Invention

[0003] In view of the above problems, this application provides an embodied intelligent motion planning method based on the VLA model to solve the problem of limited robustness and generalization ability of motion planning when executing diverse tasks under natural language instructions.

[0004] To achieve the above objectives, the inventors provide an embodied intelligent motion planning method based on a VLA model, wherein the VLA model includes a multimodal sub-model and an action generation sub-model; the method includes the following steps:

[0005] Acquire embodied agent multimodal data, which includes user commands, images collected by visual sensors on the embodied agent, and state vectors of the embodied agent end controller;

[0006] Instructions and images are input into the multimodal sub-model to obtain a multimodal fusion vector;

[0007] The embedding vector is obtained based on the multimodal fusion vector and the state vector;

[0008] The embedded vectors are input into the action generation sub-model to obtain the action sequence;

[0009] The action sequence is mapped to the vector space of the embodied agent's controller, and the action is executed.

[0010] Furthermore, the multimodal sub-model includes:

[0011] A visual encoder is used to convert the image into a visual feature vector;

[0012] A language model is used to convert the instructions into language feature vectors;

[0013] The fusion module is used to concatenate and fuse the visual feature vector and the language feature vector to generate a multimodal fusion vector.

[0014] Furthermore, the step of inputting instructions and images into the multimodal sub-model to obtain a multimodal fusion vector includes the following steps:

[0015] The image is input into a visual encoder to generate a visual feature vector;

[0016] The instruction is input into the language model, the language model's word segmenter segments the instruction into words to obtain the corresponding tokenized representation, and then the language model is used to convert the token into a language feature vector with a plan.

[0017] Map the visual feature vectors to the same embedding space as the language feature vectors generated by the language model;

[0018] The spatially aligned visual feature vectors and language feature vectors are input into the fusion module and concatenated based on the sequence dimension of the vectors to generate a multimodal fusion vector.

[0019] Furthermore, the step of obtaining the embedding vector based on the multimodal fusion vector and the state vector includes the following steps:

[0020] The time steps of the actions performed by the embodied intelligent agent are encoded using sine-cosine position encoding to obtain the time step vector;

[0021] The state vector and time step vector are concatenated and input into a multilayer perceptron, and then processed by a nonlinear activation function to obtain a spatiotemporal action feature vector.

[0022] The multimodal fusion vector and the spatiotemporal action feature vector are linearly concatenated to obtain the embedding vector.

[0023] Furthermore, the step of inputting the embedded vector into the action generation sub-model to obtain the action sequence includes the following steps:

[0024] Initialize the Gaussian noise vector n;

[0025] The embedded vector input action generation sub-model is used for noise prediction to obtain the noise velocity field v. t ;

[0026] An Euler-like method is used to perform denoising iteration on the Gaussian noise vector n;

[0027] Repeat the above iterative process to obtain the final action sequence.

[0028] Furthermore, the action generation sub-model uses a pre-trained diffusion model as its foundation. By adding noise to the pure action sequence during the training phase, the model learns the denoising process and gradually denoises from the pure noise during the inference phase to generate the action sequence.

[0029] Furthermore, during the training process of the action generation sub-model, supervision is performed based on conditional flow matching loss to guide the action generation sub-model to learn the mapping from noisy action sequences to real action sequences.

[0030] Furthermore, the step of mapping the action sequence to the vector space of the controller of the embodied intelligent agent and executing the action includes:

[0031] Obtain the model and action vector space of the embodied intelligent agent;

[0032] Based on the model and action vector space of the embodied intelligent agent, the action sequence is subjected to inverse normalization processing;

[0033] The denormalized sequence of actions is mapped to control instructions for the end effector, and the corresponding actions are executed based on the control instructions.

[0034] Furthermore, it also includes periodically determining whether the embodied intelligent agent has completed the user's instruction when performing the action steps; if the instruction is completed, the embodied intelligent agent is controlled to return to a preset pose; otherwise, all steps are repeated until the instruction is completed.

[0035] Furthermore, the step of determining whether the embodied intelligent agent has completed the user's instruction includes:

[0036] Obtain the new image and state vector;

[0037] Based on the new image and state vectors, and the instruction input based on the VLA model, it is determined whether the obtained action sequence is a zero vector; if so, the instruction is completed; otherwise, the instruction is not completed.

[0038] Unlike existing technologies, the above-mentioned technical solution effectively solves the problems of inaccurate understanding of task instructions, poor environmental adaptability, and insufficient coordination between ontology state and action in traditional motion planning methods by integrating multimodal heterogeneous data and relying on the cross-modal fusion and reasoning capabilities of the VLA model. It significantly improves the task execution accuracy, action coherence, and environmental robustness of embodied intelligent agents in unstructured scenarios, and is suitable for application scenarios such as multimodal human-computer interaction and intelligent agent action control.

[0039] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0040] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of the present invention and other related contents, and should not be considered as limitations on this application.

[0041] In the accompanying drawings of the instruction manual:

[0042] Figure 1 This is a flowchart illustrating the embodied intelligent motion planning method based on the VLA model as described in a specific implementation.

[0043] Figure 2 This is a schematic diagram of the module structure of the embodied intelligent motion planning method based on the VLA model described in a specific implementation.

[0044] Figure 3 This is a schematic diagram illustrating the process of obtaining the multimodal fusion vector as described in the specific implementation method.

[0045] Figure 4 This is a schematic diagram illustrating the process of obtaining the embedding vector as described in a specific implementation.

[0046] Figure 5 This is a schematic diagram of the process for obtaining the action sequence as described in a specific implementation method;

[0047] Figure 6 This is a schematic diagram illustrating the process of mapping an action sequence to the vector space of a controller of an embodied intelligent agent and executing the action, as described in a specific implementation. Detailed Implementation

[0048] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0049] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0050] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0051] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0052] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0053] Without further limitations, the use of terms such as “comprising,” “including,” “having,” or other similar open-ended expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0054] As understood in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0055] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0056] The processor described in the embodiments of this application can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0057] The computer program involved in the embodiments can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical, or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributed and stored in multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device, or can be connected to the device involved in the embodiments as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0058] See Figure 1As shown in the figure, an embodied intelligent motion planning method based on the VLA model acquires multimodal data of the embodied agent to achieve complete collection of task objectives, environmental perception, and agent state information. The multimodal data comprehensively covers natural language commands input by the user, environmental scene images collected by the visual sensors on the embodied agent, and state vectors of the end effector. Subsequently, the multimodal data is input into a preset VLA (Vision-Language-Action) model, which is then converted into control commands that the end effector can directly execute, driving the agent to accurately complete task actions. By integrating multimodal heterogeneous data and relying on the cross-modal fusion and reasoning capabilities of the VLA model, this method effectively solves the problems of inaccurate understanding of task commands, poor environmental adaptability, and insufficient coordination between agent state and action in traditional motion planning methods. It significantly improves the task execution accuracy, action coherence, and environmental robustness of the embodied agent in unstructured scenarios, and is suitable for application scenarios such as multimodal human-computer interaction and agent action control.

[0059] The following combination Figure 1 and Figure 2 This paper provides an implementation method for embodied intelligent motion planning based on a VLA model, wherein the VLA model includes a multimodal sub-model and an action generation sub-model; the method includes the following steps:

[0060] S10. Acquire embodied intelligent agent multimodal data, wherein the embodied intelligent agent multimodal data includes user instructions, images collected by visual sensors on the embodied intelligent agent, and state vectors of the embodied intelligent agent end controller;

[0061] S20. Input the instructions and images into the multimodal sub-model to obtain the multimodal fusion vector;

[0062] S30. Based on the multimodal fusion vector and the state vector, obtain the embedding vector;

[0063] S40. Input the embedded vector into the action generation sub-model to obtain the action sequence;

[0064] S50. Map the action sequence to the vector space of the embodied agent's controller and execute the action.

[0065] In step S10, the user's instructions can be obtained through one or more of the following interaction methods adapted to the application scenario of the embodied intelligent agent: First, natural language voice interaction, which collects the user's voice instructions through the microphone on the embodied intelligent agent and converts them into text instructions through the voice recognition module; Second, text instruction input, which allows the user to manually input text instructions through the interactive interface of a smart terminal (such as a mobile phone or tablet) and transmit them to the embodied intelligent agent through the communication module; Third, preset instruction triggering, which triggers the corresponding instruction based on predefined gestures, buttons, or specific scene events (such as detecting a target object).

[0066] In step S10, the images acquired by the visual sensors on the embodied intelligent agent can be one or more of the following: RGB images obtained from the global perspective by the fixed frontal camera of the embodied intelligent agent, RGB images obtained from the wrist perspective by the wrist camera of the embodied intelligent robotic arm, etc. To adapt to the VLA model training input format, it is preferable to crop or pad the RGB images to a size of (224, 224, 3) and convert them to a channel-priority size of (3, 224, 224).

[0067] In step S10, the state vector of the embodied intelligent agent end controller is constructed by reading the joint pose, motion speed, force torque and other body operation parameters of the end effector, and then integrating the above parameters in an orderly manner according to the preset dimension arrangement rules, so as to finally construct a state vector that can comprehensively represent the current operating state of the end effector.

[0068] In step S20, the multimodal sub-model is used to extract visual feature vectors from the image and linguistic feature vectors from the instructions, and then concatenates and fuses them to form a context-consistent multimodal fusion vector. Specifically, the multimodal sub-model includes:

[0069] A visual encoder is used to convert the image into a visual feature vector;

[0070] A language model is used to convert the instructions into language feature vectors;

[0071] The fusion module is used to concatenate and fuse the visual feature vector and the language feature vector to generate a multimodal fusion vector.

[0072] See Figure 3 As shown, specifically, the step of inputting instructions and images into the multimodal sub-model to obtain a multimodal fusion vector includes the following steps:

[0073] S201. Input the image into a visual encoder to generate a visual feature vector;

[0074] S202. The instruction is input into the language model. The word segmenter of the language model segments the instruction to obtain the corresponding tokenized representation. Then, the language model is used to convert the token into a language feature vector with a plan.

[0075] S203. Map the visual feature vectors to the same embedding space as the language feature vectors generated by the language model;

[0076] S204. The visual feature vector and language feature vector after embedding space alignment are input into the fusion module and connected based on the sequence dimension of the vectors to generate a multimodal fusion vector.

[0077] See Figure 4 As shown, step S30, the step of obtaining the embedding vector based on the multimodal fusion vector and the state vector, includes the following steps:

[0078] S301. Perform sine-cosine position encoding on the time step of the action performed by the embodied intelligent agent to obtain the time step vector;

[0079] S302. The state vector and time step vector are concatenated and input into a multilayer perceptron (MLP), and processed by a nonlinear activation function to obtain a spatiotemporal action feature vector.

[0080] S303. Linearly concatenate the multimodal fusion vector and the spatiotemporal action feature vector to obtain the embedding vector.

[0081] Specifically, a nonlinear transformation can be performed using two layers of MLP and the Swish activation function to further fuse action and time information and obtain a spatiotemporal action feature vector.

[0082] See Figure 5 As shown, step S40, which involves inputting the embedding vector into the action generation sub-model to obtain the action sequence, includes the following steps:

[0083] S401: Initialize the Gaussian noise vector n;

[0084] S402: Perform noise prediction on the embedded vector input action generation sub-model to obtain the noise velocity field v. t ;

[0085] S403: Perform denoising iterations on the Gaussian noise vector n using an Euler-like method, including:

[0086] Set the time step of the denoising process to d. t ;

[0087] Iterate step by step from the initial time step t1 to the target time step t0, using the noise velocity field v t To guide this process, follow the formula n→n+vt ·d t Update the Gaussian noise vector n;

[0088] S404: Repeat the iterative process of steps S402-S403 to obtain the final action sequence.

[0089] The aforementioned Gaussian noise vector has the same dimension as the target action sequence and serves as the initial input for generating the action sequence. Its dimension matches the degrees of freedom and action parameter dimensions of the embodied agent's end effector. The aforementioned velocity field v t Characterizes the direction and magnitude of iterative updates of the noise vector.

[0090] The iterative process of repeating steps S402-S403 is to re-predict the noise velocity field and complete the denoising calculation based on the updated vector in each round until the disordered components in the noise vector are completely eliminated; the final output denoised vector is the action sequence that meets the task requirements and is temporally coherent.

[0091] The action generation sub-model can be a diffusion-based model that receives the embedding vector as conditional input to obtain action sequences. Specifically, the action generation sub-model uses a pre-trained diffusion model as its foundation. During the training phase, noise is added to the clean action sequences to teach the model a denoising process. During the inference phase, noise is gradually removed from the pure noise to generate the action sequences. Specifically, the training of the action generation sub-model includes: adding noise to the action sequence x0 in the dataset over time from t0 to t1 to generate noisy action x. t The total noise added is u t , that is, x t = x0+ u t The training objective of the model is to predict the correct u. t The noise n to be predicted is generated by inputting the embedded vector into the model and performing a forward Markov process. t Minimize the mean square error (MSE) between predicted noise and actual noise. t ,n t During the training of the action generation sub-model, supervision is provided based on conditional flow matching loss to guide the action generation sub-model in learning the mapping from noisy action sequences to real action sequences.

[0092] The action generation sub-model based on the diffusion process, using embedding vectors as conditional inputs and employing conditional flow matching loss for supervised training, can progressively generate high-quality action sequences from noise. During the inference phase, the model generates action sequences at a high frequency, significantly improving the fluency and response speed of the embodied agent when executing complex natural language commands.

[0093] See Figure 6As shown, in step S50, the step of mapping the action sequence to the vector space of the controller of the embodied intelligent agent and executing the action includes:

[0094] S501, Obtain the model and action vector space of the embodied intelligent agent;

[0095] S502. Based on the model and action vector space of the embodied intelligent agent, the action sequence is subjected to inverse normalization processing;

[0096] S503. Map the denormalized action sequence to control instructions for the end effector, and execute the corresponding actions based on the control instructions.

[0097] The model and motion vector space of the embodied intelligent agent can be read from its configuration information. Specifically, this determines the model of its robotic arm and the specific controller space for reading the motion vector space of each joint of the robotic arm. During the execution phase, the embodied intelligent agent reads its configuration information to denormalize the motion sequence and map it to the specific controller space, achieving efficient and precise motion execution.

[0098] This application also includes, during the execution of the action step, step S60, periodically determining whether the embodied intelligent agent has completed the user's instruction; if the instruction is completed, the embodied intelligent agent is controlled to return to a preset pose; otherwise, all steps are repeated until the instruction is completed. For example, each action sequence obtained by the VLA model contains 50 time-step action vectors, which are added to the action storage stack for execution by the embodied intelligent agent. When the set time step is reached, all steps are repeated (using the newly acquired image and state vectors as input to the VLA model) to obtain a new action sequence and add it to the action storage stack to ensure the smoothness of the embodied intelligent agent's action execution. Until the instruction is completed, the action sequence no longer controls the embodied intelligent agent. Since the action vectors in the generated action sequence are relative vectors, if the instruction has been completed, the generated embedding vector will guide the action expert to generate a zero vector. Specifically, the step of determining whether the embodied intelligent agent has completed the user's instruction includes the following steps:

[0099] Obtain the new image and state vector;

[0100] Based on the new image and state vectors, and the instruction input based on the VLA model, it is determined whether the obtained action sequence is a zero vector; if so, the instruction is completed; otherwise, the instruction is not completed.

[0101] When the actuator receives a series of zero vectors, it will trigger the action to restore the default pose, stopping the original motion inference loop until a new instruction input is received.

[0102] By employing a periodic loop reasoning mechanism to dynamically update multimodal context information and combining the general reasoning ability of the multimodal model, the system determines whether the task has been completed in each loop. When the task is determined to be completed, the action sequence generated by the action generation model will indicate that the embodied agent will no longer be controlled. The embodied agent will then stop operating and reset, ensuring the efficient and accurate completion of the task.

[0103] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. An embodied intelligent motion planning method based on the VLA model, characterized in that, The VLA model includes a multimodal sub-model and an action generation sub-model; the method includes the following steps: Acquire embodied agent multimodal data, which includes user commands, images collected by visual sensors on the embodied agent, and state vectors of the embodied agent end controller; Instructions and images are input into the multimodal sub-model to obtain a multimodal fusion vector; The embedding vector is obtained based on the multimodal fusion vector and the state vector; The embedded vectors are input into the action generation sub-model to obtain the action sequence; The action sequence is mapped to the vector space of the embodied agent's controller, and the action is executed.

2. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, The multimodal sub-model includes: A visual encoder is used to convert the image into a visual feature vector; A language model is used to convert the instructions into language feature vectors; The fusion module is used to concatenate and fuse the visual feature vector and the language feature vector to generate a multimodal fusion vector.

3. The embodied intelligent motion planning method based on the VLA model according to claim 2, characterized in that, The step of inputting instructions and images into the multimodal sub-model to obtain a multimodal fusion vector includes the following steps: The image is input into a visual encoder to generate a visual feature vector; The instruction is input into the language model, the language model's word segmenter segments the instruction into words to obtain the corresponding tokenized representation, and then the language model is used to convert the token into a language feature vector with a plan. Map the visual feature vectors to the same embedding space as the language feature vectors generated by the language model; The spatially aligned visual feature vectors and language feature vectors are input into the fusion module and concatenated based on the sequence dimension of the vectors to generate a multimodal fusion vector.

4. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, The step of obtaining the embedding vector based on the multimodal fusion vector and the state vector includes the following steps: The time steps of the actions performed by the embodied intelligent agent are encoded using sine-cosine position encoding to obtain the time step vector; The state vector and time step vector are concatenated and input into a multilayer perceptron, and then processed by a nonlinear activation function to obtain a spatiotemporal action feature vector. The multimodal fusion vector and the spatiotemporal action feature vector are linearly concatenated to obtain the embedding vector.

5. The embodied intelligent motion planning method based on the VLA model according to claim 1, wherein the step of inputting the embedded vector into the action generation sub-model to obtain the action sequence includes the following steps: Initialize the Gaussian noise vector n; The embedded vector input action generation sub-model is used for noise prediction to obtain the noise velocity field v. t ; An Euler-like method is used to perform denoising iteration on the Gaussian noise vector n; Repeat the above iterative process to obtain the final action sequence.

6. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, The action generation sub-model uses a pre-trained diffusion model as its foundation. By adding noise to the clean action sequence during the training phase, the model learns the denoising process. During the inference phase, it gradually denoises the pure noise to generate the action sequence.

7. The embodied intelligent motion planning method based on the VLA model according to claim 6, characterized in that, During the training of the action generation sub-model, supervision is provided based on conditional flow matching loss to guide the action generation sub-model in learning the mapping from noisy action sequences to real action sequences.

8. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, The step of mapping the action sequence to the vector space of the controller of the embodied intelligent agent and executing the action includes: Obtain the model and action vector space of the embodied intelligent agent; Based on the model and action vector space of the embodied intelligent agent, the action sequence is subjected to inverse normalization processing; The denormalized sequence of actions is mapped to control instructions for the end effector, and the corresponding actions are executed based on the control instructions.

9. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, It also includes determining whether the embodied intelligent agent has completed the user's instruction when performing the action steps; if the instruction is completed, the embodied intelligent agent is controlled to return to a preset pose; otherwise, all steps are repeated until the instruction is completed.

10. The embodied intelligent motion planning method based on the VLA model according to claim 1, characterized in that, The step of determining whether the embodied intelligent agent has completed the user's instructions includes: Obtain the new image and state vector; Based on the new image and state vectors, and the instruction input based on the VLA model, it is determined whether the obtained action sequence is a zero vector; if so, the instruction is completed; otherwise, the instruction is not completed.

Images