VLA model language instruction noise filtering and robustness improving method oriented to body intelligence
By constructing a preprocessing pipeline based on a large language model, noise in the VLA model is identified and removed, solving the problem of the VLA model's robustness to complex language noise in real-world environments and achieving a significant improvement in task success rate.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-12
- Publication Date
- 2026-04-07
AI Technical Summary
Existing VLA models are insufficient in understanding and executing natural language instructions with complex linguistic noise in real-world environments. The lack of effective preprocessing methods in current technologies leads to a decrease in robustness.
A preprocessing pipeline based on a large language model is constructed, including instruction parsing, noise detection and filtering, and instruction reconstruction modules. It identifies and removes contextual noise that is irrelevant to the task and standardizes the core instructions into a structure that the VLA model can recognize.
It significantly improved the success rate of VLA models in real-world scenarios, restoring it to near-original levels, and enhanced the reliability and robustness of embodied intelligence systems.
Smart Images
Figure CN121811901A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of embodied intelligence technology, and to a method for improving the robustness of robots in understanding and executing natural language instructions. Specifically, it is a method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence. Background Technology
[0002] Embodied intelligence systems, through vision-language-action models, enable robots to understand and execute natural language commands, becoming a key technology for robots to integrate into everyday environments. The VLA model, by combining a large language model with visual perception, can transform user commands into specific robot action sequences, demonstrating broad application potential in real-world scenarios such as home services and industrial production.
[0003] However, in real-world human-computer interaction scenarios, user-issued natural language commands often contain complex linguistic elements, posing a significant challenge to the robust deployment of VLA models. First, when issuing commands, humans frequently incorporate contextual information unrelated to the core task, or use diverse synonyms or interrogative sentence structures. The training data for existing mainstream VLA models is mostly concise, standardized templated commands, leading to a significant decrease in their ability to understand these natural language variations; studies have observed a potential drop in task success rate of approximately 20%. Second, when commands are mixed with irrelevant content that is semantically or lexically similar to the training data, the model is more susceptible to interference, resulting in target confusion and the execution of incorrect action sequences, with success rates dropping by more than 50%. Existing research on the robustness of VLA models is typically limited to a finite number of templated command variations, lacking systematic analysis and effective solutions for the rich and complex linguistic noise in real-world scenarios.
[0004] To address these issues, directly retraining the VLA model to cover all possible language variants presents limitations such as high data collection costs, enormous computational overhead, and difficulty in guaranteeing generalization performance. Therefore, there is an urgent need for a method that can preprocess instructions before model execution, effectively filter out noise, and extract core intents to improve the practicality and reliability of existing VLA models in real-world environments at low cost and high efficiency. Summary of the Invention
[0005] The purpose of this invention is to propose a method for noise filtering and robustness enhancement of VLA models for embodied intelligence, in order to overcome the limitations of existing VLA models in real-world scenarios due to the decrease in robustness caused by natural language perturbations, and to improve the success rate and accuracy of VLA models in understanding and executing natural language instructions containing noise.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] A method for noise filtering and robustness improvement of language instructions in VLA models for embodied intelligence includes the following steps:
[0008] S1, construct an instruction parsing module based on a large language model to receive natural language instructions containing irrelevant context;
[0009] S2, through the noise detection and filtering module, perform semantic analysis on the natural language instructions to identify and remove contextual content that is irrelevant to the task;
[0010] S3 uses the instruction reconstruction module to convert the filtered core instructions into standardized structure instructions that the VLA model can recognize.
[0011] S4, the standardized instructions are input into the VLA model to drive the embodied intelligence system to perform the corresponding tasks; wherein, the instruction parsing module, the noise detection and filtering module, and the instruction reconstruction module are connected in sequence to form a preprocessing pipeline model from noisy instructions to executable instructions.
[0012] A further technical solution involves constructing an instruction parsing module based on a large language model, which receives natural language instructions containing irrelevant context as follows:
[0013] The instruction parsing module uses a pre-trained large language model to extract key actions and object information from instructions through few-shot prompts.
[0014] The pre-trained large language models include large language models with different parameter sizes: micro large language model (Flan-T5 Base, Qwen 2.5 0.5B Instruct), medium large language model (Qwen 2.5 3B Instruct, Llama 3.2 3B Instruct), and standard large language model (Meta-Llama-3-8B-Instruct).
[0015] The few-sample prompting method involves providing the large language model with example pairs containing noisy instructions and corresponding filtered instructions, with at least three examples covering different noise types.
[0016] A further technical solution involves using a noise detection and filtering module to perform semantic analysis on the natural language instructions, and then identifying and removing contextual content irrelevant to the task. The method is as follows:
[0017] The noise detection and filtering module classifies natural language instructions into random irrelevant noise, semantic noise, and task-infeasible noise based on semantic relevance and word similarity, and then performs filtering processing on each of them.
[0018] The classification includes: the first category of random irrelevant noise, which is semantically and lexically unrelated to the training instructions and has a variable context length, specifically including one or more random phrases unrelated to the robotic arm commands; the second category of semantically similar noise, which includes semantically or lexically similar to the training instructions, specifically including short phrases describing random objects in the scene, as well as information about the object's location or the action to be performed on the object; and the third category of task-infeasible noise, which has a grammatical structure similar to the training instructions but describes actions that the robotic arm cannot execute.
[0019] A further technical solution involves using an instruction refactoring module to convert the filtered core instructions into standardized structure instructions recognizable by the VLA model, as follows:
[0020] The instruction reconstruction module structures the filtered text into a format that conforms to the VLA model training instruction template, consisting of action verbs, target objects, and location information.
[0021] A further technical solution involves inputting the standardized structural instructions into the VLA model to drive the embodied intelligence system to execute corresponding tasks. The instruction parsing module, noise detection and filtering module, and instruction reconstruction module are sequentially connected, forming a preprocessing model from noisy instructions to executable instructions, as follows:
[0022] The method was validated in the simulation environments LIBERO and Habitat 2.0, and its robustness was improved on multiple different VLA models.
[0023] The LIBERO simulation environment selected for model validation is specifically designed for studying declarative and procedural knowledge transfer in large-scale simulations. It consists of four different test task groups: LIBERO Goal, LIBERO Object, LIBERO Spatial, and LIBERO Long. LIBERO Goal contains 10 short-lived tasks requiring the robot to transfer and memorize new spatial relationships; LIBERO Object contains 10 short-lived tasks centered on learning new object types, where the robot must sequentially select and place different objects; LIBERO Spatial contains 10 short-lived tasks sharing the same objects and spatial layout, differing only in their procedural objectives, used to test the transfer of motion and behavioral knowledge; and LIBERO Long contains 10 long-lived tasks for downstream evaluation of lifelong learning algorithms. Another simulation environment, Habitat 2.0, is a high-performance simulator that includes navigation tasks and can render RGB-D observation scenes and simulate rigid body dynamics at speeds exceeding 8000Hz in realistic 3D scenes.
[0024] Among the various VLA models selected for filtering model validation, four of the most state-of-the-art and popular models—OpenVLA, UniAct, MODE, and π0—were evaluated in the LIBERO simulation environment; the LLARP model was evaluated in the Habitat 2.0 simulation environment.
[0025] The noise filtering process restores the VLA model's task success rate under noisy instructions to over 98.5% of the original success rate.
[0026] The beneficial effects of this invention are as follows:
[0027] This invention proposes a method for noise filtering and robustness enhancement of VLA models for embodied intelligence. By constructing a preprocessing pipeline model based on a large language model, it can effectively identify and filter various irrelevant contextual noises in natural language instructions and standardize core instructions. Compared with directly retraining the VLA model, this invention provides a low-cost and high-efficiency robustness enhancement scheme. This method can significantly improve the reliability of VLA models in real human-computer interaction scenarios. For example, it can restore the task success rate of VLA models when subjected to semantically similar noise interference from a drop of more than 50% to near the original level. It can also effectively alleviate the performance degradation caused by instructions naturally paraphrased by humans. This invention has been verified as effective in mainstream simulation environments such as LIBERO and Habitat 2.0, as well as various VLA models such as OpenVLA, UniAct, and LLARP, demonstrating broad applicability and providing key technical support for embodied intelligence systems to understand complex human language instructions in practical deployments. Attached Figure Description
[0028] To more clearly illustrate the implementation of the present invention or the existing technical solutions, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0029] Figure 1 This is a schematic diagram of the overall processing flow of the method described in this invention;
[0030] Figure 2 This is a schematic diagram illustrating the impact of noisy commands on robot task execution in real human-computer interaction.
[0031] Figure 3 This is a schematic diagram illustrating the core classification logic principle of the noise detection and filtering module.
[0032] Figure 4 This is a diagram of an invalid robot motion sequence demonstrated in the Habit 2.0 simulator under the influence of irrelevant contextual noise.
[0033] Figure 5This is a comparison diagram of the natural language instructions input to the VLA model in the method of the present invention before and after filtering.
[0034] Figure 6 To compare the average proportions of the original commands after filtering various noise commands with LLMs filters of different parameter scales in the LIBERO benchmark simulation environment.
[0035] Figure 7 A comparison of the impact of using LLMs with different parameter sizes as filters on the success rate of the LLARP model after processing various noise instructions. Detailed Implementation
[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0037] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0038] like Figure 1 As shown in the figure, this invention discloses a method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence. The overall processing flow includes the following steps:
[0039] S1, construct an instruction parsing module based on a large language model. This module receives natural language instructions from the user, which may contain irrelevant context. For example... Figure 2 As shown, these types of noisy instructions are a common cause of robot task failure in real human-computer interaction. The core of the module is a pre-trained large language model. In one embodiment of this invention, the Meta-Llama-3-8B-Instruct model is used, guiding the model to extract the core intent from the instruction through few-shot prompts. The prompts contain three example pairs, each showing a type of noise and its filtered command.
[0040] S2, through a noise detection and filtering module, performs semantic analysis on the natural language instructions to identify and remove contextual content irrelevant to the task. For example... Figure 3As shown, this module systematically categorizes noise into three types based on the semantic and lexical similarity between the noise and the VLA model training instructions: the first type is random irrelevant noise; the second type is semantically similar noise, including descriptive and positional noise; and the third type is task-infeasible noise. The goal of this module is to accurately identify and remove these noises while retaining the core commands. Figure 4 The invalid robot action sequence shown is the result of unfiltered positional noise ("On the sofa there's an apple") interfering with the VLA model's decision-making.
[0041] S3 utilizes the instruction reconstruction module to convert the filtered core instructions into standardized instructions recognizable by the target VLA model. This module performs adaptive reconstruction based on the training data format of the downstream VLA model. Figure 5 It demonstrates the contrast between instructions that contain complex noise and those that are reconstructed into a standardized format.
[0042] S4, the reconstructed standardized instructions are input into the target VLA model to drive the embodied intelligent system to execute the corresponding task. The standardized instructions are input into the VLA model to drive the embodied intelligent system to execute the corresponding task; wherein the instruction parsing module, noise detection and filtering module, and instruction reconstruction module are connected in sequence to form a preprocessing pipeline model from noisy instructions to executable instructions.
[0043] The effectiveness of the method has been verified through systematic experiments in mainstream simulation environments. On multiple task groups of the LIBERO benchmark, the filtering framework of this invention, after application, yielded the following results: Figure 6 As shown, using LLMs of different sizes as filters can effectively restore model performance. In the Habitat 2.0 simulator, test data for the LLARP model (such as...) Figure 7 As shown in the figure, even when faced with semantically similar noise, the task success rate can be significantly recovered after filtering. Comprehensive experimental results demonstrate that this method can restore the overall task success rate of the VLA model under noisy instructions to over 98.5% of the original success rate, significantly improving the robustness and reliability of the model in practical deployments.
[0044] It should be noted that, while this filtering framework is highly effective, it may, in rare cases, mistakenly filter out important details in instructions or lose some diversity when processing natural human paraphrasing. This suggests a direction for future optimization of adaptive filtering accuracy.
Claims
1. A method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence, characterized in that, Includes the following steps: S1, construct an instruction parsing module based on a large language model to receive natural language instructions containing irrelevant context; S2, through the noise detection and filtering module, perform semantic analysis on the natural language instructions to identify and remove contextual content that is irrelevant to the task; S3 uses the instruction reconstruction module to convert the filtered core instructions into standardized structure instructions that the VLA model can recognize. S4, the standardized structural instructions are input into the VLA model to drive the embodied intelligent system to perform the corresponding tasks; wherein, the instruction parsing module, noise detection and filtering module, and instruction reconstruction module are connected in sequence to form a preprocessing pipeline model from noise instructions to executable instructions.
2. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 1, characterized in that, In S1, the instruction parsing module uses a pre-trained large language model to extract key actions and object information from instructions through few-shot prompts.
3. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 2, characterized in that, The instruction parsing module uses pre-trained large language models, including large language models with different parameter scales, namely the micro large language model (Flan-T5 Base, Qwen 2.5 0.5B Instruct), the medium large language model (Qwen 2.5 3B Instruct, Llama 3.2 3B Instruct), and the standard large language model (Meta-Llama-3-8B-Instruct).
4. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 2, characterized in that, The few-sample prompting method involves providing the large language model with example pairs containing noisy instructions and corresponding filtered instructions, with at least three examples covering different noise types.
5. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 1, characterized in that, In S2, the noise detection and filtering module classifies natural language commands into random noise, semantic noise, and task-infeasible noise based on semantic relevance and lexical similarity, and then performs filtering processing on each. Random noise contains one or more random phrases unrelated to the robotic arm commands; semantic noise contains short phrases that arbitrarily describe random objects in the scene, containing information about the object's location or the action to be performed on that object; and task-infeasible noise contains commands that the robotic arm cannot execute.
6. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 5, characterized in that, The classification includes: the first category of random irrelevant noise, which is semantically and lexically unrelated to the training instructions and has a variable context length, specifically including one or more random phrases unrelated to the robotic arm commands; the second category of semantically similar noise, which includes semantically or lexically similar to the training instructions, specifically including short phrases describing random objects in the scene, as well as information about the object's location or the action to be performed on the object; and the third category of task-infeasible noise, which has a grammatical structure similar to the training instructions but describes actions that the robotic arm cannot execute.
7. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 1, characterized in that, In S3, the instruction reconstruction module structures the filtered text into a format that conforms to the VLA model training instruction template, consisting of action verbs, target objects, and location information.
8. The method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 1, characterized in that, In S4, the method is validated in the simulation environments LIBERO and Habitat 2.0, and its robustness is improved by being validated on multiple different VLA models.
9. A method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 8, characterized in that, The LIBERO simulation environment selected for model validation is specifically designed for studying declarative and procedural knowledge transfer in large-scale simulations. It consists of four different test task groups: LIBERO Goal, LIBERO Object, LIBERO Spatial, and LIBERO Long. LIBERO Goal contains 10 short-lived tasks requiring the robot to transfer and memorize new spatial relationships; LIBERO Object contains 10 short-lived tasks centered on learning new object types, where the robot must sequentially select and place different objects; LIBERO Spatial contains 10 short-lived tasks sharing the same objects and spatial layout, differing only in their procedural objectives, used to test the transfer of motion and behavioral knowledge; and LIBERO Long contains 10 long-lived tasks for downstream evaluation of lifelong learning algorithms. Another simulation environment, Habitat 2.0, is a high-performance simulator that includes navigation tasks and can render RGB-D observation scenes and simulate rigid body dynamics at speeds exceeding 8000Hz in realistic 3D scenes.
10. A method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence as described in claim 8, characterized in that, Among the various VLA models selected for filtering model validation, four of the most state-of-the-art and popular models—OpenVLA, UniAct, MODE, and π0—were evaluated in the LIBERO simulation environment; the LLARP model was evaluated in the Habitat 2.0 simulation environment.
11. A method for noise filtering and robustness improvement of VLA model language instructions for embodied intelligence according to claim 8, characterized in that, The noise filtering process restores the VLA model's task success rate under noisy instructions to over 98.5% of the original success rate.