Target injection type fine tuning method for visual language action model

Abstract

The invention discloses a visual language action model-oriented target injection type fine tuning method, which comprises the following steps of: firstly, constructing any existing visual language action model, introducing a condition image generation model, and generating a target image with consistent semantics and vision according to an initial observation image and a task target instruction; secondly, target image features are injected into observation input through zero-initialization convolution, parameters are gradually increased from zero, it is ensured that interference noise is not introduced in the initial stage of fine adjustment to destroy a pre-training strategy, and in the training process, along with gradual optimization of the parameters, target image feature information is gradually fused into model representation, and the target image feature information is obtained; therefore, the understanding ability and the execution performance of the task target are improved. According to the method, through a lightweight target image injection mechanism and an efficient fine adjustment process, the performance of the model on various reference tasks can be remarkably improved in few training rounds, and the problem that an existing visual language action model cannot systematically introduce target image guidance is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of computer modeling technology, and in particular to a target-injection fine-tuning method for visual language action models. Background Technology

[0002] Enabling robots to complete tasks based on natural language instructions is a crucial research goal in the fields of artificial intelligence and robotics. Vision-Language-Action (VLA) models combine visual perception with language understanding to map instructions and environmental awareness to action generation. However, in recent years, the scarcity and distribution of robot data have limited the generalization ability of VLA models in out-of-distribution scenes and with novel objects. To improve the task understanding and execution capabilities of these models, some works have introduced generative models trained on large-scale internet data to generate target images as high-level visual guidance to assist low-level policies. However, existing methods either rely on small policy networks with limited capacity or cause input mismatch when directly introducing the generated target into the pre-trained VLA model, thus disrupting existing policies and incurring high computational costs. This problem highlights the urgent need for a lightweight and efficient mechanism that can seamlessly inject target image information into pre-trained VLA models without violating existing policies, thereby enhancing their task generalization and execution performance. Summary of the Invention

[0003] This invention provides a target-injection-based fine-tuning method for visual language action models, addressing the problems of high computational overhead, significant input distribution mismatch, and insufficient multi-task collaborative optimization in existing methods when introducing target images. This method achieves stable injection of target features into the multimodal representation space by introducing generative visual priors and employing a progressive fusion strategy, significantly improving the task understanding and execution capabilities of embodied agents.

[0004] This invention provides a target-injection fine-tuning method for visual language action models, comprising the following steps: Using the observed images and task target instructions acquired by the embodied intelligent agent at the initial stage of the task as input, a pre-trained conditional image generation model is used to generate a target image that is consistent with the task target state in terms of semantic understanding and visual features. The target image is input into a visual encoder to extract features, and then gradually injected into the observation channel of the visual language action model through zero-initialization convolution. The convolution parameters are gradually updated from zero, and the feature information of the target image is integrated into the multimodal representation of the visual language action model in a progressive fusion manner to obtain a trained visual language action model.

[0005] Optionally, in one embodiment of the present invention, the target image is generated by a refined image editing method. The refined image editing method uses the first frame of the task demonstration trajectory as the initial observation image and the last frame as the target reference image, and combines the task target instructions for semantic guidance to generate a target image with visual and semantic consistency.

[0006] Optionally, in one embodiment of the present invention, the conditional image generation model optimizes the semantic constraints and image consistency to ensure that the generated target image conforms to the task target description in terms of scene layout, object pose, and lighting features.

[0007] Optionally, in one embodiment of the present invention, zero-initialization convolution introduces target image features by gradually increasing the parameters, so that the visual language action model remains stable in the early stage of fine-tuning and fully integrates target image feature information in the subsequent training stage to improve the generalization performance of the visual language action model.

[0008] This invention presents a target-injection fine-tuning method for visual language action models. By introducing generated target image features based on initial observations and task instructions, and employing a progressive fusion strategy using zero-initialization convolutions, it achieves stable injection of target information. This enhances the model's understanding and execution capabilities of the task target without compromising the structure and performance stability of the pre-trained strategy. Simultaneously, it combines refined image editing methods to generate target images that are highly consistent semantically and visually, further strengthening the representation learning effect guided by the target. This method fully leverages the advantages of generative models in target image prediction and state reconstruction, and utilizes the progressive fusion mechanism of zero-initialization convolutions to stably introduce generated target image features into the pre-trained visual language action model with low perturbation, thereby significantly improving the model's task adaptability and generalization ability in complex scenarios. Experimental results show that, while maintaining the stability of the pre-training strategy and low computational overhead, this method effectively alleviates the input distribution mismatch problem when introducing target images in existing methods, and significantly improves the task success rate and cross-scene generalization performance. Compared with traditional fine-tuning methods that rely solely on initial observations and language instructions, the framework proposed in this invention has achieved significant performance improvements in multiple standard embodied intelligence benchmark tests, demonstrating stronger visual alignment capabilities and multi-task collaborative optimization effects.

[0009] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0010] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1A flowchart of a target-injection fine-tuning method for visual language action models provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a specific application embodiment of the present invention; Figure 3 This is a schematic diagram of the image editing method of the present invention. Detailed Implementation

[0011] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0012] Figure 1 This is a flowchart of a target-injection fine-tuning method for visual language action models provided according to an embodiment of the present invention.

[0013] like Figure 1 As shown, this target-injection fine-tuning method for visual language action models includes the following steps: Step 1: Using the observation images and task target instructions acquired by the embodied agent in the initial stage of the task as input, a pre-trained conditional image generation model is used to generate a target image that is consistent with the task target state in terms of semantic understanding and visual features.

[0014] The conditional image generation model optimizes the scene layout, object pose, and lighting features of the generated target images through semantic constraints and image consistency optimization, ensuring they conform to the task objective description. The target images are generated using a refined image editing method. This method uses the first frame of the task demonstration trajectory as the initial observation image and the last frame as the target reference image, combined with semantic guidance based on the task objective instructions, to generate target images with high visual and semantic consistency. Through multi-stage editing and local structure preservation optimization, the generated images conform to the actual target scene in terms of geometric relationships, color distribution, and target state in key regions, thus providing the model with a high-fidelity visual representation of the target and enhancing the semantic consistency and visual guidance of the training samples.

[0015] Specifically, the method first constructs any existing Visual-Language-Action (VLA) foundational model, which can be a multimodal Transformer structure or an embodied intelligent model containing visual encoding, language parsing, and action prediction modules. Based on this, a generative image editing module is introduced to generate target images of the task target state according to task instructions and initial observations. The conditional image generation model can employ conditional generation methods based on diffusion mechanisms or control networks. Through semantic constraint loss and reconstruction consistency constraints, the generated target images maintain consistency with the task target description in terms of scene geometry, object layout, color features, and lighting conditions. This method can further achieve local modification of the target state through refined image editing techniques. For example, using the first frame of the demonstration trajectory as the initial observation and the last frame as the target reference, combined with the task language description, local editing of the target region can generate images with high visual and semantic consistency, providing intuitive visual guidance for subsequent action planning and model optimization.

[0016] Step 2: Input the target image into the visual encoder to extract features, and then gradually inject them into the observation channel of the visual language action model through zero-initialization convolution. This allows the convolution parameters to be updated gradually from zero, integrating the target image feature information into the multimodal representation of the visual language action model in a progressive fusion manner, thus obtaining the trained visual language action model.

[0017] Zero-initialization convolutions introduce target image features through a gradual parameter growth approach. In the initial fine-tuning phase, weights are frozen or the learning rate is low, with only minor parameter updates to ensure model convergence stability in the early stages. As training epochs increase, convolution parameters are gradually unfrozen, and the learning rate is increased, allowing target features to fully participate in multimodal fusion and representation optimization in later stages. Through this phased parameter growth mechanism, the model can smoothly absorb the visual priors provided by the target image, reducing gradient oscillations and mode drift, thereby significantly improving generalization performance.

[0018] Specifically, the generated target image is input into a pre-trained visual encoder to extract a high-dimensional visual feature tensor. This feature is then mapped to the same dimensional space as the original observed features through a zero-initialization convolutional module. The zero-initialization convolutional module has all its initial parameters set to zero, meaning that in the early stages of training, its output has zero impact on the backbone network, thus preventing the pre-training strategy from being disrupted by strong perturbations in the early stages of fine-tuning. As training progresses, the convolutional parameters are gradually updated through backpropagation, allowing the target features to be slowly introduced into the model's representation space, achieving a smooth transition from "frozen" to "fused." This progressive fusion strategy enables the model to gradually learn the visual semantic features of the task target while maintaining training stability, thereby significantly improving the accuracy of action planning and the success rate of task execution. Furthermore, this mechanism effectively avoids the problem of abrupt changes in input distribution caused by target image injection, ensuring the convergence stability and generalization ability of the model during multi-task training.

[0019] Zero-initialized convolution introduces target image features by gradually increasing the parameters, keeping the model stable in the early stages of fine-tuning and preventing performance degradation due to gradient oscillations or modal drift. In subsequent training stages, by gradually unfreezing or dynamically adjusting the learning rate, the convolution parameters can fully learn the distribution of target features, achieving deep fusion of multimodal information and ultimately improving the model's generalization ability and task adaptability in complex scenarios.

[0020] For example, in a robotic arm grasping task, the initial observation image consists of multiple objects on a table, and the task instruction is "pick up the red cup and place it on the tray on the right." The system uses an image generation model to generate a target image of the red cup already placed on the tray, and then gradually injects the target features into the vision-language-action model through zero-initialization convolution. After target-injection fine-tuning, the model can accurately identify target objects under different lighting and background conditions, stably complete grasping and placement operations, and significantly improve the task execution success rate and generalization ability.

[0021] Figure 2A specific application example is demonstrated. In this embodiment, the embodied agent performs the task of "putting the chocolate in the drawer." The system first acquires environmental observation images via a camera at the initial stage of the task, including the spatial relationships between the tabletop, the drawer, and the target object (chocolate). Then, combining the verbal instruction "put the chocolate in the drawer," a pre-trained conditional image generation model is used to generate the task target image, i.e., a scene where the chocolate has been correctly placed in the drawer. Next, the system extracts features from both the initial observation image and the generated target image, obtaining corresponding visual feature vectors, and injects these features through a zero-initialization convolution module. In the early stages of training, this module's output is close to zero, causing almost no interference to the model backbone; as training iterations increase, the module parameters are gradually updated, and the target image features are progressively fused into the multimodal representation space of the vision-language-action model. After training, the model can generate precise action sequences based on task semantics and visual features, outputting grasping and placement control commands for the robotic arm through the action head module, thereby achieving stable grasping and accurate placement of the chocolate, successfully completing the target task.

[0022] Figure 3 This paper demonstrates the implementation flow of the image editing module proposed in this invention. In this embodiment, the system receives the natural language task instruction "fold clothes" and converts the instruction into a semantic feature vector through a language encoder to guide the subsequent image generation process. This semantic feature contains semantic information about the task action "folding" and the object "clothes," enabling the model to understand the task target state. Subsequently, the system inputs the observation images collected by the embodied agent in the initial stage of the task into a pre-trained text-to-image diffusion model (T2I DiT). The model first performs an inversion operation on the observation images, mapping them to a latent noise space to preserve the scene's geometric structure and lighting information; simultaneously, it samples random noise as the starting point for generation to enhance the diversity and robustness of the generation. In the diffusion generation stage, the model takes the inversion results of the observation images, random noise, and task semantic features as inputs, performs multiple rounds of denoising iterations (denoising × N), and guides the noise elimination direction according to the semantic features in each iteration, so that the generated result gradually approaches the target state of "folded clothes." Ultimately, the model outputs a target image that is semantically consistent with the task and matches the visual features of the original scene, providing a target-oriented visual supervision signal for the vision-language-action model, thereby improving the task understanding and execution capabilities of the embodied agent.

[0023] The target-injection fine-tuning method for visual-language-action models proposed in this invention first constructs an arbitrary existing visual-language-action base model and introduces a generative image generation module to generate a target image with semantically consistent visual representation based on the initial observation image and task instructions. Then, a zero-initialization convolution module is used to inject the target image features into the observation input, allowing the parameters to gradually increase from zero to ensure that no noise is introduced to interfere with the pre-training strategy in the early stages of fine-tuning. As training progresses, the convolution parameters are gradually optimized, and the target information is stably and progressively fused into the multimodal representation space, thereby improving the model's accuracy in understanding the task target and its execution performance. This invention, through a lightweight target injection mechanism and a progressive parameter update strategy, can significantly improve the model's performance on various embodied task benchmarks with very few training epochs, effectively solving the distribution mismatch and optimization instability problems existing in current VLA models when introducing target images, and achieving a dual improvement in task generalization performance and computational efficiency.

[0024] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0026] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

Claims

1. A target-injection fine-tuning method for visual language action models, characterized in that, Includes the following steps: Using the observed images and task target instructions acquired by the embodied intelligent agent at the initial stage of the task as input, a pre-trained conditional image generation model is used to generate a target image that is consistent with the task target state in terms of semantic understanding and visual features. The target image is input into a visual encoder to extract features, and then gradually injected into the observation channel of the visual language action model through zero-initialization convolution. The convolution parameters are gradually updated from zero, and the feature information of the target image is integrated into the multimodal representation of the visual language action model in a progressive fusion manner to obtain a trained visual language action model.

2. The method according to claim 1, characterized in that, The target image is generated by a refined image editing method. The refined image editing method uses the first frame of the task demonstration trajectory as the initial observation image and the last frame as the target reference image, and combines the task target instructions for semantic guidance to generate a target image with visual and semantic consistency.

3. The method according to claim 1, characterized in that, The conditional image generation model optimizes the image through semantic constraints and image consistency, ensuring that the generated target image conforms to the task objective description in terms of scene layout, object pose, and lighting features.

4. The method according to claim 1, characterized in that, Zero-initialization convolution introduces target image features by gradually increasing the parameters, which keeps the visual language action model stable in the early stages of fine-tuning and fully integrates target image feature information in subsequent training stages to improve the generalization performance of the visual language action model.

Images