Robot end side small model and VLA large model cooperative training method and device and robot system

Abstract

The invention relates to the technical field of model cooperative training, in particular to a robot end side small model and VLA large model cooperative training method and device and a robot system.The method comprises the steps that the weight coefficient of a VLA large model and a robot end side small model in cooperative training is determined based on the complexity evaluation result of a sample task; global semantic features output by the VLA large model are converted into first formatted data matched with the decision space of the robot end-side small model, and the first formatted data are transmitted to the robot end-side small model, so that the problem that the large and small models are mutually split and difficult to cooperate in traditional training is effectively solved, and the training efficiency is improved through dynamic weight distribution and a bidirectional knowledge interaction mechanism. The deep semantic understanding advantage of a large model is reserved, the real-time control characteristic of a small model is also played, a collaborative learning framework with complementary advantages is formed, and the intelligent decision-making ability and the overall performance of the robot in a complex scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model collaborative training, in particular to a robot end-side small model and VLA large model collaborative training method, device and robot system. BACKGROUND

[0002] In recent years, the trend of fusion development of robot technology and artificial intelligence technology is significant, especially the vision-language-action large model has shown strong semantic understanding and task planning ability in many vertical fields. The VLA large model provides higher level intelligent decision support for robot systems with its powerful global reasoning ability, significantly expanding the application range and control level of robots in complex scenarios.

[0003] However, directly deploying the VLA large model on the robot device end side still faces many challenges. On the one hand, the large model has a huge number of parameters and high computing resource requirements, making it difficult to run efficiently on resource-constrained end-side devices; on the other hand, the robot end side usually deploys a lightweight small model, which has the advantages of fast response and low resource occupation, but has limitations in understanding and long-term planning of complex tasks.

[0004] Currently, in the model training phase, the large model and the small model often use independent training or post-tuning methods, lacking effective collaboration mechanisms. The traditional "independent prediction / post-tuning" or "fixed weight joint training" method, due to the significant differences in calculation targets, data distribution and parameter size between the two types of models, is prone to gradient conflicts during training, or the small model is dominated by the large model, making it difficult to achieve true complementary advantages. In addition, during the actual control process of the robot, there is a lack of a mechanism that can dynamically allocate decision weights according to the complexity of the task, which cannot guarantee real-time performance while considering global semantic consistency.

[0005] Therefore, the existing technology has not yet provided a collaborative training and control architecture that can achieve deep integration of large models and small models, bidirectional knowledge transfer during the training phase, and dynamic weight adjustment during the control process. There is an urgent need for a new method that can effectively integrate the global planning capabilities of the VLA large model and the real-time control advantages of the end-side small model in the robot system to improve the overall intelligence level of the robot in complex dynamic environments. SUMMARY

[0006] Therefore, the present application aims to provide a robot end-side small model and VLA large model collaborative training method, device and robot system to solve the problem of difficulty in collaboration between large and small models and training fragmentation in the prior art.

[0007] In a first aspect, the embodiments of the present application provide a robot end-side small model and VLA large model collaborative training method, which comprises: determine the weight coefficients of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation results of the sample tasks; convert the global semantic features output by the VLA large model into first formatted data matching the decision space of the robot end-side small model, and transmit the first formatted data to the robot end-side small model, and convert the experience sequence data generated by the robot end-side small model based on the execution of the sample tasks into second formatted data matching the semantic space of the VLA large model, and transmit the second formatted data to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model; based on the collaborative training weight coefficients, fuse the first formatted data and the second formatted data to jointly construct the loss function of the VLA large model and the robot end-side small model, so as to collaboratively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

[0008] In combination with the first aspect, the step of determining the weight coefficients of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation results of the sample tasks comprises: extracting low-level features of the mixed data set through a low-level feature encoder, the mixed data set at least including visual, language and state data; inputting the low-level features into the VLA large model and the robot end-side small model to extract global task planning information and local control decision information of the low-level features; quantizing the global task planning information and the local control decision information to generate task complexity evaluation results; generating weight coefficients for the VLA large model and the robot end-side small model based on the task complexity evaluation results and the task resource status.

[0009] In combination with the first aspect, the step of quantizing the global task planning information and the local control decision information through the feature quantization module to generate the task complexity evaluation results comprises: extracting features from the global task planning information and the local control decision information based on the feature extraction module to obtain global task feature information and local strategy feature information; quantizing the global task feature information and the local strategy feature information based on the feature quantization module to obtain the task complexity evaluation results; The global task feature information includes one or more of the number of subtasks, the number of constraint conditions, the beat time, the task priority and the process level; the local strategy feature information includes one or more of the real-time requirement, the resource status, the perception information, the burst state and the call state of other devices.

[0010] According to the task complexity evaluation result and the task resource status, the weight coefficients of the VLA large model and the robot end-side small model are generated, and the steps include: Based on the comparison relationship between the task complexity evaluation result and the preset threshold, initial weight coefficients are assigned to the VLA large model and the robot end-side small model; Based on the task resource status, the initial weight coefficients are weighted and corrected to generate target weight coefficients corresponding to the VLA large model and the robot end-side small model; The task resource status includes at least one of the following: criticality of the current task, available computing resources of the system, memory occupation status, priority of the next task to be executed, density of the overall task queue, and system energy consumption status.

[0011] According to the first aspect, the bidirectional knowledge interaction channel is established through a gradient mapping layer, and the steps include: The first formatted data is mapped to the global semantic features through a gradient adapter in the gradient mapping layer; The second formatted data is compressed through an experience compressor in the gradient mapping layer.

[0012] According to the first aspect, the gradient adapter is a small MDP network model, and the experience compressor is an LSTM network or an attention network.

[0013] According to the first aspect, based on the weight coefficients, the first formatted data and the second formatted data are fused to jointly construct a loss function, and the steps include: The first formatted data and the second formatted data are weighted according to the weight coefficients; The weighted first formatted data is introduced into the loss function of the robot end-side small model as a semantic constraint term; The weighted second formatted data is introduced into the loss function of the VLA large model as an experience feedback term; The semantic constraint term is used to constrain the decision direction of the robot end-side small model, and the experience feedback term is used to optimize the planning strategy of the VLA large model.

[0014] According to the first aspect, the method further includes: Based on the trained VLA large model and the robot end-side small model, the decision weight is dynamically allocated according to the complexity evaluation result of the real-time task; If the complexity evaluation result is less than the first threshold, the robot end-side small model dominates the decision and the VLA large model provides semantic constraints; If the complexity evaluation result is greater than the second threshold, the VLA large model generates a global task plan, and the robot end-side small model is responsible for executing local action refinement; The first threshold is less than or equal to the second threshold.

[0015] In a second aspect, the embodiments of the present application also provide a device for collaborative training of a robot end-side small model and a VLA large model, configured to execute the method described above, and the device comprises: a weight determination module configured to determine a weight coefficient of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation result of the sample task; an interaction construction module configured to convert the global semantic features output by the VLA large model into first formatted data matched with the decision space of the robot end-side small model, and transmit the first formatted data to the robot end-side small model, and convert the experience sequence data generated by the robot end-side small model based on the execution of the sample task into second formatted data matched with the semantic space of the VLA large model, and transmit the second formatted data to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model; a collaborative training module configured to fuse the first formatted data and the second formatted data based on the collaborative training weight coefficient, and jointly construct a loss function of the VLA large model and the robot end-side small model, so as to collaboratively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

[0016] In a third aspect, the embodiments of the present application also provide a robot system, configured to execute the method described above, and the system comprises: a robot end-side small model deployed on a robot device, configured to execute local control decision; a VLA large model in communication connection with the robot end-side small model, configured to provide global semantic understanding and task planning; a dynamic task evaluation module configured to generate a weight coefficient of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation result of the sample task; a collaborative training architecture comprising a gradient mapping layer, and the gradient mapping layer is configured to: convert the global semantic features output by the VLA large model into the first formatted data, and transmit the first formatted data to the robot end-side small model; convert the experience sequence data generated by the robot end-side small model into the second formatted data, and transmit the second formatted data to the VLA large model; wherein the system is configured to fuse the first formatted data and the second formatted data based on the weight coefficient, so as to collaboratively train the VLA large model and the robot end-side small model.

[0017] In a fourth aspect, the present application provides an electronic device, which comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the method described above.

[0018] In a fifth aspect, the present application provides a readable storage medium, wherein the readable storage medium stores computer program instructions, and the computer program instructions are read and executed by a processor to perform the method described above.

[0019] The robot end-side small model and the VLA large model are trained in cooperation, the method comprises the following steps: based on the complexity evaluation result of the sample task, the weight coefficients of the VLA large model and the robot end-side small model in the cooperative training are determined; the global semantic features output by the VLA large model are converted into first formatted data matched with the decision space of the robot end-side small model, and are transmitted to the robot end-side small model, at the same time, the experience sequence data generated by the robot end-side small model based on the execution of the sample task are converted into second formatted data matched with the semantic space of the VLA large model, and are transmitted to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model; based on the cooperative training weight coefficients, the first formatted data and the second formatted data are fused to jointly construct the loss function of the VLA large model and the robot end-side small model, so as to cooperatively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

[0020] The application dynamically determines the cooperative training weight coefficients of the two models based on the complexity evaluation result of the sample task, converts the global semantic features of the large model into first formatted data understandable by the small model through the bidirectional knowledge interaction channel, converts the experience sequence data of the small model into second formatted data processable by the large model, and fuses the two types of formatted data based on the weight coefficients to construct a joint loss function, so as to cooperatively optimize the parameters of the two models in the iterative training, thereby effectively solving the problem that the large model and the small model are mutually isolated and difficult to cooperate in the traditional training, through the dynamic weight distribution and the bidirectional knowledge interaction mechanism, the depth semantic understanding advantage of the large model is retained, and the real-time control feature of the small model is also played, a cooperative learning framework with complementary advantages is formed, and the intelligent decision-making ability and the overall performance of the robot in a complex scene are significantly improved.

[0021] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the description, claims and drawings.

[0022] In order to make the above-mentioned objects, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are as follows. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0024] Figure 1 The flowchart of the robot end-side small model and VLA large model cooperative training method provided by the embodiment of the present application is shown in the figure. Figure 2 The principle diagram of the weight distribution process in the robot end-side small model and VLA large model cooperative training method provided by the embodiment of the present application is shown in the figure. Figure 3 The principle diagram of the weight score generation in the robot end-side small model and VLA large model cooperative training method provided by the embodiment of the present application is shown in the figure. Figure 4 The principle diagram of adjusting the weight based on the complexity score in the robot end-side small model and VLA large model cooperative training method provided by the embodiment of the present application is shown in the figure. Figure 5 The principle diagram of the robot system provided by the embodiment of the present application is shown in the figure. Figure 6 The schematic diagram of the robot end-side small model and VLA large model cooperative training device provided by the embodiment of the present application is shown in the figure. Figure 7 The structure schematic diagram of the electronic device provided by the embodiment of the present application is shown in the figure.

[0025] Reference signs: 10-weight determination module, interactive construction module-20, cooperative training module-30; 130-processor, 131-memory, 132-bus, 133-communication interface. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the present application will be described clearly and completely below in combination with the drawings. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0027] In order to facilitate the understanding of the present embodiment, the technical terms designed in the present application will be briefly introduced below.

[0028] VLA large model (Vision Language Action Model) is a multi-modal large model integrating vision, language and action decision.

[0029] After introducing the technical terms involved in the present application, next, the application scenarios and design ideas of the embodiments of the present application are briefly introduced.

[0030] Although the current VLA large model can improve the intelligence level of the robot, its high computing requirement and the real-time control advantage of the small model on the robot side are difficult to coordinate. The traditional method is easy to cause gradient conflict between models during training, and lacks a dynamic weight adjustment mechanism in the execution stage, which leads to the inability to complement each other's advantages, restricting the improvement of the overall performance of the robot.

[0031] Therefore, the embodiments of the present application provide a robot end-side small model and VLA large model cooperative training method, device and robot system to solve the problem of difficulty in coordinating large and small models and training fragmentation in the prior art.

[0032] Embodiment 1 The present application provides a robot end-side small model and VLA large model cooperative training method, which combines Figure 1 As shown in the figure, the method comprises: S110, based on the complexity evaluation result of the sample task, determining the weight coefficient of the VLA large model and the robot end-side small model in the cooperative training.

[0033] S120, converting the global semantic features output by the VLA large model into first formatted data matched with the decision space of the robot end-side small model, and transmitting to the robot end-side small model, and converting the experience sequence data generated by the robot end-side small model based on the execution of the sample task into second formatted data matched with the semantic space of the VLA large model, and transmitting to the VLA large model, to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model.

[0034] S130, based on the cooperative training weight coefficient, fusing the first formatted data and the second formatted data, and jointly constructing the loss function of the VLA large model and the robot end-side small model, to cooperatively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

[0035] Firstly, the application dynamically allocates model weights based on task complexity, then establishes a bidirectional knowledge interaction channel to realize mutual transmission of semantic knowledge of the large model and control experience of the small model, and finally constructs a joint loss function through weighted fusion to realize collaborative optimization of the two models. Masking, effectively solving the problem of the separation of the large model and the small model in the training stage, enables the two to realize knowledge complementation while maintaining their respective advantages, significantly improving the decision-making ability and execution efficiency of the robot in complex tasks.

[0036] In combination with the first aspect, step S110 comprises: S111, extracting the bottom-level features of the mixed data set by a low-level feature encoder, the mixed data set at least comprising visual, language and state data.

[0037] The low-level feature encoder refers to a general feature extraction module composed of a lightweight CNN or a ViT shallow network; the mixed data set contains multi-modal training data composed of visual images, natural language descriptions and system state data. In combination with Figure 2 As shown in step S111, the low-level feature encoder extracts task-related general bottom-level features such as object edges, spatial relationships, semantic keywords, etc. from the original multi-modal data (mixed data set) through convolution or self-attention mechanism, providing a unified feature representation for subsequent complexity evaluation. In this way, the feature unification of multi-source heterogeneous data is realized, which not only retains the key information of each modality, but also provides a standardized input format for subsequent processing.

[0038] S112, inputting the bottom-level features into the VLA large model and the robot-side small model to extract global task planning information and local control decision information of the bottom-level features.

[0039] In combination with Figure 2 As shown, the unified bottom-level features obtained in step S111 are input into models with different characteristics, and the semantic understanding ability of the VLA large model is used to generate macro task planning information, while the real-time processing ability of the end-side small model is used to generate local control strategy information.

[0040] Among them, the global task planning information refers to the structured information containing task decomposition, execution sequence and constraint conditions output by the VLA large model; the local control decision information refers to the operation information containing specific action instructions and real-time state feedback generated by the robot end-side small model.

[0041] Through step S112, the task information can be effectively separated, providing complete input information for accurate evaluation of task complexity, and fully exerting the respective advantages of the two models.

[0042] S113, information quantization processing is performed on the global task planning information and the local control decision information to generate a task complexity evaluation result.

[0043] The information quantization processing refers to a process of converting abstract task information into a calculable numerical score through a feature extraction module and a quantization module.

[0044] In step S113, structured features such as the number of subtasks and constraint conditions are extracted from the global task planning information, and dynamic features such as real-time requirements and resource demands are extracted from the local control strategy information. A pre-set quantization rule or a trained evaluation model is used to generate an accurate complexity score, i.e., a complexity evaluation result. In this way, subjective task complexity is converted into objective quantitative indicators, providing reliable basis for weight allocation and ensuring the accuracy and consistency of the evaluation result.

[0045] S114, based on the task complexity evaluation result and the task resource status, the weight coefficients for the VLA large model and the robot end-side small model are generated.

[0046] The task resource status includes the current computing resources, memory usage, energy status, and other hardware conditions of the system, as well as scheduling information such as task priority and queue density. In step S114, the initial weight allocation strategy is determined based on the complexity score, and then dynamically adjusted in combination with the real-time resource status to ensure that the weight allocation meets the task requirements and takes into account the actual running state of the system. In this way, the adaptive generation of weight coefficients is achieved, which not only guarantees the task execution effect but also effectively improves the utilization of system resources and enhances the overall adaptability.

[0047] In combination with the first aspect, step S112 includes: S1121, based on the feature extraction module, features are extracted from the global task planning information and the local control decision information to obtain global task feature information and local strategy feature information; S1122, based on the feature quantization module, the global task feature information and the local strategy feature information are quantized to obtain a task complexity evaluation result; The global task feature information includes one or more of the number of subtasks, the number of constraint conditions, the beat time, the task priority, and the process level. The local strategy feature information includes one or more of real-time requirements, resource status, perception information, burst status, and other device call status.

[0048] In this embodiment, the feature extraction module refers to a specially designed computing unit for extracting key evaluation features from task information. The global task feature information reflects the macro complexity of the task, and the local strategy feature information reflects the real-time difficulty of execution. The module uses a rule engine or a lightweight neural network in combination with Figure 3As shown, the global task feature information (static features such as the number of subtasks, the number of constraint conditions, etc.) is extracted from the structured global task planning information, and the local strategy feature information (such as resource requirements, real-time requirements, etc.) is extracted from the dynamic local control decision information, so as to establish a complete task evaluation index system, which not only considers the inherent complexity of the task, but also takes into account the dynamic factors of the execution environment, providing a comprehensive feature basis for accurate quantification.

[0049] The feature quantification module refers to a processing unit that converts abstract features into numerical scores. It can use a rule-based evaluation system or a trained prediction model. The module receives multi-dimensional feature input and generates a unified task complexity evaluation result through pre-set scoring rules (such as weighted summation, fuzzy logic judgment) or trained neural network models. In specific implementation, different complexity intervals can be set, such as low complexity (subtask number <8), medium complexity (8-12), and high complexity (>12). Then, the final weight coefficient is generated by combining the task resource status. In this way, multi-dimensional features are fused into a single comparable complexity index, eliminating the influence of different feature dimensions and providing accurate and reliable decision-making basis for subsequent weight allocation.

[0050] In combination with the first aspect, step S114 includes: S1141, based on the comparison relationship between the task complexity evaluation result and the pre-set threshold value, allocating initial weight coefficients for the VLA large model and the robot end-side small model.

[0051] It can be understood that the pre-set threshold value refers to the complexity classification standard set according to historical task data and analysis experience, which is used to divide the task into multiple levels, such as low, medium, and high complexity levels; the initial weight coefficient is the baseline weight value allocated based on the complexity level.

[0052] The task complexity score generated by S113 is compared with the pre-set threshold interval, and the pre-set weight allocation strategy is applied according to the complexity level to which the task belongs. In combination with Figure 4 As shown, the end-side small model is given a higher weight (such as 0.7:0.3) for low complexity tasks, the VLA large model is given a higher weight (such as 0.8:0.2) for high complexity tasks, and the balanced weight (such as 0.5:0.5) is used for medium complexity tasks. Through the basic mapping relationship between the task complexity evaluation result and the model weight, it is ensured that the weight allocation strategy matches the essential requirements of the task, providing a reasonable initial baseline for subsequent fine adjustment.

[0053] S1142, based on the task resource status, weighting and correcting the initial weight coefficient to generate the target weight coefficient corresponding to the VLA large model and the robot end-side small model.

[0054] The task resource condition includes at least one of the following: criticality of a current task, available computing resources of the system, memory occupation state, priority of a next task to be executed, density of a total task queue, and energy consumption state of the system.

[0055] It can be understood that the task resource condition refers to a set of environmental factors inside and outside the system that affect task execution; and the target weight coefficient is a final weight distribution result after multi-factor correction. By constructing a multi-factor correction model through step S1142, factors such as criticality of a task (increasing the weight of a large model for a critical task), system resource state (increasing the weight of a small model when resources are tight), and density of a task queue (dynamically adjusting weight distribution when the queue is dense) are comprehensively considered, and the initial weight is corrected in real time through a weighted algorithm, so as to realize the upgrade of weight distribution from "task-driven" to "task-resource dual-driven", so that the weight distribution not only meets the task demand, but also adapts to the real-time running state of the system, significantly improving the environmental adaptability and resource utilization efficiency of the system.

[0056] In combination with the first aspect, the bidirectional knowledge interaction channel is established through a gradient mapping layer, wherein: The first formatted data is mapped to the global semantic feature through a gradient adapter in the gradient mapping layer; The second formatted data is compressed to the experience sequence data through an experience compressor in the gradient mapping layer.

[0057] In combination with the first aspect, the gradient adapter is a small MDP network model; and the experience compressor is an LSTM network or an attention network.

[0058] As shown in Figure 5 The gradient mapping layer is a core component connecting the VLA large model and the robot-side small model, and realizes bidirectional knowledge conversion through the two specially designed processing units of the gradient adapter and the experience compressor. The gradient adapter adopts a small Markov decision process network model, which contains complete decision elements of state space, action space and reward function, and maps the abstract semantic features (such as "smooth grasping" and "fast movement") output by the large model to specific constraint conditions (such as maximum acceleration and joint angle range) that can be directly understood by the end-side small model, through the mapping relationship from the high-level semantic space to the low-level control space. The experience compressor can be a long short-term memory network or an attention network. The long short-term memory network (LSTM) door processes time sequence experience data, identifies important experience fragments through a gating mechanism, gradually constructs experience feature representation, and is suitable for continuous control tasks, so as to output compressed features with fixed dimensions, ensuring compatibility with the large model interface; the attention network can process the entire experience sequence in parallel, quickly identify key experience fragments, and can adaptively weight important experiences and suppress noise interference. In addition, the attention network also supports variable length input, which is suitable for different task durations.

[0059] In this embodiment, through the cooperation of the aforementioned gradient adapter and the experience compressor, the gradient adapter converts the "thinking language" of the large model into the control language executable by the small model by establishing a mapping relationship from abstract semantics to specific control parameters; at the same time, the experience compressor refines the specific operation experience of the small model into feature information with semantic value, breaking down the semantic barriers between different levels of models and forming a unified semantic understanding framework. At the same time, a complete closed loop of planning first, then executing and then feedback learning is constructed. The task planning generated by the VLA large model guides the execution process of the small model through the gradient adapter, while the experience data accumulated by the small model in the execution process is fed back to the large model through the experience compressor, prompting it to optimize the subsequent planning strategy. This two-way knowledge circulation ensures the co-evolution of the models in continuous interaction.

[0060] In combination Figure 5 As shown in the figure, a closed loop of bidirectional knowledge interaction and collaborative optimization is established between the VLA large model and the robot-side small model through the gradient mapping layer. Specifically, in the forward propagation process, the semantic understanding layer and the task planning layer of the VLA large model process global information to generate global task planning, and its knowledge extraction node extracts global semantic features. The features are sent to the gradient adapter in the gradient mapping layer, which is mapped and converted into a constraint vector matching the decision space of the robot-side small model, and is transmitted to the decision layer of the small model to constrain its decision-making behavior, and injects the global semantic intention of the large model into the real-time control of the small model. On the other hand, the robot-side small model generates experience sequence data when executing tasks. These real-time and specific local control experiences are uploaded to the experience compressor in the gradient mapping layer, which is compressed and refined into experience features matching the semantic space of the VLA large model, and uploaded to the VLA large model as its implicit constraints, so as to feed back the experience in actual execution to the VLA large model to make its planning closer to reality.

[0061] In the backward propagation and optimization process, the two models are collaboratively optimized through the matching degree signals transmitted to each other. When the small model calculates its loss function, it will consider the constraint matching degree between its decision and the received constraint vector, and the matching degree signal will be propagated back to the knowledge extraction node of the large model, prompting the large model to adjust its semantic constraints to better guide the small model. Similarly, when the VLA large model calculates its loss function, it will consider the experience matching degree between its planning and the received experience features, and the signal will be propagated back to the small model, prompting the small model to optimize its decision-making to provide more valuable experience feedback.

[0062] The whole cooperative training process is globally regulated by a dynamic task evaluation and control strategy module (indicated by "weight adjustment" at the top of the figure), which dynamically adjusts the weight coefficients of the two models in training according to the real-time task complexity, ensures that the small model dominates in simple tasks and the large model dominates in complex tasks, and thus realizes the optimal configuration of resources and the maximization of performance.

[0063] In combination with the first aspect, in step S130, based on the weight coefficients, the first formatted data and the second formatted data are fused to jointly construct a loss function, specifically including: S131, the first formatted data and the second formatted data are respectively weighted according to the weight coefficients.

[0064] According to the weight coefficients generated in step S110, the first formatted data (semantic constraint vector) and the second formatted data (empirical characteristics) obtained in step S120 are accurately weighted. The weight coefficients serve as key parameters for adjusting the intensity of knowledge transfer and directly affect the optimization direction of the two models in joint training. Specifically, a linear weighting algorithm is used to perform matrix multiplication operation on the weight coefficients and the corresponding formatted data. As shown in the formula (1), as an example, when the task complexity is high, the first formatted data is given a larger weight (such as 0.7-0.9) to strengthen the role of semantic constraints; when the system resources are tight, the weight of the second formatted data is appropriately increased (such as 0.6-0.8) to enhance the adjustment ability of real-time experience. Thus, the intensity of knowledge transfer is accurately controlled, and the optimal training balance state is ensured under different task scenarios. Figure 4

[0065] S132, the weighted first formatted data is introduced as a semantic constraint term into the loss function of the robot-side small model.

[0066] ​The weighted first formatted data is introduced as a semantic constraint term into the loss function of the robot end-side small model. The constraint term ensures that the local control decision conforms to the global semantic guidance of the large model by calculating the difference between the small model output decision and the semantic constraint vector. Specifically, a distance metric function (such as cosine similarity or KL divergence) is used to quantify the consistency of the small model decision and the semantic constraint, and it is added as a regularization term to the loss function. The specific formula is: Lsemantic = λ1 × D(Psmall, Vsemantic), where Lsemantic is the constraint loss term, which measures the loss value of the difference between the small model decision and the global semantic guidance. The smaller the value, the more the small model decision conforms to the semantic guidance of the large model; λ1 is the semantic constraint weight coefficient, which is used to adjust the importance of the semantic constraint loss term in the total loss function. The larger λ1, the stronger the requirement for the small model to follow the global semantic guidance; D() is the distance metric function, which is used to quantify the difference between two vectors or distributions. Cosine similarity measures the similarity of two vectors in direction; KL divergence measures the difference between two probability distributions; Psmall is the decision output generated by the end-side small model according to the current local state (such as sensor data), which depends on the task and may be: action vector (such as robot joint target angle), strategy distribution (such as the probability of each action), feature representation (such as embedding vector); Vsemantic is the semantic constraint vector, which is the weighted "first formatted data" generated from the large model (such as cloud-side large language model or visual model), representing the global semantic guidance for the current task, which may be: target vector or semantic embedding generated by the large model, probability distribution of expected behavior, task-related high-level instruction encoding. In this way, the end-side small model can be guided to maintain real-time performance while not deviating from the global task goal, solving the contradiction between local optimal and global optimal, and improving the accuracy and reliability of decision-making.

[0067] S133, the weighted second formatted data is introduced as an experience feedback term into the loss function of the VLA large model.

[0068] The weighted second formatted data is used as an experience feedback item to promote the planning strategy of the large model to be more consistent with the actual execution condition by evaluating the matching degree between the planning result of the large model and the actual execution experience. Specifically, a method combining feature similarity calculation and experience value evaluation can be used to construct an experience feedback loss term. The specific formula is: Lexperience = λ2 x (1-sim(Vplan, Fexperience)), where Lexperience is the experience feedback loss, which quantifies the loss value of the difference between the planning result of the large model and the actual successful experience. The smaller the value, the more consistent the planning of the large model with the actual execution condition; λ2 is the experience feedback weight coefficient, which is used to adjust the importance of the experience feedback loss term in the training or fine-tuning process of the large model. The larger λ2, the stronger the constraint force of the large model to align with the actual experience; sim() is a function for calculating the similarity between two vectors. The typical representative is cosine similarity, and the larger the value, the more similar; Vplan is the planning feature or result of the large model, which is generated by the large model based on task instructions and environment understanding. It is usually a high-level, abstract feature representation, such as: path encoding of the planning, sequence vector of task sub-targets, semantic embedding of expected state changes; Fexperience is the experience feedback feature, which is a feature vector extracted from the historical data of successful execution (“second formatted data”). It encapsulates the strategy information that has been proven effective and executable in the real physical environment, such as: perception-action pair features of successful trajectories, control mode encoding proven effective under specific execution conditions (such as specific friction coefficient, light), and features suitable for actual application extracted from successful decision-making of the end-side small model.

[0069] In this way, the large model is provided with real execution environment feedback, making its planning result both semantically reasonable and executable, significantly improving the practicality and adaptability of the planning strategy.

[0070] Among them, the semantic constraint term is used to constrain the decision direction of the robot end-side small model, and the experience feedback term is used to optimize the planning strategy of the VLA large model.

[0071] It can be understood that through steps S131-S133, a complete two-way optimization framework is constructed, in which the semantic constraint term ensures that the decision direction of the small model is consistent with the global planning of the large model, and the experience feedback term promotes the large model to optimize the planning strategy according to the actual execution effect. This two-way constraint mechanism creates a benign cooperative training cycle, enabling the two models to achieve deep cooperation while maintaining their respective characteristics, ultimately significantly improving the overall performance of the robot in complex dynamic environments.

[0072] In combination with the first aspect, the method further comprises: S210, based on the trained VLA large model and the robot end-side small model, dynamically assigning decision weights according to the complexity evaluation result of the real-time task.

[0073] S220, if the complexity evaluation result is less than the first threshold, the robot end-side small model dominates the decision, and the VLA large model provides semantic constraints; S230, if the complexity evaluation result is greater than the second threshold, the VLA large model generates a global task plan, and the robot end-side small model is responsible for executing local action refinement; Wherein, the first threshold is less than or equal to the second threshold.

[0074] Based on the model obtained by cooperative training in steps S110-S130, the task characteristics of real-time input are quickly analyzed, and a quantitative complexity score (i.e. complexity evaluation result) is generated. By comparing the score with the preset double threshold (T1, T2, where T1≤T2), the task is automatically classified into the corresponding complexity interval, thereby activating the corresponding decision mode to achieve accurate matching of task characteristics and decision architecture, avoiding resource waste or performance deficiency under fixed decision mode, and providing accurate mode switching basis for subsequent task execution.

[0075] It can be understood that in the low complexity task scenario shown in step S220, the robot end-side small model is used as the decision subject to directly process sensor data and generate control instructions, fully utilizing its low delay characteristics. At the same time, the VLA large model provides a semantic supervision cooperation mechanism to provide semantic constraint boundaries through a lightweight interface, ensuring that the end-side decision meets the basic requirements of the task, thereby maximizing the system response speed and reducing the consumption of computing resources while ensuring the quality of task completion, especially suitable for simple task scenarios with high real-time requirements.

[0076] In the high complexity task scenario shown in step S230, the VLA large model is responsible for global planning to generate a complete plan containing sub-task sequence, execution constraints and expected target, while the robot end-side small model is responsible for local refinement to convert high-level planning into specific action instructions, and keeps synchronization with the large model through real-time feedback channel during execution to form a closed loop control. Thus, the advantages of the large model in complex reasoning and long-term planning are utilized, while the real-time control capability of the small model is used to ensure execution accuracy, enabling the system to effectively handle complex tasks that require deep thinking and multi-step planning.

[0077] In a second aspect, the embodiments of the present application also provide a robot end-side small model and VLA large model cooperative training device for executing the above method, as shown in Figure 6 The device comprises a weight determination module 10, an interaction construction module 20 and a cooperative training module 30.

[0078] The weight determination module 10 is configured to determine a weight coefficient of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation result of the sample task.

[0079] The interaction construction module 20 is configured to convert the global semantic features output by the VLA large model into first formatted data matched with the decision space of the robot end-side small model, and transmit the first formatted data to the robot end-side small model, and convert the experience sequence data generated by the robot end-side small model based on the execution of the sample task into second formatted data matched with the semantic space of the VLA large model, and transmit the second formatted data to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model.

[0080] The collaborative training module 30 is configured to fuse the first formatted data and the second formatted data based on the weight coefficient, and jointly construct a loss function of the VLA large model and the robot end-side small model, so as to collaboratively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

[0081] In a third aspect, the embodiments of the present application further provide a robot system, which is combined with Figure 5 As shown in the drawings, the system is configured to perform the method as described above, and the system comprises: The robot end-side small model is deployed on the robot device end, and is configured to execute local control decisions.

[0082] The VLA large model is communicatively connected with the robot end-side small model, and is configured to provide global semantic understanding and task planning.

[0083] The dynamic task evaluation module is configured to generate a weight coefficient of the VLA large model and the robot end-side small model in the collaborative training based on the complexity evaluation result of the sample task.

[0084] The collaborative training architecture comprises a gradient mapping layer, and the gradient mapping layer is configured to.

[0085] The global semantic features output by the VLA large model are converted into the first formatted data, and are transmitted to the robot end-side small model.

[0086] The experience sequence data generated by the robot end-side small model are converted into the second formatted data, and are transmitted to the VLA large model.

[0087] The system is configured to fuse the first formatted data and the second formatted data based on the weight coefficient, so as to collaboratively train the VLA large model and the robot end-side small model.

[0088] In a third aspect, the embodiments of the present application provide an electronic device, which is combined with Figure 7As shown, the electronic device includes a memory 131 for storing a computer program and a processor 130 for running the computer program to enable the electronic device to perform the above method.

[0089] Further, in combination with Figure 7 As shown, the electronic device further includes a bus 132 and a communication interface 133, and the processor 130, the communication interface 133 and the memory 131 are connected through the bus 132.

[0090] The memory 131 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 133 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 132 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0091] The processor 130 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor 130. The processor 130 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware decoding processor for execution, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage 131, and the processor 130 reads the information in the storage 131 and combines the hardware to complete the steps of the method of the above embodiment.

[0092] In a fourth aspect, the embodiment of the present application provides a readable storage medium, and the readable storage medium stores computer program instructions. When the computer program instructions are read and run by a processor, the above method is executed.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0094] In addition, in the description of the embodiment of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood in a broad sense, for example, can be fixedly connected, can also be detachably connected, or integrally connected; can be mechanically connected, can also be electrically connected; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0095] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0096] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0097] Finally, it should be noted that: the above embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and are not limited thereto, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art within the technical scope disclosed by the present application can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for robot end-side small model and VLA large model collaborative training, characterized in that, The method comprises the steps of: determining the weight coefficients of the VLA large model and the robot end-side small model in collaborative training based on the complexity evaluation result of the sample task; converting the global semantic features output by the VLA large model into first formatted data matching the decision space of the robot end-side small model and transmitting the first formatted data to the robot end-side small model, and converting the experience sequence data generated by the robot end-side small model based on the execution of the sample task into second formatted data matching the semantic space of the VLA large model and transmitting the second formatted data to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot end-side small model; based on the collaborative training weight coefficients, fusing the first formatted data and the second formatted data to jointly construct the loss function of the VLA large model and the robot end-side small model, so as to collaboratively optimize the parameters of the VLA large model and the parameters of the robot end-side small model in the iterative training process.

2. The method of claim 1, wherein, The step of determining the weight coefficients of the VLA large model and the robot end-side small model in collaborative training based on the complexity evaluation result of the sample task comprises the steps of: extracting the bottom features of the mixed data set through a low-level feature encoder, wherein the mixed data set at least includes visual, language and state data; inputting the bottom features into the VLA large model and the robot end-side small model to extract global task planning information and local control decision information of the bottom features; performing information quantization processing on the global task planning information and the local control decision information to generate a task complexity evaluation result; generating weight coefficients for the VLA large model and the robot end-side small model based on the task complexity evaluation result and the task resource status.

3. The method of claim 2, wherein, The step of performing information quantization processing on the global task planning information and the local control decision information to generate a task complexity evaluation result comprises the steps of: based on a feature extraction module, extracting features from the global task planning information and the local control decision information to obtain global task feature information and local strategy feature information; based on a feature quantization module, quantizing the global task feature information and the local strategy feature information to obtain the task complexity evaluation result; wherein the global task feature information comprises one or more of subtask quantity, constraint condition quantity, beat time, task priority and process level; and the local strategy feature information comprises one or more of real-time requirement, resource status, perception information, burst state and other device call state.

4. The method of claim 2, wherein, The step of generating weight coefficients for the VLA large model and the robot end-side small model based on the task complexity evaluation result and the task resource status comprises the steps of: based on the comparison relationship between the task complexity evaluation result and a preset threshold, assigning initial weight coefficients to the VLA large model and the robot end-side small model; based on the task resource status, weighting and correcting the initial weight coefficients to generate target weight coefficients corresponding to the VLA large model and the robot end-side small model. The task resource condition comprises at least one of the following: criticality of a current task, available computing resources of the system, memory occupation state, priority of a next task to be executed, density of an overall task queue, and energy consumption state of the system.

5. The method of claim 1, wherein, The bidirectional knowledge interaction channel is established through a gradient mapping layer. The first formatted data is mapped to the global semantic feature through a gradient adapter in the gradient mapping layer. The second formatted data is compressed from the experience sequence data through an experience compressor in the gradient mapping layer.

6. The method of claim 5, wherein, The gradient adapter is a small MDP network model, and the experience compressor is an LSTM network or an attention network.

7. The method of claim 1, wherein, The step of fusing the first formatted data and the second formatted data based on the weight coefficient to jointly construct a loss function comprises: weighting the first formatted data and the second formatted data according to the weight coefficient; introducing the weighted first formatted data as a semantic constraint term into a loss function of the robot-side small model; introducing the weighted second formatted data as an experience feedback term into a loss function of the VLA large model; The semantic constraint term is used to constrain the decision direction of the robot-side small model, and the experience feedback term is used to optimize the planning strategy of the VLA large model.

8. The method of claim 1, wherein, The method further comprises: dynamically allocating a decision weight according to a complexity evaluation result of a real-time task based on the trained VLA large model and robot-side small model; if the complexity evaluation result is less than a first threshold, the robot-side small model dominates the decision and the VLA large model provides semantic constraints; if the complexity evaluation result is greater than a second threshold, the VLA large model generates a global task plan and the robot-side small model is responsible for executing local action refinement; wherein the first threshold is less than or equal to the second threshold.

9. A robot end-side small model and VLA large model cooperative training device, characterized by, The device is used to perform the method of any one of claims 1-8, comprising: a weight determination module configured to determine a weight coefficient of the VLA large model and the robot-side small model in collaborative training based on a complexity evaluation result of a sample task; an interaction construction module configured to convert global semantic features output by the VLA large model into first formatted data matching a decision space of the robot-side small model and transmit the first formatted data to the robot-side small model, and convert experience sequence data generated by the robot-side small model based on execution of the sample task into second formatted data matching a semantic space of the VLA large model and transmit the second formatted data to the VLA large model, so as to establish a bidirectional knowledge interaction channel between the VLA large model and the robot-side small model; a collaborative training module configured to fuse the first formatted data and the second formatted data based on the collaborative training weight coefficient to jointly construct a loss function of the VLA large model and the robot-side small model, so as to collaboratively optimize parameters of the VLA large model and parameters of the robot-side small model in an iterative training process.

10. A robot system, characterized by The method as claimed in any one of claims 1-8, comprising: a robot-side small model deployed at a robot device end for performing local control decision-making; a VLA large model in communication connection with the robot-side small model for providing global semantic understanding and task planning; a dynamic task evaluation module configured to generate a weight coefficient of the VLA large model and the robot-side small model in collaborative training based on a complexity evaluation result of a sample task; a collaborative training architecture including a gradient mapping layer configured to: convert global semantic features output by the VLA large model into first formatted data and transmit the first formatted data to the robot-side small model; and convert experience sequence data generated by the robot-side small model into second formatted data and transmit the second formatted data to the VLA large model; wherein the system is configured to fuse the first formatted data and the second formatted data based on the weight coefficient to collaboratively train the VLA large model and the robot-side small model.

Images