Robot control method, device, apparatus, and storage medium

Abstract

This application provides a robot control method, apparatus, device, and storage medium. The method includes: constructing a policy pool containing multiple pre-trained policies; generating corresponding action probability distribution information based on real-time multimodal observation data acquired by the robot; dynamically determining the optimal target pre-trained policy combination based on this information; then generating combined action distribution information by fusing the action probability distributions of each policy in the combination; and planning the target action trajectory from this information to control the robot. This application eliminates the need for online fine-tuning or additional labeled data. By reusing heterogeneous pre-trained policies and adaptively combining them according to the task context, it significantly improves the generalization ability, robustness, and task success rate of robot control, while reducing deployment costs.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a robot control method, apparatus, device, and storage medium. Background Technology

[0002] Diffusion models and flow matching models have become important methods for robot control, capable of representing complex multimodal motion distributions and applicable to scenarios such as manipulation and navigation. However, the development of these strategies is limited by the high cost of acquiring large-scale interactive data, the need for massive amounts of data to scale models, and the significant limitations of existing post-training optimization strategies, making it difficult to meet the complex task control needs of embodied humanoid robots.

[0003] Current robot control technologies include post-training optimization strategies such as supervised fine-tuning, reinforcement learning, and static model combination, as well as various pre-training strategies for different types, modalities, and input modalities. After acquiring real-time observation data, motion trajectories can be predicted by invoking pre-training strategies, and robot control can be performed based on the predicted motion sequences.

[0004] However, existing technologies struggle to achieve multi-task generalization with individually trained strategies, exhibit poor cross-modal and cross-architecture model compatibility, and rely on additional training resources for strategy optimization. Static combination methods do not consider the task dependence of weights, resulting in poor combination performance. They depend on the representational capabilities of a single model and fail to fully leverage the complementary advantages of multiple pre-trained strategies. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a robot control method, device, equipment, and storage medium to solve the problems that a single strategy in the prior art is difficult to take into account multi-task generalization, has poor cross-modal and cross-architecture model compatibility, and cannot fully utilize the complementary advantages of multiple pre-trained strategies.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: In a first aspect, this application provides a robot control method, the method comprising: During the robot's operation, real-time observation data is acquired, including data from multiple modalities. The pre-trained strategies in the strategy pool are invoked, and the action probability distribution information corresponding to each pre-trained strategy is determined based on the real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the feasibility of the robot performing actions under the pre-trained strategies. Based on the action probability distribution information of each pre-training strategy, determine the target pre-training strategy combination; Based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination, determine the combined action distribution information; The target motion trajectory is generated based on the combined motion distribution information, and the robot is controlled based on the target motion trajectory.

[0007] Optionally, the step of invoking pre-trained policies in the policy pool and determining the action probability distribution information corresponding to each pre-trained policy based on the real-time observation data includes: Invoke the pre-trained policies in the policy pool and determine the target modality data corresponding to each pre-trained policy in the real-time observation data; The target modality data corresponding to each pre-training strategy is input into each pre-training strategy, and the action probability distribution information is predicted by each pre-training strategy.

[0008] Optionally, determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy includes: The aforementioned pre-training strategies are combined to obtain multiple pre-training strategy combinations; Based on the action probability distribution information of each pre-trained strategy, determine the trajectory success rate of each combination of pre-trained strategies; The target pre-training strategy combination is determined based on the trajectory success rate of each pre-training strategy combination.

[0009] Optionally, the combination of the pre-training strategies to obtain multiple pre-training strategy combinations includes: The pre-training strategies are combined by convex combination, or by logical AND combination, or by logical OR combination, to obtain the combination of the multiple pre-training strategies.

[0010] Optionally, before determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy, the method further includes: Determine the weight values ​​of each of the pre-training strategies, and generate multiple combinations of pre-training strategies based on the weight values ​​of each of the pre-training strategies; The target task is executed in a simulation experimental environment, and the trajectory success rate of each pre-trained strategy combination is determined based on experimental observation data.

[0011] Optionally, the combination of the pre-training strategies is a convex combination; The step of determining the weight values ​​of each of the pre-training strategies and generating multiple pre-training strategy combinations based on the weight values ​​of each of the pre-training strategies includes: For each of the pre-trained strategies, each weight value is traversed within a preset weight value range. For the current weight value encountered during the traversal, the current weight value is used as the weight value of the pre-trained strategy. Each pre-training strategy is combined according to its weight value to obtain multiple pre-training strategy combinations, wherein the sum of the weight values ​​of each pre-training strategy combination is equal to 1.

[0012] Optionally, determining the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination includes: Based on the weight information of each pre-training strategy in the target pre-training strategy combination, the action probability distribution information of each pre-training strategy is linearly fused to obtain the combined action distribution information.

[0013] Secondly, this application provides a robot control device, the device comprising: The acquisition module is used to acquire real-time observation data during the robot's operation, and the real-time observation data includes data from multiple modalities; The first determining module is used to call the pre-trained strategies in the strategy pool and determine the action probability distribution information corresponding to each pre-trained strategy based on the real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the feasibility of the robot performing actions under the pre-trained strategies. The combination module is used to determine the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy; The second determining module is used to determine the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination. The control module is used to generate a target motion trajectory based on the combined motion distribution information, and to control the robot based on the target motion trajectory.

[0014] Optionally, the first determining module is specifically used for: Invoke the pre-trained policies in the policy pool and determine the target modality data corresponding to each pre-trained policy in the real-time observation data; The target modality data corresponding to each pre-training strategy is input into each pre-training strategy, and the action probability distribution information is predicted by each pre-training strategy.

[0015] Optionally, the combined module is specifically used for: The aforementioned pre-training strategies are combined to obtain multiple pre-training strategy combinations; Based on the action probability distribution information of each pre-trained strategy, determine the trajectory success rate of each combination of pre-trained strategies; The target pre-training strategy combination is determined based on the trajectory success rate of each pre-training strategy combination.

[0016] Optionally, the combined module is specifically used for: The pre-training strategies are combined by convex combination, or by logical AND combination, or by logical OR combination, to obtain the combination of the multiple pre-training strategies.

[0017] Optionally, the device further includes an experimental module, specifically used for: Determine the weight values ​​of each of the pre-training strategies, and generate multiple combinations of pre-training strategies based on the weight values ​​of each of the pre-training strategies; The target task is executed in a simulation experimental environment, and the trajectory success rate of each pre-trained strategy combination is determined based on experimental observation data.

[0018] Optionally, the combination of the pre-training strategies is a convex combination; The combined module is specifically used for: For each of the pre-trained strategies, each weight value is traversed within a preset weight value range. For the current weight value encountered during the traversal, the current weight value is used as the weight value of the pre-trained strategy. Each pre-training strategy is combined according to its weight value to obtain multiple pre-training strategy combinations, wherein the sum of the weight values ​​of each pre-training strategy combination is equal to 1.

[0019] Optionally, the second determining module is specifically used for: Based on the weight information of each pre-training strategy in the target pre-training strategy combination, the action probability distribution information of each pre-training strategy is linearly fused to obtain the combined action distribution information.

[0020] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a robot control method as described in any one of the first aspects.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a robot control method as described in any one of the first aspects.

[0022] The beneficial effects of this application are as follows: By constructing a policy pool containing multiple pre-trained policies and dynamically calling, evaluating, and combining these policies based on real-time multimodal observation data of the robot, efficient action decision-making without additional training or online interaction is achieved. This application automatically selects and fuses the optimal policy combination based on the action probability distribution output by each pre-trained policy as the confidence level, generating a high-quality combined action distribution, and thus planning a reliable target action trajectory. Furthermore, it can be compatible with heterogeneous policies of different architectures and modal inputs, effectively reusing existing model resources and avoiding the high costs of traditional fine-tuning or reinforcement learning. Simultaneously, through the dynamic combination mechanism, it enhances the system's generalization ability and robustness in open and complex environments, significantly improving the practicality and adaptability of robot control.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a robot control method provided in an embodiment of this application is shown; Figure 2 This application provides a flowchart for obtaining action probability distribution information according to an embodiment of the present application. Figure 3 This document illustrates a flowchart of a method for determining a combination of target pre-training strategies, as provided in an embodiment of this application. Figure 4 This document illustrates a flowchart of a method for determining the success rate of a trajectory, as provided in an embodiment of this application. Figure 5 A flowchart illustrating a method for generating a pre-trained strategy combination according to an embodiment of this application is shown; Figure 6 This paper shows a schematic diagram of the structure of a robot control device provided in an embodiment of this application; Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0027] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0028] Existing pre-trained policy models are highly dependent on large-scale interactive data for performance. However, collecting real robot data is costly, limiting model scaling and causing performance stagnation. Secondly, mainstream post-training optimization methods, such as supervised fine-tuning, require additional labeled data, while reinforcement learning involves complex reward design and extensive online interactions, resulting in high deployment barriers. Furthermore, existing policy combination methods often use static weights, making it impossible to dynamically adjust the contributions of each expert policy based on specific tasks, thus hindering the leveraging of complementary advantages across multiple policies. Finally, the lack of a universal compatibility mechanism between pre-trained policies of different architectures and modalities limits the efficient reuse of existing model resources. These issues collectively restrict the generalization ability and practicality of embodied intelligent systems in complex and open environments.

[0029] Based on this, this application proposes a robot control method that dynamically adjusts the weights of pre-trained strategies during the inference stage of robot control and fuses the pre-trained strategies. The optimal motion trajectory is predicted using the task success rate as an indicator, thereby effectively reusing existing resource models, avoiding the high costs of supervised fine-tuning and reinforcement learning, and significantly improving the generalization and practicality of the strategy.

[0030] Next, combine Figure 1 The robot control method of this application is described below. The subject executing this method can be a robot, such as a humanoid robot. Figure 1 As shown, the method includes: S101. Acquire real-time observation data during the robot's operation. The real-time observation data includes data from multiple modalities.

[0031] Real-time observation data refers to the environmental and self-state information collected by the robot in real time through sensors during task execution; multimodal data refers to perceptual inputs of different types or sources, such as RGB images (visual modality), depth maps, point clouds (3D geometric modality), voice commands (language modality), joint angles / velocities (prototypic perception modality), etc. Multimodal data can provide a more comprehensive understanding of the environment.

[0032] Robots can be equipped with various sensors, such as cameras, LiDAR, and microphones. These sensors simultaneously collect and fuse data streams from different modalities during operation. For example, in a grasping task, the system can simultaneously receive visual images and verbal commands, forming a Vision-Language-Action (VLA) input. This step provides context for subsequent policy invocations.

[0033] S102. Call the pre-trained strategies in the strategy pool, and determine the action probability distribution information corresponding to each pre-trained strategy based on real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the robot's ability to perform actions under the pre-trained strategies.

[0034] Among them, the policy pool refers to a collection of multiple embodied intelligent policy models that have been pre-trained. Each policy may be based on different architectures (such as diffusion models or flow matching models) or adapted to different modal inputs. The action probability distribution information refers to the probability description of the policy output about the future action sequence, which is used to reflect the probability of a certain action being sampled under the current real-time observation data, i.e., the feasibility confidence.

[0035] In one possible implementation, compatible pre-trained policies in the policy pool can be selected based on the modality types contained in the real-time observation data. If language instructions are included, the VLA policy is enabled; otherwise, the Vision-Action (VA) policy is enabled. Each invoked policy takes a corresponding subset of modalities as input and outputs its probability distribution over the action space. The probability distribution can be, for example, the mean and variance of a Gaussian distribution, or the softmax probability of a discrete action. The action probability distribution information reflects the pre-trained policy's judgment on the rationality of the action in the current context.

[0036] S103. Determine the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy.

[0037] The target pre-trained policy combination refers to the optimal combination scheme selected from all possible policy combinations for subsequent action generation. The target pre-trained policy combination includes at least two pre-trained policies. The combination of pre-trained policies can be weighted fusion, logical operations, etc. For example, multiple pre-trained policies can be combined by convex combination, AND combination, or OR combination to generate the target pre-trained policy combination.

[0038] In one possible implementation, the performance of different combinations in terms of task success rate, stability, or diversity can be evaluated based on the action probability distribution information output by each strategy. For example, the trajectory success rate of each combination can be pre-calculated through offline simulation experiments, or the combination can be dynamically selected online based on indicators such as distribution similarity and entropy. Finally, the combination with the best performance is selected as the target combination.

[0039] S104. Determine the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination.

[0040] The combined action distribution information is a new action probability distribution formed by fusing the outputs of multiple strategies, representing the integrated decision result. The combined action distribution information of the target pre-trained strategy combination can be obtained by calculating the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination according to the probability calculation method corresponding to the combination method.

[0041] In one possible implementation, if the target combination uses a convex combination (with a weight sum of 1), the action distribution is linearly weighted according to the weights of each strategy. For example, if strategy 1 outputs an action mean of μ1 and strategy 2 outputs μ2, with weights w1 and w2 respectively, then the combined mean is w1μ1 + w2μ2. This fusion preserves the advantages of each strategy and improves robustness.

[0042] S105. Generate the target motion trajectory based on the combined motion distribution information, and control the robot based on the target motion trajectory.

[0043] The target motion trajectory refers to the specific motion sequence (such as the joint angle change sequence) obtained by sampling or optimizing from the combined distribution, which is used to drive the robot to perform tasks.

[0044] In one possible implementation, the trajectory can be optimized under distribution constraints by directly sampling from the combined distribution (e.g., backsampling from a diffusion model) or using a planning algorithm (e.g., MPC). The generated trajectory is then converted into motor commands by the actuator, achieving closed-loop control. This process fully utilizes the comprehensive judgment of the integrated strategy to improve the quality of the action.

[0045] In this embodiment, a policy pool containing multiple pre-trained strategies is constructed, and these strategies are dynamically invoked, evaluated, and combined based on real-time multimodal observation data of the robot, achieving efficient action decision-making without additional training or online interaction. This application automatically selects and fuses the optimal policy combination based on the action probability distribution output by each pre-trained strategy as the confidence level, generating a high-quality combined action distribution, and thus planning a reliable target action trajectory. Furthermore, it can be compatible with heterogeneous strategies of different architectures and modal inputs, effectively reusing existing model resources and avoiding the high costs of traditional fine-tuning or reinforcement learning. Simultaneously, through the dynamic combination mechanism, the system's generalization ability and robustness in open and complex environments are enhanced, significantly improving the practicality and adaptability of robot control.

[0046] The following is a further explanation of how the probability distribution information of actions corresponding to each pre-trained strategy in the aforementioned call strategy pool is determined based on real-time observation data. Figure 2 As shown, the above step S102 includes: S201. Call the pre-trained policies in the policy pool and determine the target modality data corresponding to each pre-trained policy in the real-time observation data.

[0047] Target modality data refers to real-time observation data that matches the input requirements of a specific pre-trained strategy. Since different pre-trained strategies may only support specific modalities, such as some strategies only accepting visual images and others requiring both visual and verbal inputs, it is necessary to extract suitable target modality data from the complete multimodal observation data.

[0048] In one possible implementation, the input interface specification of each pre-trained policy in the policy pool can be parsed. For example, the required modality type can be recorded through metadata or configuration files, such as "VA" indicating vision only and "VLA" indicating vision + language. Subsequently, for the currently acquired real-time observation data, the required modality subset for each policy is selected as the target modality data. For example, if a policy is a pure vision policy, its target modality data consists only of image frames; if it is a VLA policy, the target modality data includes both images and language instructions.

[0049] S202. Input the target modality data corresponding to each pre-training strategy into each pre-training strategy, and obtain the action probability distribution information by each pre-training strategy.

[0050] Each pre-trained policy has learned a function that maps from a specific modal input to an action distribution during its training phase. During inference, the system can feed the extracted target modal data into the corresponding policy model, performing one or more forward computations. For example, a diffusion-based VLA policy receives an image and text, and outputs a Gaussian distribution parameter representing the reasonable range of grasping actions under that instruction. All invoked policies complete this process independently, obtaining the action probability distribution information for each pre-trained policy.

[0051] In this embodiment, for each pre-trained policy in the policy pool, the required "target modality data" is accurately extracted from unified multimodal real-time observations, and then each policy is driven to independently generate action probability distributions. Modality alignment ensures that each policy performs at its maximum performance under effective input, avoiding invalid inference or information mismatch.

[0052] The following is a further explanation of determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy, such as... Figure 3 As shown, the above step S103 includes: S301. Combine the various pre-training strategies to obtain multiple pre-training strategy combinations.

[0053] Among them, pre-trained policy combination refers to the combination of two or more pre-trained policies in the policy pool according to certain rules (such as weighting, logical operation, etc.) to form a new decision unit.

[0054] Optionally, the pre-trained strategies can be combined in a convex manner, or combined in a logical AND manner, or combined in a logical OR manner, to obtain a combination of multiple pre-trained strategies.

[0055] For example, different policy combinations can be enumerated or sampled. For instance, if the policy pool contains policies A, B, and C, possible combinations include {A}, {B}, {A+B}, {A+C}, {A+B+C}, etc. Combination methods can be based on convex combination, logical AND, logical OR, etc.

[0056] Optionally, convex combination of pre-trained strategies can be performed by assigning non-negative weights to multiple pre-trained strategies, with the sum of the weights of each pre-trained strategy being 1, and weighted fusion of the action probability distribution information of each pre-trained strategy according to the weights of each pre-trained strategy.

[0057] Optionally, logical combination of pre-training strategies means that an action is adopted only if all participating pre-training strategies deem it feasible. This approach is suitable for tasks with high safety requirements, where only actions unanimously agreed upon by all pre-training strategies are executed.

[0058] Optionally, logically combining pre-trained strategies means that as long as any of the pre-trained strategies involved in the decision-making process deems an action feasible, then that action can be considered a candidate action. This approach is suitable for open-domain tasks and helps to improve the diversity of actions and task coverage, avoiding strategy failure due to blind spots of a single strategy.

[0059] S302. Based on the action probability distribution information of each pre-training strategy, determine the trajectory success rate of each pre-training strategy combination.

[0060] The trajectory success rate refers to the probability estimate that a certain strategy combination can successfully complete the target task by generating an action trajectory under given observations.

[0061] In one possible implementation, mapping relationships can be pre-established through offline simulation experiments. For example, tasks can be executed on each combination in a large number of simulation scenarios, and the success rate can be statistically analyzed as the trajectory success rate of that combination. Alternatively, online estimation can be performed, for instance, by constructing surrogate metrics based on the confidence level (e.g., distribution entropy, peak probability) or consistency (e.g., KL divergence) of the action distributions of each policy. For example, if the outputs of each policy in the combination are highly consistent and have high confidence, a higher trajectory success rate can be assigned.

[0062] S303. Determine the target pre-training strategy combination based on the trajectory success rate of each pre-training strategy combination.

[0063] Optionally, among all candidate combinations, the combination with the highest trajectory success rate can be selected as the target combination. This selection process can either choose the combination with the highest trajectory success rate or combine other constraints (such as computational cost and real-time performance) for multi-objective optimization. This ensures that the robot always adopts the best-performing ensemble strategy, improving task reliability.

[0064] In this embodiment, dynamic optimization of the strategy integration scheme is achieved by constructing multiple pre-trained strategy combinations and selecting the best one based on their trajectory success rate. The effectiveness of different combinations in task completion is evaluated using action probability distribution information, and the optimal target combination is automatically selected using trajectory success rate as an objective criterion. This avoids the limitations of manually setting fixed fusion rules, enabling the robot to adaptively select the most reliable strategy collaboration method according to the specific task context, significantly improving the robot's decision robustness and task success rate in complex and uncertain environments.

[0065] Before determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy, simulation experiments can be used to determine the trajectory success rate of each pre-training strategy combination, such as... Figure 4 As shown, the process also includes: S401. Determine the weight values ​​of each pre-trained strategy and generate multiple pre-trained strategy combinations based on the weight values ​​of each pre-trained strategy.

[0066] Weight values ​​refer to the numerical coefficients assigned to each pre-trained strategy in fusion methods such as convex combination, used to characterize its relative importance in the combination. Generating multiple pre-trained strategy combinations refers to forming multiple pre-trained strategy combinations through different weight configurations.

[0067] The weights of each pre-trained strategy can be preset, randomly sampled, or traversed within a preset range. For example, for two strategies, the weights are enumerated from 0.0 to 1.0 in steps of 0.1. Each set of weights corresponds to a unique strategy combination. For example, weights [0.7, 0.3] correspond to the weighted fusion of strategies A and B.

[0068] S402. Execute the target task in a simulation experimental environment and determine the trajectory success rate of each pre-trained strategy combination based on experimental observation data.

[0069] Among them, the simulation experimental environment refers to a high-fidelity virtual robot platform that can safely and efficiently test the performance of strategies, and the trajectory success rate refers to the proportion of times the strategy combination successfully completes the target task in the total number of tasks during multiple simulation runs.

[0070] For example, for each generated strategy combination, the target task (such as a grabbing task, a door-opening task, etc.) can be repeatedly executed in a simulation environment, and the success rate can be recorded. For instance, if a combination succeeds 85 times out of 100 attempts, its task success rate is 85%.

[0071] In this embodiment, the task success rate of strategy combinations under different weight configurations is pre-evaluated in a simulation environment, and the success rate is bound to the weight information as strategy combination information, providing a reliable basis for the selection of target combinations in the online stage. This avoids the high cost and risk of online trial and error and realizes strategy optimization based on empirical performance. Furthermore, by generating and recording multiple sets of weight combinations and their effects, the optimal strategy combination can be quickly retrieved during the robot's inference stage, significantly improving the deployment efficiency, stability, and task completion capability of the robot control system.

[0072] Optionally, if the combination of pre-trained strategies is a convex combination, the process of determining the weight values ​​of each pre-trained strategy and generating multiple combinations of pre-trained strategies based on their weight values ​​is as follows: Figure 5 As shown, the above S401 step includes: S501. For each pre-trained strategy, iterate through each weight value within the preset weight value range, and for the current weight value encountered, use the current weight value as the weight value of the pre-trained strategy.

[0073] The preset weight range refers to the feasible interval of each pre-trained strategy, such as [0,1].

[0074] For example, for a combination of N policies, the system performs gridded sampling of the weight space while satisfying the constraint that the sum of the weights is 1. For instance, in the case of two policies, it iterates through w1=0.0,0.1,...,1.0, where w1=0.0,0.1,...,1.0 corresponds to w2=1. w1, w2=1 w1. For three strategies, simplex grid sampling can be used. Each time, a set of current weight values ​​is taken as the weight configuration for a pre-trained strategy combination.

[0075] S502. Combine the pre-training strategies according to their weight values ​​to obtain multiple pre-training strategy combinations, wherein the sum of the weight values ​​of the pre-training strategy combinations in each combination is equal to 1.

[0076] During the traversal, pre-training strategies can be combined according to the weight values ​​of each pre-training strategy, thereby enabling the fusion of action probability distribution information of subsequent pre-training strategies based on the weight values ​​of the pre-training strategy combination.

[0077] For example, the policy pool contains pre-trained policy A and pre-trained policy B, hereinafter referred to as policy A and policy B. Policy A is a vision-based diffusion policy with an RGB image as input, and policy B is a visual-to-speech flow matching policy with an RGB image and text instructions as input. The current task observation includes both images and language instructions, therefore policy A and policy B are invoked. First, the corresponding target modality data are input into policy A and policy B respectively. Policy A receives the RGB image and outputs μA, while policy B receives the RGB image and text instructions and outputs μB. A convex combination method is used to fuse policy B, resulting in a policy combination with weights configured as wA=0.7 and wB=0.3. The outputs of policy A and policy B are then fused according to these weights to obtain the mean μC of the combined action distribution, thus constructing the pre-trained policy combination. This pre-trained policy combination means that, given the observation data, it outputs an action distribution centered at μC. By changing the weights of policy A and policy B, various different pre-trained policy combinations can be obtained.

[0078] The process of determining the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination includes: Based on the weight information of each pre-trained strategy in the target pre-trained strategy combination, the action probability distribution information of each pre-trained strategy is linearly fused to obtain the combined action distribution information.

[0079] Optionally, the action probability distribution information output by each pre-trained strategy in the target pre-trained strategy combination can be weighted and averaged according to the weight information to generate a new comprehensive distribution. The fused distribution retains the advantageous opinions of each strategy and reflects their credibility through the weights. For example, if strategy A has a higher weight value in the grasping task, its action suggestion will dominate in the combined distribution. This distribution is then used for sampling or optimization to generate specific control instructions.

[0080] In this embodiment, the action probability distributions of each strategy are linearly fused according to the weight information of the target pre-trained strategy combination, achieving a simple, efficient, and mathematically sound action integration. This ensures that the combined action distribution reflects both the independent judgment of each strategy and its verified relative importance, avoiding the instability caused by hard switching or non-normalized fusion. Linear fusion has low computational overhead, strong compatibility, and is suitable for high-dimensional continuous or discrete action spaces.

[0081] Based on the same inventive concept, this application also provides a robot control device corresponding to the robot control method. Since the principle of the device in this application is similar to that of the robot control method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0082] Figure 6 A schematic diagram of the structure of a robot control device provided in an embodiment of this application is shown.

[0083] The acquisition module 601 is used to acquire real-time observation data during the operation of the robot. The real-time observation data includes data from multiple modalities. The first determining module 602 is used to call the pre-trained strategies in the strategy pool and determine the action probability distribution information corresponding to each pre-trained strategy based on real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the robot's ability to perform actions under the pre-trained strategies. The combination module 603 is used to determine the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy. The second determining module 604 is used to determine the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination. The control module 605 is used to generate a target motion trajectory based on the combined motion distribution information and to control the robot based on the target motion trajectory.

[0084] Optionally, the second determining module 604 is specifically used for: Invoke the pre-trained policies in the policy pool and determine the target modality data corresponding to each pre-trained policy in the real-time observation data; The target modality data corresponding to each pre-training strategy is input into each pre-training strategy, and the action probability distribution information is predicted by each pre-training strategy.

[0085] Optionally, the combination module 603 is specifically used for: By combining the various pre-training strategies, multiple pre-training strategy combinations can be obtained; Based on the action probability distribution information of each pre-training strategy, determine the trajectory success rate of each pre-training strategy combination; The target pre-training strategy combination is determined based on the trajectory success rate of each pre-training strategy combination.

[0086] Optionally, the combination module 603 is specifically used for: Multiple pre-training strategy combinations can be obtained by performing convex combination of each pre-training strategy, or by performing logical AND combination of each pre-training strategy, or by performing logical OR combination of each pre-training strategy.

[0087] Optionally, the apparatus also includes an experimental module, specifically for: Determine the weight values ​​of each pre-trained strategy, and generate multiple combinations of pre-trained strategies based on the weight values ​​of each pre-trained strategy; The target task is executed in a simulation experimental environment, and the trajectory success rate of each pre-trained strategy combination is determined based on experimental observation data.

[0088] Optionally, if the combination of pre-trained strategies is a convex combination; Optionally, the combination module 603 is specifically used for: For each pre-trained strategy, iterate through each weight value within the preset weight value range, and for the current weight value encountered, use the current weight value as the weight value of the pre-trained strategy. Each pre-training strategy is combined according to its weight value to obtain multiple pre-training strategy combinations, wherein the sum of the weight values ​​of each pre-training strategy combination is equal to 1.

[0089] Optionally, the second determining module 604 is specifically used for: Based on the weight information of each pre-trained strategy in the target pre-trained strategy combination, the action probability distribution information of each pre-trained strategy is linearly fused to obtain the combined action distribution information.

[0090] This application achieves efficient action decision-making without additional training or online interaction by constructing a policy pool containing multiple pre-trained strategies and dynamically calling, evaluating, and combining these strategies based on real-time multimodal observation data of the robot. Using the action probability distribution output by each pre-trained strategy as a confidence level, this application automatically selects and fuses the optimal strategy combination to generate a high-quality combined action distribution, thereby planning a reliable target action trajectory. Furthermore, it can accommodate heterogeneous strategies with different architectures and modal inputs, effectively reusing existing model resources and avoiding the high costs of traditional fine-tuning or reinforcement learning. Simultaneously, through a dynamic combination mechanism, it enhances the system's generalization ability and robustness in open and complex environments, significantly improving the practicality and adaptability of robot control.

[0091] Figure 7 This illustration shows a schematic diagram of an electronic device provided in an embodiment of this application, including: a processor 701, a storage medium 702, and a bus 703. The storage medium 702 stores machine-readable instructions executable by the processor 701. When the electronic device runs a robot control method as described in the embodiment, the processor 701 communicates with the storage medium 702 via the bus 703. The processor 701 executes the machine-readable instructions. The preamble of the method item of the processor 701 executes the steps in the above-described robot control method.

[0092] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor, and the processor performs the steps in the above-described robot control method.

[0093] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0094] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0097] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0099] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A robot control method, characterized in that, include: During the robot's operation, real-time observation data is acquired, including data from multiple modalities. The pre-trained strategies in the strategy pool are invoked, and the action probability distribution information corresponding to each pre-trained strategy is determined based on the real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the feasibility of the robot performing actions under the pre-trained strategies. Based on the action probability distribution information of each pre-training strategy, determine the target pre-training strategy combination; Based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination, determine the combined action distribution information; The target motion trajectory is generated based on the combined motion distribution information, and the robot is controlled based on the target motion trajectory.

2. The method according to claim 1, characterized in that, The method of invoking pre-trained strategies in the strategy pool, and determining the action probability distribution information corresponding to each pre-trained strategy based on the real-time observation data, includes: Invoke the pre-trained policies in the policy pool and determine the target modality data corresponding to each pre-trained policy in the real-time observation data; The target modality data corresponding to each pre-training strategy is input into each pre-training strategy, and the action probability distribution information is predicted by each pre-training strategy.

3. The method according to claim 1, characterized in that, The step of determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy includes: The aforementioned pre-training strategies are combined to obtain multiple pre-training strategy combinations; Based on the action probability distribution information of each pre-trained strategy, determine the trajectory success rate of each combination of pre-trained strategies; The target pre-training strategy combination is determined based on the trajectory success rate of each pre-training strategy combination.

4. The method according to claim 3, characterized in that, The combination of the aforementioned pre-training strategies yields multiple pre-training strategy combinations, including: The pre-training strategies are combined by convex combination, or by logical AND combination, or by logical OR combination, to obtain the combination of the multiple pre-training strategies.

5. The method according to claim 3, characterized in that, Before determining the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy, the process also includes: Determine the weight values ​​of each of the pre-training strategies, and generate multiple combinations of pre-training strategies based on the weight values ​​of each of the pre-training strategies; The target task is executed in a simulation experimental environment, and the trajectory success rate of each pre-trained strategy combination is determined based on experimental observation data.

6. The method according to claim 5, characterized in that, If the combination of the pre-training strategies is a convex combination; The step of determining the weight values ​​of each of the pre-training strategies and generating multiple pre-training strategy combinations based on the weight values ​​of each of the pre-training strategies includes: For each of the pre-trained strategies, each weight value is traversed within a preset weight value range. For the current weight value encountered during the traversal, the current weight value is used as the weight value of the pre-trained strategy. Each pre-training strategy is combined according to its weight value to obtain multiple pre-training strategy combinations, wherein the sum of the weight values ​​of each pre-training strategy combination is equal to 1.

7. The method according to claim 1, characterized in that, The step of determining the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination includes: Based on the weight information of each pre-training strategy in the target pre-training strategy combination, the action probability distribution information of each pre-training strategy is linearly fused to obtain the combined action distribution information.

8. A robot control device, characterized in that, include: The acquisition module is used to acquire real-time observation data during the robot's operation, and the real-time observation data includes data from multiple modalities; The first determining module is used to call the pre-trained strategies in the strategy pool and determine the action probability distribution information corresponding to each pre-trained strategy based on the real-time observation data. The strategy pool includes multiple pre-trained strategies, and the action probability distribution information is used to represent the confidence level of the feasibility of the robot performing actions under the pre-trained strategies. The combination module is used to determine the target pre-training strategy combination based on the action probability distribution information of each pre-training strategy. The second determining module is used to determine the combined action distribution information based on the action probability distribution information of each pre-trained strategy in the target pre-trained strategy combination. The control module is used to generate a target motion trajectory based on the combined motion distribution information, and to control the robot based on the target motion trajectory.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a robot control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a robot control method as described in any one of claims 1 to 7.

Images