Robot dual-arm motion planning method and device based on fusion of MCTS and MPC

Abstract

The application relates to the technical field of robot control, and discloses a robot dual-arm motion planning method and device based on MCTS and MPC fusion, the method comprising the following steps: an MCTS layer based on a hybrid planner is used to analyze real-time motion data of the robot dual arms according to an acquired initial imitation strategy, so that a candidate action sequence is obtained; an MPC layer based on the hybrid planner is used to perform state optimization operation according to the candidate action sequence, so that optimal control instructions are obtained; the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer; and the robot dual arms are controlled to perform motion operation corresponding to the optimal control instructions according to the optimal control instructions. It can be seen that the application can fuse MCTS and MCP, search for an optimal high-level strategy in an abstract state space, generate accurate bottom-level control instructions in a continuous state space, and improve the real-time performance and accuracy of robot dual-arm motion planning in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot dual-arm motion planning method and device based on the fusion of MCTS and MPC. Background Technology

[0002] With the rapid development of robotics technology, dual-arm robots, with their unique advantages, have demonstrated enormous application potential in numerous fields such as industrial manufacturing, home services, and medical care. Dual-arm robots can simulate the coordinated movements of human arms to complete complex and delicate tasks, such as collaborative handling, assembly, and fine manipulation, greatly improving the efficiency and quality of task execution.

[0003] However, current robotic dual-arm motion planning technology still has significant shortcomings. For example, while optimization-based planning methods can handle complex motion constraints and dynamic characteristics, the computational complexity increases exponentially in high-dimensional state spaces, resulting in low planning efficiency and difficulty in meeting real-time requirements in dynamic environments. Data-driven methods can improve adaptability to specific scenarios through learning, but they rely excessively on massive amounts of labeled data, and their generalization ability is insufficient when facing new, unseen scenarios, leading to a sharp decline in performance.

[0004] Therefore, it is particularly important to propose a technical solution that can improve the real-time performance and accuracy of robot dual-arm motion planning. Summary of the Invention

[0005] This invention provides a robot dual-arm motion planning method and device based on the fusion of MCTS and MPC, which can improve the real-time performance and accuracy of robot dual-arm motion planning.

[0006] To address the aforementioned technical problems, the first aspect of this invention discloses a robot dual-arm motion planning method based on the fusion of MCTS and MPC. This method is applied to a robot dual-arm control system, which controls the robot's two arms and includes a hybrid planner. The method comprises:

[0007] Based on the MCTS layer of the hybrid planner, the real-time motion data of the robot's two arms is analyzed according to the acquired initial imitation strategy to obtain candidate action sequences; wherein, the candidate action sequence includes at least one candidate action; and the real-time motion data includes real-time state data;

[0008] Based on the MPC layer of the hybrid planner, state optimization operations are performed according to the candidate action sequence to obtain the optimal control command; wherein, the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer;

[0009] According to the optimal control command, the robot's two arms are controlled to perform the motion operation corresponding to the optimal control command;

[0010] The MCTS layer based on the hybrid planner analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain candidate action sequences, including:

[0011] Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer;

[0012] The real-time state data of the robot's two arms is abstracted to obtain discrete state data corresponding to the real-time state data;

[0013] Based on the tree strategy of the MCTS layer of the hybrid planner, and according to the simulation strategy, iterative calculations are performed on the discrete state data to obtain the iterative results;

[0014] The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

[0015] As an optional implementation, in the first aspect of the invention, the initial imitation strategy is determined in the following manner:

[0016] Extract the initial imitation strategy from the acquired bi-arm movement video data;

[0017] The initial imitation strategy includes a trajectory data set, which includes multiple trajectory data. Each trajectory data includes a state-action sequence and metadata. The state-action sequence includes multiple state-action pairs, and each state-action pair includes temporally corresponding state data and action data.

[0018] As an optional implementation, in the first aspect of the present invention, the tree strategy based on the MCTS layer of the hybrid planner, and the iterative operation performed on the discrete state data according to the simulation strategy to obtain the iterative result, includes:

[0019] Create a node in the search tree to represent the discrete state data, which serves as the root node of the search tree;

[0020] Based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree;

[0021] Determine whether the first leaf node meets the preset expansion conditions;

[0022] When it is determined that the first leaf node satisfies the expansion condition, at least one second leaf node corresponding to the first leaf node is generated based on the first leaf node, and one of the second leaf nodes is determined as the simulation starting point; wherein, each second leaf node is a child node of the first leaf node;

[0023] When it is determined that the first leaf node does not meet the expansion conditions, the first leaf node is determined as the simulation starting point;

[0024] According to the simulation strategy, a simulation operation is performed on the search tree to obtain simulated episodes and corresponding reward information for the simulated episodes.

[0025] Based on the reported information, update the statistics of all nodes related to the simulated series on the search tree; wherein the statistics of each node include the number of visits to the node and the action value of the action required to reach the node.

[0026] As an optional implementation, in the first aspect of the present invention, determining whether the first leaf node satisfies a preset expansion condition includes:

[0027] Determine whether the first leaf node meets the preset calculation constraints, and determine whether the robot's two arms have a new observation state;

[0028] When it is determined that the first leaf node satisfies the calculation constraint condition, and it is determined that the robot arms do not have the newly observed state, it is determined that the first leaf node does not satisfy the preset expansion condition.

[0029] When it is determined that the robot's two arms have the newly observed state, the first leaf node is determined to meet the pre-set expansion conditions.

[0030] As an optional implementation, in the first aspect of the present invention, determining whether the first leaf node satisfies a preset calculation constraint includes:

[0031] Determine whether the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, and determine whether the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, and determine whether the first leaf node has been fully expanded.

[0032] When it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when it is determined that the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when it is determined that the first leaf node has been fully expanded, the first leaf node is determined to meet the preset computing constraints.

[0033] When it is determined that the path length from the root node to the first leaf node is less than the preset maximum tree depth, and when it is determined that the remaining computing resources for the first leaf node are greater than the preset remaining resource value, and when it is determined that the first leaf node has not been fully expanded, it is determined that the first leaf node does not meet the preset computing constraints.

[0034] As an optional implementation, in the first aspect of the present invention, the MPC layer includes a state prediction function, an optimized minimum cost function, and an optimization constraint formula;

[0035] The MPC layer based on the hybrid planner performs state optimization operations according to the candidate action sequence to obtain the optimal control command, including:

[0036] Based on the state prediction function and the optimization constraint formula, the predicted state corresponding to the candidate action sequence is determined according to the candidate action sequence.

[0037] Based on the minimum cost function and the optimization constraint formula, the optimal control sequence is determined according to the predicted state, the candidate action sequence, and the determined reference state.

[0038] The control instruction corresponding to the first step in the optimal control sequence is determined as the optimal control instruction.

[0039] As an optional implementation, in the first aspect of the invention, the hybrid planner is determined in the following manner:

[0040] Obtain the structural parameters corresponding to the robot's two arms;

[0041] Based on the structural parameters corresponding to the robot's dual arms, the model parameters corresponding to the obtained source hybrid planner are adjusted to obtain a hybrid planner adapted to the robot's dual arms, and the hybrid planner is then migrated to the robot's dual arm control system; wherein, the simulation data used by the source hybrid planner has undergone standardization processing;

[0042] Specifically, before analyzing the real-time motion data of the robot's two arms to obtain candidate action sequences based on the acquired initial imitation strategy in the MCTS layer based on the hybrid planner, the method further includes:

[0043] The system detects whether there is an energy efficiency requirement for the robot's dual arms in the robot's dual-arm control system; wherein, the energy efficiency requirement is used to indicate the need to improve the energy efficiency of the robot's dual arms.

[0044] When the robot dual-arm control system detects an energy efficiency requirement for the robot dual arms, the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer are adjusted to obtain the adjusted hybrid planner.

[0045] A second aspect of this invention discloses a robot dual-arm motion planning device based on the fusion of MCTS and MPC. The device is applied to a robot dual-arm control system, which controls the robot's two arms, and includes a hybrid planner. The device comprises:

[0046] The analysis module is used to analyze the real-time motion data of the robot's two arms based on the MCTS layer of the hybrid planner and the acquired initial imitation strategy to obtain candidate action sequences; wherein, the candidate action sequence includes at least one candidate action; and the real-time motion data includes real-time state data.

[0047] The optimization module is used to perform state optimization operations based on the MPC layer of the hybrid planner and the candidate action sequence to obtain the optimal control command; wherein the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer.

[0048] The control module is used to control the robot's two arms to perform the motion operation corresponding to the optimal control command according to the optimal control command;

[0049] The analysis module, based on the MCTS layer of the hybrid planner, analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain the specific method of candidate action sequences, including:

[0050] Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer;

[0051] The real-time state data is abstracted to obtain discrete state data corresponding to the real-time state data;

[0052] Based on the tree strategy of the MCTS layer of the hybrid planner, and according to the simulation strategy, iterative calculations are performed on the discrete state data to obtain the iterative results;

[0053] The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

[0054] As an optional implementation, in a second aspect of the invention, the initial imitation strategy is determined in the following manner:

[0055] Extract the initial imitation strategy from the acquired bi-arm movement video data;

[0056] The initial imitation strategy includes a trajectory data set, which includes multiple trajectory data. Each trajectory data includes a state-action sequence and metadata. The state-action sequence includes multiple state-action pairs, and each state-action pair includes temporally corresponding state data and action data.

[0057] As an optional implementation, in a second aspect of the invention, the analysis module performs iterative calculations on the discrete state data based on the tree strategy of the MCTS layer of the hybrid planner and according to the simulation strategy to obtain the iterative results, specifically in the following ways:

[0058] Create a node in the search tree to represent the discrete state data, which serves as the root node of the search tree;

[0059] Based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree;

[0060] Determine whether the first leaf node meets the preset expansion conditions;

[0061] When it is determined that the first leaf node satisfies the expansion condition, at least one second leaf node corresponding to the first leaf node is generated based on the first leaf node, and one of the second leaf nodes is determined as the simulation starting point; wherein, each second leaf node is a child node of the first leaf node;

[0062] When it is determined that the first leaf node does not meet the expansion conditions, the first leaf node is determined as the simulation starting point;

[0063] According to the simulation strategy, a simulation operation is performed on the search tree to obtain simulated episodes and corresponding reward information for the simulated episodes.

[0064] Based on the reported information, update the statistics of all nodes related to the simulated series on the search tree; wherein the statistics of each node include the number of visits to the node and the action value of the action required to reach the node.

[0065] As an optional implementation, in a second aspect of the present invention, the specific method by which the analysis module determines whether the first leaf node satisfies the preset expansion conditions includes:

[0066] Determine whether the first leaf node meets the preset calculation constraints, and determine whether the robot's two arms have a new observation state;

[0067] When it is determined that the first leaf node satisfies the calculation constraint condition, and it is determined that the robot arms do not have the newly observed state, it is determined that the first leaf node does not satisfy the preset expansion condition.

[0068] When it is determined that the robot's two arms have the newly observed state, the first leaf node is determined to meet the pre-set expansion conditions;

[0069] The step of determining whether the first leaf node meets the preset calculation constraints includes:

[0070] Determine whether the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, and determine whether the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, and determine whether the first leaf node has been fully expanded.

[0071] When it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when it is determined that the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when it is determined that the first leaf node has been fully expanded, the first leaf node is determined to meet the preset computing constraints.

[0072] When it is determined that the path length from the root node to the first leaf node is less than the preset maximum tree depth, and when it is determined that the remaining computing resources for the first leaf node are greater than the preset remaining resource value, and when it is determined that the first leaf node has not been fully expanded, it is determined that the first leaf node does not meet the preset computing constraints.

[0073] As an optional implementation, in a second aspect of the present invention, the MPC layer includes a state prediction function, an optimized minimum cost function, and an optimization constraint formula;

[0074] The optimization module, based on the MPC layer of the hybrid planner, performs state optimization operations according to the candidate action sequence to obtain the optimal control command in the following specific ways:

[0075] Based on the state prediction function and the optimization constraint formula, the predicted state corresponding to the candidate action sequence is determined according to the candidate action sequence.

[0076] Based on the minimum cost function and the optimization constraint formula, the optimal control sequence is determined according to the predicted state, the candidate action sequence, and the determined reference state.

[0077] The control instruction corresponding to the first step in the optimal control sequence is determined as the optimal control instruction.

[0078] As an optional implementation, in a second aspect of the invention, the hybrid planner is determined in the following manner:

[0079] Obtain the structural parameters corresponding to the robot's two arms;

[0080] Based on the structural parameters corresponding to the robot's dual arms, the model parameters corresponding to the obtained source hybrid planner are adjusted to obtain a hybrid planner adapted to the robot's dual arms, and the hybrid planner is then migrated to the robot's dual arm control system; wherein, the simulation data used by the source hybrid planner has undergone standardization processing;

[0081] The device further includes:

[0082] The detection module is used to detect whether the robot dual-arm control system has an energy efficiency requirement for the robot dual arms before the analysis module analyzes the real-time motion data of the robot dual arms based on the MCTS layer of the hybrid planner and obtains the candidate action sequence according to the acquired initial imitation strategy; wherein, the energy efficiency requirement is used to indicate that the energy efficiency of the robot dual arms needs to be improved.

[0083] An adjustment module is used to adjust the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer when the robot dual-arm control system detects that there is an energy efficiency requirement for the robot dual arms, so as to obtain an adjusted hybrid planner.

[0084] A third aspect of this invention discloses another robot dual-arm motion planning device based on the fusion of MCTS and MPC, the device comprising:

[0085] Memory containing executable program code;

[0086] A processor coupled to the memory;

[0087] The processor calls the executable program code stored in the memory to execute the robot dual-arm motion planning method based on the fusion of MCTS and MPC disclosed in the first aspect of the present invention.

[0088] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute the robot dual-arm motion planning method based on the fusion of MCTS and MPC disclosed in the first aspect of the present invention.

[0089] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0090] In this embodiment of the invention, the MCTS layer based on the hybrid planner analyzes the real-time motion data of the robot's dual arms according to the acquired initial imitation strategy to obtain a candidate action sequence; wherein, the candidate action sequence includes at least one candidate action; the real-time motion data includes real-time state data; the MPC layer based on the hybrid planner performs state optimization operations according to the candidate action sequence to obtain the optimal control command; wherein, the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer; according to the optimal control command, the robot's dual arms are controlled to execute the motion operation corresponding to the optimal control command; thus, compared with existing robot dual-arm motion planning methods, by constructing a planner that integrates MCTS and MCP, it can still search for the optimal high-level strategy in the abstract state space and generate accurate low-level control commands in the continuous state space under complex and ever-changing control environments, thereby improving the real-time performance and accuracy of robot dual-arm motion planning, thereby improving the control efficiency and control accuracy of the robot's dual arms, and thus helping to improve the efficiency and accuracy of the robot's dual arms in completing motion tasks; and, by utilizing the initial imitation strategy and performing reinforcement learning with a small amount of data, it can improve the robot's motion performance and reduce the dependence on a large amount of data. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 This is a flowchart illustrating a robot dual-arm motion planning method based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention.

[0093] Figure 2 This is a flowchart illustrating another robot dual-arm motion planning method based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention.

[0094] Figure 3 This is a schematic diagram of the structure of a robot dual-arm motion planning device based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention;

[0095] Figure 4 This is a schematic diagram of another robot dual-arm motion planning device based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention.

[0096] Figure 5 This is a schematic diagram of another robot dual-arm motion planning device based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention. Detailed Implementation

[0097] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0098] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0099] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0100] This invention discloses a robot dual-arm motion planning method and apparatus based on the fusion of MCTS and MPC. It analyzes real-time motion data of the robot's dual arms based on the MCTS layer of the hybrid planner, obtaining candidate action sequences according to the acquired initial imitation strategy. Then, based on the MPC layer of the hybrid planner, it performs state optimization operations according to the candidate action sequences to obtain optimal control commands. The robot's dual arms are then controlled to execute the corresponding motion operations according to the optimal control commands. Compared to existing robot dual-arm motion planning methods, this invention, by constructing a planner that integrates MCTS and MPC, can still search for optimal high-level strategies in the abstract state space and generate accurate low-level control commands in the continuous state space even in complex and changing control environments. This improves the real-time performance and accuracy of robot dual-arm motion planning, thereby increasing the control efficiency and accuracy of the robot's dual arms, and ultimately improving the efficiency and accuracy of the robot's dual-arm motion tasks. Furthermore, it can utilize the initial imitation strategy and perform reinforcement learning with a small amount of data, improving the robot's motion performance and reducing reliance on large amounts of data. These are described in detail below.

[0101] Example 1

[0102] Please see Figure 1 , Figure 1 This is a flowchart illustrating a robot dual-arm motion planning method based on the fusion of MCTS and MPC, as disclosed in an embodiment of the present invention. Figure 1 The described robot dual-arm motion planning method based on the fusion of MCTS and MPC can be applied to a robot dual-arm motion planning device based on the fusion of MCTS and MPC. This device may include one of a smart terminal, a smart device, a smart system, and a server, wherein the server can be a local server or a cloud server; this embodiment of the invention is not limited thereto. This method can also be applied to a robot dual-arm control system, which can be applied to a robot equipped with mechanical dual arms. This robot dual-arm control system can be used to control the robot dual arms, and it is equipped with a hybrid planner. Furthermore, this hybrid planner can be constructed based on the fusion of MCTS (Monte Carlo Tree Search) and MPC (Model Predictive Control), and it is used to implement reinforcement learning planning for the robot dual arms; this embodiment of the invention is not limited thereto. Figure 1 As shown, the robot dual-arm motion planning method based on the fusion of MCTS and MPC can include the following operations:

[0103] 101. Based on the MCTS layer of the hybrid planner, the robot's two arms are analyzed in real time according to the obtained initial imitation strategy to obtain candidate action sequences.

[0104] In this embodiment of the invention, the hybrid planner may include an MCTS layer and an MPC layer; wherein, the MCTS layer can be used to perform random simulation based on the real-time state data of the robot's two arms to generate candidate action sequences, that is, high-level decision-making can be achieved through the MCTS layer; the MPC layer can be used to predict the future state of the robot's two arms to optimize the control command input for the robot's two arms, that is, low-level control can be achieved through the MPC layer; furthermore, the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer.

[0105] In this embodiment of the invention, the real-time motion data of the robot's dual arms may include real-time state data; optionally, the real-time motion data may include real-time state data corresponding to at least one preset sampling period, which is not limited in this embodiment of the invention; further optionally, the real-time state data may include at least one state parameter; further optionally, the state parameter may include, but is not limited to, one or more combinations of normalized joint angle, joint velocity, end effector pose, end effector force / torque, object pose, visual feature points, sensor data corresponding to the base IMU (Inertial Measurement Unit), and collision flags, which is not limited in this embodiment of the invention.

[0106] In this embodiment of the invention, the candidate action sequence may include at least one candidate action; optionally, each candidate action may include at least one action parameter; further optionally, the action parameter may include, but is not limited to, one or more combinations of joint torque, position increment, end effector Cartesian velocity, stiffness / damping setting, gripper opening, and force setting, which are not limited in this embodiment of the invention; further optionally, the position increment may be calculated by inverse kinematics, which are not limited in this embodiment of the invention.

[0107] 102. The MPC layer based on the hybrid planner performs state optimization operations based on the candidate action sequence to obtain the optimal control command.

[0108] In this embodiment of the invention, optionally, an optimal control sequence can be obtained by performing a state optimization operation; wherein, the optimal control sequence includes at least one control instruction, more specifically, the optimal control sequence includes at least one control instruction corresponding to a step; further, the optimal control instruction is the control instruction corresponding to the first step in the optimal control sequence, that is, the control instruction with the earliest timing sequence in the optimal control instruction, which is not limited in this embodiment of the invention.

[0109] 103. Based on the optimal control command, control the robot's two arms to execute the motion operation corresponding to the optimal control command.

[0110] Optionally, in this embodiment of the invention, the actual motion data of the robot's two arms regarding the execution of the motion operation can be fed back to the hybrid planner to optimize the hybrid planner, thereby achieving subsequent optimization of the robot's two arm movements. This embodiment of the invention does not limit this.

[0111] As can be seen, the method described in the embodiments of the present invention can analyze the real-time motion data of the robot's dual arms based on the MCTS layer of the hybrid planner and the obtained initial imitation strategy to obtain candidate action sequences. Then, based on the MPC layer of the hybrid planner, state optimization operations are performed according to the candidate action sequences to obtain the optimal control command. The robot's dual arms are then controlled to perform the motion operation corresponding to the optimal control command. In this way, compared with the existing robot dual-arm motion planning methods, by constructing a planner that integrates MCTS and MCP, it can still search for the optimal high-level strategy in the abstract state space and generate accurate low-level control commands in the continuous state space under complex and ever-changing control environments. This improves the real-time performance and accuracy of robot dual-arm motion planning, thereby improving the control efficiency and accuracy of the robot's dual arms, which in turn helps to improve the efficiency and accuracy of the robot's dual arms in completing motion tasks. Furthermore, by utilizing the initial imitation strategy and performing reinforcement learning with a small amount of data, the robot's motion performance can be improved, reducing the dependence on a large amount of data.

[0112] In an optional embodiment, the initial imitation strategy can be determined in the following way:

[0113] An initial imitation strategy is extracted from the acquired bi-arm movement video data.

[0114] Optionally, the bi-arm movement video data may include at least one human bi-arm movement demonstration video, which is not limited in this embodiment of the invention.

[0115] The initial imitation strategy may include a trajectory data set, which may include multiple trajectory data sets. Further, each trajectory data set may include a state-action sequence and metadata. Optionally, the state-action sequence may include multiple state-action pairs. Each state-action pair may include temporally corresponding state data and action data. Optionally, the metadata may include a task label and / or a success flag. For example, the task label may be "grab," "pin," etc., and the success flag may be used to indicate whether the motion task related to that task label in the video data was successfully executed; this embodiment of the invention does not impose limitations on this. Further, optionally, trajectory data is extracted from the initial imitation strategy based on a pre-set extraction frequency (e.g., 20-50Hz); this embodiment of the invention does not impose limitations on this.

[0116] Optionally, the initial imitation strategy extracted from the acquired bi-arm movement video data can be specifically as follows:

[0117] Using binocular vision technology, the robot perceives the joint motion trajectories of the two arms from the acquired video data of the two arms' movements, and generates robot joint trajectories based on kinematic mapping, thus forming a two-arm imitation strategy.

[0118] As can be seen, this optional embodiment can extract the initial imitation strategy required by the MCTS layer from the bi-arm motion video data, which can improve the accuracy and reliability of the initial imitation strategy generation, thereby improving the accuracy of the simulation search of the subsequent MCTS layer.

[0119] In an optional embodiment, the MCTS layer based on the hybrid planner analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain candidate action sequences, which may include the following operations:

[0120] Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer;

[0121] Abstracting the real-time state data yields discrete state data corresponding to the real-time state data;

[0122] The tree strategy of the MCTS layer of the hybrid planner is used, and iterative calculations are performed on discrete state data according to the simulation strategy to obtain the iterative results;

[0123] The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node, which is used as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

[0124] Optionally, generating a simulation policy corresponding to the MCTS layer based on the obtained initial imitation policy can specifically involve: performing behavioral cloning (BC) pre-training on the initial imitation policy to obtain a behavioral cloning policy (exemplarily, the behavioral cloning policy can be represented as "π_BC"), and determining the behavioral cloning policy as the simulation policy corresponding to the MCTS layer. This embodiment of the invention is not limited in its scope. Alternatively, as another optional implementation, a random policy can be determined as the simulation policy corresponding to the MCTS layer. This embodiment of the invention is not limited in its scope.

[0125] Real-time state data is continuous, so it can be abstracted using abstract functions to obtain discrete state data, namely the discrete state data mentioned above.

[0126] As can be seen, this optional embodiment can generate a simulation strategy based on an initial imitation strategy and abstract the real-time state data to obtain the corresponding discrete state data. Then, based on the tree strategy of the MCTS layer, iterative calculations are performed on the discrete state data according to the simulation strategy to obtain the iteration results. Then, the target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence. In this way, in the hybrid planner, while improving the determination accuracy of the simulation strategy and discrete state data required by the MCTS layer, a high-level action plan is generated through the MCTS layer, providing a variety of strategy candidates. This is conducive to improving the determination accuracy of the candidate action sequence, and further conducive to improving the determination accuracy of the subsequent optimal control command.

[0127] In this optional embodiment, the tree strategy based on the MCTS layer of the hybrid planner, and the iterative operation performed on the discrete state data according to the simulation strategy to obtain the iterative result, may include the following operations:

[0128] Create nodes in the search tree to represent discrete state data, which serve as the root nodes of the search tree;

[0129] Based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree;

[0130] Determine whether the first leaf node meets the pre-defined expansion conditions;

[0131] When it is determined that the first leaf node satisfies the expansion condition, at least one second leaf node corresponding to the first leaf node is generated based on the first leaf node, and one of the second leaf nodes is determined as the simulation starting point; wherein, each second leaf node is a child node of the first leaf node;

[0132] When it is determined that the first leaf node does not meet the expansion conditions, the first leaf node is determined as the simulation starting point;

[0133] Based on the simulation strategy, simulation operations are performed on the search tree to obtain simulated episodes and corresponding reward information for those episodes.

[0134] Based on the reported information, update the statistics of all nodes related to the simulated episodes on the search tree; the statistics for each node include the number of times the node has been visited and the action value of the action required to reach the node.

[0135] The iterative process at the MCTS layer can be understood as consisting of four parts: Selection, Expansion, Simulation, and Backup. In the Selection phase, the tree is traversed from the root node using a tree strategy to select a leaf node. In the Expansion phase, depending on the application details, one or more child nodes may be added from the selected leaf node. In the Simulation phase, a complete episode simulation is run using a simulation strategy, starting from the selected node or its newly added child nodes. In the Backup phase, the rewards generated by the simulated episode are used to update or initialize the action values ​​along the tree strategy traversal path.

[0136] For example, the number of extended second leaf nodes corresponding to each first leaf node can be 1 or 3, and the embodiments of the present invention do not limit this.

[0137] The simulated episode can include a complete state-action-reward trajectory sequence generated by repeatedly executing the iterative operation of "starting from the current node state → action selection → model prediction → reward accumulation" until the terminal state or the maximum depth is reached; the reward information corresponding to the simulated episode can include the reward (R); further, the action value can be the cumulative reward for each node.

[0138] Optionally, based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree. Specifically, based on the selection function included in the tree strategy, the selection score of each node in the search tree is calculated by traversing the search tree; the node with the highest selection score is determined as the first leaf node. This embodiment of the invention does not limit the scope of the invention.

[0139] Optionally, the selection function can be the UCB1 (Upper Confidence Bound) formula, or other formulas; this embodiment of the invention does not limit the specific formula. Specifically, the UCB1 formula can be as follows:

[0140]

[0141] in, Q i It is a node i Cumulative rewards; N i It is a node i Number of visits; N p It is a node i The number of times the parent node was visited; c These are exploration parameters used to balance exploration and utilization.

[0142] Regarding the simulation operation, it is necessary to start from the simulation starting point, perform the simulation according to the simulation strategy, and continue until the termination state before the reward R can be obtained.

[0143] Optionally, all nodes related to the simulated series in the search tree can include nodes from the root node to the terminal state; further optionally, based on the reward information, the statistical information of all nodes related to the simulated series in the search tree is updated, specifically: the cumulative reward and number of visits for each node can be updated as follows:

[0144] Q i = Q i + R, N i = N i +1;

[0145] in, Q i It is a node i Cumulative rewards N i It is a node i The number of visits, R is the reward corresponding to the above simulated series.

[0146] As can be seen, this optional embodiment can first create the node corresponding to the discrete state data as the root node of the search tree, and then select the first leaf node that matches the root node by traversing the search tree based on the selection function. If the first leaf node meets the expansion condition, the child node of the first leaf node is generated as the simulation starting point; otherwise, the first leaf node is directly determined as the simulation starting point. Then, the simulation operation is executed according to the simulation strategy to obtain the simulated episode and the corresponding reward information of the simulated episode. Then, the node access count and action value on the search tree are updated. In this way, tree search exploration action control can be realized through selection, expansion, simulation, backup and other steps, avoiding local optima, enhancing exploration efficiency, and thus more efficiently and accurately determining the candidate action sequence corresponding to the current state of the robot's two arms, which is conducive to providing more accurate action strategy candidates.

[0147] In this optional embodiment, determining whether the first leaf node satisfies the preset expansion conditions may include the following operations:

[0148] Determine whether the first leaf node meets the pre-set calculation constraints, and determine whether the robot's two arms have any new observation states;

[0149] When it is determined that the first leaf node meets the calculation constraints, and it is determined that there are no new observation states for the robot's two arms, it is determined that the first leaf node does not meet the pre-set expansion conditions.

[0150] When it is determined that the robot's two arms have a new observation state, the first leaf node is determined to meet the pre-set expansion conditions.

[0151] Optionally, when it is determined that the first leaf node does not meet the calculation constraints, the extension operation can be performed or not, depending on the actual situation. This embodiment of the invention does not impose any limitations.

[0152] As can be seen, this optional embodiment can also determine that the first leaf node does not meet the expansion condition when it is determined that the first leaf node meets the computational constraints and there are no new observation states in the robot's two arms. That is, when the computational conditions are insufficient and there are no new observation states, the node can be left unexpanded. When it is determined that there are new observation states in the robot's two arms, it can directly determine that the first leaf node meets the expansion condition, that is, the new child node must be expanded. This can improve the accuracy of determining whether the first leaf node meets the expansion condition, thereby further improving the execution accuracy of the expansion operation, which in turn helps to improve the accuracy and reliability of simulated exploration in the search tree.

[0153] In this optional embodiment, determining whether the first leaf node satisfies the preset calculation constraints may include the following operations:

[0154] Determine whether the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, and determine whether the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, and determine whether the first leaf node has been fully expanded.

[0155] When it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when it is determined that the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when it is determined that the first leaf node has been fully expanded, the first leaf node is determined to meet the preset computing constraints.

[0156] When it is determined that the path length from the root node to the first leaf node is less than the preset maximum tree depth, and when it is determined that the remaining computing resources for the first leaf node are greater than the preset remaining resource value, and when it is determined that the first leaf node has not been fully expanded, it is determined that the first leaf node does not meet the preset computing constraints.

[0157] For example, computing resources may include, but are not limited to, time resources and / or computing hardware resources, and this embodiment of the invention does not impose any limitations.

[0158] As can be seen, this optional embodiment can also determine that the first leaf node meets the computational constraint conditions when it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when the first leaf node has been fully expanded. Otherwise, it can determine that the first leaf node does not meet the computational constraint conditions. This can further improve the accuracy of the judgment on whether the first leaf node meets the computational constraint conditions and the reliability of the judgment result, thereby helping to improve the accuracy of the judgment on whether the first leaf node meets the expansion conditions.

[0159] Example 2

[0160] Please see Figure 2 , Figure 2 This is a flowchart illustrating a robot dual-arm motion planning method based on the fusion of MCTS and MPC, as disclosed in an embodiment of the present invention. Figure 2 The described robot dual-arm motion planning method based on the fusion of MCTS and MPC can be applied to a robot dual-arm motion planning device based on the fusion of MCTS and MPC. This device may include one of a smart terminal, a smart device, a smart system, and a server, wherein the server can be a local server or a cloud server; this embodiment of the invention is not limited thereto. This method can also be applied to a robot dual-arm control system, which can be applied to a robot equipped with mechanical dual arms. This robot dual-arm control system can be used to control the robot dual arms, and it is equipped with a hybrid planner. Furthermore, this hybrid planner can be constructed based on the fusion of MCTS (Monte Carlo Tree Search) and MPC (Model Predictive Control), and it is used to implement reinforcement learning planning for the robot dual arms; this embodiment of the invention is not limited thereto. Figure 2 As shown, the robot dual-arm motion planning method based on the fusion of MCTS and MPC can include the following operations:

[0161] 201. Based on the MCTS layer of the hybrid planner, the robot's two arms are analyzed in real time according to the obtained initial imitation strategy to obtain candidate action sequences.

[0162] In this embodiment of the invention, the candidate action sequence includes at least one candidate action; the real-time motion data includes real-time state data; the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer; wherein, the planning time domain is used to control the search depth, and the prediction time domain is used to control the optimization range, and this embodiment of the invention does not impose any limitations.

[0163] 202. Based on the state prediction function of the MPC layer of the hybrid planner and the optimization constraint formula of the MPC layer, determine the predicted state corresponding to the candidate action sequence according to the candidate action sequence.

[0164] In this embodiment of the invention, the MPC layer of the hybrid planner may include a state prediction function, an optimized minimum cost function, and an optimization constraint formula.

[0165] 203. Based on the minimum cost function and optimization constraint formula of the MPC layer of the hybrid planner, the optimal control sequence is determined according to the predicted state, candidate action sequence and determined reference state.

[0166] 204. Determine the control command corresponding to the first step in the optimal control sequence as the optimal control command.

[0167] In this embodiment of the invention, after step 204, the optimal control command is only sent to the joint actuators corresponding to the robot's two arms. In the next cycle, the state data of the robot's two arms will be re-detected to re-execute the operations of steps 202-204, and then the rolling solution will be performed to achieve rolling optimization, forming a closed-loop control, ensuring robust performance even in a dynamically changing environment.

[0168] In this embodiment of the invention, it can be understood that the MPC layer achieves optimal control through three main steps: prediction, optimization, and application. It can handle multi-objective optimization, incorporate constraints, and predict future states, thereby generating stable and efficient motion control commands.

[0169] In this embodiment of the invention, optionally, state data can be represented by a state space, which can be expressed by the following formula:

[0170] ;

[0171] in, t For one of the moments, For time t Status data.

[0172] Optionally, motion data can be represented using motion space, which can be expressed by the following formula:

[0173] ;

[0174] in, t For one of the moments, For time t The control command can be used to represent the action data corresponding to the control command.

[0175] Optionally, the state prediction function can be expressed as follows:

[0176] .

[0177] Optionally, the candidate action sequence can be represented by the following formula:

[0178] ;

[0179] in, H m It is the planning time domain corresponding to the MCTS layer.

[0180] For example, the traditional minimum cost function J can be represented by the following equation:

[0181] ;

[0182] In the above formula, H represents the prediction time domain; The reference state is represented by Q and R, which are weight matrices. The weight matrices can be used to adjust state tracking and control energy consumption.

[0183] In this embodiment of the invention, candidate action sequences are used. As the initial solution, the optimized minimum cost function can be expressed as follows:

[0184] ;

[0185] in, H p This is the prediction time domain of MPC, where, H p ≤ H m λ is a mixing parameter, which is also a regularization parameter. It is used to balance the deviation between MPC optimization and MCTS candidate actions to ensure the consistency between the optimal control sequence output by MPC and the candidate action sequence output by MCTS. Q and R Each is a weight matrix.

[0186] Optionally, the optimization constraint formula can be as follows:

[0187] ;

[0188] in, and These are the lower and upper limits of the constraint boundaries for the action data, respectively.

[0189] 205. Based on the optimal control command, control the robot's two arms to execute the motion operation corresponding to the optimal control command.

[0190] For further detailed descriptions of steps 201 and 205 in this embodiment of the invention, please refer to the detailed descriptions of steps 101 and 103 in Embodiment 1. These descriptions will not be repeated in this embodiment of the invention.

[0191] As can be seen, the method described in the embodiments of the present invention can analyze the real-time motion data of the robot's dual arms based on the MCTS layer of the hybrid planner, obtain candidate action sequences according to the acquired initial imitation strategy, and then perform state optimization operations based on the MPC layer of the hybrid planner to obtain the optimal control command. The robot's dual arms are then controlled to perform the motion operation corresponding to the optimal control command. Thus, compared to existing robot dual-arm motion planning methods, by constructing a planner that integrates MCTS and MCP, even in complex and changing control environments, it can still search for the optimal high-level strategy in the abstract state space and generate accurate low-level control commands in the continuous state space. This improves the real-time performance and accuracy of robot dual-arm motion planning, thereby improving the control efficiency and accuracy of the robot's dual arms, and ultimately enhancing the efficiency and accuracy of the robot's dual arms in completing motion tasks. Furthermore, by utilizing the initial imitation strategy and performing reinforcement learning with a small amount of data, the robot's motion performance can be improved, reducing its dependence on large amounts of data. Furthermore, based on the state prediction function and optimization constraint formula, the predicted state corresponding to the candidate action sequence can be determined according to the candidate action sequence. Then, based on the minimum cost function and optimization constraint formula, the optimal control sequence can be determined according to the predicted state, the candidate action sequence, and the determined reference state. Finally, the control instruction corresponding to the first step in the optimal control sequence is determined as the optimal control instruction. In this way, the future state of the robot's dual arms can be predicted through the MPC layer of the hybrid planner, and the optimal control input can be calculated by optimizing the objective function, thereby improving the accuracy of the generation of the optimal control instruction. This enables fine-grained control of the robot's dual arms and improves the stability and reliability of the robot's dual arm motion.

[0192] In an optional embodiment, the hybrid planner can be determined in the following way:

[0193] Obtain the structural parameters corresponding to the robot's two arms;

[0194] Based on the structural parameters corresponding to the robot's dual arms, the model parameters corresponding to the obtained source hybrid planner are adjusted to obtain a hybrid planner adapted to the robot's dual arms, and the hybrid planner is then transferred to the robot's dual-arm control system; wherein, the simulation data used by the source hybrid planner has been standardized.

[0195] For example, structural parameters may include, but are not limited to, the number of joints and / or dynamic parameters, and are not limited in the embodiments of the present invention.

[0196] Optionally, the adjustable model parameters may include, but are not limited to, one or more of the planning time domain corresponding to the MCTS layer, the prediction time domain corresponding to the MPC layer, the weight matrix, the hybrid parameters corresponding to the MPC layer, and the exploration parameters corresponding to the MCTS layer. This embodiment of the invention does not impose any limitations.

[0197] Among them, the source hybrid planner can be obtained by training on the source platform and the model parameters can be extracted after training; in this way, the model parameters can be fine-tuned with a small amount of data on the target platform (such as the above-mentioned robot dual-arm control system) to quickly adapt to the new robot dual-arm structure.

[0198] For example, the standardized simulation data may include state-action pairs using a standardized format (e.g., normalized joint angles and torques), which is not limited in this embodiment of the invention.

[0199] As can be seen, this optional embodiment can adjust the model parameters corresponding to the source hybrid planner according to the structural parameters of the robot's dual arms, thereby obtaining a hybrid planner adapted to the robot's dual arms. This allows the hybrid planner to be migrated to the robot's dual-arm control system. In this way, existing and reliable hybrid planners can be efficiently and accurately migrated to the robot's dual-arm control system through parameter fine-tuning, realizing cross-platform migration of simulation data, improving the platform migration efficiency and flexibility of the hybrid planner. This facilitates more efficient use of the hybrid planner and related simulation data for motion planning in more diverse robot structures / platforms, achieving universal simulation data for robot dual arms with different structures. It greatly reduces the generation and debugging of simulation data, improves robot development efficiency and reduces development costs, and thus facilitates more efficient and accurate motion planning of robot dual arms.

[0200] In an optional embodiment, before analyzing the real-time motion data of the robot's two arms to obtain candidate action sequences in the MCTS layer based on the acquired initial imitation strategy, the method may further include the following operations:

[0201] The test examines whether the robot's dual-arm control system has an energy efficiency requirement for the robot's dual arms; where energy efficiency requirement indicates the need to improve the energy efficiency of the robot's dual arms.

[0202] When an energy efficiency requirement for the robot's dual-arm control system is detected, the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer are adjusted to obtain the adjusted hybrid planner.

[0203] Optionally, adjusting the action selection criteria corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer to obtain the adjusted hybrid planner may include the following operations:

[0204] Based on energy efficiency requirements, determine the energy consumption regulation method suitable for the robot's dual arms; the energy consumption regulation method includes instantaneous energy consumption regulation method and / or long-term energy consumption regulation method.

[0205] When the energy consumption regulation method includes instantaneous energy consumption regulation, adjust the minimum cost function corresponding to the MPC layer;

[0206] When the energy consumption regulation method includes long-term energy consumption regulation, adjust the action screening conditions corresponding to the MCTS layer.

[0207] As can be seen, this optional embodiment can adjust the motion screening conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer when the robot dual-arm control system detects that there is an energy efficiency requirement for the robot dual arms, so as to obtain an adjusted hybrid planner. This can achieve adaptive adjustment of the robot dual-arm motion planning based on actual energy consumption requirements, reduce the energy consumption of the robot dual arms during the motion process, and thus improve the energy efficiency of the robot dual arms.

[0208] In this optional embodiment, adjusting the action filtering conditions corresponding to the MCTS layer may include the following operations:

[0209] In the selection operation of the MCTS layer, adjust / add branch selection conditions so that the adjusted selection operation can select the branch with the minimum expected total energy of action control under the equivalent task completion time.

[0210] As can be seen, this optional embodiment can also select the branch with the minimum total energy expectation of motion control under the equivalent task completion time by adjusting the MCTS layer, which can help the robot dual-arm control achieve long-cycle energy saving and improve the energy efficiency of the robot dual-arm.

[0211] In this optional embodiment, adjusting the minimum cost function corresponding to the MPC layer may include the following operations:

[0212] Adjust the energy consumption-related terms in the minimum cost function corresponding to the MPC layer to prioritize solutions with torque less than or equal to a preset torque value and / or speed less than or equal to a preset speed;

[0213] And / or,

[0214] Adjust the weight matrix in the minimum cost function corresponding to the MPC layer;

[0215] And / or,

[0216] The power loss term is added as a soft constraint to the minimum cost function.

[0217] For example, the energy consumption-related term can be the R·u² term in the minimum cost function (as shown in the above formula). The embodiments of the present invention are not limited thereto.

[0218] The weight matrix may include a first weight matrix (as shown in the above formula, weight matrix Q) and a second weight matrix (as shown in the above formula, weight matrix R). More specifically, adjusting the weight matrix in the minimum cost function corresponding to the MPC layer can be achieved by: adaptively adjusting the second weight matrix to be larger than the first weight matrix, and the difference between the second weight matrix and the first weight matrix is ​​the determined target difference. The larger the second weight matrix is ​​compared to the first weight matrix, the more significant the energy saving, but the corresponding speed will be slower. Therefore, it can be achieved through online adaptive adjustment, which is not limited in this embodiment of the invention.

[0219] The power loss term can be used to statistically analyze the accumulated power loss in the prediction time domain. For example, the power loss term can be represented by integration, but this embodiment of the invention does not limit it. By adding the power loss term as a soft constraint, the peak current can be further reduced, thereby achieving the effect of energy saving.

[0220] For example, in a real-world dual-arm insertion task, by simultaneously optimizing the "instantaneous energy consumption" (MPC layer) and the "long-term energy consumption expectation" (MCTS layer), the overall energy efficiency was improved by 10-15%.

[0221] As can be seen, this optional embodiment can also optimize the instantaneous energy consumption of the robot's dual arms by adjusting the energy consumption-related terms in the minimum cost function, by adjusting the weight matrix in the minimum cost function, or by adding the power loss term as a soft constraint to the minimum cost function, which is beneficial to improving the energy efficiency of the robot's dual arms.

[0222] In an optional embodiment, the method may further include the following operations:

[0223] Determine whether the error between the actual motion data of the robot's two arms and the open-loop prediction result of the MPC layer is greater than or equal to a preset error value, and whether the duration for which the error value is greater than or equal to the preset error value is greater than or equal to a preset duration.

[0224] When it is determined that the error between the actual motion data of the robot's two arms and the open-loop prediction result of the MPC layer is greater than or equal to a preset error value, and the duration of this error being greater than or equal to the preset error value is greater than or equal to a preset duration, the actual state data and / or actual reward corresponding to the actual motion data of the robot's two arms are fed back to the MCTS layer to trigger the MCTS layer to perform online state replanning, thereby correcting the MCTS decision and regenerating a new candidate action sequence; and, using the actual state data as initial conditions, the optimization problem in the prediction time domain is resolved, thereby covering the original predicted trajectory and realizing closed-loop rolling optimization.

[0225] Optionally, the MCTS layer may be triggered to perform an online state replanning operation, thereby correcting the MCTS decision and regenerating a new candidate action sequence. Specifically, this can be done as follows:

[0226] The actual reward replaces the original simulated reward to update the action value of each node in the search tree corresponding to the MCTS layer; if the node state after the action value update exceeds the boundary of the abstract space, the above extension operation is triggered to realize the adaptive repair of the decision of the MCTS layer.

[0227] As can be seen, this optional embodiment can correct the MCTS decision and re-solve the MPC optimization problem based on the actual reward and actual state data when the deviation between the actual movement of the robot's two arms and the predicted result is consistently large. This enables feedback optimization of the hybrid planner, which is beneficial to improving the control accuracy and reliability of the subsequent robot's two arms control.

[0228] Example 3

[0229] Please see Figure 3 , Figure 3 This is a schematic diagram of a robot dual-arm motion planning device based on the fusion of MCTS and MPC, as disclosed in an embodiment of the present invention. Figure 3The described robot dual-arm motion planning device based on the fusion of MCTS and MPC can include one of a smart terminal, a smart device, a smart system, and a server, wherein the server can be a local server or a cloud server, and this embodiment of the invention is not limited thereto. This device can be applied to a robot dual-arm control system, which can be applied to a robot equipped with mechanical dual arms. This robot dual-arm control system can be used to control the robot's dual arms, and it is equipped with a hybrid planner. Furthermore, this hybrid planner can be constructed based on the fusion of MCTS (Monte Carlo Tree Search) and MPC (Model Predictive Control), and it is used to implement reinforcement learning planning for the robot's dual arms, and this embodiment of the invention is not limited thereto. Figure 3 As shown, the robot dual-arm motion planning device based on the fusion of MCTS and MPC may include:

[0230] Analysis module 301 is used in the MCTS layer based on the hybrid planner to analyze the real-time motion data of the robot's two arms according to the acquired initial imitation strategy, and obtain candidate action sequences; wherein, the candidate action sequence includes at least one candidate action; the real-time motion data includes real-time state data;

[0231] Optimization module 302 is used in the MPC layer based on the hybrid planner to perform state optimization operations according to the candidate action sequence to obtain the optimal control command; wherein, the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer.

[0232] The control module 303 is used to control the robot's two arms to perform the motion operations corresponding to the optimal control commands according to the optimal control commands.

[0233] As can be seen, the apparatus described in the embodiments of the present invention can, based on the MCTS layer of the hybrid planner, analyze the real-time motion data of the robot's dual arms according to the acquired initial imitation strategy, obtain candidate action sequences, and then, based on the MPC layer of the hybrid planner, perform state optimization operations according to the candidate action sequences to obtain optimal control instructions. Based on these optimal control instructions, the robot's dual arms are controlled to execute the motion operations corresponding to the optimal control instructions. Thus, compared to existing robot dual-arm motion planning technologies, by constructing a planner that integrates MCTS and MCP, it can still search for optimal high-level strategies in the abstract state space and generate accurate low-level control instructions in the continuous state space under complex and changing control environments. This improves the real-time performance and accuracy of robot dual-arm motion planning, thereby improving the control efficiency and accuracy of the robot's dual arms, and ultimately enhancing the efficiency and accuracy of the robot's dual arms in completing motion tasks. Furthermore, by utilizing the initial imitation strategy and performing reinforcement learning with a small amount of data, the motion performance of the robot can be improved, reducing its dependence on large amounts of data.

[0234] In an optional embodiment, the initial imitation strategy can be determined in the following way:

[0235] Extract the initial imitation strategy from the acquired bi-arm movement video data;

[0236] The initial imitation strategy includes a trajectory data set, which includes multiple trajectory data. Each trajectory data includes a state-action sequence and metadata. The state-action sequence includes multiple state-action pairs, and each state-action pair includes temporally corresponding state data and action data.

[0237] As can be seen, the apparatus described in this optional embodiment can extract the initial imitation strategy required for the MCTS layer from the bi-arm motion video data, which can improve the accuracy and reliability of the initial imitation strategy generation, thereby improving the accuracy of the simulation search of the subsequent MCTS layer.

[0238] In an optional embodiment, the analysis module 301, based on the MCTS layer of the hybrid planner, analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain the specific method of candidate action sequences, which may include:

[0239] Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer;

[0240] Abstracting the real-time state data yields discrete state data corresponding to the real-time state data;

[0241] The tree strategy of the MCTS layer of the hybrid planner is used, and iterative calculations are performed on discrete state data according to the simulation strategy to obtain the iterative results;

[0242] The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node, which is used as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

[0243] As can be seen, the apparatus described in this optional embodiment can generate a simulation strategy based on an initial imitation strategy and abstract the real-time state data to obtain corresponding discrete state data. Then, based on the tree strategy of the MCTS layer, iterative operations are performed on the discrete state data according to the simulation strategy to obtain iteration results. Then, the target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence. In this way, in the hybrid planner, while improving the determination accuracy of the simulation strategy and discrete state data required by the MCTS layer, a high-level action plan is generated through the MCTS layer, providing diversified strategy candidates. This is conducive to improving the determination accuracy of candidate action sequences, and further conducive to improving the determination accuracy of subsequent optimal control commands.

[0244] In this optional embodiment, the specific method by which the analysis module 301 performs iterative calculations on the discrete state data based on the tree strategy of the MCTS layer of the hybrid planner and according to the simulation strategy to obtain the iterative results may include:

[0245] Create nodes in the search tree to represent discrete state data, which serve as the root nodes of the search tree;

[0246] Based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree;

[0247] Determine whether the first leaf node meets the pre-defined expansion conditions;

[0248] When it is determined that the first leaf node satisfies the expansion condition, at least one second leaf node corresponding to the first leaf node is generated based on the first leaf node, and one of the second leaf nodes is determined as the simulation starting point; wherein, each second leaf node is a child node of the first leaf node;

[0249] When it is determined that the first leaf node does not meet the expansion conditions, the first leaf node is determined as the simulation starting point;

[0250] Based on the simulation strategy, simulation operations are performed on the search tree to obtain simulated episodes and corresponding reward information for those episodes.

[0251] Based on the reported information, update the statistics of all nodes related to the simulated episodes on the search tree; the statistics for each node include the number of times the node has been visited and the action value of the action required to reach the node.

[0252] As can be seen, the apparatus described in this optional embodiment can first create the node corresponding to the discrete state data as the root node of the search tree, and then, based on the selection function, select the first leaf node that matches the root node by traversing the search tree. If the first leaf node meets the expansion condition, then the child node of the first leaf node is generated as the simulation starting point; otherwise, the first leaf node is directly determined as the simulation starting point. Then, the simulation operation is performed according to the simulation strategy to obtain the simulated episodes and the corresponding reward information of the simulated episodes. Then, the number of visits to the nodes and the action value on the search tree are updated. In this way, tree search exploration action control can be realized through steps such as selection, expansion, simulation, and backup, avoiding local optima and enhancing exploration efficiency. This allows for more efficient and accurate determination of the candidate action sequence corresponding to the current state of the robot's two arms, which is beneficial for providing more accurate action strategy candidates.

[0253] In this optional embodiment, the specific method by which the analysis module 301 determines whether the first leaf node satisfies the preset expansion conditions may include:

[0254] Determine whether the first leaf node meets the pre-set calculation constraints, and determine whether the robot's two arms have any new observation states;

[0255] When it is determined that the first leaf node meets the calculation constraints, and it is determined that there are no new observation states for the robot's two arms, it is determined that the first leaf node does not meet the pre-set expansion conditions.

[0256] When it is determined that there is a new observation state for the robot's two arms, the first leaf node is determined to meet the pre-set expansion conditions;

[0257] The determination of whether the first leaf node meets the pre-set calculation constraints includes:

[0258] Determine whether the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, and determine whether the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, and determine whether the first leaf node has been fully expanded.

[0259] When it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when it is determined that the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when it is determined that the first leaf node has been fully expanded, the first leaf node is determined to meet the preset computing constraints.

[0260] When it is determined that the path length from the root node to the first leaf node is less than the preset maximum tree depth, and when it is determined that the remaining computing resources for the first leaf node are greater than the preset remaining resource value, and when it is determined that the first leaf node has not been fully expanded, it is determined that the first leaf node does not meet the preset computing constraints.

[0261] As can be seen, the apparatus described in this optional embodiment can also determine that the first leaf node does not meet the expansion condition when it is determined that the first leaf node meets the computational constraint condition and the robot arms do not have any new observation states. That is, when the computational conditions are insufficient and there are no new observation states, the node can be left unexpanded. When it is determined that the robot arms have new observation states, the apparatus can directly determine that the first leaf node meets the expansion condition, that is, the new child node must be expanded. This can improve the accuracy of determining whether the first leaf node meets the expansion condition, thereby further improving the execution accuracy of the expansion operation, which in turn helps to improve the accuracy and reliability of the simulation exploration in the search tree. Furthermore, it can also determine that the first leaf node meets the computational constraint condition when it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when the remaining computational resources for the first leaf node are less than or equal to the preset remaining resource value, or when the first leaf node has been fully expanded. Otherwise, it can determine that the first leaf node does not meet the computational constraint condition. This can further improve the accuracy of determining whether the first leaf node meets the computational constraint condition and the reliability of the determination result, which in turn helps to improve the accuracy of determining whether the first leaf node meets the expansion condition.

[0262] In an optional embodiment, the MPC layer may include a state prediction function, an optimized minimum cost function, and an optimization constraint formula;

[0263] The optimization module 302, based on the MPC layer of the hybrid planner, performs state optimization operations according to the candidate action sequence to obtain the optimal control command. The specific method of obtaining the optimal control command may include:

[0264] Based on the state prediction function and the optimization constraint formula, the predicted state corresponding to the candidate action sequence is determined according to the candidate action sequence.

[0265] Based on the minimum cost function and the optimization constraint formula, the optimal control sequence is determined according to the predicted state, candidate action sequence and determined reference state;

[0266] The control command corresponding to the first step in the optimal control sequence is determined as the optimal control command.

[0267] As can be seen, the apparatus described in this optional embodiment can determine the predicted state corresponding to the candidate action sequence based on the state prediction function and the optimization constraint formula, and then determine the optimal control sequence based on the predicted state, the candidate action sequence, and the determined reference state, according to the minimum cost function and the optimization constraint formula. Then, the control instruction corresponding to the first step in the optimal control sequence is determined as the optimal control instruction. In this way, the future state of the robot's dual arms can be predicted through the MPC layer of the hybrid planner, and the optimal control input can be calculated by optimizing the objective function, thereby improving the accuracy of the generation of the optimal control instruction. This enables fine-grained control of the robot's dual arms and improves the stability and reliability of the robot's dual arm motion.

[0268] In an optional embodiment, the hybrid planner can be determined in the following way:

[0269] Obtain the structural parameters corresponding to the robot's two arms;

[0270] Based on the structural parameters corresponding to the robot's dual arms, the model parameters corresponding to the obtained source hybrid planner are adjusted to obtain a hybrid planner adapted to the robot's dual arms, and the hybrid planner is then transferred to the robot's dual-arm control system; wherein, the simulation data used by the source hybrid planner has been standardized.

[0271] Among them, such as Figure 4 As shown, the device may further include:

[0272] The detection module 304 is used to detect whether there is an energy efficiency requirement for the robot's dual-arm control system before the analysis module 301 analyzes the real-time motion data of the robot's dual arms based on the MCTS layer of the hybrid planner and obtains the candidate action sequence according to the acquired initial imitation strategy. The energy efficiency requirement is used to indicate the need to improve the energy efficiency of the robot's dual arms.

[0273] The adjustment module 305 is used to adjust the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer when the robot dual-arm control system detects that there is an energy efficiency requirement for the robot dual arms, so as to obtain the adjusted hybrid planner.

[0274] As can be seen, the apparatus described in this optional embodiment can adjust the model parameters corresponding to the source hybrid planner based on the structural parameters of the robot's dual arms, thereby obtaining a hybrid planner adapted to the robot's dual arms. This allows the hybrid planner to be migrated to the robot's dual-arm control system. Through parameter fine-tuning, an existing, reliable hybrid planner can be efficiently and accurately migrated to the robot's dual-arm control system, achieving cross-platform migration of simulation data. This improves the platform migration efficiency and flexibility of the hybrid planner, thus facilitating more efficient motion planning using the hybrid planner and related simulation data in more diverse robot structures / platforms. This invention enables the universal application of simulation data for dual-arm robots with different structures, significantly reducing the generation and debugging of simulation data, improving robot development efficiency and reducing development costs. This, in turn, facilitates more efficient and precise motion planning for the dual-arm robots. Furthermore, when an energy efficiency requirement for the dual-arm robot control system is detected, the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer are adjusted to obtain an adjusted hybrid planner. This allows for adaptive adjustment of the motion planning of the dual-arm robots based on actual energy consumption requirements, reducing energy consumption during the movement of the dual-arm robots and thus improving their energy efficiency.

[0275] Example 4

[0276] Please see Figure 5 , Figure 5 This is a schematic diagram of another robot dual-arm motion planning device based on the fusion of MCTS and MPC disclosed in an embodiment of the present invention. Figure 5 As shown, the robot dual-arm motion planning device based on the fusion of MCTS and MPC may include:

[0277] Memory 401 storing executable program code;

[0278] Processor 402 coupled to memory 401;

[0279] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the robot dual-arm motion planning method based on the fusion of MCTS and MPC described in Embodiment 1 or Embodiment 2 of the present invention.

[0280] Example 5

[0281] This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute the steps in the robot dual-arm motion planning method based on the fusion of MCTS and MPC described in Embodiment 1 or Embodiment 2 of this invention.

[0282] Example 6

[0283] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the robot dual-arm motion planning method based on MCTS and MPC fusion described in Embodiment 1 or Embodiment 2.

[0284] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0285] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0286] Finally, it should be noted that the robot dual-arm motion planning method and device based on MCTS and MPC fusion disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot dual-arm motion planning method based on the fusion of MCTS and MPC, characterized in that, The method is applied to a robot dual-arm control system, which controls the robot's two arms and includes a hybrid planner; wherein the method includes: Based on the MCTS layer of the hybrid planner, the real-time motion data of the robot's two arms is analyzed according to the acquired initial imitation strategy to obtain candidate action sequences; wherein, the candidate action sequence includes at least one candidate action; and the real-time motion data includes real-time state data; Based on the MPC layer of the hybrid planner, state optimization operations are performed according to the candidate action sequence to obtain the optimal control command; wherein, the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer; According to the optimal control command, the robot's two arms are controlled to perform the motion operation corresponding to the optimal control command; The MCTS layer based on the hybrid planner analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain candidate action sequences, including: Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer; The real-time state data of the robot's two arms is abstracted to obtain discrete state data corresponding to the real-time state data; Based on the tree strategy of the MCTS layer of the hybrid planner, and according to the simulation strategy, iterative calculations are performed on the discrete state data to obtain the iterative results; The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

2. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to claim 1, characterized in that, The initial imitation strategy is determined in the following manner: Extract the initial imitation strategy from the acquired bi-arm movement video data; The initial imitation strategy includes a trajectory data set, which includes multiple trajectory data. Each trajectory data includes a state-action sequence and metadata. The state-action sequence includes multiple state-action pairs, and each state-action pair includes temporally corresponding state data and action data.

3. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to claim 1, characterized in that, The tree strategy based on the MCTS layer of the hybrid planner, and the iterative operation performed on the discrete state data according to the simulation strategy to obtain the iterative result, includes: Create a node in the search tree to represent the discrete state data, which serves as the root node of the search tree; Based on the selection function included in the tree strategy, the first leaf node that matches the root node is selected by traversing the search tree; Determine whether the first leaf node meets the preset expansion conditions; When it is determined that the first leaf node satisfies the expansion condition, at least one second leaf node corresponding to the first leaf node is generated based on the first leaf node, and one of the second leaf nodes is determined as the simulation starting point; wherein, each second leaf node is a child node of the first leaf node; When it is determined that the first leaf node does not meet the expansion conditions, the first leaf node is determined as the simulation starting point; According to the simulation strategy, a simulation operation is performed on the search tree to obtain simulated episodes and corresponding reward information for the simulated episodes. Based on the reported information, update the statistics of all nodes related to the simulated series on the search tree; wherein the statistics of each node include the number of visits to the node and the action value of the action required to reach the node.

4. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to claim 3, characterized in that, The determination of whether the first leaf node meets the preset expansion conditions includes: Determine whether the first leaf node meets the preset calculation constraints, and determine whether the robot's two arms have a new observation state; When it is determined that the first leaf node satisfies the calculation constraint condition, and it is determined that the robot arms do not have the newly observed state, it is determined that the first leaf node does not satisfy the preset expansion condition. When it is determined that the robot's two arms have the newly observed state, the first leaf node is determined to meet the pre-set expansion conditions.

5. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to claim 4, characterized in that, The determination of whether the first leaf node meets the preset calculation constraints includes: Determine whether the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, and determine whether the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, and determine whether the first leaf node has been fully expanded. When it is determined that the path length from the root node to the first leaf node is greater than or equal to the preset maximum tree depth, or when it is determined that the remaining computing resources for the first leaf node are less than or equal to the preset remaining resource value, or when it is determined that the first leaf node has been fully expanded, the first leaf node is determined to meet the preset computing constraints. When it is determined that the path length from the root node to the first leaf node is less than the preset maximum tree depth, and when it is determined that the remaining computing resources for the first leaf node are greater than the preset remaining resource value, and when it is determined that the first leaf node has not been fully expanded, it is determined that the first leaf node does not meet the preset computing constraints.

6. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to any one of claims 1-5, characterized in that, The MPC layer includes a state prediction function, an optimized minimum cost function, and an optimization constraint formula. The MPC layer based on the hybrid planner performs state optimization operations according to the candidate action sequence to obtain the optimal control command, including: Based on the state prediction function and the optimization constraint formula, the predicted state corresponding to the candidate action sequence is determined according to the candidate action sequence. Based on the minimum cost function and the optimization constraint formula, the optimal control sequence is determined according to the predicted state, the candidate action sequence, and the determined reference state. The control instruction corresponding to the first step in the optimal control sequence is determined as the optimal control instruction.

7. The robot dual-arm motion planning method based on the fusion of MCTS and MPC according to any one of claims 1-5, characterized in that, The hybrid planner is determined in the following manner: Obtain the structural parameters corresponding to the robot's two arms; Based on the structural parameters corresponding to the robot's dual arms, the model parameters corresponding to the obtained source hybrid planner are adjusted to obtain a hybrid planner adapted to the robot's dual arms, and the hybrid planner is then migrated to the robot's dual arm control system; wherein, the simulation data used by the source hybrid planner has undergone standardization processing; Specifically, before analyzing the real-time motion data of the robot's two arms to obtain candidate action sequences based on the acquired initial imitation strategy in the MCTS layer based on the hybrid planner, the method further includes: The system detects whether there is an energy efficiency requirement for the robot's dual arms in the robot's dual-arm control system; wherein, the energy efficiency requirement is used to indicate the need to improve the energy efficiency of the robot's dual arms. When the robot dual-arm control system detects an energy efficiency requirement for the robot dual arms, the action selection conditions corresponding to the MCTS layer and / or the minimum cost function corresponding to the MPC layer are adjusted to obtain the adjusted hybrid planner.

8. A robot dual-arm motion planning device based on the fusion of MCTS and MPC, characterized in that, The device is applied to a robot dual-arm control system, which controls the robot's two arms and includes a hybrid planner; wherein the device comprises: The analysis module is used to analyze the real-time motion data of the robot's two arms based on the MCTS layer of the hybrid planner and the acquired initial imitation strategy to obtain candidate action sequences; wherein, the candidate action sequence includes at least one candidate action; and the real-time motion data includes real-time state data. The optimization module is used to perform state optimization operations based on the MPC layer of the hybrid planner and the candidate action sequence to obtain the optimal control command; wherein the planning time domain corresponding to the MCTS layer is less than or equal to the prediction time domain corresponding to the MPC layer. The control module is used to control the robot's two arms to perform the motion operation corresponding to the optimal control command according to the optimal control command; The analysis module, based on the MCTS layer of the hybrid planner, analyzes the real-time motion data of the robot's two arms according to the acquired initial imitation strategy to obtain the specific method of candidate action sequences, including: Based on the obtained initial imitation strategy, generate the imitation strategy corresponding to the MCTS layer; The real-time state data of the robot's two arms is abstracted to obtain discrete state data corresponding to the real-time state data; Based on the tree strategy of the MCTS layer of the hybrid planner, and according to the simulation strategy, iterative calculations are performed on the discrete state data to obtain the iterative results; The target leaf node with the most visits is selected from the iteration results to obtain the action sequence corresponding to the target leaf node as a candidate action sequence; wherein, the candidate action sequence is the action sequence corresponding to the path formed from the root node to the target leaf node.

9. A robot dual-arm motion planning device based on the fusion of MCTS and MPC, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the robot dual-arm motion planning method based on the fusion of MCTS and MPC as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the robot dual-arm motion planning method based on the fusion of MCTS and MPC as described in any one of claims 1-7.

Images