Humanoid robot task planning method, humanoid robot and storage medium

Abstract

The embodiment of the application discloses a humanoid robot task planning method, a humanoid robot and a storage medium. The method comprises the following steps: matching a behavior tree constructed in advance according to an acquired task instruction; controlling the humanoid robot to execute a task action sequence of the task instruction according to the behavior tree; when a preset abnormal condition is met according to real-time acquired working condition information, suspending execution of the behavior tree, starting a state machine corresponding to the abnormal condition; controlling the humanoid robot based on the state machine; and when the abnormal state is removed and a recovery condition is met, exiting the state machine and resuming execution of the behavior tree until all action sequences of the task instruction are completed. Therefore, the application can effectively improve the working efficiency of the humanoid robot in dealing with daily work and abnormal scenes.

Description

Technical Field

[0001] This application belongs to the field of robot control technology, and in particular relates to a humanoid robot task planning method, a humanoid robot, and a storage medium. Background Technology

[0002] In the interdisciplinary field of intelligent manufacturing in engineering machinery and robotics, humanoid robot task planning technology is a key technology for robotic arms to complete complex tasks such as grasping, sorting, assembling, and transporting engineering machinery parts. In existing technologies, robotic arm control typically relies on behavior trees for task planning. While complex behaviors can be defined through node combinations, the response mechanisms for high-priority anomalies such as low battery levels and sudden obstacles are cumbersome, requiring the embedding of anomaly handling logic in each subtree, which can easily lead to system redundancy and response delays. How to improve the control efficiency of robotic arms is a technical problem that urgently needs to be solved by those skilled in the art.

[0003] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0004] The purpose of this application is to provide a humanoid robot task planning method, a humanoid robot, and a storage medium, which can effectively improve the work efficiency of humanoid robots in dealing with daily work and abnormal scenarios.

[0005] To achieve the above objectives: In a first aspect, embodiments of this application provide a humanoid robot task planning method, comprising the following steps: matching a pre-constructed behavior tree according to the acquired task instructions; controlling the humanoid robot to execute the task action sequence of the task instructions according to the behavior tree; when the preset abnormal conditions are met according to the real-time acquired working condition information, pausing the execution of the behavior tree and starting the state machine corresponding to the abnormal conditions; controlling the humanoid robot based on the state machine; when the abnormal state is resolved and the recovery conditions are met, exiting the state machine and resuming the execution of the behavior tree, until all action sequences of the task instructions are completed.

[0006] In an optional embodiment of this application, the method further includes: obtaining a training task set, the training task set including at least one training task; determining a root node according to the training task, splitting the training task into at least one sub-task, and generating a corresponding node for each sub-task; constructing an initial behavior tree structure according to the root node and the nodes; training the initial behavior tree structure, and determining the initial behavior tree structure that satisfies the training completion as a behavior tree.

[0007] In an optional embodiment of this application, generating corresponding nodes for each subtask includes: determining the main arm and slave arm of the humanoid robot; determining position constraints based on the main position vector, end-effector rotation matrix, and feature point coordinates of the main arm, and the slave position vector of the slave arm; determining orientation constraints based on the end-effector normals, sliding vectors, and proximity vectors of the main and slave arms, as well as a preset constant matrix; marking the position constraints and orientation constraints as constraint conditions, and verifying the generated nodes according to the constraint conditions; saving nodes that satisfy the constraint conditions; and correcting nodes that do not satisfy the constraint conditions until they satisfy the constraint conditions.

[0008] In an optional embodiment of this application, training the initial behavior tree structure includes: extracting the structural information of the initial behavior tree, processing the structural information using a genetic algorithm to obtain individuals corresponding to each initial behavior tree; determining a population based on the individuals, with each population including at least one individual; designing a fitness function based on the completion information of the initial behavior tree, the completion information including task completion time, job accuracy index, and safety index; controlling crossover and mutation operations between individuals within the population during each iteration to obtain new individuals; evaluating the fitness value of each individual based on the fitness function, and performing a selection operation based on the fitness value to update the individuals within the population; performing a migration operation between populations after a preset number of iterations; continuing to perform crossover and mutation operations and selection operations on the population after the migration operation; determining whether the individuals and the population meet a preset convergence condition after each iteration; if they meet the condition, determining the initial behavior tree corresponding to the individual as the trained behavior tree; if not, continuing to the next iteration.

[0009] In an optional embodiment of this application, the task action sequence of controlling a humanoid robot to execute task instructions according to a behavior tree includes: configuring the state of each node in the behavior tree, including one of the states of idle, running, success, failure, and interruption; initializing all nodes to the idle state; when a task instruction is received, adapting the root node to the idle state, the root node being the initial node of the behavior tree; activating each node sequentially according to the node execution order of the behavior tree, switching the state of the activated node from idle to running, and detecting its execution feedback in real time; if the task corresponding to the node is successfully completed, updating the node state to success, and activating the next node according to the node execution order of the behavior tree; if the task corresponding to the node fails, updating the node state to failure, pausing execution, and outputting a warning; if, during the execution of the node, the preset abnormal conditions are met according to the real-time acquired working condition information, updating the node state to interruption; saving the position of the interrupted node and its working condition information; if all nodes in the behavior tree are in the success state, determining that the task instruction is completed; if there is a node in the failure or interruption state, determining that the task execution is abnormal, outputting abnormal alarm information, releasing the system resources occupied by the behavior tree, and resetting the state of all nodes to the idle state.

[0010] In an optional embodiment of this application, the task action sequence of controlling the humanoid robot to execute task instructions according to the behavior tree further includes: pausing the execution of the current node when a switching instruction is obtained; saving the working condition information of the humanoid robot under the current node; matching the corresponding target behavior tree according to the switching instruction; configuring the root node of the target behavior tree to the running state, and loading the saved working condition information as context input to drive the execution of the nodes of the target behavior tree.

[0011] In an optional embodiment of this application, preset abnormal conditions include: determining that the current battery is in an abnormal state based on the battery information in the working condition information; and / or determining that there are obstacles in the working environment based on the environmental information in the working condition information; and / or determining that the humanoid robot body or end effector has collided based on the collision information in the working condition information; and / or determining that the communication between the humanoid robot and the host computer or sensor is interrupted based on the communication status in the working condition information.

[0012] In an optional embodiment of this application, pausing the execution of the behavior tree includes: determining the current task node at which the behavior tree is paused, wherein the current task node is a child node in the behavior tree; saving the working condition information when the abnormal condition is met; exiting the state machine and resuming the execution of the behavior tree includes: restoring the state of the humanoid robot according to the working condition information, and continuing to execute the unfinished task action sequence from the current task node.

[0013] Secondly, embodiments of this application provide a humanoid robot, including: a processor and a memory storing a computer program, wherein when the processor runs the computer program, it implements the steps of the above-described method.

[0014] Thirdly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method.

[0015] The embodiments of this application have the following beneficial effects: The humanoid robot task planning method provided in this application includes the following steps: matching acquired task instructions with a pre-constructed behavior tree; controlling the humanoid robot to execute the task action sequence of the task instructions according to the behavior tree; when the preset abnormal conditions are met according to the real-time acquired working condition information, pausing the execution of the behavior tree and starting the state machine corresponding to the abnormal condition; controlling the humanoid robot based on the state machine; when the abnormal state is resolved and the recovery conditions are met, exiting the state machine and resuming the execution of the behavior tree until all action sequences of the task instructions are completed. Therefore, this application can pre-construct a behavior tree and adopt a top-down control structure, consisting of a series of nodes such as selection nodes, sequence nodes, condition nodes, and action nodes. In daily work, the behavior tree is used to complete the control of the humanoid robot for daily "recognition-grasping-assembly", thereby achieving refined control of complex behaviors such as parts classification, grasping, and assembly, ensuring the continuity and reliability of the work process. Simultaneously, when an anomaly occurs, the behavior tree can precisely pause at the current task node and switch to state machine control mode to handle unexpected situations. After the anomaly is resolved, it resumes execution from the original task node based on the saved context information, achieving an efficient closed loop of "interruption-processing-recovery," effectively improving system fault tolerance and operational continuity. The collaborative mechanism of the two exhibits a clear complementarity: the state machine, with its convenient management of high-priority operations, clearly distinguishes the robot's abnormal handling state, ensuring that unexpected situations can be responded to in real time, thus compensating for the inherent defects of behavior trees in anomaly response, which require embedding anomaly handling logic in each subtree, easily leading to system redundancy and response delays; while the behavior tree, with its hierarchical and modular structural advantages and strong expressive ability of complex behaviors, focuses on the logical definition of process-oriented operations such as part grasping and assembly, ensuring clear logic and scalability of routine task execution. The combination of the two not only strengthens the system's adaptability to complex working conditions but also enhances the humanoid robot's autonomous decision-making and continuous operation capabilities in dynamic environments, effectively reducing the risk of task failure due to abnormal interruptions, and providing a highly reliable and flexible technical solution for intelligent robot control in industrial automation scenarios.

[0016] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it according to the contents of the specification, and to make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a humanoid robot task planning method provided in one embodiment.

[0019] Figure 2 A behavior tree node type diagram is provided for one embodiment.

[0020] Figure 3 This is a schematic diagram of a behavior tree construction example provided in one embodiment.

[0021] Figure 4 This is a schematic diagram illustrating the transition relationships between node states and their corresponding operations, provided as an embodiment.

[0022] Figure 5 This is a schematic block diagram of the structure of a humanoid robot provided in one embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements.

[0024] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0025] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0026] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0027] It should be noted that step designations such as S110 and S120 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S120 first and then S110, etc., but these should all be within the protection scope of this application.

[0028] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0029] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0030] Humanoid robot task planning is a core technology that combines task requirements to break down objectives, design paths and action sequences to ensure the robot completes tasks efficiently and accurately. Among these, dual-arm humanoid robots present the greatest challenge in control. Existing solutions rely solely on behavior trees for task planning. While complex behaviors can be defined through node combinations, response mechanisms for high-priority anomalies such as low battery or sudden obstacles are cumbersome, requiring the embedding of anomaly handling logic in each subtree, easily leading to system redundancy and response delays. Furthermore, traditional behavior trees suffer from single node states and control flow design flaws. The operation time of action nodes is much longer than the trigger signal interval of the root node, easily causing action preemption problems due to high-frequency signals. Moreover, the design of behavior tree structures relies on manual intervention, making autonomous learning and optimization difficult. Therefore, to overcome the shortcomings of existing technologies and improve the control efficiency of robots, especially humanoid robots, this application provides a humanoid robot task planning method. The application scenario of this application mainly involves controlling humanoid robots to perform assembly, handling, and other tasks. Specifically, this application involves controlling and planning the dual robotic arms of the humanoid robot. The focus of this invention is on the task planning and motion control of humanoid robots. For a clear description of the method provided in this embodiment, please refer to... Figures 1-4 This includes steps S110 to S140.

[0031] Step S110: Match the pre-built behavior tree according to the obtained task instructions.

[0032] In one embodiment, to achieve high efficiency and robustness in humanoid robot operations within intelligent manufacturing scenarios for engineering machinery, the control method provided in this application employs a collaborative planning mechanism combining behavior trees and state machines. Through precise division of labor and dynamic collaboration between these two functions, rapid response to high-priority abnormal events is ensured, while also supporting fine-grained planning of complex operational behaviors. It should be clarified that for routine, complex, and refined control, a behavior tree control mode is used; however, when handling abnormal situations, the behavior tree control mode is switched to a state machine for control.

[0033] Therefore, it is first necessary to clarify the control method under the behavior tree pattern, and the construction of the behavior tree is carried out in advance. This application also provides the construction process of the behavior tree.

[0034] The behavior tree adopts a traditional multi-branch tree structure, consisting of multiple nodes connected in a top-down hierarchical order. Nodes are categorized into root nodes, control nodes, condition nodes, and action nodes. The root node is the sole entry point to the behavior tree, responsible for triggering the execution flow of the entire tree; control nodes manage the execution order and logical relationships of child nodes; condition nodes determine whether the environment or internal state meets specific thresholds, thus deciding the branch direction; and action nodes encapsulate specific operation instructions, such as task units like grasping, moving, or assembling.

[0035] Any behavior tree can be encoded into an executable computer program. Its control flow nodes can be implemented using conditional statements, while execution nodes are typically user-defined. Execution of the behavior tree begins at the root node, which generates a trigger signal called "Ticks" that allows execution at a given frequency and sends this signal to its child nodes. A node is executed only if it receives a trigger signal. If a child node is currently executing, it immediately returns to a running state; if the goal has been achieved, it immediately returns to a success state; otherwise, it returns to a failure state.

[0036] In one embodiment, the method further includes: obtaining a training task set, the training task set including at least one training task; determining a root node according to the training task, splitting the training task into at least one sub-task, and generating a corresponding node for each sub-task; constructing an initial behavior tree structure according to the root node and the nodes; training the initial behavior tree structure, and determining the initial behavior tree structure that satisfies the training completion as a behavior tree.

[0037] In one embodiment, the training task set is derived from typical operating procedures in actual engineering machinery operation scenarios and consists of at least one training task. Each training task can correspond to a specific engineering machinery task, such as assembly, handling, welding and other types of tasks.

[0038] For each training task, a root node is determined, representing the highest-level objective of the entire task, such as "grab a bolt from the box and move it to the placement hole." The task objectives are obtained through task parsing, including specific part types, assembly accuracy requirements, and process priorities. Scene constraint parameters, such as workstation coordinate ranges, static obstacle outlines, and equipment interface protocols, are also acquired. These objective parameters and scene constraint parameters are loaded into the behavior tree's global parameter library to construct a 3D scene model, completing the initialization.

[0039] Subsequently, the training task is broken down into several sub-tasks. Based on the specific requirements and hierarchical relationships of the tasks, the task is progressively decomposed into more detailed sub-task nodes until the smallest execution unit is reached. Sub-task nodes can be "finding the target object," "grabbing a part," "moving an object," etc. A corresponding node is generated for each sub-task, which corresponds to one of the following: a control node, a condition node, or an action node. Action nodes include environment modeling, target object recognition, spatial positioning, gripper opening and closing, chassis movement, and posture adjustment. Appropriate action nodes need to be defined for each sub-task based on the task and actual requirements. Condition nodes select to execute or skip certain sub-nodes based on conditional judgments. In the grasping process, the gripper needs to determine condition nodes such as "whether the grasp was successful" and "whether the arm movement has been completed" as trigger conditions for executing the next action, and then proceed to the next operation according to the task logic sequence.

[0040] And configuration control nodes, which control the execution flow of the behavior tree, can be further divided into selection nodes, sequential nodes, and parallel nodes. Selection nodes select different subtasks to execute based on conditions; sequential nodes execute all sub-nodes in a predefined order; parallel nodes execute all sub-nodes simultaneously. For specific diagrams and classifications of control nodes, condition nodes, and action nodes, please refer to [reference needed]. Figure 2 The diagram shows the behavior tree node types.

[0041] In one embodiment, generating a corresponding node for each subtask includes: determining the main arm and slave arm of the humanoid robot; determining position constraints based on the main position vector, end-effector rotation matrix, and feature point coordinates of the main arm, and the slave position vector of the slave arm; determining orientation constraints based on the end-effector normals, sliding vectors, and approach vectors of the main and slave arms, as well as a preset constant matrix; marking the position constraints and orientation constraints as constraint conditions, and verifying the generated nodes according to the constraint conditions; saving nodes that satisfy the constraint conditions; and correcting nodes that do not satisfy the constraint conditions until they satisfy the constraint conditions.

[0042] In one implementation, it is important to clarify that defining precise constraints is a prerequisite for ensuring operational safety and efficiency in task planning for humanoid robot behavior tree collaborative operations. Therefore, clear constraints need to be defined for each generated node. Furthermore, considering the application scenario of the humanoid robot, which primarily involves controlling its dual robotic arms, it is also necessary to distinguish the master-slave relationship within the dual robotic arms: the master arm is responsible for leading the operational tasks, while the slave arm assists in completing auxiliary actions.

[0043] The constraints proposed in this application include position and orientation constraints of the two robotic arms, forming a strongly coupled collaborative logic, which is then transformed into condition nodes of a behavior tree to realize real-time verification and conflict avoidance of the movement of the two arms, providing underlying support for the generation of the optimal collaborative operation sequence.

[0044] Specifically, the constraints of the dual robotic arms include two dimensions: position and orientation. These two dimensions together constitute the core constraints for dual-arm collaboration. Position constraints focus on the spatial distance and relative position of the end effectors of the two arms, ensuring that the master arm (core operation) and the slave arm (support assistance) maintain a safe distance during grasping, assembly, and other processes, avoiding trajectory intersections and collisions. First, the position constraints are determined based on the master position vector, end effector rotation matrix, and feature point coordinates of the master arm, and the slave position vector of the slave arm. The mathematical representation of the position constraints can be found in the following formula.

[0045] (1) In the above formula, This represents the position vector of the end of the main arm in the base coordinate system. Let be the position vector of the arm's end in the base coordinate system; The rotation matrix at the end of the main arm. The coordinates of the main arm feature points are used to determine the allowable position range of both arms.

[0046] Orientation constraints focus on the attitude adaptability of the end effector, directly affecting gripping stability and assembly accuracy. Orientation constraints are determined by the end-effector normals, sliding vectors, and approach vectors of the master and slave arms, as well as a preset constant matrix. The mathematical expression can be found in the following formula.

[0047] (2) In the above formula, For the end normal, Let be the sliding vector. As approximation vectors, the three form an attitude coordinate system; A 3x3 constant matrix is ​​used to represent the preset posture relationship between the master and slave arms, such as parallel or perpendicular. The orientation is checked for compatibility through posture matrix operations.

[0048] Position constraints and orientation constraints are marked as constraints, forming a strongly coupled whole. The constraints are used to constrain and correct each generated node: position constraints define safety boundaries, and orientation constraints ensure posture accuracy. To be integrated into the planning process, both are transformed into core condition nodes of the behavior tree. Before executing any robotic arm movement, grasping, or other actions, joint displacement data is collected in real time and substituted into the constraint formulas shown in equations (1) and (2) for verification. If the constraints are met, execution is allowed, and the corresponding node is saved for subsequent control based on the behavior tree; if not, adjustment logic is triggered to correct nodes that do not meet the constraints, such as fine-tuning the posture, until the requirements are met.

[0049] The defined position and orientation constraints form a strongly coupled logic, which can reduce the collision rate of the two arms through real-time verification by behavior tree condition nodes; the orientation constraints ensure the adaptability of the assembly posture, improve the success rate of part picking, and solve the problems of trajectory conflict and posture deviation in traditional collaborative operations.

[0050] Once the nodes are identified, they can be connected. Based on the task and the hierarchical relationship between nodes, different nodes are connected to form a behavior tree structure. The connection method can be a parent-child relationship (parent node connects to child node) or a sibling relationship (sibling nodes are parallel). In the system, "behavior actions," "human-computer interaction," and "sensor detection" are sibling nodes, while the above nodes and "task nodes" are parent-child nodes.

[0051] Subsequently, execution conditions and strategies are set for each node. The execution conditions of a node are the preconditions or triggering conditions for node execution, and the execution strategy is the specific behavior and action of the robot when the node executes. After execution, the node returns a success, failure, or running status. For example, during the part grasping process, the movement of the robotic arm requires position and posture information from the vision detection nodes to determine whether it is within the workspace and to analyze the grasping posture of the robotic gripper.

[0052] Based on this, the initial behavior tree can be constructed, clarifying key aspects such as root node configuration, task decomposition, node configuration and connection, thereby improving the standardization and adaptability of behavior tree construction.

[0053] In one embodiment, training the initial behavior tree structure includes: extracting the structural information of the initial behavior tree; processing the structural information using a genetic algorithm to obtain individuals corresponding to each initial behavior tree; determining a population based on the individuals, with each population including at least one individual; designing a fitness function based on the completion information of the initial behavior tree, the completion information including task completion time, job accuracy index, and safety index; in each iteration, controlling the crossover and mutation operation between individuals within the population to obtain new individuals; evaluating the fitness value of each individual based on the fitness function, and performing a selection operation based on the fitness value to update the individuals within the population; after a preset number of iterations, performing a migration operation between the populations; continuing the crossover and mutation operation and selection operation on the population after the migration operation; after each iteration, determining whether the individual and the population meet a preset convergence condition; if they meet the condition, determining the initial behavior tree corresponding to the individual as the trained behavior tree; if not, continuing to the next iteration.

[0054] In one embodiment, to improve the adaptability and operational efficiency of the constructed initial behavior tree in the intelligent manufacturing scenario of engineering machinery, the behavior tree structure and parameters can be iteratively optimized using a predetermined training method. In a preferred embodiment of this application, an optimization strategy based on a genetic algorithm is used to train the initial behavior tree. By simulating the process of natural selection and evolution, adaptive adjustments are made to node connection relationships, execution order, execution conditions, and behavior strategies.

[0055] First, the initial behavior trees to be trained are initialized, generating an initial population containing multiple individuals, each corresponding to an encoded behavior tree structure. The individuals within the initial population are divided based on their structural complexity and task relevance, ensuring coverage of different topologies and strategy combinations. The initial population consists of N manually generated groups of behavior tree individuals, each group corresponding to a complete robot task planning scheme. The nodes of the behavior trees need to be encoded, using multi-level gene encoding to represent the behavior tree structure.

[0056] Based on the operational scenario design, encoding can begin at the sub-task level, with integer sequences representing the execution order, such as "visual bolt recognition → robotic arm positioning → gripper grasping → bolt placement → detection" corresponding to [1,2,3,4,5]. Next, encoding is performed using the node connection structure of the initial behavior tree, with adjacency matrices representing parent-child / sibling relationships between nodes, such as matrix element (i,j)=1 indicating that node i is the parent node of node j. Finally, the control parameters of each node are encoded, with real / integer sequences representing operational parameters, such as bolt gripping force and the safety distance between the two robotic arms. Encoding transforms the behavior tree into a chromosome form operable by a genetic algorithm, ensuring the diversity of the behavior tree. Its initial population needs to cover different operational strategies, such as different gripping postures and path planning priorities, to avoid getting trapped in local optima.

[0057] Furthermore, a fitness function is designed to evaluate the performance of behavior trees as individuals, quantifying the numerical values. The fitness function is used to evaluate the operational performance of behavior tree individuals, and needs to comprehensively consider three core indicators: time efficiency, operational accuracy, and safety. The calculation formula is shown below.

[0058] (3) In the above formula, This represents the total time to complete the task, including the time spent on grabbing, assembling, obstacle avoidance, and other steps. The shorter this time, the higher the evaluation score. This represents the accuracy of the work (e.g., if the bolt assembly deviation is ≤0.03mm, this value is 1; otherwise, points are deducted proportionally to the deviation). This represents a safety indicator. When the robot is working without collisions or grasping failures, this value is set to 1. It decreases by 0.2 for each abnormality. All of the above parameters are included in the completion information, which can be obtained directly through the simulation of the initial behavior tree. , , This represents a preset weighting coefficient that satisfies... The constraints can be adjusted according to task requirements.

[0059] The population and individuals are then iterated upon, and the behavior tree is updated using selection, crossover, and mutation operators in the genetic algorithm. During each iteration, a selection operation combining roulette wheel selection and elite retention methods can be used within the population to select high-quality individuals as parent generations. The probability of each individual being selected is related to its fitness. It is directly proportional; the probability of selection is calculated using the following formula.

[0060] (4) In the above formula, Let be the probability that individual i is selected. This represents the fitness value of the individual. This is the sum of the fitness values ​​of all individuals in the population to which the individual belongs. Under this strategy, individuals with high fitness are more likely to be selected; Elite retention: directly retain the top 10% of high-fitness individuals in the population (without participating in crossover or mutation), ensuring that no behavior trees with better structure are lost.

[0061] Crossover operations are performed on the selected parent individuals to reorganize the node structure and parameters and generate child individuals. Specifically, crossover operations can be divided into the following three levels.

[0062] The first step is subtask layer crossover. A random crossover point is selected, and the subtask sequence fragments of two parent sequences are swapped. For example, parent sequence A [1,2,3,4,5] and parent sequence B [1,3,2,5,4] are swapped at crossover point 3 to generate the child sequence [1,2,3,5,4]. Invalid sequences, such as duplicate subtasks, are corrected through deduplication.

[0063] The second is node connection layer crossover. Subtree segments are swapped, such as swapping the "grab → open gripper" subtree of parent A with the "grab → pose fine-tuning → open gripper" subtree of parent B, to optimize grasping stability.

[0064] Third is parameter layer cross-validation. Arithmetic cross-validation optimizes operation parameters. For example, the parent generation's gripping force [50,80]N and [60,90]N are cross-validated to generate the child generation [55,85]N, which is adapted to the gripping requirements of bolts of different materials.

[0065] Simultaneously, mutation operators are used to optimize the behavior tree locally. To maintain population diversity, random mutations are performed on the offspring individuals after crossover, with the mutation probability set to 0.05-0.1 to prevent excessive mutation of the behavior tree from destroying the high-quality structure. Specifically, the mutation operations can be divided into three levels: Subtask-level mutation: Randomly select two different subtask numbers and swap their positions in the sequence, for example, mutating [1,2,3,4,5] into [1,4,3,2,5]; Node connection-level mutation: Randomly modify the connection relationship of 1-2 nodes, such as changing "anti-collision" from a child node of "walking" to a sibling node, or adding or deleting condition nodes, such as adding a condition node of "part posture detection" before grasping; Parameter-level mutation: Add small perturbations to the execution condition parameters, such as mutating "grasping force threshold" from [50,80] into [53,74].

[0066] Simultaneously, after a preset number of iterations, a migration operation can be performed between populations, migrating individuals with the highest fitness values ​​(e.g., 5%) from each subpopulation to other subpopulations and replacing a corresponding proportion of low-fitness individuals, thus promoting the cross-population dissemination of superior genes. The migration cycle can be arbitrarily set, for example, once every 10 generations, to avoid frequent migrations that could lead to population convergence and premature loss of diversity.

[0067] The offspring individuals generated by crossover and mutation are merged with elite individuals to form a new population of size N; the corresponding behavior tree is generated by decoding (gene sequence → behavior tree structure), and the node connection relationship, execution order, execution conditions and behavior strategy are updated.

[0068] After each iteration, it can be determined whether the individual and the population meet the preset convergence criteria. Convergence criteria can be that the number of iterations reaches a preset threshold, or that the average fitness of the population stabilizes after 10 consecutive generations. If not, the next iteration continues until the criteria are met. When the convergence criteria are met, iteration stops and the behavior tree corresponding to the optimal individual is output as the final scheduling strategy.

[0069] The behavior tree optimized by the genetic algorithm demonstrates significant advantages in engineering machinery parts operation scenarios. First, by optimizing the sub-task sequence, it reduces invalid actions and shortens task completion time. Second, by adjusting execution condition parameters to adapt to part characteristics, it improves the grasping success rate and assembly accuracy. Third, by optimizing node connections, it reduces the collision event rate. This optimization mechanism allows the behavior tree to adapt to complex manufacturing environments without human intervention, significantly improving the operational performance of dual-arm humanoid robots. The behavior tree optimized by the genetic algorithm, through adaptive adjustments to sub-task sequence, node connections, and parameters, maintains high operational performance across different part types and operation scenarios, enhancing the robot's adaptability and performance in complex manufacturing environments without requiring manual reconfiguration of the behavior tree structure.

[0070] To facilitate understanding of the final behavior tree control strategy, based on the above principles and steps, the behavior tree construction case for a dual-arm collaborative robot's grasping and handling is as follows. Figure 3 As shown, nodes with memory are identified by adding the symbol " "Represented graphically (e.g., sequential nodes with memory are represented by "→")" (The boxes are represented graphically). The behavior tree corresponds to the task instructions; that is, each task instruction corresponds to a specific behavior tree structure. This behavior tree organizes the task execution logic in a hierarchical manner. When a task instruction is obtained, the corresponding behavior tree can be directly determined based on the task instruction.

[0071] Step S120: Control the humanoid robot to execute the task action sequence of the task instructions according to the behavior tree.

[0072] In one embodiment, the task action sequence for controlling a humanoid robot to execute task instructions based on a behavior tree includes: configuring the state of each node in the behavior tree, including one of the following states: idle, running, successful, failed, and interrupted; initializing all nodes to the idle state; when a task instruction is received, adapting the root node to the idle state, the root node being the initial node of the behavior tree; activating each node sequentially according to the node execution order of the behavior tree, switching the state of the activated node from idle to running, and detecting its execution feedback in real time; if the task corresponding to the node is successfully completed, updating the node state to successful, and activating the next node according to the node execution order of the behavior tree; if the task corresponding to the node fails, updating the node state to failed, pausing execution, and outputting a warning; if, during the execution of the node, a preset abnormal condition is met according to the real-time acquired working condition information, updating the node state to interrupted; saving the position of the interrupted node and its working condition information; if all nodes in the behavior tree are in the successful state, determining that the task instruction is completed; if any node is in the failed or interrupted state, determining that the task execution is abnormal, outputting an abnormal alarm message, releasing the system resources occupied by the behavior tree, and resetting the state of all nodes to the idle state.

[0073] In one implementation, the construction and specific process architecture of the behavior tree have been described in detail above; please refer to the preceding text for details. The behavior tree adopts a top-down hierarchical structure, consisting of a root node, control nodes, condition nodes, and action nodes. However, in actual operation of engineering machinery, it faces two core problems: First, the operation time of the action node is much longer than the trigger signal interval of the root node. For example, the bolt gripping and placement action requires 15-20 seconds to complete, while the signal sending interval is usually 0.1-0.5 seconds. High-frequency signals will cause the control flow node to repeatedly send verification signals to the running child nodes, which can easily lead to action preemption. Second, the classic behavior tree only contains three states: running, success, and failure. If the operation is restarted after an abnormal interruption, the node initialization process needs to be re-executed, such as recalibrating the robot arm posture, resulting in resource waste.

[0074] To this end, this application optimizes the task action sequence flow of a humanoid robot controlled by behavior tree to execute task instructions, expanding upon the existing three node states of running, success, and failure. The extended definition of node states includes one of the following: idle, running, success, failure, and interruption. Furthermore, the control flow nodes are set as memory nodes to adapt to the complex task requirements of the robot. Specifically, the memory node always remembers the return state of its child nodes, avoiding re-execution of child nodes until the entire sequence node or selection node is completed in success or failure. When the parent node returns success or failure, the memory is cleared so that all child nodes can be considered in the next trigger.

[0075] First, the idle state indicates that the node is not activated and is in an initial standby state. Therefore, during initialization, all nodes in the behavior tree are traversed, and their states are uniformly set to idle to ensure that no historical execution residue remains on any node when the system starts. When a task instruction is received, the system adapts to the root node that should be in the idle state and activates the root node, updating its state from idle to running. That is, when controlling according to the behavior tree, a startup operation is performed on the behavior tree to adapt to the nodes in the idle state. When the robotic arm is in standby mode, it receives the Tick signal from the parent node, triggering resource initialization, such as requesting sensor permissions and establishing tool communication. After moving to the predetermined pose, it enters the running state, such as performing bolt gripping, ensuring that there are no equipment conflicts before the operation.

[0076] Once the root node is activated, the behavior tree's child nodes can be traversed downwards according to the Tick signal. Following the execution order of the nodes in the behavior tree, each node is activated sequentially, switching its state from idle to running, and its execution feedback is monitored in real time. For activated and running nodes, that is, based on the control parameters contained in the node, corresponding action commands are executed on the humanoid robot, such as the left arm grasping a bolt, the right arm collaboratively locating the mounting hole, or determining whether the environmental conditions of the current node meet the execution requirements, such as detecting whether the target object's pose is within the grasping range, or whether the communication link is stable.

[0077] If the task corresponding to a node is successfully completed, the node status is updated to success, and the next node is activated according to the execution order of the behavior tree. Conversely, if the task corresponding to a node fails, the node status is updated to failure, execution is paused, and a warning is output to prompt the relevant personnel to intervene.

[0078] Furthermore, if a preset abnormal condition is met during node execution based on real-time acquired operational information, the node state will be updated to interrupted. The preset abnormal condition will activate the state machine control model, which will be explained in detail later. For nodes entering the interrupted state, a corresponding pause operation is performed, while for nodes in the running state, the interruption operation immediately stops the currently executing action of the humanoid robot and saves the location of the interrupted node and the current robot operational information. Additionally, when the situation corresponding to the preset abnormal condition is resolved, an activation operation can be performed, adapting to the interrupted node. By saving the location of the interrupted node and the current robot operational information, the robot's execution flow is restored, and the task continues from the original interrupted node. First, the task is restored based on the field data saved during the last interruption, such as restoring the position of the robotic arm and loading the previous execution progress. After restoration, the node re-enters the running state from the interrupted state and continues executing the unfinished task.

[0079] If all nodes in the behavior tree are in a successful state, the task instruction is considered complete. At this time, the root node state is also updated to successful, the entire behavior tree stops traversing and enters standby mode.

[0080] If any node is in a failed or interrupted state, the task execution is considered abnormal, and a stop operation is performed accordingly. The stop operation applies to all running or interrupted nodes. This operation will output an abnormal alarm message, release the system resources occupied by the behavior tree, and feed back the execution result to the parent node. After the operation is completed, the state of all nodes is reset to idle state, waiting for the next task assignment.

[0081] By controlling the daily work of humanoid robots through behavior trees, it supports the fine planning of complex work behaviors, realizes the automation and intelligent management of work processes, and improves the efficiency and stability of task execution.

[0082] In one embodiment, the task action sequence of controlling the humanoid robot to execute task instructions according to the behavior tree further includes: pausing the execution of the current node when a switching instruction is obtained; saving the working condition information of the humanoid robot under the current node; matching the corresponding target behavior tree according to the switching instruction; configuring the root node of the target behavior tree to the running state, and loading the saved working condition information as context input to drive the execution of the nodes of the target behavior tree.

[0083] In one implementation, the behavior tree control logic also includes a switching operation to respond to switching commands. Switching commands typically address the need for seamless multi-task transitions in complex operations, such as receiving a transfer task during bolt assembly. To address this, the switching operation can first perform a pause operation, stopping the action and saving the state. Then, it matches the corresponding target behavior tree based on the switching command. The target behavior tree is typically an independent behavior tree not directly related to the current task, such as switching from an assembly task to a material handling task. The system loads the root node of the target behavior tree and configures it as running, while injecting saved working condition information as the execution context to ensure the continuity and consistency of the execution environment for the new task, thereby achieving smooth switching and efficient collaboration between multiple tasks. After the switch is complete, the resources of the original behavior tree are released, and the target behavior tree begins traversal and execution. Each node makes autonomous decisions based on the input working condition information, ensuring seamless task transitions and adapting to emergency scheduling needs.

[0084] Furthermore, if it is necessary to return to the original behavior tree to continue task execution after completing the target behavior tree, the original behavior tree can be reloaded based on the previously saved work status information, and its root node can be set to running state to restore the execution progress before the interruption. After verifying the integrity of the context data, the system continues traversing from the interrupted node to ensure task continuity without having to start traversing from the beginning again.

[0085] To better understand the transition relationships between the states of each node, please refer to... Figure 4 As shown. Figure 4 It demonstrates the transition logic between five states: idle, running, interrupted, failed, and successful. It also includes the relationship between start operation, activation operation, pause operation, stop operation, and switch operation and the state of each node.

[0086] The extended five node states and memory-based control flow nodes avoid motion preemption issues caused by high-frequency signals. After an abnormal interruption, restarting does not require re-initialization, shortening the robotic arm's posture recovery time and reducing resource waste. Switching operations support seamless multi-task integration, improving emergency scheduling response efficiency.

[0087] Step S130: When the preset abnormal conditions are met according to the real-time acquired working condition information, pause the execution of the behavior tree and start the state machine corresponding to the abnormal conditions; control the humanoid robot based on the state machine.

[0088] Step S140: When the abnormal state is resolved and the recovery conditions are met, exit the state machine and resume the execution of the behavior tree until all action sequences of the task instruction are completed.

[0089] In one embodiment, preset abnormal conditions include: determining that the current battery is in an abnormal state based on battery information in the working condition information; and / or determining that there are obstacles in the working environment based on environmental information in the working condition information; and / or determining that the humanoid robot body or end effector has collided based on collision information in the working condition information; and / or determining that communication between the humanoid robot and the host computer or sensor is interrupted based on communication status in the working condition information.

[0090] In one implementation, the response mechanism for high-priority anomalies in traditional behavior tree control is cumbersome, requiring the embedding of anomaly handling logic in each subtree, which can lead to system redundancy and response delays. Therefore, this application introduces a state machine control mechanism into the behavior tree framework to handle high-priority anomaly responses and achieve rapid interruption and recovery. The state machine operates independently of the behavior tree, monitoring system status in real time. Once it determines, based on real-time robot status information, that the robot's current state meets preset anomaly conditions, the state machine can break through the behavior tree's conventional workflow, directly triggering anomaly response control, immediately pausing the currently executing behavior tree node, and switching to state machine control.

[0091] Among these, the preset exception conditions can be pre-defined high-priority exception events, which can be arbitrarily set according to actual needs. For ease of understanding, this application provides several common examples of preset exception conditions.

[0092] First, the system determines if the battery is in an abnormal state based on the battery information in the operating status data. This abnormal state includes, but is not limited to, situations such as battery level below a preset threshold, excessively high or low battery temperature, or exceeding the battery charging cycle limit, which may affect the normal operation of the humanoid robot. When such a state is detected, the system immediately triggers an abnormal response, suspends the execution of the current behavior tree, and switches to state machine control to prevent task interruption or equipment damage due to power problems. Controlling this state machine can involve planning charging paths, adjusting task priorities, or activating a low-power mode to ensure the robot can complete emergency responses safely.

[0093] Second, based on the environmental information in the work status data, it is determined whether there are obstacles in the work environment. When the sensor detects an unexpected obstacle in the work path, to avoid the risk of collision, the state machine immediately intervenes in the control flow, suspends the execution of the original behavior tree instructions, and starts the obstacle avoidance subroutine. After the obstacle is removed or the path is replanned, if it is determined that the safety recovery conditions are met, the state machine exits, returns control to the behavior tree, and continues to execute the remaining action sequence, ensuring task continuity and work safety.

[0094] Third, based on the collision information in the working status data, it is determined whether the humanoid robot body or end effector has collided. When an abnormal collision signal is detected, it is immediately identified as a high-priority anomaly. The state machine quickly interrupts the execution of the current behavior tree, triggers emergency braking, and switches to a safe mode to prevent mechanical damage or personnel injury. Self-check and reset operations are performed. After confirming that the external environment is safe and there is no continued risk of collision, the state machine actively releases control and resumes the execution of the behavior tree.

[0095] Fourth, the system determines whether there is a network outage or excessive signal delay based on the communication status in the operational information. When a communication interruption with the main control system is detected or the delay exceeds a preset threshold, behavior tree execution is immediately paused, and state machine control is initiated. State machine control can switch to local emergency control mode to execute predetermined safety shutdown or return-to-home strategies. Once communication is restored and link stability is confirmed, the state machine verifies the system synchronization status. If the recovery conditions are met, control is returned, ensuring operational continuity and the reliability of remote control.

[0096] In one embodiment, pausing the execution of the behavior tree includes: determining the current task node at which the behavior tree is paused, wherein the current task node is a child node in the behavior tree; saving the working condition information when the abnormal condition is met; exiting the state machine and resuming the execution of the behavior tree includes: restoring the state of the humanoid robot according to the working condition information, and continuing to execute the unfinished task action sequence from the current task node.

[0097] In one implementation, similar to the behavior tree control mode described above, when an interruption occurs and a pause operation is performed, the current task node and operational information are saved, and the system switches to a state machine for exception handling. After completing the emergency response under state machine control, various status parameters are monitored in real time. Once it is confirmed that the exception has been resolved and the safety recovery conditions are met, control is released, and the system returns to the behavior tree. The behavior tree will then trigger an activation operation, loading the saved operational information and task node, restoring the task execution context, and continuing to execute the remaining action sequence from that task node to ensure task continuity and operational efficiency.

[0098] The rationale behind the collaborative control mechanism of behavior trees and state machines lies in the fact that, taking high-priority operations such as a robot needing to immediately return to recharge when its battery is low as an example, implementing this through behavior trees would require embedding charging logic in any task subtree, making the operation extremely cumbersome. State machines, however, can easily achieve this by simply establishing a unified trigger path from all other actions to the charging action, fully demonstrating their simplicity and efficiency in high-priority state management. Through the collaborative architecture of state machines and behavior trees, state machines can quickly respond to high-priority anomalies such as low battery and obstacles, with low response latency; behavior trees precisely control complex action sequences such as part recognition and grasping, improving the operational efficiency of traditional single-planning methods. Through architectural innovation, constraint definition, state optimization, and algorithm optimization, the operational efficiency, accuracy, safety, and adaptability of humanoid robots in intelligent manufacturing scenarios for engineering machinery have been comprehensively improved, providing core technical support for achieving full automation and intelligence in the engineering machinery manufacturing process.

[0099] Therefore, the humanoid robot task planning method provided in this application includes the following steps: matching the acquired task instructions with a pre-constructed behavior tree; controlling the humanoid robot to execute the task action sequence of the task instructions according to the behavior tree; when the preset abnormal conditions are met according to the real-time acquired working condition information, pausing the execution of the behavior tree and starting the state machine corresponding to the abnormal conditions; controlling the humanoid robot based on the state machine; when the abnormal state is resolved and the recovery conditions are met, exiting the state machine and resuming the execution of the behavior tree until all action sequences of the task instructions are completed. Therefore, this application can pre-construct a behavior tree, adopting a top-down control structure composed of a series of nodes such as selection nodes, sequence nodes, condition nodes, and action nodes. In daily work, the behavior tree is used to complete the control of the humanoid robot for daily "recognition-grasping-assembly," thereby achieving refined control of complex behaviors such as parts classification, grasping, and assembly, ensuring the continuity and reliability of the work process. Simultaneously, when an anomaly occurs, the behavior tree can precisely pause at the current task node and switch to state machine control mode to handle unexpected situations. After the anomaly is resolved, it resumes execution from the original task node based on the saved context information, achieving an efficient closed loop of "interruption-processing-recovery," effectively improving system fault tolerance and operational continuity. The collaborative mechanism of the two exhibits a clear complementarity: the state machine, with its convenient management of high-priority operations, clearly distinguishes the robot's abnormal handling state, ensuring that unexpected situations can be responded to in real time, thus compensating for the inherent defects of behavior trees in anomaly response, which require embedding anomaly handling logic in each subtree, easily leading to system redundancy and response delays; while the behavior tree, with its hierarchical and modular structural advantages and strong expressive ability of complex behaviors, focuses on the logical definition of process-oriented operations such as part grasping and assembly, ensuring clear logic and scalability of routine task execution. The combination of the two not only strengthens the system's adaptability to complex working conditions but also enhances the humanoid robot's autonomous decision-making and continuous operation capabilities in dynamic environments, effectively reducing the risk of task failure due to abnormal interruptions, and providing a highly reliable and flexible technical solution for intelligent robot control in industrial automation scenarios.

[0100] Figure 5 An internal structural diagram of a humanoid robot in one embodiment is shown. This humanoid robot can specifically be a terminal or a server. Figure 5 As shown, the device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 5 The processor 310 shown in the diagram does not indicate that there is only one processor 310, but only indicates the positional relationship of the processor 310 relative to other devices. In practical applications, there can be one or more processors 310; similarly, Figure 5The memory 311 illustrated herein has the same meaning, that is, it is only used to indicate the positional relationship of memory 311 relative to other devices. In practical applications, there can be one or more memories 311. When the processor 310 runs the computer program, the method applied to the above-mentioned device is implemented.

[0101] The device may also include at least one network interface 312. The various components of the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to implement communication between these components. In addition to a data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general designated all buses as Bus System 313.

[0102] The memory 311 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 311 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0103] The memory 311 in this embodiment of the invention is used to store various types of data to support the operation of the device. Examples of this data include: any computer programs used to operate on the device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment of the invention can be included in the application.

[0104] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a storage medium storing a computer program. The storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the storage medium is run by a processor, it implements the above method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A task planning method for a humanoid robot, characterized in that, Includes the following steps: Match the pre-built behavior tree according to the obtained task instructions; The humanoid robot is controlled to execute the task action sequence of the task instructions based on the behavior tree; When the preset abnormal conditions are met based on the real-time acquired working condition information, the execution of the behavior tree is paused, and the state machine corresponding to the abnormal condition is started; the humanoid robot is controlled based on the state machine. When the abnormal state is resolved and the recovery conditions are met, exit the state machine and resume the execution of the behavior tree until all action sequences of the task instruction are completed; The behavior tree is obtained through training, and the training of the behavior tree includes: Obtain a training task set, which includes at least one training task; determine a root node based on the training task; split the training task into at least one subtask; generate a corresponding node for each subtask; construct an initial behavior tree structure based on the root node and the nodes; train the initial behavior tree structure; and determine the initial behavior tree structure that satisfies the training completion requirement as the behavior tree. The generation of a corresponding node for each of the subtasks includes: The process involves: determining the main arm and slave arm of the humanoid robot; determining position constraints based on the main arm's master position vector, end-effector rotation matrix, and feature point coordinates, and the slave arm's slave position vector; determining orientation constraints based on the end-effector normals, sliding vectors, and approach vectors of the main and slave arms, and a preset constant matrix; marking the position and orientation constraints as constraint conditions; verifying the generated nodes based on the constraint conditions; saving nodes that satisfy the constraint conditions; and correcting nodes that do not satisfy the constraint conditions until they do. The task action sequence for controlling the humanoid robot to execute the task instructions based on the behavior tree includes: Each node in the behavior tree is configured with a state, including one of the following: idle, running, successful, failed, and interrupted. All nodes are initialized to the idle state. When a task instruction is received, the root node, which is in the idle state, is adapted; the root node is the initial node of the behavior tree. According to the execution order of the nodes in the behavior tree, each node is activated sequentially, switching its state from idle to running, and its execution feedback is monitored in real time. If the task corresponding to a node is successfully completed, the node's state is updated to successful, and the next node is activated according to the execution order of the behavior tree. If the task corresponding to a node fails, the node's state is updated to failed, execution is paused, and a warning is output. If, during the execution of a node, a preset abnormal condition is met based on real-time acquired operating information, the node's state is updated to interrupted. The location of the interrupted node and the operating information are saved. If all nodes in the behavior tree are in the successful state, the task instruction is considered complete. If any node is in a failed or interrupted state, the task execution is considered abnormal, an abnormal alarm is output, the system resources occupied by the behavior tree are released, and the state of all nodes is reset to the idle state.

2. The humanoid robot task planning method as described in claim 1, characterized in that, Training the initial behavior tree structure includes: The structural information of the initial behavior tree is extracted, and the structural information is processed by a genetic algorithm to obtain the individual corresponding to each initial behavior tree; the population is determined by the individuals, and each population includes at least one of the individuals; A fitness function is designed based on the completion information of the initial behavior tree, wherein the completion information includes task completion time, job accuracy index, and safety index; During each iteration, crossover and mutation operations are performed between individuals in the population to obtain new individuals; the fitness value of each individual is evaluated according to the fitness function, and a selection operation is performed based on the fitness value to update the individuals in the population; after a preset number of iterations, migration operations are performed between the populations; the population after the migration operation is then subjected to the crossover and mutation operations and the selection operation again. After each iteration, it is determined whether the individual and the population meet the preset convergence condition; if they do, the initial behavior tree corresponding to the individual is determined as the trained behavior tree; if they do not meet the condition, the next iteration continues.

3. The humanoid robot task planning method as described in claim 1, characterized in that, The task action sequence for controlling the humanoid robot to execute the task instructions based on the behavior tree further includes: When a switching instruction is received, the execution of the current node is paused; the working status information of the humanoid robot under the current node is saved. Match the corresponding target behavior tree according to the switching instruction; Configure the root node of the target behavior tree to run, and load the saved operating condition information as context input to drive the execution of the nodes of the target behavior tree.

4. The humanoid robot task planning method as described in claim 1, characterized in that, The preset abnormal conditions include: Based on the battery information in the operating condition information, it is determined that the current battery is in an abnormal state; and / or, Based on the environmental information in the aforementioned working condition information, it is determined that there are obstacles in the working environment; and / or... Based on the collision information in the operating condition information, it is determined that the humanoid robot body or the end effector has collided; and / or Based on the communication status in the working condition information, it is determined that the humanoid robot's communication with the host computer or sensors is interrupted.

5. The humanoid robot task planning method as described in claim 1, characterized in that, The suspension of the execution of the behavior tree includes: Determine the current task node at which the behavior tree pauses, where the current task node is a child node in the behavior tree; save the working condition information when the abnormal condition is met; The step of exiting the state machine and resuming the execution of the behavior tree includes: The humanoid robot's state is restored based on the working condition information, and the unfinished task action sequence is continued from the current task node.

6. A humanoid robot, characterized in that, Including processor and memory; The processor is used to execute the computer program stored in the memory to implement the humanoid robot task planning method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the humanoid robot task planning method as described in any one of claims 1 to 5.

Images