A robotic arm execution method for dynamic task tree traversal

Abstract

The present invention discloses a method for executing a dynamic task tree traversal by a mechanical arm, comprising the following steps: S1: receiving nodes of a task sequence of a tree topology structure after parsing; S2: traversing the task nodes in an iterative manner, determining the task type, and dynamically adjusting the task type priority of the node using a machine learning model; S3: executing the mechanical arm action defined by the node based on the adjusted priority, and feeding back the execution result in real time; S4: monitoring the execution state of the node, and triggering the exception handling mechanism for adjustment when a deviation or anomaly is detected; S5: analyzing the operator's behavior pattern through intention recognition technology, predicting the next operation intention, and dynamically adjusting the subsequent nodes of the task tree; S6: after the node is executed, feeding back the execution result to the machine learning model to optimize the decision, until all task nodes are executed. The present invention improves the efficiency of task execution and the reliability of the mechanical arm in a complex and changeable task environment through dynamic adjustment and real-time feedback.

Description

Technical Field

[0001] The present invention relates to the technical field of automated robotic arms, and in particular to a robotic arm execution method for dynamic task tree traversal. Background Art

[0002] With the development of industrial automation and intelligent manufacturing, robotic arms are increasingly used in various manufacturing and logistics distribution scenarios. However, existing robotic arm task execution methods often find it difficult to achieve efficient and flexible task scheduling and exception handling when faced with complex and changing task environments. Therefore, it is particularly important to develop a robotic arm execution method that can dynamically adjust task priorities and optimize execution paths in real time.

[0003] Traditional robotic arm task execution methods usually use fixed task sequences and preset execution paths. The advantage of this method is that it is simple, easy to implement and maintain. However, its disadvantages are also very obvious: lack of flexibility, inability to dynamically adjust task priorities according to changes in the task environment, and inability to promptly handle abnormal situations that occur during task execution. This leads to great limitations on the execution efficiency and reliability of the robotic arm when facing a complex and changing production environment. Although the existing technical solutions have introduced machine learning technology to a certain extent, there are still many shortcomings. They can only optimize the task sequence statically and cannot adjust the task priority in real time during task execution.

[0004] Chinese patent CN114536333A discloses a robot task planning system and application method based on behavior tree, including: trajectory segmentation module, target posture detection module, behavior tree design module, dynamic motion primitive generalization module, host computer system and KinovaJaco robot. On the one hand, the KinovaJaco robot is controlled by teleoperation to perform multiple grasping task demonstrations. The host computer system obtains action primitives from the collected demonstration data through the trajectory segmentation module and constructs an action primitive library; on the other hand, according to the grasping task execution logic, the grasping task behavior tree is created through the behavior tree design module; after the Kinect sensor captures the object depth image and estimates the object position and posture through the target posture detection module, it is combined with the corresponding action primitive in the action primitive library, and then generalized by the dynamic motion primitive generalization module, and the generalized action primitive is used as the behavior tree action node. The behavior tree transmits the execution instruction to the host computer system to control the KinovaJaco robot to perform the grasping task. However, the above patent mainly relies on pre-defined behavior trees and action primitive libraries, and lacks a mechanism for dynamically adjusting task priorities. Although intent recognition technology is mentioned, it mainly focuses on the generalization of action primitives.

[0005] Therefore, the technical solution of the present application solves the deficiencies in the prior art by introducing a dynamic task tree traversal method and real-time intent recognition technology. The machine learning model is used to dynamically adjust the task priority and optimize the task execution efficiency. The operator's behavioral intention is predicted in real time through the intent recognition technology to dynamically adjust the subsequent nodes of the task tree. Compared with the prior art, this solution not only improves the flexibility and adaptability of task execution, but also significantly improves the execution efficiency and reliability of the robot arm in complex and changeable task environments. Summary of the invention

[0006] A method for executing a dynamic task tree traversal by a robot arm comprises the following steps:

[0007] S1: receiving nodes of a tree topology task sequence processed by parsing logic, wherein the nodes of the task sequence include at least one parent node and multiple child nodes; the parent node is the overall goal of the task sequence, and the child nodes are specific steps to achieve the overall goal;

[0008] S2: Iterate through the nodes in the task sequence, determine the task type of the node, and use the machine learning model to dynamically adjust the task type priority of the node to optimize the overall task execution efficiency;

[0009] S3: Based on the adjusted task type priority of the node, the robot arm action defined by the node is executed, and the execution result is fed back in real time using the machine learning model;

[0010] S4: Monitor the execution status of the node in real time, and use the machine learning model to determine whether the robot arm executes correctly according to the predetermined order and dependency relationship. If an execution deviation or anomaly is detected, the exception handling mechanism is intelligently triggered and adjusted according to the exception recovery mechanism;

[0011] S5: During the node execution process, the intention recognition technology is used to analyze the operator's behavior pattern changes, predict the next operation intention, and dynamically adjust the subsequent nodes of the task tree according to the prediction results to optimize the execution path;

[0012] S6: After the node is executed, the next node to be executed is adjusted based on the prediction of the intent recognition technology. For multiple candidate nodes in the task tree, the optimal node is selected for continued execution based on the priority of intent recognition, and the execution results are fed back to the machine learning model to continuously optimize the decision until all nodes of the task sequence are executed.

[0013] Preferably, the sub-nodes in S1 define one or more state modules of the robot arm;

[0014] The state module includes but is not limited to TCP position and posture, kinematic parameters and peripheral IO operations.

[0015] Preferably, the iteration method in S2 is to visit layer by layer from top to bottom, starting from the root node, traversing each task node thereunder in turn, and then deeply visiting each state module to determine the task type of each task node.

[0016] Preferably, the construction of the machine learning model in S2 includes the following steps:

[0017] S2.1: Collect data related to the execution of robot arm tasks, including the time of task execution, resource usage and final execution results. Data under abnormal conditions should also be included in the collection scope to provide a basis for subsequent abnormality detection;

[0018] S2.2: Use the long short-term memory network algorithm to build a machine learning model, design the input and output of the machine learning model, ensure that the input can fully reflect the characteristics of the task, and the output is used to guide the adjustment of task priorities;

[0019] S2.3: Use the collected data sets to train the machine learning model, evaluate the performance of the machine learning model through cross-validation, adjust the parameters of the machine learning model to optimize the prediction accuracy and response speed, and ensure that the machine learning model can quickly predict the time and resource consumption of task execution;

[0020] S2.4: Integrate the trained machine learning model into the robotic arm control process to achieve real-time dynamic adjustment of task type priority.

[0021] Preferably, the machine learning model implements real-time dynamic adjustment of task type priority by rating the execution time and resource consumption of tasks according to the execution time and resource usage of the tasks;

[0022]

[0023] Where P(t) is the priority score of task t, T(t) is the time of task t, R(t) is the resource consumption of task t, w1 and w2 are weight coefficients;

[0024] The higher the priority score is, the higher the priority of the task is.

[0025] Preferably, the exception handling mechanism includes an exception detection module, an exception recording module and an exception triggering module;

[0026] The abnormality detection module determines the execution deviation error by monitoring the robot arm's failure to reach the specified position, the posture deviation exceeding the threshold, the speed abnormality, the acceleration abnormality and the peripheral device operation failure;

[0027] The exception recording module records the node where the exception occurs, the exception type, the exception time, and saves the status of the task node;

[0028] The exception trigger module receives the execution deviation error signal of the exception detection module, transmits the information of the exception recording module to the exception recovery mechanism, and adjusts the execution state of the task node according to the exception recovery mechanism.

[0029] Preferably, the abnormal recovery mechanism includes retrying, jumping to a standby node, and falling back to a safe location;

[0030] The retry is when the system re-executes the current task node after detecting an abnormality;

[0031] The jump to the backup node is that when it is detected that the current task node cannot continue to execute, the system jumps to the predefined backup task node;

[0032] The system moves the robot arm to a preset safe position when it detects that there is a greater risk in the current task node.

[0033] Preferably, the intention recognition technology collects operator behavior change data, uses a neural network algorithm to analyze the collected behavior data changes, establishes an intention recognition model, predicts the operator's next operation intention through the intention recognition model, dynamically adjusts subsequent nodes of the task tree, and optimizes the task execution path.

[0034] Preferably, the intention recognition model comprises the following steps:

[0035] S5.1: Collect operator behavior change data, record the operator's action sequence, operation time interval, and operation frequency;

[0036] S5.2: Extract feature data, extract features from the collected behavior change data, including but not limited to the key action points and action sequence of the action sequence, the standard deviation of the operation time interval, and the number of operation frequencies;

[0037] S5.3: Establish an intention recognition model. Use a convolutional neural network to analyze the extracted feature data and establish an intention recognition model. The input of the intention recognition model is the operator's behavior characteristics, and the output is the operator's next operation intention.

[0038] S5.4: Real-time prediction and adjustment of the task tree. During the task execution, the current operation status is collected in real time and input into the intention recognition model to predict the operator's next operation intention, dynamically adjust the subsequent nodes of the task tree, and optimize the execution path.

[0039] Preferably, S6 comprises the following steps:

[0040] S6.1: Conduct a preliminary evaluation of multiple candidate task nodes from the task tree, including the complexity of the task, the time required, and the resource consumption;

[0041] S6.2: Calculate the priority of the candidate nodes, and calculate the priority of each candidate node according to the priority rule; the priority rule includes the urgency of the task, the success rate of the task, the execution time of the task and the resource consumption;

[0042] S6.3: Select the node with the highest priority from the candidate nodes as the next execution node and continue to execute the task. After the task is completed, the execution result is fed back to the machine learning model. The machine learning model continuously optimizes the intent recognition model and priority selection strategy based on the new data to improve the efficiency and accuracy of task execution.

[0043] Compared with the prior art, the technical solution of this application has the following technical effects:

[0044] The present invention solves the problem of low task execution efficiency by using the long short-term memory network algorithm to construct a machine learning model and dynamically adjusts task priorities, significantly improving the task execution efficiency. The machine learning model is used to dynamically adjust task priorities according to the task execution time and resource consumption, thereby optimizing the task execution order and reducing invalid waiting and resource waste.

[0045] The present invention analyzes the operator's behavior data by using a neural network algorithm, establishes an intention recognition model, and predicts the operator's next operation intention. This solves the problem of rigid task execution path, realizes flexible optimization of task execution path, predicts the operator's behavior intention in real time, dynamically adjusts subsequent nodes of the task tree, and improves the flexibility and adaptability of task execution.

[0046] The present invention solves the exception handling problem in the task execution process by designing an exception detection module, an exception recording module and an exception triggering module, and intelligently triggers the exception handling mechanism, thereby improving the high reliability and robustness of task execution, and can intelligently detect and handle execution deviations or exceptions, and make adjustments according to the exception recovery mechanism to ensure the stability and reliability of task execution.

[0047] The present invention solves the optimization problem in the task execution process by using machine learning models to provide real-time feedback on execution results during task execution, and continuously optimizes the intent recognition model and priority selection strategy according to new data, thereby achieving continuous optimization of task execution. Through the real-time feedback mechanism, the model is continuously optimized according to new data, thereby improving the adaptability and intelligence level of task execution.

[0048] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings as follows.

[0049] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0051] Figure 1 Execute the flow chart for the robot arm;

[0052] Figure 2 Flowchart for machine learning model;

[0053] Figure 3 It is an iterative flow chart;

[0054] Figure 4 Flowchart of the intent recognition model;

[0055] Figure 5 It is the candidate node flow chart. Specific embodiments

[0056] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all of the embodiments. In the following description, specific details such as specific configuration and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures is omitted in the embodiment.

[0057] It should be understood that the references to "one embodiment" or "this embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "one embodiment" or "this embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0058] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0059] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that there can be two relationships. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0060] The term "at least one" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0061] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusions.

[0062] Example 1

[0063] This embodiment mainly describes a method for executing a dynamic task tree traversal by a robot arm;

[0064] Further, if Figure 1 As shown, a method for executing a dynamic task tree traversal by a robot arm includes the following steps:

[0065] S1: receiving nodes of a tree topology task sequence processed by parsing logic, wherein the nodes of the task sequence include at least one parent node and multiple child nodes; the parent node is the overall goal of the task sequence, and the child nodes are specific steps to achieve the overall goal;

[0066] S2: Iterate through the nodes in the task sequence, determine the task type of the node, and use the machine learning model to dynamically adjust the task type priority of the node to optimize the overall task execution efficiency;

[0067] S3: Based on the adjusted task type priority of the node, the robot arm action defined by the node is executed, and the execution result is fed back in real time using the machine learning model;

[0068] S4: Monitor the execution status of the node in real time, and use the machine learning model to determine whether the robot arm executes correctly according to the predetermined order and dependency relationship. If an execution deviation or anomaly is detected, the exception handling mechanism is intelligently triggered and adjusted according to the exception recovery mechanism;

[0069] S5: During the node execution process, the intention recognition technology is used to analyze the changes in the operator's behavior pattern, predict the next operation intention, and dynamically adjust the subsequent nodes of the task tree according to the prediction results to optimize the execution path;

[0070] S6: After the node is executed, the next node to be executed is adjusted based on the prediction of the intent recognition technology. For multiple candidate nodes in the task tree, the optimal node is selected for continued execution based on the priority of intent recognition, and the execution results are fed back to the machine learning model to continuously optimize the decision until all nodes of the task sequence are executed.

[0071] By decomposing the task sequence into nodes of a tree topology structure, each node contains a parent node and child nodes, the parent node represents the overall goal, and the child node represents the specific steps, hierarchical management of tasks is achieved; the machine learning model is used to dynamically adjust the task type priority of the node to optimize the overall task execution efficiency, and the accuracy and timeliness of task execution are ensured by real-time feedback of execution results, thereby improving the flexibility and adaptability of task execution.

[0072] Further, the sub-nodes in S1 define one or more state modules of the robot arm;

[0073] The state module includes but is not limited to TCP position and posture, kinematic parameters and peripheral IO operations.

[0074] The fine control of the above-mentioned state modules enables the robotic arm to accurately execute each task step, improving the accuracy and reliability of task execution, while enhancing the versatility and adaptability of the system, enabling it to perform a variety of tasks.

[0075] Furthermore, the iteration method in S2 is to visit layer by layer from top to bottom, starting from the root node, traversing each task node thereunder in turn, and then deeply visiting each state module to determine the task type of each task node.

[0076] Adopt a top-down layer-by-layer access method, starting from the root node, and traversing each task node in turn to ensure the systematicness and integrity of the task; deeply access each status module to determine the task type of each task node, ensure the refinement and concretization of the task, improve the accuracy of task execution, and avoid missing any task node.

[0077] Further, if Figure 2 As shown, the construction of the machine learning model in S2 includes the following steps:

[0078] S2.1: Collect data related to the execution of robot arm tasks, including the time of task execution, resource usage and final execution results. Data under abnormal conditions should also be included in the collection scope to provide a basis for subsequent abnormality detection;

[0079] S2.2: Use the long short-term memory network algorithm to build a machine learning model, design the input and output of the machine learning model, ensure that the input can fully reflect the characteristics of the task, and the output is used to guide the adjustment of task priorities;

[0080] S2.3: Use the collected data sets to train the machine learning model, evaluate the performance of the machine learning model through cross-validation, adjust the parameters of the machine learning model to optimize the prediction accuracy and response speed, and ensure that the machine learning model can quickly predict the time and resource consumption of task execution;

[0081] S2.4: Integrate the trained machine learning model into the robotic arm control process to achieve real-time dynamic adjustment of task type priority.

[0082] The machine learning model dynamically adjusts the task type priority of the node to optimize the overall task execution efficiency. The machine learning model can intelligently adjust the task execution order based on historical data and real-time data to improve the efficiency and success rate of task execution; in addition, the model can predict the execution time and resource consumption of the next task to further optimize the task execution path.

[0083] Furthermore, the machine learning model implements real-time dynamic adjustment of the priority of task types by rating the duration of task execution and resource consumption according to the execution time and resource usage of the tasks;

[0084]

[0085] Where P(t) is the priority score of task t, T(t) is the time of task t, R(t) is the resource consumption of task t, w1 and w2 are weight coefficients;

[0086] The higher the priority score is, the higher the priority of the task is.

[0087] The machine learning model predicts the execution time and resource consumption of the next task based on the time consumption and resource usage of the current task, and rates the tasks according to their duration and resource consumption; tasks with long execution time and high resource consumption are downgraded, while tasks with short execution time and low resource consumption are promoted to priority, so as to reasonably allocate resources and improve task execution efficiency and resource utilization.

[0088] Further, the exception handling mechanism includes an exception detection module, an exception recording module and an exception triggering module;

[0089] The abnormality detection module determines the execution deviation error by monitoring the robot arm's failure to reach the specified position, the posture deviation exceeding the threshold, the speed abnormality, the acceleration abnormality and the peripheral device operation failure;

[0090] The exception recording module records the node where the exception occurs, the exception type, the exception time, and saves the status of the task node;

[0091] The exception trigger module receives the execution deviation error signal of the exception detection module, transmits the information of the exception recording module to the exception recovery mechanism, and adjusts the execution state of the task node according to the exception recovery mechanism.

[0092] The exception detection module monitors the execution status of the robot arm in real time to ensure that the tasks are executed correctly according to the predetermined order and dependencies; the exception triggering module intelligently triggers the exception handling mechanism, adjusts the execution status of the task node according to the exception recovery mechanism, and promptly discovers and handles execution deviations or exceptions, thereby improving the stability and security of task execution.

[0093] Further, the abnormal recovery mechanism includes retrying, jumping to a backup node, and falling back to a safe location;

[0094] The retry is when the system re-executes the current task node after detecting an abnormality;

[0095] The jump to the backup node is that when it is detected that the current task node cannot continue to execute, the system jumps to the predefined backup task node;

[0096] The system moves the robot arm to a preset safe position when it detects that there is a greater risk in the current task node.

[0097] The abnormal recovery mechanism includes multiple strategies such as retry, jumping to backup nodes and rolling back to a safe position, ensuring that the task can be quickly recovered when encountering an abnormality and improving the robustness of task execution. When it is detected that there is a greater risk in the current task node, the system will move the robotic arm to a preset safe position to ensure the safety of the operation and prevent accidents.

[0098] Furthermore, the intention recognition technology collects operator behavior change data, uses a neural network algorithm to analyze the changes in the collected behavior data, establishes an intention recognition model, predicts the operator's next operation intention through the intention recognition model, dynamically adjusts subsequent nodes of the task tree, and optimizes the task execution path.

[0099] By collecting operator behavior change data, using neural network algorithms to analyze the collected data, establishing an intention recognition model, and predicting the operator's next possible operation intention. According to the prediction results, the subsequent nodes of the task tree are dynamically adjusted to optimize the task execution path, prepare the corresponding task nodes in advance, reduce the time of task switching, and improve the efficiency and accuracy of task execution.

[0100] Furthermore, the intention recognition model includes the following steps:

[0101] S5.1: Collect operator behavior change data, record the operator's action sequence, operation time interval, and operation frequency;

[0102] S5.2: Extract feature data, extract features from the collected behavior change data, including but not limited to the key action points and action sequence of the action sequence, the standard deviation of the operation time interval, and the number of operation frequencies;

[0103] S5.3: Establish an intention recognition model. Use a convolutional neural network to analyze the extracted feature data and establish an intention recognition model. The input of the intention recognition model is the operator's behavior characteristics, and the output is the operator's next operation intention.

[0104] S5.4: Real-time prediction and adjustment of the task tree. During the task execution, the current operation status is collected in real time and input into the intention recognition model to predict the operator's next operation intention, dynamically adjust the subsequent nodes of the task tree, and optimize the execution path.

[0105] Key features are extracted from the collected behavior change data, including the key action points of the action sequence, the order of actions, the standard deviation of the operation time interval, and the number of operation frequencies. The extracted feature data are analyzed using a convolutional neural network to establish an intent recognition model. Through structures such as convolutional layers, activation functions, pooling layers, and fully connected layers, the model can effectively extract and analyze features, improve the accuracy of intent prediction, and optimize the task execution path.

[0106] Further, if Figure 5 As shown, the S6 comprises the following steps:

[0107] S6.1: Conduct a preliminary evaluation of multiple candidate task nodes from the task tree, including the complexity of the task, the time required, and the resource consumption;

[0108] S6.2: Calculate the priority of the candidate nodes, and calculate the priority of each candidate node according to the priority rule; the priority rule includes the urgency of the task, the success rate of the task, the execution time of the task and the resource consumption;

[0109] S6.3: Select the node with the highest priority from the candidate nodes as the next execution node and continue to execute the task. After the task is completed, the execution result is fed back to the machine learning model. The machine learning model continuously optimizes the intent recognition model and priority selection strategy based on the new data to improve the efficiency and accuracy of task execution.

[0110] This embodiment of the robot arm execution method through dynamic task tree traversal of the present application has significant advantages in task decomposition, dynamic adjustment, real-time feedback, exception handling, intention recognition, etc., and can significantly improve the efficiency and accuracy of task execution.

[0111] Example 2

[0112] This embodiment is based on Embodiment 1 and mainly describes how to iterate through the nodes in the task sequence in S2:

[0113] like Figure 3 As shown, in step S2, the nodes in the task sequence are iterated and the task type of the node is determined. The following steps can be followed:

[0114] S2.1.1. Initialize the task sequence node, starting from the root node of the task sequence, which is the entry point of the entire task sequence. There are multiple child nodes below, which are the basic components of the task sequence;

[0115] S2.1.2, traverse the task nodes, for each task node in the array or list, these task nodes appear in the form of child nodes of the keyword, traverse these task nodes and process them one by one;

[0116] S2.1.3, access the state module, for each node in the array or list under each task node, access the TCP position and posture, speed, acceleration, path constraint, and delay time attributes contained therein;

[0117] S2.1.4. Determine the task type. During the process of accessing each node, determine the task type of the node, including but not limited to moving to a certain location, performing a certain action, and interacting with other devices;

[0118] S2.1.5. Recursively process nested tasks. If a node is found to point to another task configuration file during the traversal process, it is necessary to recursively enter the file and repeat the above traversal and type determination process.

[0119] S2.1.6. End condition: When all nodes have been visited and their task types have been determined, the iterative traversal ends.

[0120] Through the above steps, it can be ensured that each task node is properly identified and processed, so that the robot can execute each step in the task sequence in the expected way. This iterative traversal method ensures that even in the face of complex task sequences, each task node can be executed in an orderly manner.

[0121] Example 3

[0122] This embodiment introduces in detail the application of the intention recognition technology in the robot arm execution method of dynamic task tree traversal based on Embodiment 1;

[0123] like Figure 4 As shown in the figure, the intention recognition technology collects the operator's behavior change data, uses a neural network algorithm to analyze the collected data, and establishes an intention recognition model. The intention recognition model can predict the operator's next operation intention, thereby dynamically adjusting the subsequent nodes of the task tree and optimizing the task execution path.

[0124] Step S5.1: Collecting operator behavior change data

[0125] Action sequence: record the operator's action sequence, grab, place, rotate;

[0126] Operation time interval: record the time interval between each action of the operator;

[0127] Operation frequency: records the number of times an operator performs a certain action within a certain period of time;

[0128] Step S5.2: Extract feature data

[0129] Key action points: extract key action points in the operator's action sequence, such as grabbing, placing, and rotating;

[0130] Action duration: record the duration of each action and calculate the mean and standard deviation;

[0131] Action frequency: Calculate the frequency of each action, that is, the number of times the operator performs an action within a certain period of time;

[0132] Action sequence: record the operator's action sequence, such as grab first and then place, rotate first and then adjust, etc.

[0133] Operation time interval characteristics:

[0134] Average time interval: calculate the average time interval between each action of the operator;

[0135] Time interval standard deviation: Calculate the standard deviation of the operation time interval to reflect the stability of the operator's operation time;

[0136] Shortest and longest time interval: record the shortest and longest time interval between each action of the operator;

[0137] Operating frequency characteristics:

[0138] Total number of operations: records the total number of manipulation actions performed by the operator during the task execution;

[0139] Frequency of each action: Calculate the frequency of each action (grab, place, rotate) separately;

[0140] Changes in operating frequency: Calculate the change trend of operating frequency over time;

[0141] Standardization: In order to eliminate the influence of different feature dimensions, the features are standardized so that all features can be compared on the same scale.

[0142] Step S5.3: Build an intent recognition model

[0143] Convolutional Neural Networks:

[0144] Input layer: The input feature vector contains the operator's behavioral characteristics, action sequence, operation time interval, operation frequency, etc.

[0145] Convolutional layer: extracts local features of input features through convolution operation;

[0146] Activation function: Use the ReLU activation function to introduce nonlinearity and enhance the expressiveness of the model;

[0147] Pooling layer: Reduce the dimension of features and reduce the amount of calculation through the maximum pooling operation;

[0148] Fully connected layer: maps the pooled feature vector to the final output;

[0149] Output layer: Use the softmax function to convert the output of the fully connected layer into a probability distribution, indicating the operator's next operation intention;

[0150] Model training:

[0151] Loss function: Use the cross entropy loss function to measure the difference between the model's predicted value and the true value and optimize the model's parameters;

[0152] Gradient descent: Update model parameters through gradient descent method to minimize the loss function and improve the prediction accuracy of the model;

[0153] Step S5.4: Real-time prediction and adjustment of task tree

[0154] Real-time data collection:

[0155] During the task execution, the current operation status is collected in real time, including the operator's behavioral characteristics and changes in the task environment;

[0156] Intent Prediction:

[0157] The operation status collected in real time is input into the intention recognition model to predict the operator's next operation intention.

[0158] Dynamically adjust the task tree:

[0159] According to the predicted operation intention, the subsequent nodes of the task tree are dynamically adjusted to optimize the task execution path; it is predicted that the operator will make fine adjustments to the position, and the system can prepare the corresponding task nodes in advance to reduce the time of task switching;

[0160] The task execution results are fed back to the intent recognition model, and the model continuously optimizes intent recognition and priority selection strategies based on new data to improve the efficiency and accuracy of task execution.

[0161] Through the above-mentioned embodiment steps, an intention recognition model based on a convolutional neural network is established, which can predict the operator's next operation intention in real time, and dynamically adjust the subsequent nodes of the task tree according to the prediction results to optimize the task execution path. This technology has important application value in the robot arm execution method of dynamic task tree traversal, and can significantly improve the efficiency and accuracy of task execution.

[0162] Example 4

[0163] Based on Example 1, this implementation introduces in detail a specific implementation method of a robot arm execution method for dynamic task tree traversal:

[0164] Implementation background:

[0165] A robotic arm is responsible for grabbing electronic components from a tray and installing them on a circuit board. In order to ensure efficient and accurate execution of tasks, the technical solution of this application is used to control the operation of the robotic arm;

[0166] 1. Nodes of the tree topology task sequence that receive parsing logic processing;

[0167] Define the task tree:

[0168] Parent node: completes the component assembly of the circuit board.

[0169] Subnodes: move to the tray position, grab electronic components, move to the circuit board position, install electronic components, and return to the initial position;

[0170] Subnode status module: TCP position and posture, kinematic parameters, peripheral IO operations;

[0171] 2. Traverse the nodes in the task sequence in an iterative manner;

[0172] Iteration traversal:

[0173] Starting from the root node "Complete the assembly of components on the circuit board", visit each child node layer by layer;

[0174] Determine the task type of each sub-node. "Move to the tray position" is a motion task, and "Grab the electronic component" is an operation task.

[0175] Dynamically adjust priorities through machine learning models;

[0176] Data Collection:

[0177] Task execution time: The total time from the start to the end of each task is recorded. The average time of 10 "moving to the tray position" is 3.2 seconds, and the standard deviation is 0.5 seconds.

[0178] Resource usage: The energy consumption and CPU usage of the robot arm when performing tasks are recorded. The average energy consumption of 10 "grasping electronic components" is 1.5 kWh and the CPU usage is 70%.

[0179] Final execution result: record whether the task was completed successfully, including whether there were any errors or exceptions, and record that 1 out of 10 "moves to the circuit board position" failed to successfully reach the target position.

[0180] Abnormal conditions: record the specific conditions of the robot arm failing to reach the specified position, posture deviation exceeding the threshold, abnormal speed, abnormal acceleration and failure of peripheral equipment operation, and record that the posture deviation exceeded 5 degrees 2 times out of 10 times of "installing electronic components".

[0181] Model construction:

[0182] Build machine learning models using the Long Short-Term Memory Network algorithm;

[0183] Input data includes task execution time, resource usage, final execution results, and exceptions;

[0184] The output data is used to guide the adjustment of task priorities.

[0185] Model training:

[0186] Use the collected datasets to train machine learning models;

[0187] Evaluate the performance of the model through cross-validation and adjust the model parameters to optimize prediction accuracy and response speed;

[0188] During the training process, the mean square error was used as the loss function and the parameters were updated through the back propagation algorithm. After 100 epochs of training, the mean square error was reduced from 0.15 to 0.05.

[0189] Model Ensemble:

[0190] Integrate the trained machine learning model into the robotic arm control process;

[0191] Realize real-time dynamic adjustment of task type priority and adjust the execution order of tasks according to the model's prediction results; the model predicts that the priority score of "moving to the tray position" is 85, while the priority score of "grabbing electronic components" is 70.

[0192] 3. Based on the adjusted node’s task type priority, execute the robot action defined by the node;

[0193] Task execution: Execute the robot action defined by each node according to the adjusted priority;

[0194] “Move to the tray location” is rated as high priority by the priority model, and the robot will perform this task first;

[0195] Real-time feedback: When the robot arm performs a task, it can provide real-time feedback of the execution results to the machine learning model;

[0196] Feedback data includes the actual time of task execution, resource usage, and final results;

[0197] The actual time to execute the "move to the tray position" task was 3.1 seconds, the energy consumption was 1.4 kWh, and the target position was successfully reached.

[0198] If an exception occurs during the task execution, it will be recorded and fed back to the exception handling module in a timely manner. When executing the "install electronic components" task, the posture deviation is 6 degrees, which is recorded as an exception.

[0199] 4. Real-time monitoring of the execution status of the node;

[0200] Anomaly detection: monitor the current position and posture of the robot arm to ensure that it meets expectations. Set the posture deviation threshold to 5 degrees. If it exceeds 5 degrees, it is considered abnormal;

[0201] Monitor the speed and acceleration of the robot arm to ensure that they are within the normal range, setting the maximum speed to 1 m / s and the maximum acceleration to 2 m / s2;

[0202] Monitor the operating status of peripheral devices to ensure that no operation fails. If it is detected that the peripheral IO device is not responding, it is judged as abnormal.

[0203] Exception record: record the node where the exception occurred, the exception type, the exception time, and save the status of the task node;

[0204] The node ID is "Install electronic components", the exception type is "posture deviation", the exception occurred 3.5 seconds ago, the current TCP position is (100, 150, 200), and the posture is (0, 0, 6).

[0205] Abnormal triggering: Receives the execution deviation error signal from the abnormal detection module, passes the information of the abnormal recording module to the abnormal recovery mechanism, and adjusts the execution status of the task node according to the abnormal recovery mechanism.

[0206] Abnormal recovery mechanism:

[0207] Retry: After detecting an anomaly, the system re-executes the current task node; re-executes the "install electronic components" task;

[0208] Jump to backup node: When the system detects that the current task node cannot continue to execute, it jumps to the predefined backup task node; jumps to the "Check installation location" node;

[0209] Fall back to a safe position: When the system detects that there is a greater risk in the current task node, it moves the robotic arm to a preset safe position; moves the robotic arm to a safe position with coordinates (0, 0, 0).

[0210] 5. During the node execution process, analyze the operator's behavior pattern and changes in the task environment through intention recognition technology;

[0211] Data collection: Collect the operator's behavior change data, record the operator's action sequence, operation time interval, and operation frequency;

[0212] Record the operator's hand movements (open fingers, closed fingers), head turn (left turn, right turn), and body posture (lean forward, lean back);

[0213] Specific data:

[0214] Action sequence: [open fingers, close fingers, turn left, turn right]

[0215] Operation time interval: [2.1 seconds, 1.8 seconds, 3.0 seconds, 2.5 seconds]

[0216] Operating frequency: 5 times per minute;

[0217] Feature extraction:

[0218] Features are extracted from the collected behavior change data, including but not limited to the key action points and action order of the action sequence, the standard deviation of the operation time interval, and the number of operation frequencies.

[0219] Specific features:

[0220] Key points of action: fingers opening and fingers closing;

[0221] Action sequence: [open fingers, close fingers, turn left, turn right]

[0222] Standard deviation of operation time interval: 0.5 seconds;

[0223] The number of operating frequency: 5 times per minute;

[0224] Model building:

[0225] Use convolutional neural networks to analyze the extracted feature data and establish an intent recognition model.

[0226] The input of the intention recognition model is the operator's behavioral characteristics, and the output is the operator's possible next operation intention.

[0227] Specific data:

[0228] Input features: [finger open, finger closed, turn left, turn right, 0.5 seconds, 5 times / minute]

[0229] Output intention: Grab the next electronic component;

[0230] Real-time prediction: During task execution, the current operation status is collected in real time and input into the intent recognition model;

[0231] Predict the operator's next possible operation intention, dynamically adjust the subsequent nodes of the task tree, and optimize the execution path;

[0232] Current state: fingers open, turn left;

[0233] Predicting intention: preparing to grab the next electronic component;

[0234] Adjust the task tree: increase the priority of the "Move to tray position" node to 90;

[0235] 6. After the node is executed, adjust the next node to be executed based on the prediction of the intent recognition technology;

[0236] Initial Assessment:

[0237] Conduct a preliminary evaluation of multiple candidate task nodes from the task tree, including the complexity of the task, the time required, and the resource consumption;

[0238] Specific assessment:

[0239] “Move to the tray position”: complexity 1, time required 3.2 seconds, resource consumption 1.5 kWh;

[0240] “Grab electronic components”: complexity 2, time required 4.0 seconds, resource consumption 2.0 kWh;

[0241] “Move to circuit board position”: complexity 1, time required 3.5 seconds, resource consumption 1.8 kWh;

[0242] Priority calculation:

[0243] According to the priority rule, the priority of each candidate node is calculated.

[0244] Priority rules include the urgency of the task, the success rate of the task, the execution time of the task, and the resource consumption.

[0245] Specific calculation:

[0246] "Move to tray position": priority = 0.4*1+0.3*90%+0.2*3.2+0.1*1.5=85;

[0247] "Grab electronic components": priority = 0.4*2+0.3*85%+0.2*4.0+0.1*2.0=70;

[0248] "Move to circuit board position": priority = 0.4*1+0.3*95%+0.2*3.5+0.1*1.8=88;

[0249] Node selection:

[0250] Select the node with the highest priority from the candidate nodes as the next execution node to continue executing the task.

[0251] The node with the highest priority: "Move to board position" (priority 88);

[0252] Feedback optimization:

[0253] After the task is completed, the execution results are fed back to the machine learning model.

[0254] The machine learning model continuously optimizes the intent recognition model and priority selection strategy based on new data to improve the efficiency and accuracy of task execution.

[0255] The actual time to perform the “move to circuit board position” task was 3.4 seconds, the energy consumption was 1.7 kWh, and the target position was successfully reached.

[0256] The model adjusts the priority calculation formula based on new data and optimizes the intent recognition model.

[0257] Through the above implementation steps, the adaptability and execution efficiency of the robot arm in complex environments can be effectively improved, ensuring a smoother and more efficient production process. This method can not only dynamically adjust the priority of tasks, but also handle abnormal situations in real time and make intelligent adjustments based on the operator's behavior patterns and changes in the task environment.

[0258] The above are only preferred embodiments of the present invention, which do not limit the scope of protection of the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any changes, modifications, replacements, integrations and parameter changes to these embodiments within the spirit and principles of the present invention through conventional substitutions or without departing from the principles and spirit of the present invention fall within the scope of protection of the present invention.

Claims

1. A method for executing a dynamic task tree traversal by a robot arm, characterized in that: The following steps are involved: S1: receiving nodes of a tree topology task sequence processed by parsing logic, wherein the nodes of the task sequence include at least one parent node and multiple child nodes; the parent node is the overall goal of the task sequence, and the child nodes are specific steps to achieve the overall goal; S2: Iterate through the nodes in the task sequence, determine the task type of the node, and use the machine learning model to dynamically adjust the task type priority of the node to optimize the overall task execution efficiency; S3: Based on the adjusted task type priority of the node, the robot arm action defined by the node is executed, and the execution result is fed back in real time using the machine learning model; S4: Monitor the execution status of the node in real time, and use the machine learning model to determine whether the robot arm executes correctly according to the predetermined order and dependency relationship. If an execution deviation or anomaly is detected, the exception handling mechanism is intelligently triggered and adjusted according to the exception recovery mechanism; S5: During the node execution process, the intention recognition technology is used to analyze the operator's behavior pattern changes, predict the next operation intention, and dynamically adjust the subsequent nodes of the task tree according to the prediction results to optimize the execution path; S6: After the node is executed, the next node to be executed is adjusted based on the prediction of the intent recognition technology. For multiple candidate nodes in the task tree, the optimal node is selected for continued execution based on the priority of intent recognition, and the execution results are fed back to the machine learning model to continuously optimize the decision until all nodes of the task sequence are executed.

2. A method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The subnodes in S1 define one or more state modules of the robot arm; The state module includes but is not limited to TCP position and posture, kinematic parameters and peripheral IO operations.

3. A method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The iteration method in S2 is to visit layer by layer from top to bottom, starting from the root node, traversing each task node thereunder in turn, and then deeply visiting each state module to determine the task type of each task node.

4. The method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The construction of the machine learning model in S2 includes the following steps: S2.1: Collect data related to the execution of robot arm tasks, including the time of task execution, resource usage and final execution results. Data under abnormal conditions should also be included in the collection scope to provide a basis for subsequent abnormality detection; S2.2: Use the long short-term memory network algorithm to build a machine learning model, design the input and output of the machine learning model, ensure that the input can fully reflect the characteristics of the task, and the output is used to guide the adjustment of task priorities; S2.3: Use the collected data sets to train the machine learning model, evaluate the performance of the machine learning model through cross-validation, adjust the parameters of the machine learning model to optimize the prediction accuracy and response speed, and ensure that the machine learning model can quickly predict the time and resource consumption of task execution; S2.4: Integrate the trained machine learning model into the robotic arm control process to achieve real-time dynamic adjustment of task type priority.

5. A method for executing a dynamic task tree traversal by a robot arm according to claim 4, characterized in that: The machine learning model implements real-time dynamic adjustment of task type priority by rating the execution time and resource consumption of tasks according to the execution time and resource usage of the tasks; Where P(t) is the priority score of task t, T(t) is the time of task t, R(t) is the resource consumption of task t, w1 and w2 are weight coefficients; The higher the priority score is, the higher the priority of the task is.

6. A method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The exception handling mechanism includes an exception detection module, an exception recording module and an exception triggering module; The abnormality detection module determines the execution deviation error by monitoring the robot arm's failure to reach the specified position, the posture deviation exceeding the threshold, the speed abnormality, the acceleration abnormality and the peripheral device operation failure; The exception recording module records the node where the exception occurs, the exception type, the exception time, and saves the status of the task node; The exception trigger module receives the execution deviation error signal of the exception detection module, transmits the information of the exception recording module to the exception recovery mechanism, and adjusts the execution state of the task node according to the exception recovery mechanism.

7. A method for executing a dynamic task tree traversal by a robot arm according to claim 6, characterized in that: The abnormal recovery mechanism includes retrying, jumping to a standby node, and falling back to a safe location; The retry is when the system re-executes the current task node after detecting an abnormality; The jump to the backup node is that when it is detected that the current task node cannot continue to execute, the system jumps to the predefined backup task node; The system moves the robot arm to a preset safe position when it detects that there is a greater risk in the current task node.

8. The method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The intention recognition technology collects operator behavior change data, uses a neural network algorithm to analyze the changes in the collected behavior data, establishes an intention recognition model, predicts the operator's next operation intention through the intention recognition model, dynamically adjusts subsequent nodes of the task tree, and optimizes the task execution path.

9. A method for executing a dynamic task tree traversal by a robot arm according to claim 8, characterized in that: The intent recognition model includes the following steps: S5.1: Collect operator behavior change data, record the operator's action sequence, operation time interval, and operation frequency; S5.2: Extract feature data, extract features from the collected behavior change data, including but not limited to the key action points and action sequence of the action sequence, the standard deviation of the operation time interval, and the number of operation frequencies; S5.3: Establish an intention recognition model. Use a convolutional neural network to analyze the extracted feature data and establish an intention recognition model. The input of the intention recognition model is the operator's behavior characteristics, and the output is the operator's next operation intention. S5.4: Real-time prediction and adjustment of the task tree. During the task execution, the current operation status is collected in real time and input into the intention recognition model to predict the operator's next operation intention, dynamically adjust the subsequent nodes of the task tree, and optimize the execution path.

10. A method for executing a dynamic task tree traversal by a robot arm according to claim 1, characterized in that: The S6 comprises the following steps: S6.1: Conduct a preliminary evaluation of multiple candidate task nodes from the task tree, including the complexity of the task, the time required, and the resource consumption; S6.2: Calculate the priority of the candidate nodes, and calculate the priority of each candidate node according to the priority rule; the priority rule includes the urgency of the task, the success rate of the task, the execution time of the task and the resource consumption; S6.3: Select the node with the highest priority from the candidate nodes as the next execution node and continue to execute the task. After the task is completed, the execution result is fed back to the machine learning model. The machine learning model continuously optimizes the intent recognition model and priority selection strategy based on the new data to improve the efficiency and accuracy of task execution.

Images