An embodied intelligent decision-making control method and system based on the motion trajectory of a robotic arm

Abstract

The present application proposes an embodied intelligent decision-making control method and system based on the motion trajectory of a robotic arm, in the field of embodied intelligent technology, wherein the method includes: executing the first method and the second method simultaneously respectively; including: according to the motion trajectory data of the second robotic arm, using an improved dynamic time warping algorithm to obtain the similarity between the first real-time trajectory and the standard action curve; converting the motion trajectory data of the second robotic arm into a sequential decision problem, and according to the value of the reward function in the sequential decision problem, obtaining the similarity between the second real-time trajectory and the standard action curve; when the similarity is less than or equal to a preset similarity threshold, using a third method to dynamically correct the motion trajectory data of the robotic arm. The method of the present application can more accurately measure the degree of conformity between the actual task execution of the model and the expectation under the premise of meeting the real-time requirements.

Description

Technical Field

[0001] The present invention belongs to the field of embodied intelligence technology, and specifically relates to an embodied intelligence decision-making control method and system based on the motion trajectory of a robotic arm. Background Art

[0002] In today's era of rapid technological innovation, embodied intelligence, a highly promising and challenging research area within artificial intelligence, is attracting significant attention from researchers worldwide. Embodied intelligence aims to empower intelligent entities with the ability to perceive, make decisions, and perform actions in real-world physical environments. This requires them to not only interpret complex and dynamically changing environmental information, but also make appropriate decisions based on this information and precisely execute corresponding actions, thereby achieving efficient interaction with their environment.

[0003] For embodied intelligent systems, determining whether a model has successfully completed a given action is a crucial and currently severely lacking capability. In the conceptual framework of embodied intelligence, the ability to drive behavioral execution is similar to the function of the human cerebellum, which is responsible for precisely regulating the body movements of intelligent entities to achieve the desired actions; while the ability to drive judgment and decision-making is like the human brain, which makes reasonable decisions based on perceived environmental information and task objectives to guide the behavior of intelligent entities. An efficient and reliable embodied intelligent system requires close collaboration and precise coordination between the brain and cerebellum. However, due to the lack of an effective action completion judgment mechanism, even if the cerebellum can prompt intelligent entities to perform actions, it cannot know for sure whether these actions are in line with the desired goals, which greatly weakens the performance of the entire embodied intelligent system.

[0004] Take the use of robotic arms in industrial production as an example. On the production line, robotic arms must complete a series of complex operations, such as material handling and parts processing. Existing models often struggle to accurately determine whether the operation is successfully completed when manipulating the robotic arm to perform these operations. For example, after installing a part in a specified position, the model cannot determine whether the part has been correctly and securely installed. This can lead to quality defects in subsequent production steps or even equipment failure. This uncertainty is exacerbated in large-scale production environments, seriously affecting production efficiency and product quality. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, this application proposes an embodied intelligent decision-making control method and system based on the motion trajectory of a robotic arm.

[0006] In the first aspect, the present application proposes an embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm, comprising:

[0007] Step S1: The first robotic arm and the second robotic arm receive the same task instruction set;

[0008] Step S2: controlling the first robotic arm using a remote control method according to the order of the task instructions in the task instruction set;

[0009] Step S3: collecting motion trajectory data of the first robotic arm;

[0010] Step S4: fitting a standard motion curve according to the motion trajectory data of the first robotic arm;

[0011] Step S5: According to the order of the task instructions in the task instruction set, the second robot arm performs the task action of the current instruction;

[0012] Step S6: acquiring the motion trajectory data of the second robotic arm in real time;

[0013] Step S7: Executing the first method and the second method simultaneously; the first method includes: using an improved dynamic time warping algorithm based on the motion trajectory data of the second robotic arm to obtain the similarity between the first real-time trajectory and the standard action curve; the second method includes: converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve based on the value of the reward function in the sequential decision problem;

[0014] Step S8: When the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold, the second robotic arm completes the task action of the current instruction and returns to step S5 to execute the task action of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thereby completing the embodied intelligent decision-making control of the second robotic arm;

[0015] Step S9: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to a preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, and a third method is used to dynamically correct the motion trajectory data of the second robotic arm, and return to step S6 to re-execute the task action of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and using the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

[0016] The motion trajectory data of the first robotic arm or the motion trajectory data of the second robotic arm includes: joint positions of the robotic arm, end coordinates of the robotic arm, and joint accelerations of the robotic arm.

[0017] The improved dynamic time warping algorithm comprises:

[0018] The motion trajectory data of the second robot arm is segmented according to a preset time interval, and each trajectory segment has N real-time trajectory points;

[0019] In each trajectory, the distance between each real-time trajectory point and each trajectory point in the standard motion curve is calculated, and all distances are combined into a distance matrix;

[0020] By using the dynamic programming method, the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each trajectory is calculated in the distance matrix;

[0021] Normalize the shortest path;

[0022] The similarity between the normalized result and the standard action curve is calculated to obtain the similarity between the first real-time trajectory and the standard action curve.

[0023] Converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision problem, includes:

[0024] The motion trajectory data of the second robot arm is segmented according to a preset time interval, and each trajectory segment has N real-time trajectory points;

[0025] Treat each trajectory as the state space of a sequential decision problem;

[0026] The joint velocity and planned path of the second manipulator corresponding to each trajectory segment are used as the action space of the sequential decision problem;

[0027] According to the state space and action space, the values ​​of the trajectory similarity reward function, the task completion reward function and the smoothness reward function are calculated;

[0028] Accumulate the values ​​of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated in each trajectory segment;

[0029] The accumulated function value is used as the similarity between the second real-time trajectory and the standard action curve.

[0030] The trajectory similarity reward function is calculated as follows:

[0031] ;

[0032] in, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second manipulator and the standard action curve is greater than the preset distance threshold, the value of the trajectory similarity reward function is reduced by 1. If the distance between the motion trajectory data of the second manipulator and the standard action curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard action curve, and DTW is the improved dynamic time warping algorithm.

[0033] Task completion reward function, including:

[0034] When the end coordinate of the robot arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is increased by 1;

[0035] When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is increased or decreased by 1.

[0036] The smoothness reward function is calculated as follows:

[0037] ;

[0038] in, is a smoothness reward function. When the joint acceleration of the manipulator suddenly changes to be greater than the preset acceleration threshold, the value of the smoothness reward function is reduced by 1. When the joint acceleration of the manipulator suddenly changes to be less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the manipulator at the tth moment, is the joint acceleration of the robot arm at the t-1th moment, and N is the number of real-time trajectory points.

[0039] In the second aspect, the present application proposes an embodied intelligent decision-making and control system based on the motion trajectory of a robotic arm, comprising: an instruction receiving module, a remote control operation module, a first data acquisition module, a standard curve fitting module, an action execution module, a second data acquisition module, a similarity calculation module, and a decision control module;

[0040] The instruction receiving module is connected to the remote control operation module and the action execution module respectively, the remote control operation module is connected to the first data acquisition module, the first data acquisition module is connected to the standard curve fitting module, the action execution module is connected to the second data acquisition module, the second data acquisition module is connected to the similarity calculation module, the similarity calculation module is connected to the standard curve fitting and the decision control module respectively, and the decision control module is connected to the second data acquisition module and the action execution module respectively;

[0041] An instruction receiving module, used for the first robotic arm and the second robotic arm to receive the same task instruction set;

[0042] a remote control operation module, configured to control the first robotic arm using a remote control operation method according to the sequence of task instructions in the task instruction set;

[0043] A first data acquisition module, used to acquire motion trajectory data of the first robotic arm;

[0044] A standard curve fitting module, used for fitting a standard motion curve according to the motion trajectory data of the first robotic arm;

[0045] An action execution module, configured to cause the second robotic arm to perform the task action of the current instruction according to the order of the task instructions in the task instruction set;

[0046] A second data acquisition module is used to obtain the motion trajectory data of the second robotic arm in real time;

[0047] a similarity calculation module, configured to simultaneously execute a first method and a second method; the first method comprising: using an improved dynamic time warping algorithm based on the motion trajectory data of the second robotic arm to obtain a similarity between the first real-time trajectory and the standard action curve; and the second method comprising: converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining a similarity between the second real-time trajectory and the standard action curve based on a value of a reward function in the sequential decision problem;

[0048] A decision control module is used to, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than a preset similarity threshold, enable the second robotic arm to complete the task action of the current instruction, return to the action execution module to execute the task action of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, and complete the embodied intelligent decision control of the second robotic arm; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, adopts a third method to dynamically correct the motion trajectory data of the second robotic arm, returns to the second data acquisition module to re-execute the task action of the current instruction, and the third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and adopting the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

[0049] In a third aspect, the present application proposes an electronic device comprising: one or more processors, and a memory, wherein the memory is used to store instructions. When the instructions are executed by the one or more processors, the one or more processors execute the embodied intelligent decision-making and control method based on the motion trajectory of the robotic arm.

[0050] In a fourth aspect, the present application proposes a computer-readable storage medium storing executable instructions, which, when executed, enable a processor to execute the embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm.

[0051] Beneficial effects:

[0052] This application proposes an embodied intelligent decision-making and control method and system based on the motion trajectory of a robotic arm. This application uses an improved dynamic time warping (DTW) algorithm to calculate the similarity between the real-time trajectory and the standard trajectory. This algorithm fully considers the dynamic changes in the trajectory over time. Through a reward function approach, it enhances autonomous decision-making capabilities and deeply couples trajectory monitoring with control strategies. Compared to simple trajectory comparison methods, this method can more accurately measure the degree of conformity between the model's actual task execution and expectations, while still meeting real-time requirements. Compared to the original DTW algorithm, this method significantly improves real-time performance and reduces noise and the impact of outliers on matching results. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of an embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to an embodiment of the present application;

[0054] Figure 2 This is a flow chart of an embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to an embodiment of the present application;

[0055] Figure 3 This is a principle block diagram of an embodied intelligent decision-making control system based on the motion trajectory of a robotic arm according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The specific implementation of the present application is further described in detail below with reference to the accompanying drawings and examples.

[0057] The uncertainty mentioned in the background technology will continue to intensify in a large-scale production environment. It is difficult to find a better implementation plan in the existing technology. In addition, the implementation of each task action in the embodied intelligent decision-making control needs to take real-time into consideration. Therefore, many existing technologies have been tried and it is difficult to meet the needs of a large-scale production environment. In this case, the present application proposes an embodied intelligent decision-making control method and system based on the motion trajectory of a robotic arm. Three different methods are used to cooperate with each other and make up for each other's shortcomings. It not only meets the real-time requirements, but also can more accurately measure the degree of compliance between the actual task execution of the model and the expectations.

[0058] Example 1:

[0059] In the first aspect, this application proposes an embodied intelligent decision-making control method based on the motion trajectory of a robotic arm, such as Figure 1、 Figure 2 Shown, including:

[0060] Step S1: The first robotic arm and the second robotic arm receive the same task instruction set;

[0061] Step S2: controlling the first robotic arm using a remote control method according to the order of the task instructions in the task instruction set;

[0062] Step S3: collecting motion trajectory data of the first robotic arm;

[0063] Step S4: fitting a standard motion curve according to the motion trajectory data of the first robotic arm;

[0064] In this embodiment, it is first necessary to fit a standard action curve, such as Figure 2 As shown, the process starts at the "Start 1" node. The primary purpose of this stage is to construct a standard motion curve. Assuming the first and second robotic arms receive the same set of task instructions, the first robotic arm is teleoperated: the operator controls the first robotic arm using a teleoperation device (such as a joystick or control software interface) to perform the required actions. During this process, the operator precisely controls the robot to complete a series of actions according to the task requirements. For example, in an industrial assembly scenario, this could simulate grasping and installing a part in a specified location. The motion trajectory data of the first robotic arm is then collected. During the teleoperation of the first robotic arm, the system utilizes encoders, position sensors, and external vision sensors installed at the robot joints to collect various data in real time during the first robotic arm's motion, including joint angle changes, the spatial position coordinates of the end effector, joint velocity, and joint acceleration. This data serves as the basis for subsequent analysis. Finally, a standard motion curve is fitted. After sufficient data has been collected, specialized algorithms such as the least squares method are used to process the data. By minimizing the sum of squares of the errors between the measurement points and the fitting curve, the coefficients of a suitable function form (such as a polynomial function) are determined, thereby fitting a curve that can represent the standard movement of the robot. This curve serves as a reference standard for subsequent task judgments.

[0065] Specifically, the least squares method is taken as an example for detailed description: During the operation of the embodied intelligent system, taking the ACT model reasoning task as an example, the present invention focuses on the motion state of the robotic arm. In the early stage, the motion trajectory of the robotic arm when performing various tasks is comprehensively collected to form a rich and representative data set. In the stage of fitting the standard trajectory, the professional algorithm of the least squares method in curve fitting is adopted. Specifically, it is assumed that the motion trajectory of the robotic arm in Cartesian space can be approximated by the function y=f(x) (where x usually represents time or the number of motion steps, and y represents the position coordinates of the robotic arm in space). There is a series of discrete measurement points in the data set. . The goal of the least squares method is to find a function f(x) that minimizes the sum of the squares of the errors between the measurement points and the fitting curve. For robot arm trajectory fitting, a polynomial function is often used as the fitting function form. By solving the normal equations of the above minimization problem, a curve that can best fit the robot arm motion trajectory is obtained, that is, the standard trajectory line. The process of fitting the standard motion curve belongs to the prior art and will not be repeated in this application.

[0066] Step S5: According to the order of the task instructions in the task instruction set, the second robot arm performs the task action of the current instruction;

[0067] Step S6: acquiring the motion trajectory data of the second robotic arm in real time; the motion trajectory data of the first robotic arm or the motion trajectory data of the second robotic arm includes: the joint position of the robotic arm, the end coordinates of the robotic arm, and the joint acceleration of the robotic arm.

[0068] Step S7: Executing the first method and the second method simultaneously; the first method includes: using an improved dynamic time warping (DTW) algorithm based on the motion trajectory data of the second robotic arm to obtain the similarity between the first real-time trajectory and the standard action curve; the second method includes: converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve based on the value of the reward function in the sequential decision problem;

[0069] Step S8: When the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold, the second robotic arm completes the task action of the current instruction and returns to step S5 to execute the task action of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thereby completing the embodied intelligent decision-making control of the second robotic arm;

[0070] Step S9: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to a preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, and a third method is used to dynamically correct the motion trajectory data of the second robotic arm, and return to step S6 to re-execute the task action of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and using the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

[0071] In this embodiment, during the task execution and determination phase, corresponding to "Start 2," the process starts at the "Start 2" node, at which point the actual task execution and determination phase begins. First, the second robotic arm autonomously executes the corresponding action based on the model inference results or the received task instructions in the task instruction set. For example, in a logistics sorting task, the robot autonomously plans a path and grabs the goods based on the identified cargo information. While the robot executes the action, the system continuously collects the robot's actual motion trajectory data in real time, namely, the motion trajectory data of the second robotic arm. The first and second methods are simultaneously executed, and the real-time collected trajectory is compared with the previously fitted standard motion curve to calculate the similarity between the two. If the similarity between the first real-time trajectory and the standard motion curve and the similarity between the second real-time trajectory and the standard motion curve are both greater than 90%, the second robotic arm completes the task action for the current instruction. The system can then, according to a preset strategy, such as gracefully terminating the inference process to release computing resources, return to step S5 to execute the task action of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, completing the embodied intelligent decision-making control of the second robotic arm. When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to ninety percent, the second robotic arm has not completed the task action of the current instruction, and the motion trajectory data of the second robotic arm is dynamically corrected using the third method, and the process returns to step S6 to re-execute the task action of the current instruction.

[0072] This application adopts three different methods, which cooperate with each other to meet the requirements of real-time and accuracy control.

[0073] Specifically, the first method needs to be executed simultaneously with the second method, otherwise the real-time requirements cannot be met, and both the first method and the second method use an improved dynamic time warping algorithm as the basis for calculation.

[0074] Among them, the improved dynamic time warping algorithm includes:

[0075] Step S71.1: segmenting the motion trajectory data of the second robotic arm according to a preset time interval, with each trajectory segment having N real-time trajectory points;

[0076] Step S71.2: In each trajectory segment, calculate the distance between each real-time trajectory point and each trajectory point in the standard motion curve, and form a distance matrix with all the distances;

[0077] Step S71.3: Calculate the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each trajectory segment in the distance matrix using a dynamic programming method;

[0078] Step S71.4: Normalize the shortest path;

[0079] Step S71.5: Calculate the similarity between the normalized result and the standard action curve to obtain the similarity between the first real-time trajectory and the standard action curve.

[0080] In this embodiment, during model inference, real-time sensors continuously collect the motion trajectory data of the second robotic arm. A modified dynamic time warping (DTW) algorithm is used to calculate the similarity between the real-time trajectory and the fitted standard trajectory. The core idea of ​​the DTW algorithm is to perform nonlinear warping of two sequences on the time axis. Within the traditional DTW algorithm framework, piecewise approximation is used to find the optimal alignment between the sequences by dividing them into subsequences. GPUs are used to perform parallel computations on the distance matrix, allowing for a distributed framework to process large sequences. The improved DTW algorithm can better compare trajectory data and, through parallel computing, meet real-time requirements.

[0081] The second method converts motion trajectory data into a sequential decision-making problem. Through reinforcement learning, the intelligent agent can autonomously learn "how to judge whether the task is completed" in interaction with the environment, and dynamically adjust the control strategy to optimize the trajectory.

[0082] Converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision problem, includes:

[0083] Step S72.1: Segment the motion trajectory data of the second robotic arm into segments according to a preset time interval, with each segment having N real-time trajectory points;

[0084] Step S72.2: Treat each trajectory as the state space of a sequential decision problem;

[0085] Step S72.3: The joint velocities and planned paths of the second manipulator corresponding to each trajectory segment are used as the action space of the sequential decision problem;

[0086] Step S72.4: Calculate the value of the trajectory similarity reward function, the value of the task completion reward function, and the value of the smoothness reward function based on the state space and the action space;

[0087] Step S72.5: Accumulate the values ​​of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated for each trajectory segment;

[0088] Step S73.6: The accumulated function value is used as the similarity between the second real-time trajectory and the standard action curve.

[0089] Among them, the trajectory similarity reward function is calculated as follows:

[0090] ;

[0091] in, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second manipulator and the standard action curve is greater than a preset distance threshold (5 mm in this embodiment), the value of the trajectory similarity reward function is reduced by 1. If the distance between the motion trajectory data of the second manipulator and the standard action curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard action curve, and DTW is the improved dynamic time warping algorithm.

[0092] Task completion reward function, including:

[0093] When the end coordinate of the robot arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is increased by 1;

[0094] When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is increased or decreased by 1.

[0095] The smoothness reward function is calculated as follows:

[0096] ;

[0097] in, is a smoothness reward function. When the joint acceleration of the manipulator suddenly changes to be greater than a preset acceleration threshold (1 meter per second in this embodiment), the value of the smoothness reward function is reduced by 1. When the joint acceleration of the manipulator suddenly changes to be less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the manipulator at the tth moment, is the joint acceleration of the robot arm at the t-1th moment, and N is the number of real-time trajectory points.

[0098] In this embodiment, when the similarity between the second real-time trajectory and the standard action curve is greater than the accumulated function value of 90% of the segment trajectories, it is determined that the second real-time trajectory is similar to the standard trajectory of the standard action curve.

[0099] Third method:

[0100] The motion trajectory data of the second manipulator is input into the reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and the motion trajectory data of the second manipulator is dynamically corrected using the trajectory adjustment parameter, including:

[0101] Offline strategy learning: Using algorithms such as DDPG and SAC, and using the historical motion trajectory data of the second robot arm for training, we can obtain the optimal control strategy under different states and obtain the reinforcement learning algorithm model.

[0102] Online real-time optimization: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the reinforcement learning algorithm model is triggered to generate trajectory adjustment parameters, so that the robot arm dynamically corrects the path during execution. The trajectory adjustment parameters can dynamically correct the motion trajectory data of the second robot arm.

[0103] Exploration and Utilization Balance: Furthermore, through ε-greedy or entropy regularization, feasible trajectories are explored in new scenarios while leveraging existing successful experiences for rapid convergence.

[0104] Among them, the calculation formula and internal structure data of the reinforcement learning algorithm model itself are prior art and will not be described in detail in this embodiment.

[0105] Strategy optimization for task completion judgment: Furthermore, the traditional judgment rule of "similarity ≥ 90% and joint position tolerance meeting the standard" is converted into the output of the policy network, and the judgment threshold is automatically optimized through reinforcement learning. For example, in noisy environments, the policy will adaptively increase the weight of force feedback and reduce the sensitivity to trajectory similarity.

[0106] This embodiment proposes an embodied intelligent decision-making and control method based on the motion trajectory of the robot arm. By fitting the standard motion trajectory of the robot arm through the least squares method, the motion characteristics of the robot arm under the ideal task execution state can be accurately characterized. The similarity between the real-time trajectory and the standard trajectory is calculated in combination with the improved dynamic time warping algorithm, which fully considers the dynamic change characteristics of the trajectory in the time dimension. Through the reinforcement learning reward function method, the autonomous decision-making ability is improved, and the trajectory monitoring and control strategy are deeply coupled. A complete closed loop is formed. Compared with the simple trajectory comparison method, it can more accurately measure the degree of compliance between the actual task execution of the model and the expected situation. Compared with the traditional dynamic time warping algorithm, there are significant improvements in real-time performance and reducing noise and reducing the impact of outliers on matching results.

[0107] Example 2:

[0108] This embodiment proposes an embodied intelligent decision-making control system based on the motion trajectory of the robot arm, such as Figure 3 Shown, including:

[0109] Instruction receiving module, remote control operation module, first data acquisition module, standard curve fitting module, action execution module, second data acquisition module, similarity calculation module, decision control module;

[0110] The instruction receiving module is connected to the remote control operation module and the action execution module respectively, the remote control operation module is connected to the first data acquisition module, the first data acquisition module is connected to the standard curve fitting module, the action execution module is connected to the second data acquisition module, the second data acquisition module is connected to the similarity calculation module, the similarity calculation module is connected to the standard curve fitting and the decision control module respectively, and the decision control module is connected to the second data acquisition module and the action execution module respectively;

[0111] An instruction receiving module, used for the first robotic arm and the second robotic arm to receive the same task instruction set;

[0112] a remote control operation module, configured to control the first robotic arm using a remote control operation method according to the sequence of task instructions in the task instruction set;

[0113] A first data acquisition module, used to acquire motion trajectory data of the first robotic arm;

[0114] A standard curve fitting module, used for fitting a standard motion curve according to the motion trajectory data of the first robotic arm;

[0115] An action execution module, configured to cause the second robotic arm to perform the task action of the current instruction according to the order of the task instructions in the task instruction set;

[0116] A second data acquisition module is used to obtain the motion trajectory data of the second robotic arm in real time;

[0117] a similarity calculation module, configured to simultaneously execute a first method and a second method; the first method comprising: using an improved dynamic time warping algorithm based on the motion trajectory data of the second robotic arm to obtain a similarity between the first real-time trajectory and the standard action curve; and the second method comprising: converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining a similarity between the second real-time trajectory and the standard action curve based on a value of a reward function in the sequential decision problem;

[0118] A decision control module is used to, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than a preset similarity threshold, enable the second robotic arm to complete the task action of the current instruction, return to the action execution module to execute the task action of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, and complete the embodied intelligent decision control of the second robotic arm; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, adopts a third method to dynamically correct the motion trajectory data of the second robotic arm, returns to the second data acquisition module to re-execute the task action of the current instruction, and the third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and adopting the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

[0119] Example 3:

[0120] This embodiment proposes an electronic device, comprising: one or more processors, and a memory, wherein the memory is used to store instructions. When the instructions are executed by the one or more processors, the one or more processors execute the embodied intelligent decision-making and control method based on the motion trajectory of the robotic arm.

[0121] The electronic device can be a mobile phone, computer, or tablet computer, and includes a memory and a processor. The memory stores a computer program that, when executed by the processor, implements the embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm, as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and a communication component.

[0122] The processor is configured to execute all or part of the steps of the embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm as described in the above-described embodiment. The memory is configured to store various types of data, such as instructions for any application or method in the electronic device, as well as data related to the application.

[0123] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the embodied intelligent decision-making control method based on the motion trajectory of the robotic arm described in the above embodiment.

[0124] Example 4:

[0125] This embodiment provides a computer-readable storage medium storing executable instructions. When the instructions are executed, if they are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0126] The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of an embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm described in various embodiments of the present application.

[0127] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (for example, SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application, abbreviation of application software) application store and other media that can store program verification codes, on which a computer program is stored. When the computer program is executed by the processor, it can implement the above-mentioned steps of the embodied intelligent decision-making control method based on the motion trajectory of the robotic arm.

[0128] The various embodiments in this application are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0129] The scope of protection of this application is not limited to the above-described embodiments. Obviously, those skilled in the art may make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, the disclosure is intended to include such modifications and variations.

Claims

1. An embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm, characterized in that: include: Step S1: The first robotic arm and the second robotic arm receive the same task instruction set; Step S2: controlling the first robotic arm using a remote control method according to the order of the task instructions in the task instruction set; Step S3: collecting motion trajectory data of the first robotic arm; Step S4: fitting a standard motion curve according to the motion trajectory data of the first robotic arm; Step S5: According to the order of the task instructions in the task instruction set, the second robot arm performs the task action of the current instruction; Step S6: acquiring the motion trajectory data of the second robotic arm in real time; Step S7: Execute the first method and the second method simultaneously; The first method includes: using an improved dynamic time warping algorithm based on the motion trajectory data of the second manipulator to obtain the similarity between the first real-time trajectory and the standard action curve; the second method includes: converting the motion trajectory data of the second manipulator into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve based on the value of the reward function in the sequential decision problem; Step S8: When the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold, the second robotic arm completes the task action of the current instruction and returns to step S5 to execute the task action of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thereby completing the embodied intelligent decision-making control of the second robotic arm; Step S9: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to a preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, and a third method is used to dynamically correct the motion trajectory data of the second robotic arm, and return to step S6 to re-execute the task action of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and using the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

2. The embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to claim 1, characterized in that: The motion trajectory data of the first robotic arm or the motion trajectory data of the second robotic arm includes: joint positions of the robotic arm, end coordinates of the robotic arm, and joint accelerations of the robotic arm.

3. The embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to claim 1, characterized in that: The improved dynamic time warping algorithm comprises: The motion trajectory data of the second robot arm is segmented according to a preset time interval, and each trajectory segment has N real-time trajectory points; In each trajectory, the distance between each real-time trajectory point and each trajectory point in the standard motion curve is calculated, and all distances are combined into a distance matrix; By using the dynamic programming method, the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each trajectory is calculated in the distance matrix; Normalize the shortest path; The similarity between the normalized result and the standard action curve is calculated to obtain the similarity between the first real-time trajectory and the standard action curve.

4. The embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to claim 1, characterized in that: Converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision problem, includes: The motion trajectory data of the second robot arm is segmented according to a preset time interval, and each trajectory segment has N real-time trajectory points; Treat each trajectory as the state space of a sequential decision problem; The joint velocity and planned path of the second manipulator corresponding to each trajectory segment are used as the action space of the sequential decision problem; According to the state space and action space, the values ​​of the trajectory similarity reward function, the task completion reward function and the smoothness reward function are calculated; Accumulate the values ​​of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated in each trajectory segment; The accumulated function value is used as the similarity between the second real-time trajectory and the standard action curve.

5. The embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to claim 4, characterized in that: The trajectory similarity reward function is calculated as follows: ； in, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second manipulator and the standard action curve is greater than the preset distance threshold, the value of the trajectory similarity reward function is reduced by 1. If the distance between the motion trajectory data of the second manipulator and the standard action curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard action curve, and DTW is the improved dynamic time warping algorithm.

6. The embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm according to claim 4, characterized in that: Task completion reward function, including: When the end coordinate of the robot arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is increased by 1; When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is increased or decreased by 1.

7. The method for embodied intelligent decision-making and control based on the motion trajectory of a robotic arm according to claim 4, characterized in that: The smoothness reward function is calculated as follows: ； in, is a smoothness reward function. When the joint acceleration of the manipulator suddenly changes to be greater than the preset acceleration threshold, the value of the smoothness reward function is reduced by 1. When the joint acceleration of the manipulator suddenly changes to be less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the manipulator at the tth moment, is the joint acceleration of the robot arm at the t-1th moment, and N is the number of real-time trajectory points.

8. An embodied intelligent decision-making and control system based on the motion trajectory of a robotic arm, characterized in that: include: Instruction receiving module, remote control operation module, first data acquisition module, standard curve fitting module, action execution module, second data acquisition module, similarity calculation module, decision control module; The instruction receiving module is connected to the remote control operation module and the action execution module respectively, the remote control operation module is connected to the first data acquisition module, the first data acquisition module is connected to the standard curve fitting module, the action execution module is connected to the second data acquisition module, the second data acquisition module is connected to the similarity calculation module, the similarity calculation module is connected to the standard curve fitting and the decision control module respectively, and the decision control module is connected to the second data acquisition module and the action execution module respectively; An instruction receiving module, used for the first robotic arm and the second robotic arm to receive the same task instruction set; a remote control operation module, configured to control the first robotic arm using a remote control operation method according to the sequence of task instructions in the task instruction set; A first data acquisition module, used to acquire motion trajectory data of the first robotic arm; A standard curve fitting module, used for fitting a standard motion curve according to the motion trajectory data of the first robotic arm; An action execution module, configured to cause the second robotic arm to perform the task action of the current instruction according to the order of the task instructions in the task instruction set; A second data acquisition module is used to obtain the motion trajectory data of the second robotic arm in real time; a similarity calculation module, configured to simultaneously execute a first method and a second method; the first method comprising: using an improved dynamic time warping algorithm based on the motion trajectory data of the second robotic arm to obtain a similarity between the first real-time trajectory and the standard action curve; and the second method comprising: converting the motion trajectory data of the second robotic arm into a sequential decision problem, and obtaining a similarity between the second real-time trajectory and the standard action curve based on a value of a reward function in the sequential decision problem; A decision control module is used to, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than a preset similarity threshold, enable the second robotic arm to complete the task action of the current instruction, return to the action execution module to execute the task action of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, and complete the embodied intelligent decision control of the second robotic arm; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm fails to complete the task action of the current instruction, adopts a third method to dynamically correct the motion trajectory data of the second robotic arm, returns to the second data acquisition module to re-execute the task action of the current instruction, and the third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and adopting the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

9. An electronic device, characterized in that: include: One or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that It stores executable instructions, which, when executed, enable the processor to execute an embodied intelligent decision-making and control method based on the motion trajectory of a robotic arm as described in any one of claims 1 to 7.

Images