Body intelligent decision-making control method and system based on motion trail of mechanical arm

Abstract

The invention provides an intelligent decision-making control method and system based on the motion trail of a mechanical arm, and belongs to the technical field of intelligent. The method comprises the steps that a first method and a second method are executed at the same time; comprising the steps that according to motion track data of a second mechanical arm, an improved dynamic time warping algorithm is adopted, and the similarity between a first real-time track and a standard action curve is obtained; the motion track data of the second mechanical arm is converted into a sequence decision problem, and the similarity between a second real-time track and a standard action curve is obtained according to the value of a reward function in the sequence decision problem; and under the condition that the similarity is smaller than or equal to a preset similarity threshold value, a third method is adopted for dynamically correcting the motion track data of the mechanical arm. According to the method, on the premise that the real-time performance is met, the conformity degree of the actual task execution condition of the model and the expectation can be measured more accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of embodied intelligence, and particularly relates to an embodied intelligence decision-making control method and system based on the motion trajectory of a robotic arm. Background Art

[0002] In the current period of rapid technological innovation, embodied intelligence, as a highly potential and challenging research direction within the field of artificial intelligence, is attracting great attention from global scientific researchers. Embodied intelligence aims to endow intelligent entities with the ability to perceive, make decisions, and perform actions in a real physical environment. This requires intelligent entities not only to be able to interpret complex and dynamically changing environmental information, but also to make appropriate decisions based on this information and precisely execute corresponding actions to achieve efficient interaction with the environment.

[0003] For an embodied intelligence system, determining whether a model has successfully completed a given action is a crucial and currently severely lacking ability. In the conceptual framework of embodied intelligence, the ability to drive behavior execution is similar to the function of the human cerebellum, responsible for precisely regulating the body movements of intelligent entities to achieve the expected actions; while the ability to drive judgment and decision-making is like the human brain, making reasonable decisions based on the perceived environmental information and task goals to guide the behavior of intelligent entities. An efficient and reliable embodied intelligence system requires close cooperation and precise coordination between the brain and the cerebellum. However, currently, due to the lack of an effective action completion determination mechanism, even though the cerebellum can prompt intelligent entities to execute actions, it cannot accurately know whether these actions meet the expected goals, which greatly weakens the performance of the entire embodied intelligence system.

[0004] Taking the application of a robotic arm in industrial production as an example, on a production line, the robotic arm needs to complete a series of complex operations, such as material handling, part processing, etc. Existing models often have difficulty accurately determining whether an operation has been successfully completed when controlling the robotic arm to perform these operations. For example, after installing a part at a specified position, the model cannot determine whether the part has been correctly and firmly installed, which may lead to quality defects in subsequent production processes and even cause equipment failures. This uncertainty will continuously intensify in a large-scale production environment, seriously affecting production efficiency and product quality. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present application proposes an embodied intelligence decision-making control method and system based on the motion trajectory of a robotic arm.

[0006] In a first aspect, the present application proposes an embodied intelligence decision-making control method based on the motion trajectory of a robotic arm, including:

[0007] Step S1: The first robotic arm and the second robotic arm receive the same task instruction set;

[0008] Step S2: Control the first robotic arm by means of remote control according to the order of the task instructions in the task instruction set;

[0009] Step S3: Collect the motion trajectory data of the first robotic arm;

[0010] Step S4: Fit a standard action curve based on the motion trajectory data of the first robotic arm;

[0011] Step S5: According to the order of the task instructions in the task instruction set, the second robotic arm performs the task actions of the current instruction;

[0012] Step S6: Obtain the motion trajectory data of the second robotic arm in real time;

[0013] Step S7: Execute the first method and the second method simultaneously; the first method includes: obtaining the similarity between the first real-time trajectory and the standard action curve by using an improved dynamic time warping algorithm according to the motion trajectory data of the second robotic arm; the second method includes: transforming the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem;

[0014] Step S8: When the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold, the second robotic arm completes the task actions of the current instruction, returns to Step S5 to execute the task actions of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, and completes the embodied intelligent decision-making control of the second robotic arm;

[0015] Step S9: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm has not completed the task actions of the current instruction, and uses a third method to dynamically correct the motion trajectory data of the second robotic arm, and returns to Step S6 to re-execute the task actions of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and using the trajectory adjustment parameter to dynamically correct the motion trajectory data of the second robotic arm.

[0016] The motion trajectory data of the first robotic arm or the motion trajectory data of the second robotic arm includes: the joint positions of the robotic arm, the end coordinates of the robotic arm, and the joint accelerations of the robotic arm.

[0017] The improved dynamic time warping algorithm includes:

[0018] Segment the motion trajectory data of the second robotic arm at preset time intervals, with each segment of the trajectory having N real-time trajectory points;

[0019] In each segment of the trajectory, calculate the distance between each real-time trajectory point and each trajectory point on the standard action curve, and form a distance matrix with all the distances;

[0020] Using the dynamic programming method, calculate the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each segment of the trajectory in the distance matrix;

[0021] Normalize the shortest path;

[0022] Calculate the similarity between the normalized result and the standard action curve to obtain the similarity between the first real-time trajectory and the standard action curve.

[0023] The conversion of the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem, includes:

[0024] Segment the motion trajectory data of the second robotic arm at preset time intervals, with each segment of the trajectory having N real-time trajectory points;

[0025] Take each segment of the trajectory as the state space of the sequential decision-making problem;

[0026] Take the joint velocity of the second robotic arm and the planned path corresponding to each segment of the trajectory as the action space of the sequential decision-making problem;

[0027] According to the state space and the action space, calculate the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function;

[0028] Accumulate the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated for each segment of the trajectory;

[0029] Take the accumulated function value as the similarity between the second real-time trajectory and the standard action curve.

[0030] The trajectory similarity reward function, the calculation formula is as follows:

[0031] ;

[0032] Where, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second robotic arm and the standard action curve is greater than the preset distance threshold, the value of the trajectory similarity reward function is decreased by 1. If the distance between the motion trajectory data of the second robotic arm and the standard action curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard action curve, and DTW is the improved dynamic time warping algorithm.

[0033] The task completion reward function includes:

[0034] When the end coordinate of the robotic arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is increased by 1;

[0035] When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is increased or decreased by 1.

[0036] The smoothness reward function is calculated as follows:

[0037] ;

[0038] Where, is the smoothness reward function. When the joint acceleration mutation of the robotic arm is greater than the preset acceleration threshold, the value of the smoothness reward function is decreased by 1. When the joint acceleration mutation of the robotic arm is less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the robotic arm at the t-th moment, is the joint acceleration of the robotic arm at the (t - 1)-th moment, and N is the number of real-time trajectory points.

[0039] In a second aspect, the present application proposes an embodied intelligent decision control system based on the motion trajectory of a robotic arm, including: an instruction receiving module, a remote control operation module, a first data acquisition module, a standard curve fitting module, an action execution module, a second data acquisition module, a similarity calculation module, and a decision control module;

[0040] Among them, the instruction receiving module is respectively connected to the remote control operation module and the action execution module, the remote control operation module is connected to the first data acquisition module, the first data acquisition module is connected to the standard curve fitting module, the action execution module is connected to the second data acquisition module, the second data acquisition module is connected to the similarity calculation module, the similarity calculation module is respectively connected to the standard curve fitting and the decision control module, and the decision control module is respectively connected to the second data acquisition module and the action execution module;

[0041] The instruction receiving module is used to receive the same task instruction set for the first robotic arm and the second robotic arm;

[0042] A remote control operation module, configured to control the first robotic arm by means of remote control operation according to the sequence of task instructions in the task instruction set;

[0043] A first data acquisition module, configured to acquire the motion trajectory data of the first robotic arm;

[0044] A standard curve fitting module, configured to fit a standard action curve according to the motion trajectory data of the first robotic arm;

[0045] An action execution module, configured to perform the task actions of the current instruction on the second robotic arm according to the sequence of task instructions in the task instruction set;

[0046] A second data acquisition module, configured to obtain the motion trajectory data of the second robotic arm in real time;

[0047] A similarity calculation module, configured to execute the first method and the second method simultaneously; the first method includes: obtaining the similarity between the first real-time trajectory and the standard action curve according to the motion trajectory data of the second robotic arm by using an improved dynamic time warping algorithm; the second method includes: converting the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem;

[0048] A decision control module, configured to, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than a preset similarity threshold, cause the second robotic arm to complete the task actions of the current instruction, return to the action execution module to execute the task actions of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, and complete the embodied intelligent decision control of the second robotic arm; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm has not completed the task actions of the current instruction, and the motion trajectory data of the second robotic arm is dynamically corrected by using a third method, and the third method is returned to the second data acquisition module to re-execute the task actions of the current instruction, and the third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and dynamically correcting the motion trajectory data of the second robotic arm by using the trajectory adjustment parameter.

[0049] In a third aspect, the present application provides an electronic device, including: one or more processors, and a memory, where the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors are caused to execute the above-mentioned method for embodied intelligent decision control based on the motion trajectory of a robotic arm.

[0050] Fourth aspect, the present application proposes a computer-readable storage medium storing executable instructions, which when executed cause a processor to execute the method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm as described above.

[0051] Beneficial effects:

[0052] The present application proposes a method and system for embodied intelligent decision-making control based on the motion trajectory of a robotic arm. The improved dynamic time warping (DTW) algorithm is used in the present application to calculate the similarity between the real-time trajectory and the standard trajectory, fully considering the dynamic change characteristics of the trajectory in the time dimension. Through the reward function method, the autonomous decision-making ability is improved, and the trajectory monitoring and control strategy are deeply coupled. Compared with simple trajectory comparison methods, it can more accurately measure the degree of compliance between the actual task execution of the model and the expectation on the premise of meeting real-time requirements. Compared with the original DTW algorithm, there are significant improvements in both real-time performance and reducing the influence of noise and outliers on the matching result. Description of the drawings

[0053] Figure 1 is a flowchart of a method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to an embodiment of the present application;

[0054] Figure 2 is a schematic diagram of the process of a method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to an embodiment of the present application;

[0055] Figure 3 is a block diagram of the principle of a system for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to an embodiment of the present application. Detailed implementation manners

[0056] The following further describes in detail the specific implementation manners of the present application with reference to the drawings and embodiments.

[0057] Regarding the problem that the uncertainty mentioned in the background art will continuously intensify in a large-scale production environment, it is difficult to find a better implementation solution in the prior art. Coupled with the fact that real-time performance needs to be considered for the implementation of each task action in embodied intelligent decision-making control, it is very difficult to meet the requirements of a large-scale production environment after trying many existing technologies. In this case, the present application proposes a method and system for embodied intelligent decision-making control based on the motion trajectory of a robotic arm, which uses three different methods, cooperating with each other, complementing each other's advantages and disadvantages, not only meeting the real-time requirements, but also being able to more accurately measure the degree of compliance between the actual task execution of the model and the expectation.

[0058] Embodiment 1:

[0059] First aspect, the present application proposes a method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm, as Figure 1, Figure 2 as shown, including:

[0060] Step S1: The first robotic arm and the second robotic arm receive the same task instruction set;

[0061] Step S2: According to the order of the task instructions in the task instruction set, the first robotic arm is controlled by means of remote operation;

[0062] Step S3: Collect the motion trajectory data of the first robotic arm;

[0063] Step S4: Fit a standard motion curve based on the motion trajectory data of the first robotic arm;

[0064] In this embodiment, first, a standard motion curve needs to be fitted. As shown in Figure 2 , the process starts from the "Start 1" node. The main purpose of this stage is to construct a standard motion curve. On the premise that the first robotic arm and the second robotic arm receive the same task instruction set, remotely operate the first robotic arm: The operator controls the first robotic arm to perform relevant actions through a remote operation device (such as a joystick, a control software interface, etc.). During this process, the operator precisely controls the robot to complete a series of actions according to the task requirements. For example, in an industrial assembly scenario, simulate the actions of grasping and installing parts at a specified position. Then collect the motion trajectory data of the first robotic arm: During the remote operation of the first robotic arm, the system uses devices such as encoders, position sensors installed at the robot joints, and external vision sensors to collect various data during the movement of the first robotic arm in real time, including joint angle changes, spatial position coordinates of the end effector, joint movement speed, joint acceleration, and other information. These data will serve as the basis for subsequent analysis. Finally, perform the fitting of the standard motion curve: After collecting sufficient data, use professional algorithms such as the least squares method to process the data. By minimizing the sum of the squares of the errors between the measurement points and the fitted curve, determine the coefficients of a suitable functional form (such as a polynomial function), thereby fitting a curve that can represent the standard motion of the robot. This curve is used as a reference standard for subsequent task determination.

[0065] Specifically, taking the least squares method as an example, a detailed description is as follows: During the operation of the embodied intelligent system, taking the ACT model inference task as an example, the present invention focuses on the motion state of the robotic arm. In the early stage, comprehensively collect the motion trajectories of the robotic arm when performing various tasks to form a rich and representative data set. In the stage of fitting the standard trajectory, use the professional algorithm of the least squares method in curve fitting. Specifically, assume that the motion trajectory of the robotic arm in the Cartesian space can be approximately represented by the function y = f(x) (where x usually represents time or the number of motion steps, and y represents the position coordinates of the robotic arm in space). There is a series of discrete measurement points in the data set​​ The goal of the least squares method is to find a function f(x) that minimizes the sum of the squares of the errors between the measurement points and the fitting curve. For the trajectory fitting of a robotic arm, a polynomial function is often selected as the form of the fitting function. By solving the normal equations of the above minimization problem, a curve that can best fit the motion trajectory of the robotic arm, that is, the standard trajectory line, can be obtained. The process of fitting the standard motion curve belongs to the prior art and will not be elaborated in this application.

[0066] Step S5: According to the order of the task instructions in the task instruction set, the second robotic arm performs the task actions of the current instruction.

[0067] Step S6: Real-time obtain the motion trajectory data of the second robotic arm; the motion trajectory data of the first robotic arm or the second robotic arm includes: the joint positions of the robotic arm, the end coordinates of the robotic arm, and the joint accelerations of the robotic arm.

[0068] Step S7: Execute the first method and the second method simultaneously; the first method includes: according to the motion trajectory data of the second robotic arm, using an improved dynamic time warping algorithm (DTW, Dynamic Time Warping) to obtain the similarity between the first real-time trajectory and the standard motion curve; the second method includes: converting the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard motion curve according to the value of the reward function in the sequential decision-making problem.

[0069] Step S8: When the similarity between the first real-time trajectory and the standard motion curve and the similarity between the second real-time trajectory and the standard motion curve are both greater than the preset similarity threshold, the second robotic arm completes the task actions of the current instruction, returns to step S5 to execute the task actions of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, and the embodied intelligent decision control of the second robotic arm is completed.

[0070] Step S9: When the similarity between the first real-time trajectory and the standard motion curve or the similarity between the second real-time trajectory and the standard motion curve is less than or equal to the preset similarity threshold, the second robotic arm has not completed the task actions of the current instruction. Use the third method to dynamically correct the motion trajectory data of the second robotic arm, and return to step S6 to re-execute the task actions of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain trajectory adjustment parameters, and using the trajectory adjustment parameters to dynamically correct the motion trajectory data of the second robotic arm.

[0071] In this embodiment, in the task execution and determination stage, corresponding to the start of "Start 2", the process starts from the "Start 2" node, and at this time, it enters the actual task execution and determination link. First, the second robotic arm autonomously executes corresponding actions according to the model inference result or the task instructions in the received task instruction set. For example, in a logistics sorting task, the robot autonomously plans a path and grabs goods based on the identified cargo information. While the robot is executing actions, the system continuously and real-time collects the actual motion trajectory data of the robot, that is, obtains the motion trajectory data of the second robotic arm in real time, and simultaneously executes the first method and the second method respectively. The real-time collected trajectory is compared with the standard action curve obtained by fitting before, and the similarity between the two is calculated; if the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than ninety percent, the second robotic arm completes the task actions of the current instruction. The system can, according to a preset strategy, such as gracefully terminating the inference program, releasing computing resources, and returning to step S5 to execute the task actions of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thus completing the embodied intelligent decision-making control of the second robotic arm. In the case where the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to ninety percent, the second robotic arm does not complete the task actions of the current instruction, and the third method is used to dynamically correct the motion trajectory data of the second robotic arm, and it returns to step S6 to re-execute the task actions of the current instruction.

[0072] This application adopts three different methods and cooperates with each other to meet the requirements of real-time and accurate control.

[0073] Specifically, the first method needs to be executed simultaneously with the second method, otherwise the real-time requirement cannot be met, and both the first method and the second method adopt an improved dynamic time warping algorithm as the basis for calculation.

[0074] Among them, the improved dynamic time warping algorithm includes:

[0075] Step S71.1: Segment the motion trajectory data of the second robotic arm at preset time intervals, and each segment of the trajectory has N real-time trajectory points;

[0076] Step S71.2: In each segment of the trajectory, calculate the distance between each real-time trajectory point and each trajectory point in the standard action curve, and form a distance matrix with all the distances;

[0077] Step S71.3: Through the dynamic programming method, in the distance matrix, calculate the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each segment of the trajectory;

[0078] Step S71.4: Normalize the shortest path;

[0079] Step S71.5: Calculate the similarity between the normalized result and the standard action curve to obtain the similarity between the first real-time trajectory and the standard action curve.

[0080] In this embodiment, during model inference, the real-time sensor continuously collects the motion trajectory data of the second robotic arm, and uses the improved dynamic time warping (DTW) algorithm to calculate the similarity between the real-time trajectory and the fitted standard trajectory. The core idea of the DTW algorithm is to perform non-linear warping on two sequences on the time axis. Under the framework of the traditional DTW algorithm, piecewise approximation is used, that is, the method of dividing the sequence into subsequences is used to find the best alignment between them, and the GPU is used to perform parallel calculations on the relevant operations of the distance matrix, and a distributed framework is used to process large-scale sequences. The improved DTW algorithm can better compare the trajectory data, and through parallel calculation, it can meet the real-time requirements.

[0081] The second method is to transform the motion trajectory data into a sequential decision-making problem, and let the intelligent agent autonomously learn "how to judge the completion of the task" in the interaction with the environment through reinforcement learning, and dynamically adjust the control strategy to optimize the trajectory.

[0082] The transformation of the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem, includes:

[0083] Step S72.1: Segment the motion trajectory data of the second robotic arm at a preset time interval, and each segment of the trajectory has N real-time trajectory points;

[0084] Step S72.2: Take each segment of the trajectory as the state space of the sequential decision-making problem;

[0085] Step S72.3: Take the joint velocity of the second robotic arm and the planned path corresponding to each segment of the trajectory as the action space of the sequential decision-making problem;

[0086] Step S72.4: Calculate the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function according to the state space and the action space;

[0087] Step S72.5: Accumulate the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated for each segment of the trajectory;

[0088] Step S73.6: Take the accumulated function value as the similarity between the second real-time trajectory and the standard action curve.

[0089] Among them, the trajectory similarity reward function has the following calculation formula:

[0090] ;

[0091] Among them, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second robotic arm and the standard action curve is greater than the preset distance threshold (5 millimeters in this embodiment), the value of the trajectory similarity reward function is decreased by 1. If the distance between the motion trajectory data of the second robotic arm and the standard action curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard action curve, and DTW is the improved dynamic time warping algorithm.

[0092] The task completion reward function includes:

[0093] When the end coordinate of the robotic arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is increased by 1;

[0094] When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is increased or decreased by 1.

[0095] The smoothness reward function has the following calculation formula:

[0096] ;

[0097] Among them, is the smoothness reward function. When the joint acceleration mutation of the robotic arm is greater than the preset acceleration threshold (1 meter per second in this embodiment), the value of the smoothness reward function is decreased by 1. When the joint acceleration mutation of the robotic arm is less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the robotic arm at the t-th moment, is the joint acceleration of the robotic arm at the (t - 1)-th moment, and N is the number of real-time trajectory points.

[0098] In this embodiment, when the similarity between the second real-time trajectory and the standard action curve is greater than the cumulative function value of the segmented trajectory of 90%, it is determined that the second real-time trajectory is similar to the standard trajectory of the standard action curve.

[0099] The third method:

[0100] Input the motion trajectory data of the second robotic arm into the reinforcement learning algorithm model to obtain the trajectory adjustment parameters, and dynamically correct the motion trajectory data of the second robotic arm using the trajectory adjustment parameters, including:

[0101] Offline Policy Learning: Using algorithms such as DDPG and SAC, training with the motion trajectory data of the historical second robotic arm to obtain the optimal control policy for learning in different states, and obtaining a reinforcement learning algorithm model.

[0102] Online Real-time Optimization: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, trigger the reinforcement learning algorithm model to generate trajectory adjustment parameters, so that the robotic arm dynamically corrects the path during execution. The trajectory adjustment parameters can dynamically correct the motion trajectory data of the second robotic arm.

[0103] Exploration-Exploitation Balance: Further, through ε-greedy or entropy regularization, explore feasible trajectories in new scenarios while quickly converging using existing successful experiences.

[0104] Among them, the calculation formula and internal structure data of the reinforcement learning algorithm model itself are prior arts, and will not be elaborated in this embodiment.

[0105] Policy Optimization for Task Completion Judgment: Further, transform the traditional judgment rule of "similarity ≥ 90% and joint position tolerance meets the standard" into the output of the policy network, and automatically optimize the judgment threshold through reinforcement learning. For example, in a noisy environment, the policy will adaptively increase the weight of force feedback and reduce the sensitivity to trajectory similarity.

[0106] This embodiment proposes an embodied intelligent decision-making control method based on the motion trajectory of a robotic arm. By fitting the standard motion trajectory of the robotic arm using the least squares method, it can accurately depict the motion characteristics of the robotic arm in the ideal task execution state. Combining the improved dynamic time warping algorithm to calculate the similarity between the real-time trajectory and the standard trajectory, fully considering the dynamic change characteristics of the trajectory in the time dimension, and improving the autonomous decision-making ability through the reinforcement learning reward function method, deeply coupling trajectory monitoring and control strategies to form a complete closed-loop. Compared with simple trajectory comparison methods, it can more accurately measure the degree of compliance between the actual task execution situation of the model and the expectation. Compared with traditional dynamic time warping algorithms, it has a greater improvement in real-time performance and reducing the impact of noise and outliers on the matching results.

[0107] Embodiment 2:

[0108] This embodiment proposes an embodied intelligent decision-making control system based on the motion trajectory of a robotic arm, as Figure 3 shown, including:

[0109] Instruction receiving module, remote control operation module, first data acquisition module, standard curve fitting module, action execution module, second data acquisition module, similarity calculation module, decision-making control module;

[0110] Among them, the instruction receiving module is respectively connected to the remote control operation module and the action execution module. The remote control operation module is connected to the first data acquisition module. The first data acquisition module is connected to the standard curve fitting module. The action execution module is connected to the second data acquisition module. The second data acquisition module is connected to the similarity calculation module. The similarity calculation module is respectively connected to the standard curve fitting and the decision control module. The decision control module is respectively connected to the second data acquisition module and the action execution module;

[0111] The instruction receiving module is used for the first robotic arm and the second robotic arm to receive the same task instruction set;

[0112] The remote control operation module is used to control the first robotic arm by means of remote control according to the order of the task instructions in the task instruction set;

[0113] The first data acquisition module is used to collect the motion trajectory data of the first robotic arm;

[0114] The standard curve fitting module is used to fit the standard action curve according to the motion trajectory data of the first robotic arm;

[0115] The action execution module is used for the second robotic arm to perform the task actions of the current instruction according to the order of the task instructions in the task instruction set;

[0116] The second data acquisition module is used to obtain the motion trajectory data of the second robotic arm in real time;

[0117] The similarity calculation module is used to execute the first method and the second method simultaneously; The first method includes: according to the motion trajectory data of the second robotic arm, using an improved dynamic time warping algorithm to obtain the similarity between the first real-time trajectory and the standard action curve; The second method includes: transforming the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem;

[0118] The decision control module is used to make the second robotic arm complete the task actions of the current instruction and return to the action execution module to execute the task actions of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thereby completing the embodied intelligent decision control of the second robotic arm, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm does not complete the task actions of the current instruction, and a third method is used to dynamically correct the motion trajectory data of the second robotic arm, and then return to the second data acquisition module to re-execute the task actions of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain trajectory adjustment parameters, and dynamically correcting the motion trajectory data of the second robotic arm by using the trajectory adjustment parameters.

[0119] Embodiment 3:

[0120] This embodiment provides an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to execute the described embodied intelligent decision control method based on the motion trajectory of a robotic arm.

[0121] The electronic device may be a mobile phone, a computer, a tablet computer, etc., including a memory and a processor, and a computer program is stored on the memory, and when the computer program is executed by the processor, it implements the embodied intelligent decision control method based on the motion trajectory of a robotic arm as described in the embodiment. It can be understood that the electronic device may further include an input / output (I / O) interface and a communication component.

[0122] Among them, the processor is used to execute all or part of the steps in the described embodied intelligent decision control method based on the motion trajectory of a robotic arm as in the above embodiment. The memory is used to store various types of data, which may include, for example, instructions of any application program or method in the electronic device, and data related to the application program.

[0123] The processor may be implemented by an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is used to execute the method for embodied intelligent decision-making control based on the motion trajectory of the robotic arm described in the above embodiments.

[0124] Embodiment 4:

[0125] This embodiment provides a computer-readable storage medium storing executable instructions, which, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium.

[0126] This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method for embodied intelligent decision-making control based on the motion trajectory of the robotic arm described in various embodiments of the present application.

[0127] The foregoing storage medium includes: flash memory, hard disk, multimedia card, card-type memory (such as SD (Secure Digital Memory Card) or DX (abbreviation for Memory Data Register, MDR), memory data register, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, server, APP (abbreviation for Application, application software), application mall, and other media that can store program check codes. A computer program is stored thereon, and when the computer program is executed by a processor, it can implement each step of the method for embodied intelligent decision-making control based on the motion trajectory of the robotic arm described above.

[0128] The various embodiments in the present application are described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments.

[0129] The protection scope of this application is not limited to the above embodiments. Obviously, those skilled in the art can make various changes and deformations to the present disclosure without departing from the scope and spirit of the present disclosure. If these changes and deformations fall within the scope of the claims of the present disclosure and their equivalent technologies, the intention of the present disclosure also includes these changes and deformations.

Claims

1. An embodied intelligent decision-making control method based on the motion trajectory of a robotic arm, characterized in that, Including: Step S1: The first robotic arm and the second robotic arm receive the same task instruction set. Step S2: According to the order of the task instructions in the task instruction set, control the first robotic arm using the remote operation method. Step S3: Collect the motion trajectory data of the first robotic arm. Step S4: Fit the standard action curve based on the motion trajectory data of the first robotic arm. Step S5: According to the order of the task instructions in the task instruction set, the second robotic arm performs the task actions of the current instruction. Step S6: Real-time obtain the motion trajectory data of the second robotic arm. Step S7: Simultaneously execute the first method and the second method respectively. The first method includes: According to the motion trajectory data of the second robotic arm, using the improved dynamic time warping algorithm to obtain the similarity between the first real-time trajectory and the standard action curve; The second method includes: Transforming the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem. Step S8: When the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than the preset similarity threshold, the second robotic arm completes the task actions of the current instruction, returns to step S5 to execute the task actions of the next instruction in the task instruction set, until the task actions of all instructions in the task instruction set are completed, and the embodied intelligent decision-making control of the second robotic arm is completed. Step S9: When the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm has not completed the task actions of the current instruction, and uses the third method to dynamically correct the motion trajectory data of the second robotic arm, returns to step S6 to re-execute the task actions of the current instruction. The third method includes: Inputting the motion trajectory data of the second robotic arm into the reinforcement learning algorithm model to obtain the trajectory adjustment parameters, and using the trajectory adjustment parameters to dynamically correct the motion trajectory data of the second robotic arm.

2. The embodied intelligent decision-making control method based on the motion trajectory of the robotic arm according to claim 1, characterized in that, The motion trajectory data of the first robotic arm or the motion trajectory data of the second robotic arm includes: the joint positions of the robotic arm, the end coordinates of the robotic arm, and the joint accelerations of the robotic arm.

3. A method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to claim 1, characterized in that The improved dynamic time warping algorithm includes: Segment the motion trajectory data of the second robotic arm at a preset time interval, and each segment of the trajectory has N real-time trajectory points. In each segment of the trajectory, calculate the distance between each real-time trajectory point and each trajectory point in the standard action curve, and form a distance matrix with all the distances. Through the dynamic programming method, in the distance matrix, calculate the shortest path from the first real-time trajectory point to the Nth real-time trajectory point in each segment of the trajectory. Normalize the shortest path. Calculate the similarity between the normalized result and the standard action curve to obtain the similarity between the first real-time trajectory and the standard action curve.

4. An embodied intelligent decision-making control method based on the motion trajectory of a robotic arm according to claim 1, characterized in that, Converting the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem, including: Segmenting the motion trajectory data of the second robotic arm at a preset time interval, with N real-time trajectory points in each segment of the trajectory; Regarding each segment of the trajectory as the state space of the sequential decision-making problem; Regarding the joint speed of the second robotic arm and the planned path corresponding to each segment of the trajectory as the action space of the sequential decision-making problem; Calculating the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function according to the state space and the action space; Accumulating the values of the trajectory similarity reward function, the task completion reward function, and the smoothness reward function calculated for each segment of the trajectory; Taking the accumulated function value as the similarity between the second real-time trajectory and the standard action curve.

5. A method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to claim 4, characterized in that The trajectory similarity reward function, the calculation formula is as follows: ； Among them, is the trajectory similarity reward function. If the distance between the motion trajectory data of the second robotic arm and the standard motion curve is greater than the preset distance threshold, the value of the trajectory similarity reward function is decreased by 1. If the distance between the motion trajectory data of the second robotic arm and the standard motion curve is less than or equal to the preset distance threshold, the value of the trajectory similarity reward function is increased by 1. is the motion trajectory data of the second robotic arm, is the standard motion curve, and DTW is the improved dynamic time warping algorithm.

6. The embodied intelligent decision-making control method based on the motion trajectory of the robotic arm according to claim 4, wherein The task completion reward function, including: When the end coordinate of the robotic arm is less than or equal to the preset tolerance threshold, the value of the task completion reward function is incremented by 1; When the end coordinate of the robotic arm is greater than the preset tolerance threshold, the value of the task completion reward function is incremented or decremented by 1.

7. A method for embodied intelligent decision-making control based on the motion trajectory of a robotic arm according to claim 4, characterized in that The smoothness reward function, the calculation formula is as follows: ； Among them, is the smoothness reward function. When the joint acceleration mutation of the robotic arm is greater than the preset acceleration threshold, the value of the smoothness reward function is decreased by 1. When the joint acceleration mutation of the robotic arm is less than or equal to the preset acceleration threshold, the value of the smoothness reward function is increased by 1. is the joint acceleration of the robotic arm at the t-th moment, is the joint acceleration of the robotic arm at the (t - 1)-th moment, and N is the number of real-time trajectory points.

8. An embodied intelligent decision-making control system based on the motion trajectory of a robotic arm, characterized in that, Including: An instruction receiving module, a remote control operation module, a first data acquisition module, a standard curve fitting module, an action execution module, a second data acquisition module, a similarity calculation module, and a decision control module; Among them, the instruction receiving module is respectively connected to the remote control operation module and the action execution module, the remote control operation module is connected to the first data acquisition module, the first data acquisition module is connected to the standard curve fitting module, the action execution module is connected to the second data acquisition module, the second data acquisition module is connected to the similarity calculation module, the similarity calculation module is respectively connected to the standard curve fitting and the decision control module, and the decision control module is respectively connected to the second data acquisition module and the action execution module; The instruction receiving module is used to receive the same task instruction set for the first robotic arm and the second robotic arm; The remote control operation module is used to control the first robotic arm by means of remote control according to the order of the task instructions in the task instruction set; The first data acquisition module is used to acquire the motion trajectory data of the first robotic arm; The standard curve fitting module is used to fit the standard action curve according to the motion trajectory data of the first robotic arm; The action execution module is used to perform the task actions of the current instruction for the second robotic arm according to the order of the task instructions in the task instruction set; The second data acquisition module is used to obtain the motion trajectory data of the second robotic arm in real time; A similarity calculation module, configured to simultaneously execute the first method and the second method respectively; the first method includes: obtaining the similarity between the first real-time trajectory and the standard action curve according to the motion trajectory data of the second robotic arm by using an improved dynamic time warping algorithm; the second method includes: converting the motion trajectory data of the second robotic arm into a sequential decision-making problem, and obtaining the similarity between the second real-time trajectory and the standard action curve according to the value of the reward function in the sequential decision-making problem. A decision-making control module, configured to, when the similarity between the first real-time trajectory and the standard action curve and the similarity between the second real-time trajectory and the standard action curve are both greater than a preset similarity threshold, cause the second robotic arm to complete the task action of the current instruction, and return to the action execution module to execute the task action of the next instruction in the task instruction set until the task actions of all instructions in the task instruction set are completed, thereby completing the embodied intelligent decision-making control of the second robotic arm; when the similarity between the first real-time trajectory and the standard action curve or the similarity between the second real-time trajectory and the standard action curve is less than or equal to the preset similarity threshold, the second robotic arm does not complete the task action of the current instruction, and the motion trajectory data of the second robotic arm is dynamically corrected by using a third method, and the second data acquisition module is returned to re-execute the task action of the current instruction. The third method includes: inputting the motion trajectory data of the second robotic arm into a reinforcement learning algorithm model to obtain a trajectory adjustment parameter, and dynamically correcting the motion trajectory data of the second robotic arm by using the trajectory adjustment parameter.

9. An electronic device, characterized in that, Comprising: One or more processors, and a memory for storing instructions, which when executed by the one or more processors, cause the one or more processors to execute an embodied intelligent decision-making control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions, which when executed cause the processor to execute an embodied intelligent decision-making control method according to any one of claims 1 to 7.

Images