Mechanical arm trajectory planning method and system and medium

Abstract

The invention discloses a mechanical arm trajectory planning method and system and a medium, and belongs to the field of mechanical arm trajectory planing.The method comprises the steps that an image set of injection objects in different postures is collected, and trajectory data of mechanical arm injection are collected; marking an injection area in the image set, and training a target recognition model based on YOLOv8 by using the marked image set; recognizing an injection area in the current image set by using a target recognition model, and taking the center point of the injection area as an injection point; an improved dynamic motion primitive DMP algorithm is used for planning the motion track of the mechanical arm returning to the current position after injection is completed from the current position to the injection point; the improved dynamic motion primitive DMP algorithm is obtained by introducing a speed item scaling factor and superposing a plurality of nonlinear Gaussian radial basis functions; in view of easy failure of a mechanical arm motion trajectory planning algorithm in the prior art, the adaptability of the algorithm to the situation that the head and the tail of the trajectory are the same is improved.

Description

Technical Field

[0001] The present application relates to the field of robotic arm planning, and more specifically, to a robotic arm trajectory planning method, system, and medium. Background Art

[0002] With the rapid development of artificial intelligence technology, robots have become one of the important symbols of intelligent manufacturing. Robot technology has received extensive attention. Especially in fields such as healthcare and services, the application of robots has become increasingly in-depth, posing higher requirements for the intelligence level of robots. To improve the environmental adaptability of robots and the flexibility of task execution, it is urgent to study more efficient and robust robot trajectory planning methods.

[0003] Imitation learning is a research hotspot in the field of robotics. By teaching, robots learn human motion trajectories and generalize the trajectories so that robots can master corresponding motion skills. Among them, the Dynamic Movement Primitive (DMP) algorithm proposed by Ijspeert et al. in 2002 is a new type of imitation learning method. Compared with traditional methods, DMP has the advantages of continuous and smooth trajectories, easy implementation of multi-degree-of-freedom coupling, and low computational complexity, showing good universality in the field of non-linear trajectory learning. However, the classical DMP algorithm still has some deficiencies. For example, when the starting and ending points of the running trajectory are the same, the algorithm is prone to failure, resulting in a low success rate of trajectory planning.

[0004] Chinese Patent Application, Application No. CN111618847B, Publication Date September 4, 2020, discloses a robotic arm autonomous grasping method based on deep reinforcement learning and dynamic movement primitives, including the following steps: installing a camera module, ensuring that the recognition area is not blocked, preprocessing the image of the grasping target area, and sending it as state information to the deep reinforcement learning agent; constructing a local policy proximal optimization training model based on the state and deep reinforcement learning principle; fusing dynamic movement primitives and imitation learning to construct a new hybrid movement primitive model; training the robotic arm to autonomously grasp an object based on the model. The present application can effectively solve the problem of uneven joint movement of the robotic arm based on traditional deep reinforcement learning. By combining the dynamic movement primitive algorithm, the learning problem of meta-parameters is transformed into a reinforcement learning problem, and the training method of deep reinforcement learning can be used to enable the robotic arm to complete the autonomous grasping task. However, although this solution optimizes the parameter learning of the dynamic movement primitive using deep reinforcement learning and improves the smoothness of trajectory generation, it does not specifically address the problem of algorithm failure caused by the coincidence of the beginning and end of the running trajectory. When the grasping task requires the robotic arm to complete a closed trajectory movement and return to the initial position, this method may face the risk of planning failure. Summary of the Invention

[0005] 1. Technical problems to be solved

[0006] Aiming at the problem that the robotic arm motion trajectory planning algorithm in the prior art is prone to failure, this application provides a robotic arm trajectory planning method, system and medium. At the same time, the controller and trajectory shape learner of the dynamic movement primitive algorithm are improved, and the adaptability of the algorithm to the situation where the beginning and end of the trajectory are the same is improved.

[0007] 2. Technical solutions

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] One aspect of this application provides a robotic arm trajectory planning method, including: S1, collecting an image set of injection objects in different postures, and collecting the trajectory data of the robotic arm injection; the image set includes RGB images and depth images; S2, preprocessing the collected trajectory data to obtain an optimal reference path; S3, annotating the injection area of the preprocessed image set, and training a target recognition model based on YOLOv8 using the annotated image set; S4, training an improved dynamic movement primitive DMP algorithm using the optimal reference path; S5, using the target recognition model to identify the injection area in the current image set, and taking the center point of the injection area as the injection point; S6, using the trained improved dynamic movement primitive DMP algorithm to plan the movement trajectory of the robotic arm from the current position to the injection point and then returning to the current position after injection.

[0010] Among them, the depth image is a special form of image that records the distance information from each pixel point in the image to the camera. In the robotic arm trajectory planning method of this application, the depth image and the RGB image together constitute a complete image set. By analyzing the color image and the depth image, the position and posture information of the injection object in the three-dimensional space can be obtained. These information are crucial for the motion planning of the robotic arm, because the robotic arm needs to adjust its joint angles and the pose of the end effector according to the three-dimensional coordinates of the injection point to achieve precise injection operations. The spatial information provided by the combination of the color image and the depth image enables the robotic arm to accurately locate the injection target in a complex environment, improving the success rate and stability of injection.

[0011] The Dynamic Movement Primitive (DMP) algorithm is a machine learning algorithm for trajectory encoding and generation. In the robotic arm trajectory planning method of this application, a velocity term scaling factor is introduced into the PD controller of the DMP, enhancing the velocity regulation ability; multiple non-linear Gaussian radial basis functions are superimposed in the trajectory shape learner, improving the fitting accuracy of the trajectory shape. By preprocessing the collected trajectory data, an optimal reference path is obtained, and the improved piecewise DMP algorithm is trained with it, enabling the algorithm to learn the motion characteristics of the taught injection trajectory.

[0012] Further, in S4, the improved piecewise dynamic movement primitive DMP algorithm includes: decomposing the dynamic movement primitive DMP algorithm into a PD controller and a trajectory shape learner f; the PD controller is used to control the position state and velocity of the robotic arm; the trajectory shape learner f is used to control the shape characteristics of the trajectory; by setting a velocity term scaling factor in the velocity, an improved PD controller is obtained; by superimposing multiple non-linear Gaussian radial basis functions in the trajectory shape learner f, an improved trajectory shape learner f is obtained; according to the improved PD controller and the improved trajectory shape learner f, an improved piecewise dynamic movement primitive DMP algorithm is obtained.

[0013] Among them, the PD controller (Proportional-Derivative Controller) is a feedback controller. In the DMP algorithm, the PD controller is used to control the position state and velocity of the robotic arm to make it track the reference trajectory. The trajectory shape learner is the core component of the DMP algorithm, used to learn and encode the shape characteristics of the taught trajectory, and reconstruct a non-linear term similar to the shape of the taught trajectory during trajectory generation. In the traditional DMP algorithm, the trajectory shape learner usually uses the form of weighted linear combination. In the improved DMP algorithm proposed in this application, the trajectory shape learner is improved by introducing multiple non-linear Gaussian radial basis functions to obtain an improved trajectory shape learner.

[0014] On the one hand, the PD controller of the traditional DMP algorithm only adjusts the system state through the linear combination of the position error and the velocity error, and the adjustment ability is limited. By introducing a scaling factor in the velocity term to form an improved PD controller, additional scaling adjustment can be applied to the velocity term while ensuring the convergence of the position error. This enables the controller to more flexibly control the motion velocity of the robotic arm, speed up or slow down the velocity according to actual needs, and improve the accuracy and stability of trajectory tracking. The enhanced velocity regulation ability expands the application scenarios of the DMP algorithm, enabling it to adapt to tasks with special requirements for the velocity of the robotic arm.

[0015] On the other hand, the traditional DMP algorithm uses the form of weighted linear combination to represent the trajectory shape learner. When the running trajectory is relatively complex, it is often difficult to guarantee the fitting accuracy. By superimposing multiple non-linear Gaussian radial basis functions in the trajectory shape learner, the fitting ability of the learner is significantly improved. The Gaussian radial basis function is a powerful local receptive field function and is highly sensitive to local changes in input variables. By arranging multiple Gaussian kernel functions in the phase space and performing non-linear superposition, the improved trajectory shape learner can accurately approximate any complex continuous function. Even if there are sharp changes or strong non-linearity in the running trajectory, high-precision coding learning can be achieved. The improvement in the fitting ability enables the improved DMP algorithm to handle more diverse trajectory planning tasks and meet the requirements of different processes for trajectory shapes.

[0016] On the other hand, in some application scenarios, the starting and ending states of the running trajectory may be exactly the same. For example, a robotic arm needs to move along a closed trajectory and return to the initial position. For the traditional DMP algorithm, when the starting and ending states are the same, its phase space will degenerate into a single attractor point, losing the ability to generate continuous trajectories and causing the algorithm to fail. The improved DMP algorithm cleverly uses the path splicing strategy, taking the ending state of one trajectory as the starting parameter of another trajectory to construct a composite trajectory with continuous states. In this way, the improved algorithm breaks through the limitations of the traditional DMP, realizes continuous and stable planning of closed trajectories, effectively solves the problem of algorithm failure caused by the same start and end of the running trajectory, and greatly expands the applicable range of the DMP algorithm.

[0017] Furthermore, the improved PD controller is:

[0018]

[0019] where y represents the position state, τ is the time scaling factor, α y and β y are the adjustment parameters in the PD controller, g represents the target position of the robotic arm, and are the first derivative and the second derivative of y respectively.

[0020] The improved trajectory shape learner f is:

[0021]

[0022] where ω i represents the weight value corresponding to each basis function; s is the phase variable of the first-order system; X is the number of basis functions, is the radial basis function;

[0023]

[0024] where, σ and c i respectively represent the width and center of the basis function respectively.

[0025] In the traditional dynamic movement primitive DMP algorithm, the trajectory shape learner usually uses linearly weighted radial basis functions to represent, which has the defect of insufficient expressive ability and is difficult to fit complex trajectories well. In this application, by stacking multiple non-linear Gaussian radial basis functions in the trajectory shape learner, on the one hand, the Gaussian radial basis function is a commonly used non-linear function. By stacking multiple Gaussian radial basis functions with different widths and centers, various complex non-linear trajectories can be fitted, greatly enhancing the expressive ability of the trajectory shape learner. On the other hand, each Gaussian radial basis function only has a large response near its center, and the response decays rapidly at a distance from the center. This characteristic of local response enables each basis function to depict the characteristics of the local trajectory shape respectively, thereby improving the local generalization performance of the trajectory shape learner.

[0026] Further, in S5, using the target recognition model to identify the injection area in the current image set, and taking the center point of the injection area as the injection point, includes: using the target recognition model to identify the injection area in the RGB image, and extracting the center coordinates of the injection area; calculating the three-dimensional coordinates of the center coordinates in the camera coordinate system according to the depth information corresponding to the center coordinates of the injection area in the depth image; converting the three-dimensional coordinates of the center coordinates in the camera coordinate system to the robotic arm coordinate system to obtain the three-dimensional coordinates of the center point of the injection area in the robotic arm coordinate system as the injection point.

[0027] Further, in S6, using the improved segmented dynamic movement primitive DMP algorithm after training to plan the movement trajectory of the robotic arm from the current position to the injection point and then return to the current position after injection, includes: taking the current position of the robotic arm as the starting point and the injection point as the target point, and using the improved segmented dynamic movement primitive DMP algorithm to plan the movement trajectory of the robotic arm from the starting point to the target point as the first path; taking the injection point as the starting point and the current position of the robotic arm as the target point, and using the improved segmented dynamic movement primitive DMP algorithm to plan the movement trajectory of the robotic arm from the starting point to the target point as the second path; taking the state parameters of the first path at the target point as the state parameters of the second path at the starting point; connecting the first path and the second path to obtain the movement trajectory from the current position of the robotic arm to the injection point, completing the injection and returning to the current position.

[0028] Further, the state parameters include position, velocity and phase;

[0029] Further, in S2, preprocessing the collected trajectory data to obtain the optimal reference path, includes:

[0030] The collected trajectory data is smoothed using the Moving Average Filter (MAF) algorithm. The originally collected running trajectory data often contains noise and jitter. Directly using this data for trajectory planning is likely to cause the generated trajectory to oscillate or distort, affecting the smoothness of the robotic arm's movement. By smoothing the trajectory data using the moving average filter algorithm, high-frequency noise can be effectively suppressed, making the trajectory curve smoother and more continuous, providing a more reliable input for subsequent processing.

[0031] The Dynamic Time Warping (DTW) algorithm is used to align the time steps of the smoothed trajectory data. In some embodiments, by collecting artificial running trajectory data as training data, in this kind of trajectory data, due to the speed changes and pauses during the manual teaching process, the time step lengths of different running trajectories are usually inconsistent. Directly encoding and learning these trajectories will cause a mismatch in the time dimension, affecting the learning effect. Using the Dynamic Time Warping (DTW) algorithm to align the time steps of the smoothed trajectory can eliminate the differences in time scales, making different trajectories synchronized in the time dimension, creating conditions for subsequent encoding and regression.

[0032] Multiple trajectory data with aligned time steps are processed using Gaussian Mixture Modeling (GMM) encoding. Inputting the multiple trajectory data with aligned time into the Gaussian mixture model for encoding can extract the probability distribution characteristics of the trajectories from the trajectory data. The Gaussian mixture model is a general density estimation method that can fit any complex probability distribution through the linear combination of multiple Gaussian components. Through GMM encoding processing, the discrete running trajectory data is transformed into a continuous probability distribution function, which not only reduces the data storage volume but also facilitates the probability sampling and generation of trajectories.

[0033] Based on the trajectory data encoded by GMM, a Gaussian Mixture Regression (GMR) model is used to generate a trajectory with uniformly distributed trajectory points as the optimal reference path. On the basis of GMM encoding, using the Gaussian mixture regression model to regress the encoded trajectory data can obtain a reference path with uniformly distributed trajectory points. Compared with the original running trajectory, this reference path inherits the motion characteristics of the teaching data, but the node distribution is more uniform and the arc length parameterization effect is better. This provides high-quality training samples for the subsequent parameter estimation of the trajectory shape learner, helping to improve the fitting accuracy and generalization ability of the shape learner. At the same time, the reference path can also be used as the target state of trajectory planning to guide the robotic arm to achieve smooth and accurate movement.

[0034] Another aspect of the embodiments of this specification also provides a robotic arm trajectory planning system, including an acquisition module that acquires an image set including RGB images and depth images of an injection object in different poses, as well as trajectory data of the robotic arm injection; a preprocessing module that performs trajectory smoothing processing and trajectory time step alignment processing on the trajectory data and obtains an optimal reference trajectory; an image annotation module that annotates the injection areas in the image set; and a path planning module that improves the dynamic movement primitive (DMP) algorithm and uses the improved piecewise movement primitive (DMP) algorithm to plan the movement trajectory of the robotic arm from the current position to the center point of the injection area to complete the injection and return to the current position.

[0035] 3. Beneficial effects

[0036] Compared with the prior art, the advantages of this application are as follows:

[0037] (1) A scaling factor α is introduced into the velocity term of the PD controller y , forming an improved controller equation. The traditional PD controller only adjusts the system state through the linear combination of the position error and the velocity error, and its adjustment ability is limited. After introducing the velocity scaling factor, while maintaining the convergence of the position error, an additional scaling adjustment can be applied to the velocity term, enabling the controller to more flexibly control the movement speed of the robotic arm. The velocity scaling factor can be set according to actual needs. When α y > 1, the movement of the robotic arm can be accelerated; when α y < 1, the movement can be slowed down; when α y = 1, it is equivalent to the traditional PD controller. By reasonably adjusting α y , the robotic arm can exhibit matching movement speed characteristics at different stages, such as appropriately decelerating when approaching the target point, improving the accuracy and stability of trajectory tracking. Therefore, the improved PD controller endows the DMP algorithm with stronger speed adjustment ability, enabling it to adapt to a wider range of application scenarios.

[0038] (2) In the trajectory shape learner f, multiple Gaussian radial basis functions are superimposed to fit the shape features of the running trajectory. The traditional DMP algorithm usually represents the shape learner in the form of a weighted linear combination. When the running trajectory is complex, it is difficult to guarantee the fitting accuracy. After introducing the Gaussian radial basis function, the fitting ability of the learner is greatly improved. The Gaussian radial basis function is a local receptive field function, which is very sensitive to the local changes of input variables. By arranging multiple Gaussian kernel functions in the phase space, any continuous function can be well approximated. At the same time, the Gaussian radial basis function also has good mathematical properties, and its parameters can be estimated by simple least squares method. The training process is efficient and easy to implement. By non-linearly combining multiple Gaussian radial basis functions, the improved trajectory shape learner can accurately describe the shape features of the running trajectory. Even for complex trajectories with sharp changes or strong non-linearity, high-precision coding learning can be achieved.

[0039] (3) For the special case where the start and end points of the running trajectory coincide, the traditional DMP algorithm is prone to problems of trajectory planning failure. This is due to the inherent time evolution mechanism of the classical DMP algorithm. When the start and end states are the same, the phase space of the DMP degenerates into a single attractor point, losing the ability to generate continuous trajectories. To address this problem, the improved DMP algorithm cleverly utilizes the idea of path splicing. By setting the end state of one trajectory as the start parameters of another trajectory, a composite trajectory with continuous states is constructed. This strategy breaks through the limitations of the classical DMP algorithm, enabling the robotic arm to autonomously plan a circular closed trajectory and smoothly return to the initial position. Description of the Drawings

[0040] Figure 1 is a flowchart of a robotic arm trajectory planning method of the present application;

[0041] Figure 2 is the performance curve of YOLOv8 adopted by the present application;

[0042] Figure 3 is the optimal trajectory diagram of the robotic arm obtained by the present application;

[0043] Figure 4 is a demonstration diagram of the learning generalization of the improved dynamic movement primitive algorithm of the present application. Detailed Embodiment

[0044] The present application will be described in detail below in conjunction with the drawings in the specification and specific embodiments.

[0045] Refer to Figure 1, A robotic arm trajectory planning method, comprising the following steps: Step 1: Collect RGB images and Depth images of injection objects in different poses, which are used as the training set and validation set for target recognition; Step 2: Annotate the injection areas in the dataset images and input them into the YOLOv8 target recognition network model for recognition training to obtain a target recognition model that can locate the center point position of the injection area on the injection object; Step 3: Collect the process trajectory of an expert dragging the robotic arm to complete the injection in manual mode, preprocess the obtained trajectory, and use it as the operating trajectory to provide a reference for the robotic arm motion trajectory planning; Step 4: Use the improved segmented dynamic movement primitive algorithm to encode and learn the collected operating trajectory to obtain a robotic arm motion trajectory model with expert injection characteristics; Step 5: Use the target recognition model to recognize the injection area in the current image of the camera, take the center point position of the injection area as the segmentation point of the robotic arm motion trajectory, obtain the current position of the robotic arm end, and plan the motion trajectory of the robotic arm end from the current position to the injection point to complete the injection and then return to the current position through the robotic arm motion trajectory model; Step 6: The robotic arm starts to move, starting from the current position, following the imitation trajectory learned through algorithm encoding, controlling the robotic arm end to move to the injection point in a specific pose imitating the expert injection process, and after completing the injection task, returning to the initial position of the robotic arm according to the planned trajectory, waiting to execute the next task.

[0046] Refer to Figure 2 , Steps 1 and 2 are to use the YOLOv8 algorithm to train a target recognition model to recognize the injection area. The YOLO algorithm is a real-time target detection system that predicts target bounding boxes and categories through a single neural network, and uses a single CNN model to achieve end-to-end target detection. It is built on the success of previous YOLO versions, introducing new features and improvements to further enhance performance and flexibility. Specific innovations include a new backbone network, a new Anchor-Free detection head, and a new loss function. Compared with the previous YOLOv5 and YOLOv7, the YOLOv8 algorithm achieves an excellent balance between speed and accuracy and is suitable for application scenarios that require fast detection.

[0047] Refer to Figure 3 , Step 3 includes: In this embodiment, the process trajectory of an expert dragging the robotic arm to complete the injection in manual mode is collected as trajectory data, and the trajectory data is used to train the improved DMP algorithm to control the robotic arm to execute the injection task. The trajectory recording format is: ε i ={i,ε s,i} N , where N represents the current number of demonstration trajectories, i = {1, 2,..., T} is the time step sequence demonstrated by the expert, T represents the total number of time steps in this demonstration, and ε iRepresents the robotic arm demonstration trajectory data obtained corresponding to the i-th time step of the N-th demonstration.

[0048] Refer to Figure 3 , in step three, the preprocessing of the multi-segment expert demonstration robotic arm trajectory data collected includes trajectory smoothing processing, trajectory time step alignment processing, and obtaining the optimal trajectory. The moving average filter MAF algorithm is used to smooth the expert's running trajectory to eliminate the zigzag disturbances in the trajectory caused by environmental factors during sampling of the expert's running trajectory and reduce the impact on the subsequent generalized trajectory. The main idea of MAF is to set a sliding window with a fixed width. Assume that the continuously obtained N sampling values are regarded as a queue with a fixed length. Each time a new data is sampled and placed at the end of the queue, while discarding a data at the head of the queue. Then, an arithmetic average operation is performed on the N data in the queue to obtain a new filtering result, that is, the output filtering sequence. Let the width be N = 2k + 1, and the input and output sequences be x(n) and y(n) respectively. The filtered sequence is:

[0049]

[0050] Refer to Figure 3 , in step three, the dynamic time warping DTW algorithm is used to perform time alignment processing on the trajectory to avoid the final trajectory learning generalization effect being affected because some trajectory sequences with similar spatial characteristics cannot be aligned in time steps. Multiple running trajectories have similar motion characteristics. The Gaussian Mixture Model (GMM) is used to encode the running trajectories, and then the GMR regression model is used to generate an optimal motion trajectory that includes the motion characteristics of the running trajectories and has a uniform distribution of trajectory points.

[0051] Refer to Figure 4 , step four includes: using the improved dynamic movement primitive DMP algorithm to perform encoding learning on the collected running trajectories. The dynamic movement primitive DMP algorithm can be regarded as the superposition of a proportional-derivative controller (PD controller) and a trajectory shape learner. The most commonly used and simplest PD controller can be represented by a spring-damper system:

[0052]

[0053] Among them, y represents the state of the system, and are the first derivative and the second derivative of y respectively, g represents the target state to which the system finally converges, and the parameters α y and β y can be regarded as two adjustment parameters in the PD controller, and τ is the time scaling factor. Assume that f is a non-linear function, representing the trajectory shape learner, then the DMP can be expressed as:

[0054]

[0055] By changing the target end point g and the nonlinear term f in equation (3), the end point and shape of the learned trajectory can be adjusted. To obtain the desired trajectory shape, different nonlinear terms f need to be constructed. Therefore, it can be achieved by normalized linear superposition of multiple nonlinear basis functions, and the function can be expressed as:

[0056]

[0057] where s is a phase variable from a 1st-order system, ω i Represents the weight value corresponding to each basis function, X is the number of basis functions, Defined as a radial basis function, its function is expressed as:

[0058]

[0059] Where σ and c i Represents basis functions By changing the width and center of different basis functions and their corresponding weights, complex trajectories can be obtained by weighting, so as to obtain the desired trajectory shape. Generally speaking, the complexity of the trajectory is proportional to the number of basis functions. If you want to change the convergence speed of the trajectory, you also need to change the first-order derivative of the system state representing the speed. Add a scaling term τ to the above, and the complete formula is expressed as:

[0060]

[0061] Through formula (6), we can get the learning trajectory of different target endpoints and convergence speeds. Formula (6) can also be regarded as the basic formula of the DMP algorithm. Through this algorithm, the running trajectory is encoded and learned to obtain a new trajectory. This trajectory has the action characteristics of the running trajectory and has good generalization performance.

[0062] When applying the dynamic motion primitive algorithm to the vaccine injection scenario, in order to simplify the operation process, the robot arm is defined to return to the initial position after completing the task. At this time, the starting point and end point of the robot arm's motion trajectory are the same, the generalized trajectory is always the origin, and the dynamic motion primitive method fails. Combined with the above formula for analysis, it can be seen from the properties of the dynamic motion primitive that the motion process of the learning trajectory is determined by the forcing function f. At that time, the gy term in formula (6) is always 0, so the forcing function is always 0, so that the forcing function cannot change the motion trend of the learning trajectory. The learning trajectory moves from point y to the end point g according to the initial motion law of the second-order system, that is, the motion law of the critical damping model. Since the two points are in the same position, the obtained learning trajectory is always a point.

[0063] When the starting and ending points of the robotic arm's movement are at the same point, the dynamic motion algorithm fails and cannot learn and generalize new motion trajectories. Considering the characteristics of vaccine injection, this paper proposes a robotic arm trajectory planning method based on vision and dynamic motion primitives. Combining the characteristics of the vaccine injection process, the injection points recognized by vision are used as the trajectory segmentation point set, and the dynamic motion primitive algorithm is used to complete the robotic arm task trajectory planning and the robotic arm injection task.

[0064] As can be seen from Equation (2), this equation is a process equation. If is used as the state variable and written in matrix form as:

[0065]

[0066] The real part of the poles of the coefficient matrix in the formula needs to select appropriate α y and β y parameters to make it less than 0 to ensure the convergence of the formula. As time goes by during the learning process, the equation will converge to:

[0067]

[0068] As can be seen from Equation (8), the state variable will finally converge to the target end point g, that is, using the DMP algorithm can ensure that the system finally converges to the target state. On this premise, can be obtained by using the given f in Equation (4). After obtaining it, set the new starting position and target end point and reset the phase variable s, and then use Equation (6) to obtain the learning trajectory to the newly set target end point.

[0069] Therefore, for complex application scenarios, combining the characteristic that the DMP algorithm can converge to a specified target point, this paper proposes to use an improved segmented DMP algorithm for encoding and learning. Let the given running trajectory be Y ori , the starting point of the learning trajectory to be generated is y 0 , the target end point is g, and a learning trajectory reaching the specified target point can be obtained through encoding and generalization learning. At the same time, calculate the segmentation point b of the injection point to be reached in the scenario, and divide the running trajectory into 2 segments with the segmentation point as the demarcation point:

[0070] Y ori =(y ori1 ,y ori2 ) (9)

[0071] Set the starting point and target end point coordinates of the two trajectories respectively as:

[0072] Y 0 =[y o1 y o2 T (10)

[0073] G = [g 1 g 2 T (11)

[0074] During the learning process, the target end point g of the front-segment learning trajectory is set according to the target injection point 1 , so that it generates a motion trajectory that can reach the injection point and is as similar as possible to the running trajectory. Then, when learning the rear-segment trajectory, the target end point g 2 is set as the end point of the original running trajectory to ensure that the reproduced trajectory finally returns to the original trajectory and converges to the original target point. Since the DMP algorithm converges to the target end point g during the dynamic control process, when performing segmented learning, only the end point of the front-segment learning trajectory needs to be recorded and used as the initial state quantity of the rear-segment learning trajectory, and a complete learning trajectory can be directly obtained.

[0075] Referring to Figure 4 , Step 5 includes obtaining an image through the RGB-D camera on the robotic arm, the target recognition model recognizes the injection area in the color image, and the three-dimensional coordinates of the injection point in the camera coordinates can be obtained through the center coordinates of the injection area and its depth information in the depth image. The three-dimensional coordinates of the injection point are converted to obtain the three-dimensional coordinates in the robotic arm coordinates. At the same time, this coordinate point is the segmentation point of the robotic arm motion trajectory, and the end effector of the robotic arm completes the injection task at this point; according to the optimal motion trajectory model, start planning the trajectory in the current state to obtain the motion trajectory of the end of the robotic arm, and the corresponding continuous trajectories of other joints of the robotic arm are obtained through the inverse kinematics of the robotic arm.

[0076] Step 6 includes: the robotic arm starts to execute the injection task according to the planned trajectory: the robotic arm is already in a posture suitable for observing the injection object. Starting from the current position, the end effector of the robotic arm moves to the injection point according to the motion characteristics of the expert injection process, and returns to the initial position of the robotic arm after completing the injection task, waiting for the next injection process.

[0077] ​The present invention and its implementation manners are schematically described above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Any reference signs in the claims should not limit the claimed claims. Therefore, if those of ordinary skill in the art are inspired by it and, without departing from the gist of the present invention, design similar structural manners and embodiments to the technical solution without creative efforts, they shall fall within the protection scope of this patent. In addition, the term "including" does not exclude other elements or steps, and the term "a" before an element does not exclude including "a plurality of" such elements. The plurality of elements stated in the product claims can also be implemented by one element through software or hardware. The terms such as "first" and "second" are used to indicate names and do not indicate any particular order.

Claims

1. A robot arm trajectory planning method, characterized in that: include: Collect image sets of injection objects in different postures, and collect trajectory data of robot arm injection; The image set includes an RGB image and a depth image; Label the injection area in the image set and use the labeled image set to train the YOLOv8-based object recognition model; Using the target recognition model to identify the injection area in the current image set, and taking the center point of the injection area as the injection point; An improved dynamic motion primitive DMP algorithm is used to plan the motion trajectory of the robot arm from the current position to the injection point and then returning to the current position after completing the injection; the improved dynamic motion primitive DMP algorithm is obtained by introducing a velocity term scaling factor and superimposing multiple nonlinear Gaussian radial basis functions.

2. The robot arm trajectory planning method according to claim 1, characterized in that: Improved dynamic motion primitive (DMP) algorithm, including: The dynamic motion primitive DMP algorithm is decomposed into a PD controller and a trajectory shape learner f; the PD controller is used to control the position state and speed of the robot arm; the trajectory shape learner f is used to control the shape characteristics of the trajectory; By setting the speed term scaling factor in the speed, an improved PD controller is obtained; By superimposing multiple nonlinear Gaussian radial basis functions in the trajectory shape learner f, an improved trajectory shape learner f is obtained; According to the improved PD controller and the improved trajectory shape learner f, an improved segmented dynamic motion primitive DMP algorithm is obtained.

3. The robot arm trajectory planning method according to claim 2, characterized in that: The improved PD controller is: Among them, y represents the position state, τ is the time expansion factor; α y and β y is the adjustment parameter in the PD controller; g represents the target position of the robot; and are the first and second derivatives of y respectively.

4. The robot arm trajectory planning method according to claim 2, characterized in that: The improved trajectory shape learner f is: Among them, ω i represents the weight value corresponding to each basis function; s is the phase variable of the first-order system; X is the number of basis functions, is the radial basis function; Among them, σ and c i Represents basis functions Width and center.

5. The robot arm trajectory planning method according to claim 1, characterized in that: The target recognition model is used to identify the injection area in the current image set, including: The target recognition model is used to identify the injection area in the RGB image and extract the center coordinates of the injection area; According to the depth information corresponding to the center coordinates of the injection area in the depth image, the three-dimensional coordinates of the center coordinates in the camera coordinate system are calculated; The three-dimensional coordinates of the center coordinates in the camera coordinate system are converted to the robot arm coordinate system, and the three-dimensional coordinates of the center point of the injection area in the robot arm coordinate system are obtained as the injection point.

6. The robot arm trajectory planning method according to claim 5, characterized in that: The improved dynamic motion primitive DMP algorithm is used to plan the motion trajectory of the robot arm from the current position to the injection point and then return to the current position after completing the injection, including: Taking the current position of the robot as the starting point and the injection point as the target point, the improved segmented dynamic motion primitive DMP algorithm is used to plan the motion trajectory of the robot from the starting point to the target point as the first path; Taking the injection point as the starting point and the current position of the robot as the target point, the improved segmented dynamic motion primitive DMP algorithm is used to plan the motion trajectory of the robot from the target point back to the initial position as the second path; The state parameters of the first path at the target point are used as the state parameters of the second path at the starting point; The first path and the second path are connected to obtain a motion trajectory starting from the current position of the robot arm to the injection point to complete the injection and then return to the current position.

7. The robot arm trajectory planning method according to claim 6, characterized in that: State parameters include position, velocity and phase.

8. The robot arm trajectory planning method according to any one of claims 2 to 7, characterized in that: Before planning the motion trajectory using the improved dynamic motion primitive DMP algorithm, it also includes: The collected trajectory data is smoothed using the moving average filter (MAF) algorithm; The dynamic time warping (DTW) algorithm is used to align the time steps of the smoothed trajectory data. Perform Gaussian mixture coding (GMM) processing on multiple trajectory data after time step alignment; According to the trajectory data after GMM coding, the Gaussian mixture regression model GMR is used to generate a trajectory with uniformly distributed trajectory points as the optimal reference path; The improved segmented dynamic motion primitive (DMP) algorithm is trained using the optimal reference path.

9. A robot arm trajectory planning system, characterized in that: include: The acquisition module collects image sets including RGB images and depth images of injection objects in different postures, as well as trajectory data of the robot arm injection; The preprocessing module performs trajectory smoothing and trajectory time step alignment on the trajectory data to obtain the optimal reference trajectory; An image annotation module that annotates the injection area in the image set; The path planning module improves the dynamic motion primitive DMP algorithm and uses the improved segmented dynamic motion primitive DMP algorithm to plan the motion trajectory of the robot arm from the current position to the center point of the injection area to complete the injection and return to the current position.

10. A computer-readable storage medium storing computer instructions, which implement the method according to any one of claims 1 to 8 when executed by a processor.

Images