Compliant picking motion planning method, system, product, device and storage medium

Abstract

This invention relates to a compliant harvesting motion planning method, system, product, equipment, and storage medium, belonging to the field of humanoid compliant harvesting technology for robotic arms. It solves the technical problem that existing technologies cannot flexibly adjust the trajectory and control the posture of individual fruits in different positions and postures, leading to conflicts between pose requirements and joint execution in harvesting scenarios, and weak adaptability to complex trajectories. The method involves collecting demonstration data; constructing a policy function, inputting the current end-effector pose state of the robotic arm to obtain the probability distribution of robotic arm movements, and generating the current action; constructing a multi-dimensional sensory neural reward network to calculate the current total reward; constructing a value function, combining the current total reward to obtain the value advantage of the current action; calculating the value loss and policy loss, updating the policy function and value function, and then using the robotic arm to execute the current action. This invention is used to achieve compliant harvesting of fresh fruit.

Description

Technical Field

[0001] This invention relates to the field of robotic arm humanoid compliant harvesting technology, specifically to compliant harvesting motion planning methods, systems, products, equipment, and storage media. Background Technology

[0002] Fruit picking robots have become a core element of smart agriculture due to their unique advantage of replacing manual labor. By developing and applying fresh fruit picking robots, the production and application needs of standardized greenhouse and orchard planting scenarios can be met.

[0003] In the prior art, Chinese patent document CN116985150A discloses a "method and apparatus for planning the compliant collection trajectory of bunched fruits". This method includes: determining a target spatial point set on the collection path of the bunched fruits to be collected, the target spatial point set including start and end spatial points and at least one intermediate spatial point; using the target spatial point set to segment the reference trajectory for collecting the bunched fruits, obtaining multiple segmented trajectories and a reference trajectory segment corresponding to each segmented trajectory; using each segmented trajectory to perform imitation learning calculations on its corresponding reference trajectory segment, determining the optimal learning trajectory corresponding to each segmented trajectory, and splicing together each optimal learning trajectory to generate the optimal collection trajectory for the bunched fruits to be collected. However, this technical solution is a motion trajectory morphology description method based on teaching data, emphasizing the structural expression and reproduction of the trajectory. When facing different positions and postures of each fruit, it cannot perform flexible trajectory adjustment and posture control for harvesting planning. In the harvesting scenario, there is a conflict between pose requirements and joint execution, resulting in weak adaptability to complex trajectories.

[0004] In the prior art, Chinese patent document CN114029945A discloses "a grasping path control method for a robotic arm for picking spherical fruits." When the obtained end-effector posture cannot yield an inverse kinematic solution, a genetic algorithm is used to solve for the end-effector posture that can reach the fruit location. This is then repeated by moving the picking robot to improve the success rate of obtaining the inverse kinematic solution, thereby increasing the picking success rate. However, this technical solution uses quaternions to calculate a fixed posture through coordinate transformation and vector rotation formulas, outputting a unique solution. It is only suitable for scenarios with a clear target and an unchanging environment.

[0005] In summary, existing technologies have limitations in flexibly adjusting the trajectory and controlling the posture of individual fruits in different positions and postures during harvesting. This results in conflicts between position and posture requirements and joint execution in harvesting scenarios, and also leads to technical problems such as weak adaptability to complex trajectories. Summary of the Invention

[0006] This invention solves the technical problem that existing technologies cannot flexibly adjust the trajectory and control the posture of a single fruit in different positions and postures when planning the picking process, which leads to conflicts between the position and posture requirements and joint execution in the picking scenario, and has a weak ability to adapt to complex trajectories.

[0007] The compliant harvesting movement planning method of the present invention includes the following steps:

[0008] Step 1: Initialize the robotic arm's execution environment and parameters, and load sample data;

[0009] Step 2: Construct a strategy function, input the current end-effector pose state, obtain the probability distribution of the robot arm's actions, and generate the current action based on the probability distribution of the robot arm's actions;

[0010] Step 3: Construct a multidimensional sensory neural reward network and calculate the current total reward;

[0011] Step 4: Construct a value function, calculate the state value, and combine it with the current total reward to obtain the value advantage of the current action.

[0012] Step 5: Based on the value advantage of the current action and the probability distribution of the robotic arm's actions, calculate the value loss and the strategy loss. After updating the corresponding strategy function and value function using the strategy loss and value loss, use the robotic arm to execute the current action.

[0013] Step 6: Determine whether the current action has reached the target position. If yes, output the optimal strategy; otherwise, repeat steps 2 to 5.

[0014] Furthermore, in one embodiment of the present invention, the desired end-effector pose in step 2 is the position coordinates of the end effector and the quaternion pose of the end effector.

[0015] Furthermore, in one embodiment of the present invention, step 3, which involves constructing a multidimensional sensory neural reward network and calculating the current total reward, specifically includes:

[0016] Step 31: Obtain the current state of the robotic arm's end effector;

[0017] Step 32: Calculate the demonstration quaternion corresponding to the normal vector of the target in two consecutive frames in the demonstration data, and input the current state, demonstration data and demonstration quaternion into the demonstration reward branch of the multidimensional sensory neural reward network to obtain the demonstration reward;

[0018] Step 33: Calculate the actual pose quaternion of the end effector in the current state and the normal vector of the end effector in the next state to obtain the corresponding predicted quaternion. Input the current state, the current action and the predicted quaternion into the agent action reward branch of the multidimensional sensory neural reward network to obtain the agent action reward.

[0019] Step 34: Perform a weighted fusion of the demonstration reward and the agent action reward to obtain the current total reward.

[0020] Furthermore, in one embodiment of the present invention, both the demonstration reward branch and the agent action reward branch are multi-layer fully connected neural networks.

[0021] Furthermore, in one embodiment of the present invention, the multilayer fully connected neural network includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, and an output layer connected in sequence. The first hidden layer consists of 128 neurons with ReLU, the second hidden layer consists of 64 neurons with ReLU, the third hidden layer consists of 32 neurons with ReLU, and the fourth hidden layer consists of 16 neurons with ReLU.

[0022] The compliant harvesting motion planning system of the present invention is constructed based on the above method and includes the following modules:

[0023] The initialization module initializes the robotic arm's execution environment and parameters, and loads sample data.

[0024] The motion generation module constructs a strategy function, takes the current end-effector pose state as input, obtains the motion probability distribution of the robotic arm, and generates the current motion based on the motion probability distribution of the robotic arm.

[0025] The reward module constructs a multidimensional sensory neural reward network to calculate the current total reward.

[0026] The value estimation module constructs a value function, calculates the state value, and combines it with the current total reward to obtain the value advantage of the current action.

[0027] The execution module calculates the value loss and strategy loss based on the value advantage of the current action and the probability distribution of the robotic arm's actions. After updating the corresponding strategy function and value function using the strategy loss and value loss, the robotic arm executes the current action.

[0028] The judgment module determines whether the current action has reached the target position. If so, it outputs the optimal strategy; otherwise, it re-executes the initialization module and then the execution module.

[0029] The computer program product of the present invention includes a computer program or instructions, which, when executed by a processor, implement any of the above-described compliant picking motion planning methods.

[0030] The electronic device of the present invention includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0031] Memory, used to store computer programs;

[0032] When a processor executes a program stored in memory, it implements any of the above-described compliant picking motion planning methods.

[0033] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the aforementioned compliant picking motion planning methods.

[0034] This invention solves the technical problem in existing technologies where, when dealing with individual fruits in different positions and postures, flexible trajectory adjustment and posture control in harvesting planning are impossible, leading to conflicts between pose requirements and joint execution in harvesting scenarios, and weak adaptability to complex trajectories. Specific beneficial effects include:

[0035] 1. This invention proposes a compliant harvesting motion planning method. Addressing the core problem of varying fruit positions and postures during harvesting tasks, making compliant harvesting difficult, this invention constructs a multi-dimensional perception system for position and posture. This system forms a multi-dimensional perception inverse reinforcement learning mechanism completely independent of general spatial control, specifically designed for the compliant harvesting needs of robotic arms. Position perception uses a visual sensor to collect the three-dimensional coordinates of the fruit's center point, providing basic positioning information for pose decision-making. Utilizing the fruit surface normal vector and the quaternion posture perception at the robotic arm's end effector, it can accurately identify the fruit's spatial state. By calculating and updating the strategy loss and value loss, it guides the strategy function to output the most suitable harvesting posture, avoiding contact damage and plant damage. This solves the pain points of traditional methods that rely solely on position information for rigid harvesting, causing damage to the fruit, plant, and environment, and are unable to cope with changes in fruit posture, as well as the disconnect between perception and harvesting task requirements.

[0036] 2. This invention proposes a compliant harvesting motion planning method. The strategy function directly outputs the action-level decision-making mode of the robotic arm's joint angles, constructing a collaborative framework for mapping end-effector pose decision to inverse kinematics. This addresses the pain points of traditional inverse reinforcement learning in harvesting scenarios, such as conflicts between pose requirements and joint execution, and weak adaptability to complex trajectories. The strategy function can flexibly optimize the end-effector pose, ensuring accurate matching with the fruit's position and posture. The inverse kinematics module guarantees the feasibility of joint angles and avoids invalid execution based on joint state verification. This mechanism enables the robotic arm to make accurate decisions based on different fruit postures, significantly improving its adaptability to complex harvesting scenarios, effectively reducing fruit damage and harvesting failures, avoiding overtravel risks, and enhancing the strategy's adaptability to different harvesting scenarios.

[0037] 3. This invention proposes a compliant harvesting motion planning method, designing a reward fusion mechanism that combines demonstration similarity rewards and agent action rewards. Through a multi-dimensional perceptual neural reward network, it autonomously learns reward weights, guiding the strategy to learn accurate, compliant, and feasible pose movements. This addresses the pain points of traditional inverse reinforcement learning's pose decision-making and inverse solution execution disconnect, and reward design not adapting to harvesting scenarios. Reward synergy ensures that the strategy output pose adapts to different fruit poses and harvesting environments, while also possessing trajectory compliance, accuracy, and feasibility, thus improving strategy convergence speed and actual harvesting results.

[0038] This invention focuses on a harvesting action generation strategy under state perception, addressing the motion planning problem in compliant operation. It involves the planning of different fruit positions and postures, harvesting environment, current robotic arm state, trajectory and posture, and emphasizes decision-making mechanism and strategy autonomy. Attached Figure Description

[0039] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0040] Figure 1 This is a schematic diagram of the multidimensional sensory neural reward network described in Implementation Method 1. Detailed Implementation

[0041] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0042] Implementation Method 1: Traditional reinforcement learning methods often focus on modeling position coordinates, neglecting the perception and utilization of spatial orientation information. This makes it difficult to effectively cope with complex spatial interaction scenarios, resulting in a mismatch between the robotic arm's execution and the harvesting requirements, affecting harvesting efficiency and fruit integrity.

[0043] To address the aforementioned issues, this implementation proposes a compliant harvesting motion planning method. It designs a dedicated multi-dimensional sensory neural reward network and strategy optimization for human-like compliant harvesting operations of a robotic arm. Focusing on compliant harvesting tasks, this method ensures the robot possesses flexible trajectory adjustment and posture control capabilities when dealing with individual fruits in different positions and postures. This solves the technical problem in existing technologies where flexible trajectory adjustment and posture control are not possible for individual fruits in different positions and postures, leading to conflicts between pose requirements and joint execution in harvesting scenarios, and weak adaptability to complex trajectories. Specifically, it includes the following steps:

[0044] Step 1: Initialize the robotic arm's execution environment and parameters, and load sample data;

[0045] In this embodiment, the execution environment and execution parameters of the robotic arm are initialized, specifically including: building a physical environment model for the operation of the robotic arm, clarifying environmental information such as workspace boundaries and obstacle distribution; setting basic operating parameters of the robotic arm, such as joint range of motion, upper limit of movement speed, load threshold, control cycle, etc.; initializing the core parameters of the robotic arm agent, including the initial weights of neural networks such as hierarchical coordinate perception neural feature extractor, policy function, multidimensional perception neural reward network and value function, as well as key parameters such as the capacity of the experience buffer, laying the foundation for the subsequent stable operation and learning process of the robotic arm.

[0046] In this embodiment, the demonstration data This indicates the state of the robotic arm's end effector at each point in time, including the demonstrated action. This data was collected in advance and provided as part of the training dataset.

[0047] Step 2: Construct a strategy function, input the current end-effector pose state, obtain the probability distribution of the robot arm's actions, and generate the current action to be executed based on the probability distribution of the robot arm's actions;

[0048] The policy function uses a deep neural network structure to perform a nonlinear mapping on the current end-effector pose state and outputs the probability distribution of the robot arm's actions through a Gaussian distribution.

[0049] In this embodiment, the current end-effector pose is input into a pre-constructed strategy function. The strategy function outputs the probability distribution of the robot arm's actions based on the input features, focuses on the high-dimensional task objective (the fruit picking point position and the fruit posture), and outputs the desired end-effector pose (position coordinates + quaternion pose). The inverse kinematics solution transforms the desired end-effector pose into the joint angles of the robot arm, which directly drive the actuator action.

[0050] The strategy function employs a deep neural network structure, which performs non-linear mapping on features and ultimately outputs a probability density function of continuous actions through a Gaussian distribution, i.e., the probability distribution of the robotic arm's actions. Based on the output probability distribution of the robotic arm's actions, random sampling is used to obtain the current action performed by the robotic arm. This enables feature-based action generation.

[0051] Step 3: Construct a multidimensional sensory neural reward network and calculate the current total reward;

[0052] Step 4: Construct a value function, calculate the state value, and combine it with the current total reward to obtain the value advantage of the current action, specifically:

[0053] ;

[0054] in, The value advantage of the current action, As a discount factor, This is the current total reward. For state value, For interval Time index, For time-series correlation weights, The function for calculating the Pearson correlation coefficient. This is the current state.

[0055] This implementation method is a time-series-related advantage function estimation. Based on the current total reward and the state value output by the value function, it calculates the advantage function value and quantifies the value advantage of the current action. Among them, time-series correlation weight Used to control the impact of temporal feature associations on advantage assessment The Pearson correlation coefficient is a function used to measure... Time and The temporal correlation of moments ensures the continuity of the action sequence.

[0056] Step 5: Based on the value advantage of the current action and the probability distribution of the robotic arm's actions, calculate the policy loss and value loss. After updating the corresponding policy function and value function using the policy loss and value loss, use the robotic arm to execute the current action.

[0057] The calculation of strategy loss and value loss based on the value advantage of the currently executed action and the probability distribution of the robotic arm's actions is as follows:

[0058] ;

[0059] ;

[0060] ;

[0061] in, For strategic losses, For policy network parameters, As an experience buffer, For the experience buffer Mid-sampling Find the expected value. For the currently executing action, This represents the probability distribution of the robotic arm's actions. Weights are used to constrain motion smoothness. For the first Rate of change of movement of the step For motion smoothness constraints, For policy entropy weights, For policy entropy, Weighting for the smoothness of movement. For the loss of value, As a value parameter, This is for entropy regularization.

[0062] This implementation calculates the strategy loss to maximize expected returns while ensuring the new strategy does not significantly deviate from the old strategy. It calculates the strategy loss of the policy function by combining the value advantage of the current action and the probability distribution of the robotic arm's actions. The policy network parameters are used in this calculation. The control strategy outputs the probability of actions based on the state, and the experience buffer is used. Store data generated during the interaction between the intelligent agent and the environment, and use it as an experience buffer. Mid-sampled state-action pairs Find the expected value Eliminate single-sample noise, ensure statistical stability of loss calculation, and avoid parameter oscillations and rate of change of action. Measuring motion change is the core quantitative indicator of smoothness constraints; motion smoothness constraint terms Constraining abrupt changes in actions, simulating the consistency of human operations, and policy entropy. It measures the uncertainty of the probability distribution of actions (the higher the entropy value, the stronger the exploratory nature).

[0063] When calculating the value loss, a simple and efficient multilayer perceptron (MLP) structure is used, and a regularization term is added to avoid overfitting.

[0064] The total loss function is a unified optimization objective used to simultaneously update policy parameters and value parameters, and it calculates the total loss:

[0065] ;

[0066] in, This represents the total loss value.

[0067] Step 6: Determine whether the current action has reached the target position. If yes, output the optimal strategy; otherwise, repeat steps 2 to 5.

[0068] Implementation Method Two: This implementation method further defines the compliant picking motion planning method described in Implementation Method One. In step 3, a multidimensional sensory neural reward network is constructed to calculate the current total reward, specifically as follows:

[0069] Step 31: Obtain the current state of the robotic arm's end effector;

[0070] Step 32: Calculate the demonstration quaternion corresponding to the normal vector of the target in two consecutive frames in the demonstration data, and input the current state, demonstration data and demonstration quaternion into the demonstration reward branch of the multidimensional sensory neural reward network to obtain the demonstration reward;

[0071] Step 33: Calculate the actual posture quaternion of the current state of the robotic arm end effector (i.e. the result after executing the previous action) and the next state normal vector of the robotic arm end effector to obtain the corresponding predicted quaternion. Input the current state, the current action, and the predicted quaternion into the agent action reward branch of the multidimensional sensory neural reward network to obtain the agent action reward.

[0072] Step 34: Perform a weighted fusion of the demonstration reward and the agent action reward to obtain the current total reward.

[0073] Attitude describes the spatial position and orientation of an object or robot relative to a reference coordinate system in three-dimensional space. Accurate attitude description is crucial when it comes to motion planning, object manipulation, and task execution. For harvesting tasks, it is necessary not only to perceive the attitude of the robotic arm itself, but also to perceive the dynamic changes of the surface normal vector of the target fruit in real time, and to coordinate these changes with the attitude adjustment of the robotic arm.

[0074] However, traditional inverse reinforcement learning methods typically utilize expert demonstration data for trajectory imitation, with inputs often consisting of joint angles or end-effector positions, lacking consideration for posture quaternions. While some research has used posture in kinematics calculations (e.g., for robotic arm end-effector posture representation), it is often treated merely as static input, failing to establish a dynamic relationship between target posture changes, robotic arm posture adjustments, and future posture alignment. Specifically, this manifests in several ways: focusing only on the posture of a single object (e.g., only the robotic arm itself or only the target fruit), failing to collaboratively handle the posture mapping relationship between the two; using posture only for inverse kinematics, failing to use posture changes as the core guiding signal for the reward function; and in reinforcement learning frameworks, common reward designs are based solely on task completion (e.g., whether the target point has been reached), ignoring the impact of posture continuity and compliance on harvesting success rate.

[0075] To achieve compliant harvesting with continuous posture, an attempt was made to directly map the normal vector of the target fruit to the target posture of the robotic arm, but the following technical problems still exist:

[0076] 1. Posture disconnect problem. Using only the predicted quaternion generated by the target normal vector to guide the movement will cause the robotic arm to ignore the continuity between the current actual posture and the target posture during the movement, resulting in severe shaking or non-compliant trajectory, which can easily damage the fruit;

[0077] 2. The challenge of multi-source information fusion. How to effectively fuse the actual quaternion in the current state (i.e., the execution result of the previous action) with the predicted quaternion in the next state (i.e., the expected pose based on the target normal vector) so that the reward function can both evaluate the immediate effect of the action and guide the policy toward the target pose optimization has become a technical bottleneck.

[0078] 3. Conflict between demonstration data and real-time decision-making. Simply imitating demonstration data (step 32) is difficult to adapt to the diversity of different fruit postures, while relying solely on real-time prediction (step 33) may deviate from the compliant characteristics in expert experience.

[0079] Therefore, to solve the above-mentioned technical problems, this embodiment proposes a multidimensional sensory neural reward network with dual-branch position and posture inputs, using state-action pairs (including current state-agent action pairs) as the basis. Current state - demonstration action pair Using the input as input, the agent's action reward and demonstration reward are calculated separately. Finally, the two reward values ​​are fused in a weighted manner, which takes into account both the guiding role of the demonstration and the evaluation of the agent's actions. Therefore, it is more learning-instructive and can automatically learn a potential reward function through demonstration behavior samples, replacing the manually designed reward mechanism, thereby improving training efficiency and policy quality.

[0080] This embodiment constructs a multidimensional sensory neural reward network. The demonstrative reward branch in this embodiment is specifically:

[0081] In the demonstration reward calculation path, the input includes the current state of the robotic arm's end effector. Demonstration actions in demonstration data And pose information. The pose information of the demonstration reward branch is calculated using the demonstration quaternion from the normal vectors of two consecutive frames. This is used to represent the changing trend of spatial rotation direction, ensuring that the policy learning follows a smooth trajectory under ideal posture changes. These inputs are sequentially fed into a four-layer fully connected neural network with 128, 64, 32, and 16 neurons respectively, all using ReLU (Rectified Linear Function) activation functions. Each layer employs 25% Dropout to suppress overfitting. The network output is the demonstration reward value corresponding to the current demonstration motion. The current state refers to the set of observation information of the robotic arm's end effector at the current moment.

[0082] In this embodiment, the changes in picking posture in the demonstration data are determined by the dynamic rotation of the fruit surface normal vector. During the agent's learning process, it does not simply reproduce the demonstration trajectory, but learns how to autonomously plan the corresponding actuator end-effector trajectory and posture based on current perceived information (the real-time pose of the target fruit, environmental conditions, and the state of the robotic arm itself). Therefore, although the goal of each picking task is to complete it smoothly, the planned motion is unique in both time and space due to changes in input conditions, demonstrating the strategy's adaptability to diverse scenarios.

[0083] The agent action reward branch is specifically as follows:

[0084] In the agent action reward calculation path, the input includes the current state of the robotic arm's end effector. The action currently being performed by the intelligent agent And pose information. The pose information for the agent's action reward branch is obtained using the actual pose quaternion based on the current state of the robotic arm's end effector. The predicted quaternion obtained from the next state normal vector of the robotic arm's end effector This allows the agent to evaluate in real time whether its actions both conform to the continuity of the current actual posture and effectively approximate the target posture. The branch network structure is consistent with the demonstration path, outputting the agent's action reward value corresponding to the current action. .

[0085] The total reward is expressed as follows:

[0086] ;

[0087] in, This is the current total reward. The reward weighting coefficient is used to balance the priority between demonstration similarity and goal achievement. To calculate the current state of the agent Demonstration actions and quaternion pose (Based on the state of the demonstration location) Current normal vector and demonstration action The similarity reward obtained from the corresponding normal vector generation, To calculate the current state Quaternion posture (quaternion of current pose) (and the next state normal vector), action position information generated by the policy network. The reward for the agent's actions.

[0088] This implementation method simultaneously inputs agent action-state and demonstration action-state pairs, and balances demonstration similarity with task goal achievement through weighted fusion, avoiding policy bias driven by a single reward.

[0089] In this embodiment, the demonstration similarity reward is based on multi-dimensional calculation of position and posture to measure the similarity between the agent's pose trajectory and the expert demonstration, ensuring that the picking action conforms to human operating habits. The agent's action reward is to evaluate the smoothness of the end position increment and the rate of change of posture. The constraint strategy outputs a continuous and smooth pose trajectory, laying the foundation for the joint movement compliance after inverse kinematics.

[0090] Traditional methods only plan the position trajectory of the end effector for fresh fruit placement tasks, using only XYZ position information as input. Their core focus is on spatial path planning, with no strict requirements for posture accuracy and no consideration of the technical issues related to posture, force, and fruit damage during the picking process. This proposed method, however, targets the highly posture-sensitive task of fresh fruit picking. For the first time, it combines the temporal changes of the normal vector of a manually demonstrated trajectory with the robot's own historical posture. Posture changes are derived from the differences in normal vectors between consecutive frames, and quaternions are calculated as network input. Its core contribution lies not in the mathematical conversion between normal vectors and quaternions themselves, but in the integrated modeling of the geometric constraints, temporal changes, and human-like decision-making mechanisms of the picking action.

[0091] Through the aforementioned dual-branch position and pose (quaternion) input design, this implementation achieves closed-loop coordination of pose perception, action execution, and future prediction. This implementation, for the first time, establishes a multi-dimensional correlation between the current pose, target pose, and predicted pose at the reward function level, enabling the robotic arm to dynamically adjust its strategy based on the pose of different fruits, overcoming the limitation of existing methods that only focus on pose at a single moment.

[0092] Furthermore, this implementation method can incorporate force feedback to fuse actual contact force information with posture prediction, thereby further improving compliance.

[0093] Implementation Method 3: The compliant harvesting movement planning system described in this implementation method is constructed based on the method described in Implementation Method 1, and includes the following modules:

[0094] The initialization module initializes the robotic arm's execution environment and parameters, and loads sample data.

[0095] The motion generation module constructs a strategy function, takes the current end-effector pose state as input, obtains the motion probability distribution of the robotic arm, and generates the current motion based on the motion probability distribution of the robotic arm.

[0096] The reward module constructs a multidimensional sensory neural reward network to calculate the current total reward.

[0097] The value estimation module constructs a value function, calculates the state value, and combines it with the current total reward to obtain the value advantage of the current action.

[0098] The execution module calculates the value loss and strategy loss based on the value advantage of the current action and the probability distribution of the robotic arm's actions. After updating the corresponding strategy function and value function using the strategy loss and value loss, the robotic arm executes the current action.

[0099] The judgment module determines whether the current action has reached the target position. If so, it outputs the optimal strategy; otherwise, it re-executes the action generation module and then the execution module.

[0100] Implementation Method 4: A computer program product, comprising a computer program or instructions that, when executed by a processor, implement any of the above-described compliant picking motion planning methods.

[0101] Implementation Method 5: An electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0102] Memory, used to store computer programs;

[0103] When a processor executes a program stored in memory, it implements any of the above-described compliant picking motion planning methods.

[0104] Implementation Method Six: A computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described compliant picking motion planning methods.

[0105] The foregoing has provided a detailed description of the compliant harvesting motion planning method, system, product, equipment, and storage medium proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A compliant picking motion planning method, characterized in that, Includes the following steps: Step 1: Initialize the robotic arm's execution environment and parameters, and load sample data; Step 2: Construct a strategy function, input the current end-effector pose state, obtain the probability distribution of the robot arm's actions, and generate the current action based on the probability distribution of the robot arm's actions; Step 3: Construct a multidimensional sensory neural reward network and calculate the current total reward, specifically as follows: Step 31: Obtain the current state of the robotic arm's end effector; Step 32: Calculate the demonstration quaternion corresponding to the normal vector of the target in two consecutive frames in the demonstration data, and input the current state, demonstration data and demonstration quaternion into the demonstration reward branch of the multidimensional sensory neural reward network to obtain the demonstration reward; Step 33: Calculate the actual pose quaternion of the end effector in the current state and the normal vector of the end effector in the next state to obtain the corresponding predicted quaternion. Input the current state, the current action and the predicted quaternion into the agent action reward branch of the multidimensional sensory neural reward network to obtain the agent action reward. Step 34: Perform a weighted fusion of the demonstration reward and the agent action reward to obtain the current total reward; Step 4: Construct a value function, calculate the state value, and combine it with the current total reward to obtain the value advantage of the current action. Step 5: Based on the value advantage of the current action and the probability distribution of the robotic arm's actions, calculate the value loss and the strategy loss. After updating the corresponding strategy function and value function using the strategy loss and value loss, use the robotic arm to execute the current action. Step 6: Determine whether the current action has reached the target position. If yes, output the optimal strategy; otherwise, repeat steps 2 to 5.

2. The compliant picking motion planning method of claim 1, wherein, In step 2, the desired end-effector pose is the position coordinates of the end effector and the quaternion pose of the end effector.

3. The compliant picking motion planning method of claim 1, wherein, Both the demonstration reward branch and the agent action reward branch are multi-layer fully connected neural networks.

4. The compliant picking motion planning method of claim 3, wherein, The multilayer fully connected neural network includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, and an output layer connected in sequence. The first hidden layer consists of 128 neurons with ReLU, the second hidden layer consists of 64 neurons with ReLU, the third hidden layer consists of 32 neurons with ReLU, and the fourth hidden layer consists of 16 neurons with ReLU.

5. A compliant picking motion planning system, said system is constructed based on the method of claim 1, characterized in that, Includes the following modules: The initialization module initializes the robotic arm's execution environment and parameters, and loads sample data. The motion generation module constructs a strategy function, takes the current end-effector pose as input, obtains the motion probability distribution of the robotic arm, and generates the current motion based on the motion probability distribution of the robotic arm. The reward module constructs a multidimensional sensory neural reward network to calculate the current total reward. The value estimation module constructs a value function, calculates the state value, and combines it with the current total reward to obtain the value advantage of the current action. The execution module calculates the value loss and strategy loss based on the value advantage of the current action and the probability distribution of the robotic arm's actions. After updating the corresponding strategy function and value function using the strategy loss and value loss, the robotic arm executes the current action. The judgment module determines whether the current action has reached the target position. If so, it outputs the optimal strategy; otherwise, it re-executes the action generation module and then the execution module.

6. A computer program product comprising computer programs or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the compliant picking motion planning method according to any one of claims 1-4.

7. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the compliant picking motion planning method according to any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the compliant picking motion planning method according to any one of claims 1-4.

Images