Agricultural product picking method and equipment based on multi-finger picking manipulator

Abstract

The invention discloses an agricultural product picking method and device based on a multi-finger picking manipulator, and relates to the field of agricultural automation, and the method comprises the steps: building a three-dimensional model of the multi-finger picking manipulator and an agricultural product production base; performing contact dynamics simulation based on the three-dimensional model to determine a preliminary picking track; controlling the multi-finger picking manipulator to execute a picking task based on the preliminary picking track; in the picking process, real-time postures and surrounding environment information of all components of the multi-finger picking manipulator are collected; according to the real-time attitude, determining a predicted picking track by using a bidirectional long-short-term memory network; determining a safe picking track by adopting a local path planning algorithm according to the surrounding environment information and the preliminary picking track; and adjusting the preliminary picking track according to the predicted picking track and the safe picking track so as to adjust the posture of each component of the multi-finger picking manipulator in real time. The accuracy of the motion trail in the picking process of the multi-finger picking mechanical arm is improved, and then the damage rate of agricultural products in the picking process is reduced.

Description

Technical Field

[0001] The present application relates to the field of agricultural automation, and in particular to a method and device for picking agricultural products based on a multi-finger picking robot. Background Art

[0002] With the development of the economy and society, agricultural production is constantly shifting towards scale, intensiveness and precision. In terms of agricultural product picking, in order to improve the picking efficiency, there have been relevant studies on automatic picking through robots to improve the automation level of agricultural product picking and thus improve the picking efficiency. At present, robots generally pick agricultural products through picking manipulators, but due to the fragile characteristics of agricultural products, during the process of picking agricultural products by the picking manipulator, the overshoot of its gripping force will scratch the agricultural products, and the gripping force response speed is too slow to hold the agricultural products firmly. In addition, during the movement of the robot, deformation and vibration will occur based on factors such as the flatness of the ground. At this time, the rigid manipulator is transformed into a flexible manipulator, which makes the control of the picking manipulator more difficult, thereby increasing the damage rate of agricultural products. Summary of the invention

[0003] The purpose of this application is to provide an agricultural product picking method and equipment based on a multi-finger picking robot, which can improve the accuracy of the motion trajectory of the multi-finger picking robot during the picking process, thereby reducing the damage rate of agricultural products during the picking process.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides an agricultural product picking method based on a multi-finger picking manipulator, the multi-finger picking manipulator includes a stepper motor, a base and a plurality of gripping parts, each gripping part includes a plurality of flexible finger parts, the stepper motor is respectively connected to the base and each finger part, and the agricultural product picking method based on the multi-finger picking manipulator includes:

[0006] Establish a 3D model of a multi-finger picking robot and a 3D model of an agricultural product production base;

[0007] Based on the three-dimensional model of the multi-finger picking manipulator and the three-dimensional model of the agricultural product production base, the contact dynamics simulation of the multi-finger picking manipulator and the agricultural products in the agricultural product production base is performed to determine a preliminary picking trajectory; the preliminary picking trajectory includes the posture of the base, the posture of each gripping part and the posture of each finger part at each moment during the picking process;

[0008] Based on the preliminary picking trajectory, the multi-finger picking manipulator is controlled to perform the picking task at the agricultural product production base, and during the picking process, the real-time posture of the base, the real-time posture of each gripping part, the real-time posture of each finger part and the surrounding environment information are collected;

[0009] According to the real-time posture of the base, the real-time posture of each grasping part and the real-time posture of each finger, a bidirectional long short-term memory network is used to determine the predicted picking trajectory within the future set time period;

[0010] According to the surrounding environment information and the preliminary picking trajectory, a local path planning algorithm is used to determine a collision-free safe picking trajectory within a future set time period;

[0011] According to the predicted picking trajectory and the collision-free safe picking trajectory within the future set time period, the preliminary picking trajectory is adjusted to adjust the posture of the base, the posture of each grasping part and the posture of each finger of the multi-finger picking manipulator in real time.

[0012] Optionally, based on the three-dimensional model of the multi-finger picking manipulator and the three-dimensional model of the agricultural product production base, a contact dynamics simulation is performed between the multi-finger picking manipulator and the agricultural products in the agricultural product production base to determine a preliminary picking trajectory, specifically including:

[0013] Controlling the three-dimensional model of the multi-finger picking manipulator to perform a picking task in the three-dimensional model of the agricultural product production base;

[0014] Solving the dynamic parameters of the three-dimensional model of the multi-finger picking manipulator based on non-collision dynamics; the dynamic parameters include the displacement, velocity and acceleration of each finger;

[0015] When the three-dimensional model of the multi-finger picking manipulator collides with the agricultural products in the three-dimensional model of the agricultural product production base, a collision pair is generated, and the state of the collision pair is determined, and the dynamic response amount of the three-dimensional model of the multi-finger picking manipulator is determined according to the state of the collision pair;

[0016] In the process of performing the picking task, the posture of the base, the posture of each gripping part and the posture of each finger part at each moment are determined in real time according to the dynamic parameters or the dynamic response amount to obtain a preliminary picking trajectory.

[0017] Optionally, the dynamic response quantity includes vibration, deformation, normal contact force, tangential contact force and collision contact duration of the three-dimensional model of the multi-finger picking manipulator.

[0018] Optionally, the states of the collision pair include a sticky state, a forward sliding state, a reverse sliding state, a sticky-sliding switching state, a sliding-sticky switching state, and a forward-reverse sliding switching state;

[0019] Determining the dynamic response of the three-dimensional model of the multi-finger picking manipulator according to the state of the collision pair specifically includes:

[0020] If the state of the collision pair is a sticky state, the sticky state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator;

[0021] If the state of the collision pair is a forward sliding state or a reverse sliding state, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator;

[0022] If the state of the collision pair is a stick-slip switching state or a sliding-stick switching state, based on the sticky state constraint condition, the sticky state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator;

[0023] If the state of the collision pair is a forward-reverse sliding switching state, based on the sliding state constraint condition, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator.

[0024] Optionally, the posture includes position, velocity and acceleration.

[0025] Optionally, according to the surrounding environment information and the preliminary picking trajectory, a local path planning algorithm is used to determine a collision-free safe picking trajectory within a future set period of time, specifically including:

[0026] According to the surrounding environment information, determining whether there are obstacles around the multi-finger picking robot;

[0027] If there are no obstacles around the multi-finger picking manipulator, the collision-free safe picking trajectory in the future set time period includes the posture of the base, the posture of each gripping part and the posture of each finger part in the corresponding time period of the preliminary picking trajectory;

[0028] If there are obstacles around the multi-finger picking robot, a local starting point and a local ending point are determined in the preliminary picking trajectory by a local path planner;

[0029] Between the local starting point and the local ending point, with the goal of minimizing the potential field, the trajectory points between the local starting point and the local ending point are determined to obtain a collision-free safe picking trajectory within a future set time period.

[0030] Optionally, a particle swarm optimization algorithm is used to determine the trajectory point between the local starting point and the local ending point.

[0031] Optionally, the local path planning algorithm is a dynamic window method.

[0032] Optionally, the agricultural product picking method based on the multi-finger picking manipulator further includes:

[0033] During the picking process, the vibration signals of each finger are collected in real time;

[0034] According to the vibration signal of each finger, the Q-Learning algorithm is used to adjust the speed of the stepper motor and the corresponding duration to suppress the vibration of each finger.

[0035] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned agricultural product picking method based on a multi-finger picking robot.

[0036] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0037] The present application provides a method and device for picking agricultural products based on a multi-finger picking robot. By modeling the multi-finger picking robot and the agricultural product production base, the contact dynamics simulation of the multi-finger picking robot and the agricultural products is performed, which provides a basis for the motion control of the multi-finger picking robot. Furthermore, in the actual picking process, according to the real-time posture of the multi-finger picking robot, a bidirectional long short-term memory network is used to predict the picking trajectory. At the same time, local path planning is performed according to the surrounding environment information to determine a collision-free safe picking trajectory. The preliminary picking trajectory obtained by simulation is adjusted based on the predicted picking trajectory and the collision-free safe picking trajectory, which improves the accuracy of the motion trajectory of the multi-finger picking robot during the picking process, thereby reducing the damage rate of the agricultural products during the picking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a schematic diagram of a multi-finger picking manipulator;

[0040] Figure 2 A schematic flow chart of a method for picking agricultural products based on a multi-finger picking robot is provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0043] In an exemplary embodiment, Figure 1 As shown, the multi-finger picking manipulator includes a stepper motor (not shown in the figure), a base 101 and a plurality of gripping parts 103, each gripping part 103 includes a plurality of flexible finger parts 104, and the stepper motor is respectively connected to the base 101 and each finger part 104. When the multi-finger picking manipulator performs a picking task, the posture of the base 101 and each finger part 104 is controlled by controlling the speed and duration of the stepper motor.

[0044] Furthermore, an extension arm 102 is installed on the base 101, and each gripping part 103 is installed at the end of the extension arm 102, and the extension arm 102 is connected to the base 101 through a rotating joint, and the extension arm 102 is also connected to each gripping part 103 through a rotating joint. Each finger part 104 is a flexible rod. All the finger parts 104 of the same gripping part 103 form an open chain structure, and every two finger parts 104 are connected through a rotating joint. Each rotating joint is driven by a stepping motor to enable the multi-finger picking manipulator to achieve a desired posture.

[0045] In an exemplary embodiment, a method for picking agricultural products based on a multi-finger picking manipulator is provided, and the method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. The method for picking agricultural products based on a multi-finger picking manipulator can be applied to fruit picking scenarios.

[0046] In the embodiments of the present application, Figure 2 As shown, the agricultural product picking method based on the multi-finger picking robot includes the following steps 201 to 206.

[0047] Step 201, establishing a three-dimensional model of a multi-finger picking robot and a three-dimensional model of an agricultural product production base.

[0048] Step 202, based on the three-dimensional model of the multi-finger picking robot and the three-dimensional model of the agricultural product production base, perform contact dynamics simulation of the multi-finger picking robot and the agricultural products in the agricultural product production base to determine a preliminary picking trajectory.

[0049] The preliminary picking trajectory includes the posture of the base, the posture of each gripping part and the posture of each finger at each moment during the picking process. The posture includes position, speed and acceleration.

[0050] In a specific example, step 202 includes the following steps 301 to 304 .

[0051] Step 301 : Control the three-dimensional model of the multi-finger picking robot to perform a picking task in the three-dimensional model of the agricultural product production base.

[0052] Step 302: solving the dynamic parameters of the three-dimensional model of the multi-finger picking manipulator based on non-collision dynamics. The dynamic parameters include the displacement, velocity and acceleration of each finger.

[0053] Specifically, based on the kinetic equation Calculate the dynamic parameters of the finger; where J is the generalized acceleration array, is the generalized mass matrix, and R is the non-collision generalized force matrix.

[0054] Since the three-dimensional model of the multi-finger picking robot and the three-dimensional model of the agricultural product production base are in a non-contact state at the beginning, the three-dimensional model of the multi-finger picking robot starts to move and collides with the three-dimensional model of the agricultural product production base. Therefore, during this period of time, the dynamic parameters of the three-dimensional model of the multi-finger picking robot are solved based on non-collision dynamics.

[0055] Step 303, when the three-dimensional model of the multi-finger picking robot collides with the agricultural products in the three-dimensional model of the agricultural product production base, a collision pair is generated, and the state of the collision pair is determined, and the dynamic response of the three-dimensional model of the multi-finger picking robot is determined according to the state of the collision pair.

[0056] The dynamic response includes vibration, deformation, normal contact force, tangential contact force and collision contact duration of the three-dimensional model of the multi-finger picking manipulator.

[0057] First, the impulse-momentum method equation Solve the velocity jump condition at the initial moment of collision; where, is the normal velocity array, is the tangential velocity array, is the normal impulse potential vector matrix, is the tangential impulse potential vector array, and z is the displacement array.

[0058] Then, based on the increase and decrease constraint method, the dynamic equation of the three-dimensional model of the multi-finger picking manipulator with friction and multi-point collision is established:

[0059]

[0060] in, is the normal constraint generalized force, is the generalized force of viscous friction, is the generalized force of sliding friction, n C is the number of collision pairs in continuous contact, n H is the number of collision pairs in the tangential sticking state, is Φ zk The transposed matrix, Φ zk is the Jacobian matrix of the normal constraint of the kth collision pair, is Γ zk The transposed matrix of zk is the tangential constraint Jacobian matrix of the kth collision pair, λ Nk is the normal contact constraint reaction force of the kth collision pair, λ Tk is the tangential contact constraint reaction force of the kth collision pair, for The augmented matrix of .

[0061] Then, the state of the collision pair is determined according to the stick-slip switching criterion, and the states of the collision pair include a sticky state, a forward sliding state, a reverse sliding state, a stick-slip switching state (a critical state from stickiness to sliding), a sliding-sticky switching state (a critical state from sliding to stickiness), and a forward-reverse sliding switching state.

[0062] When |λ Tk |<μ k λ Nk ,(k∈I H ), the collision pair is in a sticky state, and the tangential relative velocity

[0063] When |λ Tk |>μ k λ Nk ,(k∈(I C -I H )), the collision pair is in a sliding state, and the tangential relative velocity is

[0064] when And ||λ Tk |-μk λ Nk |→0,(k∈(I C -I H )), the collision pair is in a forward and reverse sliding switching state;

[0065] when And |λ Tk |-μ k λ Nk ,(k∈I H ) changes from positive to close to 0, the collision pair is in a sliding-stick switching state;

[0066] when And |λ Tk |-μ k λ Nk ,(k∈(I C -I H When the value of )) changes from negative to 0, the collision pair is in a stick-slip switching state;

[0067] Among them, μ k is the friction coefficient, I C is the set of collision pairs in continuous contact state, I H is the set of collision pairs that are in a sticky state.

[0068] (1) If the state of the collision pair is a viscous state, the viscous state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking robot.

[0069] (2) If the state of the collision pair is a forward sliding state or a reverse sliding state, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking robot.

[0070] (3) If the state of the collision pair is a stick-slip switching state or a sliding-stick switching state, based on the sticky state constraint condition, the sticky state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking robot.

[0071] (4) If the state of the collision pair is a forward-reverse sliding switching state, based on the sliding state constraint condition, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking robot.

[0072] The above viscous state dynamic equation is:

[0073] The above sliding state dynamic equation is:

[0074] Step 304, during the picking task, the posture of the base, the posture of each gripping part and the posture of each finger part at each moment are determined in real time according to the dynamic parameters or the dynamic response amount to obtain a preliminary picking trajectory.

[0075] This application establishes a variable topology dynamic equation based on the increase and decrease constraint method, uses the impulse-momentum method to solve the initial velocity jump condition of the collision, constructs contact constraints according to different contact states, and uses complementary conditions to describe the dynamic contact conditions, thereby obtaining a concise, accurate and easy-to-program collision dynamics formula, which can capture the movement speed and deformation of the fingers, the collision force, the position of the collision point, the multiple stick-slip switching of the collision point during the contact process, the sticky positive and reverse micro-slipping and other contact state switching, and provides a basis for the motion control of a multi-finger picking robot.

[0076] Step 203, based on the preliminary picking trajectory, the multi-finger picking manipulator is controlled to perform the picking task at the agricultural product production base, and during the picking process, the real-time posture of the base, the real-time posture of each gripping part, the real-time posture of each finger part and the surrounding environment information are collected. The real-time posture includes position, speed and acceleration. The surrounding environment information includes the distance between the multi-finger picking manipulator and all objects within the set range, which can be collected by radar or obtained by image recognition.

[0077] Step 204, based on the real-time posture of the base, the real-time posture of each gripping part and the real-time posture of each finger part, a bidirectional long short-term memory network is used to determine the predicted picking trajectory within a future set time period.

[0078] Specifically, a bidirectional long short-term memory network is pre-trained, and the input of the bidirectional long short-term memory network is the posture of the base, the posture of each grasping part, and the posture of each finger at each moment in a period of time before the current moment, and the output is the posture of the base, the posture of each grasping part, and the posture of each finger at each moment in a period of time after the current moment.

[0079] The bidirectional long short-term memory network can process sequence data and maintain long-term memory, while taking into account past and future information, enabling the model to better capture contextual relationships in sequence data.

[0080] Step 205, based on the surrounding environment information and the preliminary picking trajectory, a local path planning algorithm is used to determine a collision-free safe picking trajectory within a future set time period.

[0081] In a specific example, step 205 includes the following steps 401 to 404 .

[0082] Step 401: determine whether there are obstacles around the multi-finger picking robot according to the surrounding environment information.

[0083] Step 402, if there are no obstacles around the multi-finger picking robot, the collision-free safe picking trajectory in the future set time period includes the posture of the base, the posture of each gripping part and the posture of each finger part in the corresponding time period of the preliminary picking trajectory.

[0084] Step 403: If there are obstacles around the multi-finger picking robot, a local starting point and a local ending point are determined in the preliminary picking trajectory by a local path planner.

[0085] Step 404, between the local starting point and the local ending point, with the goal of minimizing the potential field, determines the trajectory points between the local starting point and the local ending point, so as to obtain a collision-free safe picking trajectory within a future set time period.

[0086] Among them, the smaller the distance between the trajectory point to be solved and the local termination point, the smaller the potential field corresponding to the trajectory point to be solved, and the larger the distance between the trajectory point to be solved and the obstacle, the smaller the potential field corresponding to the trajectory point to be solved.

[0087] Specifically, a particle swarm optimization algorithm is used to determine the trajectory point between the local starting point and the local ending point.

[0088] This application adopts a combination of artificial potential field algorithm and particle swarm algorithm for local path planning, which can generate a more optimal spatial path while completing effective obstacle avoidance and ensuring the smoothness of the movement of the multi-finger picking robot.

[0089] In another specific example, the local path planning algorithm is a dynamic window method.

[0090] The dynamic window method is a local path planning algorithm for mobile robots, especially suitable for dynamic environments where speed and steering are considered simultaneously. It is based on the robot's dynamic model and perception information, and dynamically adjusts speed and steering to adapt to changes in the environment, thereby achieving safe and efficient path planning.

[0091] The path planning steps of the dynamic window method are as follows:

[0092] 1) Search for feasible speed and direction.

[0093] Based on the dynamic model of the multi-finger picking manipulator and the perception information of the environment, all the speed and steering combinations that the multi-finger picking manipulator can take at the current moment are determined. These speed and steering ranges are called dynamic windows, which are usually defined by parameters such as minimum speed, maximum speed, minimum steering angular velocity, and maximum steering angular velocity.

[0094] 2) Trajectory generation.

[0095] For each searched speed and steering combination, a series of possible trajectories are generated in a short time according to the dynamic model of the multi-finger picking manipulator. These trajectories represent the possible movement paths that the multi-finger picking manipulator may take, including linear motion, curved motion, etc.

[0096] 3) Trajectory evaluation.

[0097] For each generated trajectory, three indicators are evaluated: the deviation angle between the trajectory and the obstacle direction (the smaller the angle, the higher the score), the speed of the multi-finger picking robot in the trajectory (the higher the speed, the higher the score), and the distance between the trajectory and the obstacle (the larger the distance, the higher the score). Based on the above three items, a comprehensive score is given, and the evaluation can be based on the kinematic characteristics, the obstacle information of the environment, and the optimization goal of the path.

[0098] 4) Select the optimal trajectory.

[0099] According to the evaluation results of the trajectories, the trajectory with the highest score is selected as the optimal trajectory of the multi-finger picking manipulator. The optimal trajectory is usually the trajectory that can approach the target path to the greatest extent and avoid obstacles.

[0100] The dynamic window method can quickly generate paths in a real-time environment, and can adjust the speed and steering of the multi-finger picking manipulator in real time according to changes in the dynamic environment. It can dynamically adjust the speed and steering according to the dynamic window of the multi-finger picking manipulator, and achieve flexible adaptation in different environments and tasks. It has a wide range of applicability. By evaluating the contact between the generated trajectory and obstacles, it can effectively avoid collisions and avoid obstacles, thereby improving the safety and stability of the multi-finger picking manipulator. Moreover, the implementation of the dynamic window method is relatively simple, and does not require complex map modeling and path planning algorithms. It can only perform path planning and navigation based on the dynamic model and perception information of the multi-finger picking manipulator.

[0101] Step 206, adjusting the preliminary picking trajectory according to the predicted picking trajectory and the collision-free safe picking trajectory within the future set time period, so as to adjust the posture of the base, the posture of each gripping part and the posture of each finger part of the multi-finger picking manipulator in real time.

[0102] Specifically, if the predicted picking trajectory and / or the safe picking trajectory is different from the trajectory in the corresponding time period in the preliminary picking trajectory, the predicted picking trajectory and / or the safe picking trajectory is used to replace the trajectory in the corresponding time period in the preliminary picking trajectory.

[0103] In the process of a multi-finger picking robot performing a picking task, the present application combines the simulated picking trajectory, the predicted picking trajectory and the locally planned picking trajectory, thereby improving the accuracy of the motion trajectory of the multi-finger picking robot during the picking process, thereby avoiding damage to agricultural products by the multi-finger picking robot.

[0104] In order to further improve the picking accuracy of the multi-finger picking robot, the agricultural product picking method based on the multi-finger picking robot also includes the following steps 207 and 208.

[0105] Step 207: During the picking process, the vibration signal of each finger is collected in real time.

[0106] Step 208 , according to the vibration signal of each finger, the Q-Learning algorithm is used to adjust the speed of the stepper motor and the corresponding duration to suppress the vibration of each finger.

[0107] This application adopts the Q-Learning algorithm to adjust the speed and corresponding duration of the stepper motor during the picking process. On the basis of accurate trajectory planning, it suppresses the vibration of each finger, further improves the stability of the multi-finger picking robot, and thus reduces the damage rate of agricultural products during the picking process.

[0108] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0109] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0110] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0113] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for picking agricultural products based on a multi-finger picking manipulator, wherein the multi-finger picking manipulator comprises a stepper motor, a base and a plurality of gripping parts, each gripping part comprises a plurality of flexible finger parts, the stepper motor is respectively connected to the base and each finger part, and is characterized in that: The agricultural product picking method based on the multi-finger picking manipulator includes: Establish a 3D model of a multi-finger picking robot and a 3D model of an agricultural product production base; Based on the three-dimensional model of the multi-finger picking manipulator and the three-dimensional model of the agricultural product production base, the contact dynamics simulation of the multi-finger picking manipulator and the agricultural products in the agricultural product production base is performed to determine a preliminary picking trajectory; the preliminary picking trajectory includes the posture of the base, the posture of each gripping part and the posture of each finger part at each moment during the picking process; Based on the preliminary picking trajectory, the multi-finger picking manipulator is controlled to perform the picking task at the agricultural product production base, and during the picking process, the real-time posture of the base, the real-time posture of each gripping part, the real-time posture of each finger part and the surrounding environment information are collected; According to the real-time posture of the base, the real-time posture of each grasping part and the real-time posture of each finger, a bidirectional long short-term memory network is used to determine the predicted picking trajectory within the future set time period; According to the surrounding environment information and the preliminary picking trajectory, a local path planning algorithm is used to determine a collision-free safe picking trajectory within a future set time period; According to the predicted picking trajectory and the collision-free safe picking trajectory within the future set time period, the preliminary picking trajectory is adjusted to adjust the posture of the base, the posture of each grasping part and the posture of each finger of the multi-finger picking manipulator in real time.

2. The agricultural product picking method based on a multi-finger picking manipulator according to claim 1 is characterized in that: Based on the three-dimensional model of the multi-finger picking manipulator and the three-dimensional model of the agricultural product production base, a contact dynamics simulation is performed between the multi-finger picking manipulator and the agricultural products in the agricultural product production base to determine a preliminary picking trajectory, specifically including: Controlling the three-dimensional model of the multi-finger picking manipulator to perform a picking task in the three-dimensional model of the agricultural product production base; Solving the dynamic parameters of the three-dimensional model of the multi-finger picking manipulator based on non-collision dynamics; the dynamic parameters include the displacement, velocity and acceleration of each finger; When the three-dimensional model of the multi-finger picking manipulator collides with the agricultural products in the three-dimensional model of the agricultural product production base, a collision pair is generated, and the state of the collision pair is determined, and the dynamic response amount of the three-dimensional model of the multi-finger picking manipulator is determined according to the state of the collision pair; In the process of performing the picking task, the posture of the base, the posture of each gripping part and the posture of each finger part at each moment are determined in real time according to the dynamic parameters or the dynamic response amount to obtain a preliminary picking trajectory.

3. The agricultural product picking method based on a multi-finger picking manipulator according to claim 2 is characterized in that: The dynamic response quantity includes vibration, deformation, normal contact force, tangential contact force and collision contact duration of the three-dimensional model of the multi-finger picking manipulator.

4. The agricultural product picking method based on a multi-finger picking manipulator according to claim 2, characterized in that: The states of the collision pair include a sticky state, a forward sliding state, a reverse sliding state, a sticky-sliding switching state, a sliding-sticky switching state, and a forward-reverse sliding switching state; Determining the dynamic response of the three-dimensional model of the multi-finger picking manipulator according to the state of the collision pair specifically includes: If the state of the collision pair is a sticky state, the sticky state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator; If the state of the collision pair is a forward sliding state or a reverse sliding state, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator; If the state of the collision pair is a stick-slip switching state or a sliding-stick switching state, based on the sticky state constraint condition, the sticky state dynamic equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator; If the state of the collision pair is a forward-reverse sliding switching state, based on the sliding state constraint condition, the sliding state dynamics equation is used to solve the dynamic response of the three-dimensional model of the multi-finger picking manipulator.

5. The agricultural product picking method based on a multi-finger picking manipulator according to claim 1, characterized in that: The posture includes position, velocity and acceleration.

6. The agricultural product picking method based on a multi-finger picking manipulator according to claim 1, characterized in that: According to the surrounding environment information and the preliminary picking trajectory, a local path planning algorithm is used to determine a collision-free safe picking trajectory within a future set period of time, specifically including: According to the surrounding environment information, determining whether there are obstacles around the multi-finger picking robot; If there are no obstacles around the multi-finger picking manipulator, the collision-free safe picking trajectory in the future set time period includes the posture of the base, the posture of each gripping part and the posture of each finger part in the corresponding time period of the preliminary picking trajectory; If there are obstacles around the multi-finger picking robot, a local starting point and a local ending point are determined in the preliminary picking trajectory by a local path planner; Between the local starting point and the local ending point, with the goal of minimizing the potential field, the trajectory points between the local starting point and the local ending point are determined to obtain a collision-free safe picking trajectory within a future set time period.

7. The agricultural product picking method based on a multi-finger picking manipulator according to claim 6, characterized in that: A particle swarm optimization algorithm is used to determine the trajectory point between the local starting point and the local ending point.

8. The agricultural product picking method based on a multi-finger picking manipulator according to claim 1, characterized in that: The local path planning algorithm is a dynamic window method.

9. The agricultural product picking method based on a multi-finger picking manipulator according to claim 1, characterized in that: The agricultural product picking method based on the multi-finger picking manipulator also includes: During the picking process, the vibration signals of each finger are collected in real time; According to the vibration signal of each finger, the Q-Learning algorithm is used to adjust the speed of the stepper motor and the corresponding duration to suppress the vibration of each finger.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the agricultural product picking method based on a multi-finger picking robot as described in any one of claims 1 to 9.

Images