Double-arm picking robot control system and method based on neurodynamics

Abstract

The invention relates to a double-arm picking robot control system and method based on neurodynamics, and belongs to the technical field of mechanical arm control. The system is composed of a moving platform, a left multi-joint mechanical arm, a right multi-joint mechanical arm and a central control system. The method comprises the following steps: S1, acquiring sensor data in real time; s2, identifying a picking target and calculating the position of the picking target; s3, calculating a current self state; s4, task decomposition is conducted, and expected states of the mobile platform and the tail ends of the two multi-joint mechanical arms are obtained; s5, building a dynamic neural network solver and performing training; s6, loading the dynamic neural network solver to a central control system; and S7, calculating a control strategy by using the trained dynamic neural network solver, and driving a servo motor to control and execute a picking task. According to the method, the dynamic neural network solver adopting the gradient descent method with speed compensation is innovatively designed, and accurate control over the zero position error can be theoretically achieved.

Description

Technical Field

[0001] This invention relates to a control system and method for a dual-arm harvesting robot based on neurodynamics, belonging to the field of robotic arm control technology, and is particularly applicable to the control of a dual-arm harvesting robot based on neurodynamics. Background Technology

[0002] The application of dual-arm robot control systems in the harvesting field is mainly reflected in their ability to efficiently and accurately complete the harvesting tasks of fruits, vegetables, and other crops. Through advanced sensor technologies, such as visual recognition systems and force sensors, it perceives the position, ripeness, and relative position of the fruit to its surrounding environment in real time. The two arms work in tandem; one arm is responsible for fixing or adjusting the position of the fruit-bearing branch, while the other arm precisely performs the harvesting action, avoiding damage to the fruit and the plant. Its technical features include a high degree of automation and intelligence, adapting to fruits of different shapes, sizes, and growth stages; good flexibility and adaptability, allowing parameter adjustments based on different crops and planting environments; and optimized motion planning algorithms to achieve efficient harvesting paths, improving harvesting efficiency and quality, reducing labor costs, and providing strong support for the modernization and intelligent development of agriculture.

[0003] Dual-arm mobile robots combine the advantages of a mobile platform and a robotic arm, increasing the system's degrees of freedom (DOFs) and enabling the execution of more complex tasks. Through a physically constrained velocity-coordinated control scheme, dual-arm mobile robots can achieve coordinated movement of the mobile platform and the robotic arm, improving task execution efficiency and accuracy. Existing research mainly focuses on the task execution of the end effector, neglecting the precise control of the mobile platform, resulting in unpredictable and inaccurate movement of the mobile platform during task execution. Current research on dual-arm mobile robots is difficult to directly apply to agricultural production scenarios such as harvesting because the control precision and response speed cannot yet be compared with manual operation. Summary of the Invention

[0004] In view of this, in order to solve the control problem of dual-arm harvesting robots in agricultural product harvesting, this invention provides a control system and method for dual-arm harvesting robots based on neurodynamics. First, sensors are used to identify the robot's own state and target position. Then, a speed cooperative control with physical constraints is proposed. The design optimization criteria are used to coordinate the adjustment of the mobile platform and the robotic arm. Finally, a dynamic neural network is used to build a dynamic neural network to achieve efficient solution of the control strategy, thereby realizing fast and accurate control of the dual-arm harvesting robot.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A neurodynamic dual-arm harvesting robot control system, combined with Figure 1The system comprises a mobile platform, a left multi-joint robotic arm, a right multi-joint robotic arm, and a central control system. The mobile platform is a trolley driven by dual servo motors. The left and right multi-joint robotic arms are identical, consisting of multiple joints driven by an independent servo motor. The central control system is a computer with a processor of CPU, GPU, or NPU, equipped with a communication module and a dynamic neural network solver in its memory. The mobile platform has a battery that powers itself, the two multi-joint robotic arms, and the central control system. The two multi-joint robotic arms are symmetrically mounted on either side of the vertical axis of the drive shaft. The central control system is mounted on the mobile platform and communicates with both the mobile platform and the two multi-joint robotic arms via I / O ports to transmit control signals and receive sensor signals.

[0007] Preferably, the mobile platform is equipped with an image sensor and communicates with the central control system via an I / O port to calculate the spatial coordinates of the target to be picked, using image segmentation and positioning technologies; the image sensor is a depth camera; the mobile platform is equipped with a three-axis gyroscope and an accelerometer, and communicates with the central control system via an I / O port to identify the position and speed of the mobile platform; each servo motor of the multi-joint robotic arm is equipped with a position sensor and a speed sensor, and communicates with the central control system via an I / O port to obtain joint angles and angular velocities.

[0008] Preferably, the two multi-joint robotic arms select different end effectors depending on the different harvesting tasks.

[0009] A neurodynamic-based control method for a dual-arm harvesting robot, applied to a neurodynamic-based dual-arm harvesting robot control system, combining... Figure 2 Specifically, it includes the following steps:

[0010] S1: The central control system acquires sensor data in real time;

[0011] S2: The central control system uses target detection methods to identify the harvesting target based on sensor data and calculates the location of the harvesting target;

[0012] S3: The central control system calculates its current state by combining sensor data;

[0013] S4: The central control system decomposes the task based on the location of the picking target and its own status to obtain the expected state of the mobile platform and the expected end state of the two multi-joint robotic arms.

[0014] S5: Based on the current state and the desired state, build a dynamic neural network solver and train it using historical data;

[0015] S6: Load the trained dynamic neural network solver into the central control system via the communication module;

[0016] S7: The central control system uses a trained dynamic neural network solver to calculate the control strategy in real time and drives the servo motor to perform the harvesting task.

[0017] For ease of description, this invention uses two coordinate systems to describe the stationary and moving coordinates respectively. An absolute coordinate system OXYZ is established in the workspace of the neurodynamics-based dual-arm harvesting robot control system, containing the origin O and the X, Y, and Z axes.

[0018] Furthermore, the sensor data mentioned in step S1 includes: image data from the image sensor, encoded data from the position sensor and rotational speed data from the speed sensor, and position and speed data of the moving platform calculated by the three-axis gyroscope and accelerometer.

[0019] Furthermore, the target detection method described in step S2 is a target detection method for image data, including the YOLO algorithm, etc. It uses a frame to anchor the target, which is an existing technology that can be directly called; combined with a depth camera, the position of the detected target can be directly calculated, which is also an existing technology.

[0020] Furthermore, the current state of itself described in step S3 at the current time t includes two parts:

[0021] (1) Data directly measured by sensors, including: angles of all servo motors in the neurodynamics-based dual-arm harvesting robot control system. and angular velocity And the position coordinates of the connection point between the mobile platform and the left multi-joint robotic arm in the three directions of the absolute coordinate system OXYZ. and speed in, These are the angular velocities and angles of the current left and right drive wheels, respectively. These are vectors representing the angular velocities and angles of the current left and right multi-joint robotic arms, respectively. m represents the number of joints in a single multi-joint robotic arm. The subscript l corresponds to the left multi-joint robotic arm, and the subscript r corresponds to the right multi-joint robotic arm.

[0022] (2) The data calculated based on data directly measured by sensors and the geometric dimensions of the neurodynamic dual-arm harvesting robot control system includes: the linear velocity vector of the end effector in the absolute coordinate system OXYZ of the neurodynamic dual-arm harvesting robot control system. and end position and Jacobi matrix The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix of the Jacobian matrix. in, These are the velocity vectors of the end effectors corresponding to the left and right multi-joint robotic arms, respectively. These are the end positions of the end effectors corresponding to the left and right multi-joint robotic arms, respectively.

[0023] Furthermore, the desired state of the mobile platform described in step S4 is the desired coordinate position of the connection point between the mobile platform and the left multi-joint robotic arm in the absolute coordinate system OXYZ when performing the picking task. The desired end-effector state of the multi-joint robotic arm is the desired linear velocity vector of the end effector. Desired end position And the corresponding left multi-joint robotic arm, and the expected joint angles of the left multi-joint robotic arm. and

[0024] The dynamic neural network solver consists of an auxiliary network for a neural differential equation cascaded with a main network, where the network weights do not require training. Specifically, the input to the neural differential equation is the Jacobian matrix K(t) and the desired linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the inputs of the main network are the hidden state s(t), the Jacobian matrix K(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix K of the Jacobian matrix. + (t), the output is the angular velocity control signal of the servo motor. Where the projection equation For activation function, ω L ω U Angular velocity of all servo motors The values ​​of ζ and α are lower and upper bounds, respectively, and I is the identity matrix.

[0025] Preferably, due to the pseudo-inverse matrix K of the Jacobian matrix + The calculation of (t) may encounter a situation where the Jacobian matrix K(t) is singular and cannot be calculated. The dynamic neural network solver consists of an auxiliary network of a neural differential equation connected in series with a main network, where the network weights do not require training. Specifically, the input of the neural differential equation is the expected linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the input of the main network is the hidden state s(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment is given, and the output is the angular velocity control signal of the servo motor. in For the projection equation, ω L ω U Angular velocity of all servo motors The values ​​of ζ and α are lower and upper bounds, respectively. ζ and α are hyperparameters set by the user. W1, W2, W3, and W4 are weight matrices with training parameters.

[0026] To minimize the performance metric of the neurodynamics-based dual-arm harvesting robot control system, the minimum angular velocity vector of all servo motors is required. The quadratic function, i.e., minimizing the performance index, is the optimization objective matrix for state adjustment. in, It is a symmetric positive definite matrix. To optimize the vector, Λ and κ are determined by the objective to be optimized.

[0027] Preferably, when the task performed by the neurodynamics-based dual-arm harvesting robot control system does not involve the end effector task, i.e., it is considered as a forward kinematics problem in the harvesting process, the dynamic neural network solver sets the desired linear velocity vector of the end effector. The zero vector is ζ, which is set to 0 to simplify the calculation and improve efficiency.

[0028] The beneficial effects of this invention are as follows: This invention provides a control system and method for a dual-arm harvesting robot based on neurodynamics. For harvesting operations, a control system for a dual-arm harvesting robot based on neurodynamics is proposed. It innovatively designs a dynamic neural network solver using a gradient descent method with speed compensation. This solver can theoretically achieve zero position error control, including precise control of the moving platform, enabling it to complete linear movement or rotation in place and accurately reach the predetermined position. At the same time, under the premise of considering joint constraints and preset tasks, it can precisely control the control system of the dual-arm harvesting robot based on neurodynamics. Attached Figure Description

[0029] To illustrate the objectives and technical solutions of this invention, the following figures are provided:

[0030] Figure 1 This is an architecture diagram of the neurodynamic-based dual-arm harvesting robot control system in this invention; where dashed lines represent circuit connections and solid lines represent mechanical connections.

[0031] Figure 2This is a flowchart of the control method for the dual-arm harvesting robot based on neurodynamics in this invention;

[0032] Figure 3 This is an architecture diagram of the dynamic neural network solver in Embodiment 2 of the present invention;

[0033] Figure 4 This is a schematic diagram of the equivalent optimization problem of the dynamic neural network solver in Embodiment 2 of the present invention;

[0034] Figure 5 This is a diagram showing the joint angle changes of the two multi-joint robotic arms in Embodiment 2 of the present invention; where the horizontal axis represents time (unit: seconds).

[0035] Figure 6 This is a diagram showing the position change of the mobile platform in Embodiment 2 of the present invention; where the horizontal axis represents time (unit: seconds). Detailed Implementation

[0036] To make the objectives and technical solutions of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0037] Example 1: In large-scale citrus cultivation, traditional manual harvesting faces challenges such as labor shortages, high costs, and risks associated with working at heights. To address this, mobile harvesting robots have been introduced into orchard settings: a mobile platform equipped with a robotic arm autonomously navigates to the fruit-bearing area in complex terrain. Existing technologies primarily capture images of the fruit trees using binocular cameras, employing target detection algorithms to identify citrus fruits in real time, generating frames to anchor the harvesting target, and identifying the three-dimensional spatial coordinates of the fruit, providing precise positioning data for the robotic arm. However, current motion control of mobile harvesting robots mainly focuses on the task execution of the end effector, neglecting the precise control of the mobile platform, and suffers from poor real-time performance. Therefore, this invention proposes a "neurodynamics-based dual-arm harvesting robot control system." For ease of description, two coordinate systems are used to describe the stationary and moving coordinates respectively. An absolute coordinate system OXYZ, with an origin at O ​​and the X, Y, and Z axes, is established in the workspace of the neurodynamics-based dual-arm harvesting robot control system.

[0038] Specifically, the aforementioned neurodynamics-based dual-arm harvesting robot control system, combined with Figure 1The system consists of a mobile platform (1), a left multi-joint robotic arm (2), a right multi-joint robotic arm (3), and a central control system (4). The mobile platform (1) is a small car with two servo motors driving the left and right wheels respectively. The left multi-joint robotic arm (2) and the right multi-joint robotic arm (3) are the same, consisting of six robotic arms driven by independent servo motors for each joint. The central control system (4) is a computer with a processor of one of CPU, GPU, or NPU, equipped with a communication module and containing a dynamic neural network solver in its memory. The mobile platform (1) is equipped with a battery that supplies power to itself, the left multi-joint robotic arm (2), the right multi-joint robotic arm (3), and the central control system (4). The left multi-joint robotic arm (2) and the right multi-joint robotic arm (3) are symmetrically mounted on both sides of the vertical line of the drive shaft. The central control system (4) is mounted on the mobile platform and communicates with the mobile platform (1), the left multi-joint robotic arm (2), and the right multi-joint robotic arm (3) through I / O ports to send control signals and receive sensor signals.

[0039] The mobile platform (1) is equipped with an image sensor (11) and communicates with the central control system through an I / O port. It calculates the spatial coordinates of the target to be picked by combining image segmentation and positioning technologies. The image sensor (11) is a depth camera. The mobile platform (1) is equipped with a three-axis gyroscope (12) and an accelerometer (13) and communicates with the central control system (4) through an I / O port to identify the position and speed of the mobile platform. Each servo motor of the left multi-joint robotic arm (2) is equipped with a position sensor 1 (21) and a speed sensor 1 (22) and communicates with the central control system through an I / O port to obtain the joint angle and angular velocity. Each servo motor of the right multi-joint robotic arm (3) is equipped with a position sensor 2 (31) and a speed sensor 2 (32) and communicates with the central control system through an I / O port to obtain the joint angle and angular velocity.

[0040] The end effector of the left multi-joint robotic arm (2) is a pair of scissors, and the end effector of the right multi-joint robotic arm (3) is a mechanical gripper.

[0041] In detail, the neurodynamic-based dual-arm harvesting robot control system uses image recognition technology via image sensor (11) to detect and locate citrus fruits that meet the required maturity level. Then, it combines data from a three-axis gyroscope (12), accelerometer (13), position sensor 1 (21), speed sensor 1 (22), position sensor 2 (31), and speed sensor 2 (32) to determine its own position and speed information, i.e., its own state. Then, it combines a path planning algorithm to realize the path planning of the mobile platform (1), the left multi-joint robotic arm (2), and the right multi-joint robotic arm (3), and converts the path information into control signals for each servo motor to achieve the citrus harvesting operation.

[0042] Example 2: Regarding the scenario and apparatus of Example 1, to ensure the accuracy and speed of citrus harvesting operations, while considering the kinematic equations of the robotic arm, the kinematic relationship of the end effector's orientation, and the limitations of joint physical constraints, this example provides a "control method for a dual-arm harvesting robot based on neurodynamics," combined with... Figure 2 It includes the following steps:

[0043] Step 1: The central control system (4) acquires sensor data in real time.

[0044] The sensor data includes: image data from image sensor (11), encoded data from position sensor 1 (21) and position sensor 2 (31) coded by servo motor, rotational speed data from rotational speed sensor 1 (22) and rotational speed sensor 2 (32), and position and speed data of the moving platform calculated by three-axis gyroscope (12) and accelerometer (13).

[0045] Step 2: The central control system (4) uses the target detection method to identify the picking target based on the sensor data and calculates the location of the picking target.

[0046] The target detection method described is an image data-based target detection method, including algorithms such as YOLOv5s, which uses a frame to anchor the picking target; combined with a binocular depth camera, the location of the detected picking target can be directly calculated. The ripeness of citrus fruits can be detected using a ripeness detection method for crabapples based on hyperspectral imaging technology and convolutional neural networks.

[0047] Step 3: The central control system (4) calculates its current state by combining sensor data.

[0048] The current state of itself described in step S3 at the current time t includes two parts:

[0049] (1) Data directly measured by sensors, including: angles of all servo motors in the neurodynamics-based dual-arm harvesting robot control system. and angular velocity And the position coordinates of the connection point between the mobile platform and the left multi-joint robotic arm on the horizontal plane of the absolute coordinate system OXYZ. and speed in, These are the angular velocities and angles of the current left and right drive wheels, respectively. These are vectors representing the angular velocities and angles of the current left and right multi-joint robotic arms, respectively. The subscript *l* corresponds to the left multi-joint robotic arm, and the subscript *r* corresponds to the right multi-joint robotic arm. At time t=0, φ(0)=φ r (0) = 0 rad, θ l (0)=θ r (0) = [0, 0.67, 0, 1.3, 0, 0.5] T rad.

[0050] (2) The data calculated based on data directly measured by sensors and the geometric dimensions of the neurodynamic dual-arm harvesting robot control system includes: the linear velocity vector of the end effector in the absolute coordinate system OXYZ of the neurodynamic dual-arm harvesting robot control system. and the terminal position r(t) = [p l (t),p r [t]; and Jacobian matrix The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix of the Jacobian matrix. in, These are the velocity vectors of the end effectors corresponding to the left and right multi-joint robotic arms, respectively. These are the end positions of the end effectors corresponding to the left and right multi-joint robotic arms, respectively.

[0051] Step 4: Central Control System (4) Decomposes the task according to the location of the picking target and its own state to obtain the expected state of the mobile platform and the expected state of the end of the two multi-joint robotic arms.

[0052] The desired state of the mobile platform is the desired coordinate position [x] of the connection point between the mobile platform and the left multi-joint robotic arm in the absolute coordinate system OXYZ when performing the picking task. p ,y p ] = [1, 0.5], where the desired end-effector state of the multi-joint robotic arm is the desired linear velocity vector of the end effector. Desired end position And the corresponding left multi-joint robotic arm, and the expected joint angle θ of the left multi-joint robotic arm. p =θ rp =[0,0,0,0,0,0]T rad.

[0053] Step 5: Based on the current state and the desired state, build a dynamic neural network solver and train it using historical data.

[0054] Combination Figure 3 The dynamic neural network solver consists of an auxiliary network for a neural differential equation connected in series with a main network, where the network weights do not require training. Specifically, the input to the neural differential equation is the Jacobian matrix K(t) and the desired linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the inputs of the main network are the hidden state s(t), the Jacobian matrix K(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix K of the Jacobian matrix. + (t), the output is the angular velocity control signal of the servo motor. Where the projection equation For activation function, ω L ω U Angular velocity of all servo motors The lower and upper bounds of the value of ζ are given, ζ = 10. 5 α = 10 is a hyperparameter, and I is the identity matrix.

[0055] To minimize the performance metric of the neurodynamics-based dual-arm harvesting robot control system, the minimum angular velocity vector of all servo motors is required. The quadratic function, i.e., minimizing the performance index, is the optimization objective matrix for state adjustment. in, It is a symmetric positive definite matrix.

[0056] To optimize the vector, Λ and κ are determined by the objective to be optimized.

[0057] Combination Figure 4 The principle of the dynamic neural network solver of the present invention is explained in detail. For the optimization function of the dual-arm picking robot control system based on neurodynamics, the corresponding objective function, constraint equation and restriction condition can be transformed into solving the velocity compensation, feedback term and projection equation, avoiding the complex Jacobian matrix inversion operation.

[0058] Step 6: Load the trained dynamic neural network solver into the central control system via the communication module (4).

[0059] Step 7: The central control system (4) uses the trained dynamic neural network solver to calculate the control strategy in real time and drives the servo motor to perform the picking task.

[0060] Combination such as Figure 5 , Figure 6 The execution results shown demonstrate that the mobile platform (1), the left multi-joint robotic arm (2), and the right multi-joint robotic arm (3) can all accurately follow the desired position and have a fast response speed.

[0061] Step 8: Repeat step 7 until all harvesting is complete.

[0062] In practical applications, steps five and six can be performed offline to train the model, while the remaining steps can be executed online.

[0063] To better demonstrate the beneficial effects of the present invention, a comparative experiment was conducted in this embodiment with the method of reference [1]. The comparison results are shown in Table 1.

[0064] [1]Z.Zhang, S.Yang, S.Chen, Y.Luo, H.Yang, and Y.Liu,Avector-basedconstrained obstacle avoidance scheme for wheeled mobile redundant robotmanipulator,IEEE Trans.Cogn.Dev.Syst.,vol.13,no.3,pp.465–474,Sep.2021.

[0065] Table 1 Comparison of experimental results

[0066]

[0067] It is evident that the method of the present invention is several orders of magnitude more accurate than the prior art in reference [1].

[0068] Example 3: The pseudo-inverse matrix K of the Jacobian matrix in Example 2 + The calculation of (t) may encounter situations where the Jacobian matrix K(t) is singular and cannot be calculated. This embodiment provides a "control method for a dual-arm harvesting robot based on neurodynamics", which combines... Figure 2 The detailed steps that are the same as in Example 2 will not be repeated here. The difference is that the following steps are included:

[0069] Step 1: The central control system (4) acquires sensor data in real time.

[0070] Step 2: The central control system (4) uses the target detection method to identify the picking target based on the sensor data and calculates the location of the picking target.

[0071] Step 3: The central control system (4) calculates its current state by combining sensor data.

[0072] Step 4: Central Control System (4) Decomposes the task according to the location of the picking target and its own state to obtain the expected state of the mobile platform and the expected state of the end of the two multi-joint robotic arms.

[0073] Step 5: Based on the current state and the desired state, build a dynamic neural network solver and train it using historical data.

[0074] The dynamic neural network solver consists of an auxiliary network for a neural differential equation connected in series with a main network, where the network weights do not require training. Specifically, the input to the neural differential equation is the desired linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the input of the main network is the hidden state s(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment is given, and the output is the angular velocity control signal of the servo motor. in For the projection equation, ω L ω U Angular velocity of all servo motors The values ​​of ζ and α are lower and upper bounds, respectively. ζ and α are hyperparameters set by the user. W1, W2, W3, and W4 are weight matrices with training parameters.

[0075] Step 6: Load the trained dynamic neural network solver into the central control system via the communication module (4).

[0076] Step 7: The central control system (4) uses the trained dynamic neural network solver to calculate the control strategy in real time and drives the servo motor to perform the picking task.

[0077] Step 8: Repeat step 7 until all harvesting is complete.

[0078] Preferably, when the task performed by the neurodynamics-based dual-arm harvesting robot control system does not involve the end effector task, i.e., it is considered as a forward kinematics problem in the harvesting process, the dynamic neural network solver sets the desired linear velocity vector of the end effector. The zero vector is ζ, which is set to 0 to simplify the calculation and improve efficiency.

[0079] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A control system for a dual-arm harvesting robot based on neurodynamics, characterized in that, The system comprises a mobile platform, a left multi-joint robotic arm, a right multi-joint robotic arm, and a central control system. The mobile platform is a trolley driven by dual servo motors. The left and right multi-joint robotic arms are identical, consisting of multiple joints driven by an independent servo motor. The central control system is a computer with a processor of CPU, GPU, or NPU, equipped with a communication module and a dynamic neural network solver in its memory. The mobile platform has a battery that powers itself, the two multi-joint robotic arms, and the central control system. The two multi-joint robotic arms are symmetrically mounted on either side of the drive shaft. The central control system is mounted on the mobile platform and communicates with both the mobile platform and the two multi-joint robotic arms via I / O ports to transmit control signals and receive sensor signals.

2. The control system for the neurodynamic-based dual-arm harvesting robot according to claim 1, characterized in that, The mobile platform is equipped with an image sensor and communicates with the central control system via an I / O port. It uses image segmentation and positioning technologies to calculate the spatial coordinates of the target to be harvested. The image sensor is a depth camera. The mobile platform is also equipped with a three-axis gyroscope and an accelerometer, which communicate with the central control system via an I / O port to identify the position and speed of the mobile platform. Each servo motor of the multi-joint robotic arm is equipped with a position sensor and a speed sensor, which communicate with the central control system via an I / O port to obtain the joint angle and angular velocity.

3. The control system for the neurodynamic-based dual-arm harvesting robot according to claim 1, characterized in that, The two multi-joint robotic arms select different end effectors depending on the harvesting task.

4. A neurodynamic-based dual-arm harvesting robot control method applied to the neurodynamic-based dual-arm harvesting robot control system according to any one of claims 1 to 3, characterized in that, Includes the following steps: S1: The central control system acquires sensor data in real time; S2: The central control system uses target detection methods to identify the harvesting target based on sensor data and calculates the location of the harvesting target; S3: The central control system calculates its current state by combining sensor data; S4: The central control system decomposes the task based on the location of the picking target and its own status to obtain the expected state of the mobile platform and the expected end state of the two multi-joint robotic arms. S5: Based on the current state and the desired state, build a dynamic neural network solver and train it using historical data; S6: Load the trained dynamic neural network solver into the central control system; S7: The central control system uses a trained dynamic neural network solver to calculate the control strategy in real time and drives the servo motor to perform the harvesting task.

5. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that, The current state of itself described in step S3 at the current time t includes two parts: (1) Data directly measured by sensors, including: angles of all servo motors in the neurodynamics-based dual-arm harvesting robot control system. and angular velocity And the position coordinates of the connection point between the mobile platform and the left multi-joint robotic arm in the three directions of the absolute coordinate system OXYZ. and speed in, These are the angular velocities and angles of the current left and right drive wheels, respectively. These are vectors representing the angular velocities and angles of the current left and right multi-joint robotic arms, respectively, where m is the number of joints in a single multi-joint robotic arm, and the subscripts are... The left multi-joint robotic arm corresponds to the left multi-joint robotic arm, and the subscript r corresponds to the right multi-joint robotic arm. (2) The data calculated based on data directly measured by sensors and the geometric dimensions of the neurodynamic dual-arm harvesting robot control system includes: the linear velocity vector of the end effector in the absolute coordinate system OXYZ of the neurodynamic dual-arm harvesting robot control system. and end position and Jacobi matrix The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix of the Jacobian matrix. in, These are the velocity vectors of the end effectors corresponding to the left and right multi-joint robotic arms, respectively. These are the end positions of the end effectors corresponding to the left and right multi-joint robotic arms, respectively.

6. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that, The desired state of the mobile platform described in step S4 is the desired coordinate position of the connection point between the mobile platform and the left multi-joint robotic arm in the absolute coordinate system OXYZ when performing the picking task. The desired end-effector state of the multi-joint robotic arm is the desired linear velocity vector of the end effector. Desired end position And the corresponding left multi-joint robotic arm, and the expected joint angles of the left multi-joint robotic arm. and 7. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that, The dynamic neural network solver consists of an auxiliary network for a neural differential equation cascaded with a main network, where the network weights do not require training. Specifically, the input to the neural differential equation is the Jacobian matrix K(t) and the desired linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the inputs of the main network are the hidden state s(t), the Jacobian matrix K(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment, and the pseudo-inverse matrix K of the Jacobian matrix. + (t), the output is the angular velocity control signal of the servo motor. Where the projection equation For activation function, ω L ω U Angular velocity of all servo motors The values ​​of ζ and α are lower and upper bounds, respectively, and I is the identity matrix.

8. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that, The dynamic neural network solver consists of an auxiliary network for a neural differential equation connected in series with a main network, where the network weights do not require training. Specifically, the input to the neural differential equation is the desired linear velocity vector of the end effector. The output is the derivative of the hidden state s(t). Where σ(·) is the sigmoid activation function; the input of the main network is the hidden state s(t), the terminal position r(t), and the desired terminal position r. p The optimization objective matrix L(t) for state adjustment is given, and the output is the angular velocity control signal of the servo motor. in For the projection equation, ω L ω U Angular velocity of all servo motors The values ​​of ζ and α are lower and upper bounds, respectively. ζ and α are hyperparameters set by the user. W1, W2, W3, and W4 are weight matrices with training parameters.

9. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that, The optimization objective matrix for state adjustment in, It is a symmetric positive definite matrix. To optimize the vector, Λ and κ are determined by the objective to be optimized.

10. The control method for a dual-arm harvesting robot based on neurodynamics according to claim 4, characterized in that... When the task performed by the neurodynamics-based dual-arm harvesting robot control system does not involve the end effector task, i.e., it is considered as a forward kinematics problem in the harvesting process, the dynamic neural network solver sets the desired linear velocity vector of the end effector. The zero vector is ζ, which is set to 0 to simplify the calculation and improve efficiency.

Images