Picking robot control method and system based on three-dimensional reconstruction and reinforcement learning

Abstract

The application discloses a kind of based on three-dimensional reconstruction and reinforcement learning's picking robot control method, including the following steps: collection orchard multi-view image, using 3D Gaussian splashing rendering method will orchard multi-view image convert virtual orchard scene;Fruit positioning is carried out based on virtual orchard scene, and fruit picking task database is established;According to picking robot parameter, the digital twin model corresponding to picking robot is established;Using virtual orchard scene and fruit picking task database, based on SAC algorithm, the digital twin model is trained to reinforcement learning, and the optimal picking path and picking action are obtained;Picking path and picking action are mapped to picking robot, and picking robot executes fruit picking task according to picking path and picking action;Feedback data in the process of using picking robot to execute fruit picking task is used to update the reinforcement learning training parameters of digital twin model, and picking robot executes fruit picking according to optimized picking path and picking action.

Description

Technical Field

[0001] This invention relates to the field of intelligent equipment technology, and more specifically, to an intelligent control method and system for an orchard harvesting robot based on multi-view 3D reconstruction and reinforcement learning. Background Technology

[0002] In recent years, with the rapid development of smart agriculture and digital twin technology, orchard harvesting robots have gradually become an important research direction for intelligent agricultural equipment. The combination of computer vision and robot control to achieve automatic fruit identification, positioning, and harvesting is one of the key technologies driving intelligent agricultural production. However, existing orchard harvesting robots still face the following technical bottlenecks in complex natural environments:

[0003] Insufficient accuracy in 3D environment modeling. Traditional 3D reconstruction methods mainly rely on sparse point cloud reconstruction based on monocular or binocular vision. In environments like orchards with high-density vegetation, complex lighting, and occlusion features, the reconstructed models often suffer from sparse, discontinuous point clouds and high noise, making it difficult to accurately reflect the spatial structural features of fruit trees, branches, leaves, and fruits. This leads to insufficient accuracy in subsequent fruit recognition and localization, severely affecting the accuracy of robotic arm grasping decisions.

[0004] Fruit recognition and localization lack robustness. Current fruit detection methods mostly employ two-dimensional image recognition algorithms or shallow point cloud segmentation, which are poorly adapted to complex lighting conditions, dense foliage, and overlapping fruits. When a fruit is partially occluded or its color is similar to the background, the recognition accuracy drops significantly. Furthermore, fruit localization methods based on geometric projection have limited accuracy, making it difficult to obtain the fruit's true pose and radius information in three-dimensional space, thus affecting the accuracy and success rate of robotic arm grasping.

[0005] Reinforcement learning policies exhibit weak generalization ability. Existing reinforcement learning control methods based on algorithms such as PPO (Proximal Policy Optimization) and DDPG (Deep Deterministic Policy Gradient) can achieve path planning and motion control of robotic arms, but they are prone to problems such as training instability, slow policy convergence, or oscillations in high-dimensional continuous motion spaces. Furthermore, these algorithms have poor policy generalization; when the fruit position, posture, or ambient lighting changes, the model performance significantly degrades, making it difficult to adapt to the complex dynamic environment of orchards.

[0006] The coupling between virtual simulation and physical systems is insufficient. Currently, most harvesting robots still rely primarily on virtual simulation platforms for reinforcement learning training. Due to differences between the simulation environment and the real orchard in physical properties such as light, friction, fruit elasticity, and noise, the control strategies obtained through virtual training often cannot be directly replicated in reality. This leads to a significant degradation in "simulation-to-real" transfer performance, resulting in insufficient robustness of the system in real-world operations.

[0007] In summary, existing orchard picking robot systems have limitations in areas such as 3D environment reconstruction, intelligent grasping strategy learning, and virtual-real fusion control. Summary of the Invention

[0008] The purpose of this invention is to overcome the defects and shortcomings of the existing technology and provide a control method and system for a picking robot based on 3D reconstruction and reinforcement learning. By introducing the 3D Gaussian splash rendering method, continuous visual reconstruction of the orchard scene is achieved; reinforcement learning training based on the SAC algorithm is used to improve the stability and generalization of the robotic arm's strategy learning; and through bidirectional data interaction between the robot operating system and the simulation platform, virtual-real fusion and closed-loop optimization are achieved, thereby enabling the orchard picking robot to achieve high-precision perception, autonomous decision-making and intelligent control in complex natural environments.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A control method for a harvesting robot based on 3D reconstruction and reinforcement learning includes the following steps:

[0011] By collecting multi-view images of the orchard using drones and mobile robots, and using the 3D Gaussian splash rendering method for three-dimensional reconstruction, the multi-view images of the orchard are transformed into a virtual orchard scene containing fruit trees, branches and leaves and fruits.

[0012] Based on a virtual orchard scene, fruit area detection and spatial positioning are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture and maturity, and to establish a fruit picking task database.

[0013] Based on the geometric, kinematic, and dynamic parameters of the real picking robot, a digital twin model corresponding to the real picking robot is established in a virtual orchard scene to simulate the fruit picking path and picking actions.

[0014] By using a virtual orchard scene and a fruit picking task database, reinforcement learning training of the digital twin model is performed based on the SAC algorithm to obtain the optimal picking path and picking action for the picking robot to perform fruit picking tasks.

[0015] The picking path and picking actions are mapped to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality.

[0016] By using feedback data from the fruit-picking process of the picking robot, the reinforcement learning training parameters of the digital twin model are updated, and the picking robot performs fruit picking according to the optimized picking path and picking actions.

[0017] Furthermore, a 3D Gaussian splash rendering method is used for 3D reconstruction, transforming the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches, and fruits. Specifically:

[0018] Photometric correction and distortion correction were performed on multi-view images of the orchard;

[0019] Sparse point clouds are generated using the SFM algorithm;

[0020] The color, depth, and density information of each point in the point cloud are mapped to a three-dimensional Gaussian distribution. Continuous volume rendering is achieved through 3D Gaussian splashing to generate a visualized virtual orchard scene.

[0021] Furthermore, based on the virtual orchard scene, fruit area detection and spatial localization are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture, and maturity, and a fruit picking task database is established. Specifically:

[0022] Semantic segmentation was performed on the virtual orchard scene generated by 3D Gaussian splashing, and the PointNet++ network was used to divide the point cloud into fruit, branches and leaves and background.

[0023] An improved DBSCAN algorithm was used to cluster the fruit point cloud to calculate the three-dimensional center coordinates, orientation, and radius of the fruit.

[0024] Fruit surface color features were extracted based on the HSV color space for maturity assessment, and a fruit harvesting task database was established.

[0025] Furthermore, utilizing a virtual orchard scenario and a database of fruit picking tasks, the digital twin model is trained using reinforcement learning based on the SAC algorithm, specifically as follows:

[0026] Using a virtual orchard scenario as the background, a digital twin model of a harvesting robot as the intelligent agent, and a fruit harvesting task database as the task objective, the digital twin model is trained using reinforcement learning based on the SAC algorithm. The trained digital twin model is then output, which is used by the harvesting robot to generate corresponding control commands based on real-time status information during actual operation.

[0027] Furthermore, the objective function of the SAC algorithm is defined as:

[0028] ;

[0029] Where J(π) represents the objective function value of the policy function π; E represents the mathematical expectation; s∈S represents the state variable in the environment, where S is the state space; a∈A represents the action variable in the environment, where A is the action space; α is the entropy adjustment coefficient; Q(s,a) is the value function, representing the value of performing action a in state s; π(a|s) is the policy probability distribution function, representing the probability density of performing action a in state s.

[0030] Furthermore, a reinforcement learning environment is defined as:

[0031] State space S = {robotic arm joint angles, end effector pose, coordinates of fruit relative to the robotic arm, obstacle position, gripper state};

[0032] Motion space A = {joint angular velocity, gripper opening / closing range};

[0033] The reinforcement learning reward function is:

[0034] ;

[0035] Where R represents the total reward value; R grasp To retrieve the reward item upon successful capture; R collision For collision penalty items; R smooth λ is the motion smoothness reward; β is the collision penalty weight coefficient; and β is the smoothness reward weight coefficient.

[0036] Furthermore, the harvesting path and harvesting actions are mapped to the harvesting robot, specifically as follows:

[0037] The robot operating system enables data interaction between virtual simulation and real harvesting robots: the virtual end acts as the control command issuing end, outputting control commands for the harvesting robot; the real end acts as the execution end, receiving control commands through the robot operating system bridge middleware and driving the various servo motors of the harvesting robot to perform actions.

[0038] A harvesting robot control system based on 3D reconstruction and reinforcement learning, applying any one of the above-mentioned harvesting robot control methods based on 3D reconstruction and reinforcement learning, includes:

[0039] The 3D reconstruction module is used to collect multi-view images of the orchard through drones and mobile robots, and to perform 3D reconstruction using the 3D Gaussian splash rendering method, transforming the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches and leaves and fruits.

[0040] The fruit localization module is used to detect and spatially locate fruit areas based on a virtual orchard scene. It extracts fruit features through point cloud segmentation and cluster analysis, obtains fruit location, size, posture and maturity, and establishes a fruit picking task database.

[0041] The digital twin module is used to create a digital twin model of the real picking robot in a virtual orchard scene based on the geometric, kinematic and dynamic parameters of the real picking robot, and to simulate the fruit picking path and picking action.

[0042] The reinforcement learning module is used to train the digital twin model using the virtual orchard scene and fruit picking task database based on the SAC algorithm, so as to obtain the optimal picking path and picking action of the picking robot to perform fruit picking tasks.

[0043] The virtual-real control module is used to map the picking path and picking actions to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality.

[0044] The feedback optimization module is used to update the reinforcement learning training parameters of the digital twin model using feedback data from the fruit-picking robot during the fruit-picking process. The fruit-picking robot then performs fruit picking according to the optimized picking path and actions.

[0045] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the harvesting robot control method based on 3D reconstruction and reinforcement learning as described above.

[0046] A storage medium, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the harvesting robot control method based on 3D reconstruction and reinforcement learning as described above.

[0047] Compared with existing technologies, this invention achieves continuous rendering and spatial reconstruction of fruit trees, branches and leaves and fruits through 3D Gaussian splashing technology, overcomes the technical bottleneck of traditional dense reconstruction, completes high-fidelity modeling of fine structures such as orchard branches and leaves, and significantly improves the realism and geometric accuracy of the virtual environment under complex lighting and occlusion.

[0048] This invention utilizes the SAC algorithm for reinforcement learning training, enabling the robotic arm of a harvesting robot to possess more stable learning capabilities and policy generalization in a high-dimensional continuous motion space. This allows for high-success-rate grasping under varying fruit distributions and obstacle layouts. By using ROS communication to map virtual policies to the actual robotic arm's motion commands, simulation training results can be directly applied to actual harvesting operations, significantly improving the accuracy of policy transfer and control consistency.

[0049] This invention utilizes a self-learning process based on closed-loop feedback to dynamically correct virtual model parameters and control strategies, achieving continuous optimization and adaptive evolution. It is not only applicable to automated harvesting operations in orchards for lychees, apples, and citrus fruits, but can also be extended to other agricultural robotic scenarios, such as sorting, pruning, and fertilization, demonstrating significant potential for industrial application and economic value. Attached Figure Description

[0050] Figure 1 This is a flowchart of a harvesting robot control method based on 3D reconstruction and reinforcement learning.

[0051] Figure 2 This is a schematic diagram illustrating the construction of a virtual orchard scene.

[0052] Figure 3 A schematic diagram illustrating the construction of a digital twin model of a harvesting robot.

[0053] Figure 4 A flowchart illustrating the process of enhancing learning and training.

[0054] Figure 5 This is a schematic diagram illustrating the interaction between a virtual environment and a harvesting robot.

[0055] Figure 6 This is a schematic diagram of the harvesting path and posture of the harvesting robot. Detailed Implementation

[0056] The following description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the harvesting robot control method and system based on 3D reconstruction and reinforcement learning of the present invention.

[0057] Please see Figure 1 This invention discloses a control method for a harvesting robot based on 3D reconstruction and reinforcement learning, comprising the following steps:

[0058] S1: Collect multi-view images of the orchard using drones and mobile robots, and use the 3D Gaussian splash rendering method for three-dimensional reconstruction to transform the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches and fruits.

[0059] S2: Based on the virtual orchard scene, fruit area detection and spatial positioning are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture and maturity, and to establish a fruit picking task database.

[0060] S3: Based on the geometric, kinematic and dynamic parameters of the real picking robot, establish a digital twin model corresponding to the real picking robot in the virtual orchard scene to simulate the fruit picking path and picking action;

[0061] S4: Using a virtual orchard scene and a fruit picking task database, the digital twin model is trained by reinforcement learning based on the SAC algorithm to obtain the optimal picking path and picking action for the picking robot to perform the fruit picking task.

[0062] S5: Map the picking path and picking actions to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality.

[0063] S6: Utilize feedback data from the fruit-picking process performed by the picking robot to update the reinforcement learning training parameters of the digital twin model. The picking robot then performs fruit picking according to the optimized picking path and picking actions.

[0064] Step S1: Collect multi-view images of the orchard using drones and mobile robots, and perform three-dimensional reconstruction using the 3D Gaussian splash rendering method to transform the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches, leaves, and fruits.

[0065] like Figure 2 As shown, firstly, based on the layout and extent of the lychee orchard, the area to be sampled is determined and the sampling path is planned. After a high-definition camera is installed on a mobile robot, the robot takes two-dimensional images of the orchard from different angles along the set path to obtain precise distance information between the fruit trees and the fruit. Meanwhile, a drone takes images at different altitudes along a prescribed flight path to obtain two-dimensional images of the tree canopies, ensuring complete collection of the orchard's three-dimensional information.

[0066] Subsequently, the captured images underwent photometric calibration and distortion correction. A sparse point cloud was generated using the SFM (Structure from Motion) algorithm, and each point was converted into a 3D Gaussian ellipsoid as an initial setting. The color, depth, and density information of the points were mapped to a 3D Gaussian distribution, thus establishing a set of Gaussian ellipsoids. An adaptive density control strategy was employed to dynamically adjust the number and distribution of Gaussian ellipsoids based on the view space gradient, ultimately generating a virtual orchard scene with continuous visualization capabilities, containing high-fidelity geometric and textural information of fruit trees, branches, leaves, and fruits.

[0067] Step S2: Based on the virtual orchard scene, fruit area detection and spatial positioning are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture and maturity, and to establish a fruit picking task database.

[0068] Specifically, the virtual orchard scene was converted into dense point cloud data, with the point cloud density controlled at 1 cm³. A pre-trained PointNet++ semantic segmentation network was used to segment the point cloud into three categories: fruit, branches and leaves, and background. The input features of the segmentation network included: spatial coordinates, RGB color, and normal vector direction.

[0069] The segmented fruit point clouds were processed using an improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, and the following feature parameters were calculated for each cluster: 3D center coordinates: obtained through centroid calculation; fruit posture: determined by principal component analysis (PCA); radius parameter: calculated using the minimum bounding sphere; maturity assessment: based on HSV (Hue-Saturation-Value) color space analysis. Finally, a fruit picking task database was established based on the above parameters.

[0070] Step S3: Based on the geometric, kinematic, and dynamic parameters of the real picking robot, establish a digital twin model corresponding to the real picking robot in the virtual orchard scene to simulate the fruit picking path and picking actions.

[0071] like Figure 3 As shown, firstly, based on the physical parameters of the actual harvesting robot, a standard URDF (Unified Robot Description Format) description file is written. In the URDF file, the following are precisely defined: Link geometry: shape, size, mass, inertia matrix; Joint type: rotary joint, prismatic joint, etc.; Kinematic constraints: joint limits, velocity limits, torque capacity; Visual and collision attributes: Mesh geometry file, collision detection parameters.

[0072] Subsequently, using the Unity Robotics Hub plugin and the URDFImporter tool, the URDF file was directly imported into the Unity scene. This automatically generated the corresponding GameObject hierarchy, maintaining the parent-child relationship of the robotic arm, and configured the ArticulationBody component parameters to ensure consistency with the dynamic characteristics of a real robotic arm. Next, collision detection parameters were set, and the physical materials for the interaction between the gripper and the fruit were configured. Finally, forward kinematics testing was conducted to verify the accuracy of the end effector pose calculation.

[0073] Step S4: Using the virtual orchard scene and fruit picking task database, the digital twin model is trained by reinforcement learning based on the SAC algorithm to obtain the optimal picking path and picking action for the picking robot to perform the fruit picking task.

[0074] like Figure 4 As shown, reinforcement learning training is a multi-step process, mainly including environment modeling, network structure design, reward function determination, training, and evaluation. First, a virtual orchard scenario is used as the physical interaction environment (providing obstacle constraints), and a digital twin model of the picking robot is used as the agent. The fruit picking task database is used as the "target point set" for the training task. During training, fruit coordinates are randomly indexed from the fruit picking task database and set as the grasping target for the current round.

[0075] The reinforcement learning environment is defined with a state space of S = {robotic arm joint angles, end effector pose, fruit coordinates relative to the robotic arm, obstacle position, gripper state} and a motion space of A = {joint angular velocity, gripper opening / closing amount}. Core task requirements include: precise harvesting: accurately identifying and grasping ripe fruit; safe obstacle avoidance: avoiding collisions with fruit tree branches and other obstacles; efficient operation: completing the harvesting task in the shortest possible time; and trajectory optimization: generating smooth and energy-efficient motion trajectories.

[0076] Secondly, a suitable reinforcement learning algorithm (such as the SAC algorithm) is selected based on environmental modeling to assist the harvesting robot in decision-making and action. The objective function of the SAC algorithm can be expressed as:

[0077] ;

[0078] Where J(π) represents the objective function value of the policy function π; E represents the mathematical expectation; s∈S represents the state variable in the environment, where S is the state space; a∈A represents the action variable in the environment, where A is the action space; α is the entropy adjustment coefficient; Q(s,a) is the value function, representing the value of performing action a in state s; π(a|s) is the policy probability distribution function, representing the probability density of performing action a in state s.

[0079] Given the characteristics of the picking task, a carefully designed reward function is needed to guide the agent's learning. The reward function takes the following form:

[0080] ;

[0081] Where R represents the total reward value; Rgrasp is the reward for successful grasping; Rcollision is the collision penalty; Rsmooth is the smoothness reward; λ is the collision penalty weight coefficient; and β is the smoothness reward weight coefficient. This reward function quantifies the quality of each action of the robot, thereby optimizing the strategy during training.

[0082] The SAC algorithm is built on an Actor-Critic architecture, which includes a policy network (Actor) for generating action decisions and a value network (Critic) for evaluating the value of actions. As shown in Figure 4, the reinforcement learning training process mainly includes two core stages: environment interaction and agent training.

[0083] During the environmental interaction phase, the agent interacts with the virtual orchard scenario based on the current policy network, and at each time step, it adjusts its actions according to the current state s. t Output the corresponding action a t and receive instant rewards from the environment. t And the state s at the next moment t+1 The empirical data generated during the interaction process is presented in the form of quadruplets (s t ,a t ,r t ,s t+1 The samples are stored in the form of experience replay pool to eliminate temporal correlation between samples and improve training stability.

[0084] During the agent training phase, small batches of data are randomly sampled from the experience replay pool to update the network parameters. To alleviate the Q-value overestimation problem, this invention employs a dual-evaluation network structure (soft-Q1 and soft-Q2) and its corresponding target evaluation networks (target-soft-Q1 and target-soft-Q2). In the calculation of the target Q-value, the minimum value Min(Q1,Q2) of the outputs of the two evaluation networks is taken as the target estimate, and a loss function is constructed based on the difference between the target value and the current predicted value. The network parameters are then updated using the backpropagation algorithm.

[0085] This invention utilizes ML-Agents tools, combined with a virtual orchard scenario built based on digital twins, to conduct interactive training around target fruits in a fruit-picking task database. During training, the SAC algorithm synchronously updates the parameters of the policy network and the value network based on the feedback signal of the reward function, making the value network's evaluation of action rewards more accurate and guiding the policy network to learn and generate better joint motion paths and control commands.

[0086] After training convergence, the performance of the obtained policy is evaluated in a virtual environment. Since the value network is only used to assist policy learning during the training phase and does not participate in actual decision-making, only the trained policy network needs to be retained and exported during model deployment. During the runtime phase, this policy network generates corresponding action outputs online based on the real-time perceived state information of the picking robot, serving as input for subsequent trajectory planning and actual machine control, thereby enabling the picking robot to perform autonomous grasping operations in a real orchard environment.

[0087] Step S5: Map the picking path and picking actions to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained from reinforcement learning training, realizing synchronous picking control from virtual to reality.

[0088] like Figure 5 and Figure 6As shown, a virtual-real hybrid control system is established to achieve "virtual control of the real, with real-time feedback." First, in the Unity3D virtual simulation platform, based on the optimal action strategy output from reinforcement learning training, optimal trajectory planning and control instructions are generated to obtain the optimal picking path and actions of the picking robot. Data interaction between the virtual simulation and the real picking robot is achieved through the Robot Operating System (ROS): the virtual end (Unity simulation environment) acts as the control instruction issuing end, outputting joint control instructions, end effector action instructions, and gripper control parameters for the picking robot; the real end (the picking robot) acts as the execution end, receiving instructions through the ROS-Bridge (Robot Operating System Bridge) middleware and driving the various servo motors of the picking robot to perform actions.

[0089] Step S6: Using the feedback data from the fruit-picking process of the picking robot, update the reinforcement learning training parameters of the digital twin model, and the picking robot performs fruit picking according to the optimized picking path and picking actions.

[0090] Specifically, during the harvesting operation, the sensor devices on the harvesting robot (including RGB-D cameras, force sensors, joint encoders, and IMU inertial navigation structures) collect data on the robot's operating status and the external environment in real time, and publish this data to the virtual simulation environment through the ROS topic mechanism. The digital twin receives feedback information in real time and dynamically updates the virtual orchard scene and the robot's posture, achieving synchronous fitting between the virtual twin model and the physical entity's state.

[0091] An RGB-D camera is installed at the end of the robotic arm of the harvesting robot, and a force sensor is deployed at the gripper connection to monitor the grasping contact force. The joint encoder built into the servo motor provides real-time feedback of the joint angle. An IMU (Inertial Measurement Unit) inertial navigation structure is configured on the base to assist in attitude calculation.

[0092] When deviations are detected (such as end-effector pose error, abnormal gripper force, fruit slippage, etc.), the virtual terminal automatically triggers the local strategy optimization module, calls the pre-trained strategy network for rapid motion replanning, generates correction instructions, and sends them to the harvesting robot in real time, achieving closed-loop control. Simultaneously, a built-in two-way data recording and optimization mechanism is implemented: the state data of the harvesting robot during its execution phase (including motion trajectory, grasping success rate, and obstacle avoidance path) is transmitted back to the virtual twin for retraining and performance self-learning of the digital twin model; historical task data is used to periodically fine-tune the strategy network, achieving continuous strategy optimization and generalization.

[0093] This virtual-real integrated control achieves a seamless mapping from virtual simulation to real execution, enabling the optimal harvesting strategy in the virtual environment to directly drive the real harvesting robot. Based on actual feedback, the control parameters are dynamically adjusted, forming a closed-loop iterative control mode of "virtual decision-making - real execution - data feedback - strategy optimization".

[0094] This invention also discloses a harvesting robot control system based on 3D reconstruction and reinforcement learning, which applies the harvesting robot control method based on 3D reconstruction and reinforcement learning described above, including:

[0095] The 3D reconstruction module is used to collect multi-view images of the orchard through drones and mobile robots, and to perform 3D reconstruction using the 3D Gaussian splash rendering method, transforming the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches and leaves and fruits.

[0096] The fruit localization module is used to detect and spatially locate fruit areas based on a virtual orchard scene. It extracts fruit features through point cloud segmentation and cluster analysis, obtains fruit location, size, posture and maturity, and establishes a fruit picking task database.

[0097] The digital twin module is used to create a digital twin model of the real picking robot in a virtual orchard scene based on the geometric, kinematic and dynamic parameters of the real picking robot, and to simulate the fruit picking path and picking action.

[0098] The reinforcement learning module is used to train the digital twin model using the virtual orchard scene and fruit picking task database based on the SAC algorithm, so as to obtain the optimal picking path and picking action of the picking robot to perform fruit picking tasks.

[0099] The virtual-real control module is used to map the picking path and picking actions to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality.

[0100] The feedback optimization module is used to update the reinforcement learning training parameters of the digital twin model using feedback data from the fruit-picking robot during the fruit-picking process. The fruit-picking robot then performs fruit picking according to the optimized picking path and actions.

[0101] This invention constructs a virtual-real integrated, closed-loop optimized intelligent control system by combining high-precision 3D modeling with deep reinforcement learning. The system first performs efficient and secure policy training in a digital twin virtual environment, then applies the optimized policy to a real harvesting robot, and continuously iterates through real-time data feedback. This systematically solves the problems of insufficient perception accuracy, weak decision-making generalization ability, and low operational efficiency in existing technologies under complex environments.

[0102] The harvesting robot control system based on 3D reconstruction and reinforcement learning of the present invention can execute the harvesting robot control method based on 3D reconstruction and reinforcement learning of the present invention, and can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0103] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the harvesting robot control method based on 3D reconstruction and reinforcement learning described above. The electronic device of this invention can execute the harvesting robot control method based on 3D reconstruction and reinforcement learning of this invention, and can execute any combination of the steps of the method embodiments, possessing the corresponding functions and beneficial effects of the method.

[0104] This invention also discloses a storage medium, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the harvesting robot control method based on 3D reconstruction and reinforcement learning described above. The computer-readable storage medium of this invention can execute the harvesting robot control method based on 3D reconstruction and reinforcement learning of this invention, and can execute any combination of the steps of the method embodiments, possessing the corresponding functions and beneficial effects of the method.

[0105] In summary, this invention achieves continuous rendering and spatial reconstruction of fruit trees, branches, leaves, and fruits through 3D Gaussian splashing technology, overcoming the technical bottleneck of traditional dense reconstruction, completing high-fidelity modeling of fine structures such as orchard branches and leaves, and significantly improving the realism and geometric accuracy of the virtual environment under complex lighting and occlusion.

[0106] This invention utilizes the SAC algorithm for reinforcement learning training, enabling the robotic arm of a harvesting robot to possess more stable learning capabilities and policy generalization in a high-dimensional continuous motion space. This allows for high-success-rate grasping under varying fruit distributions and obstacle layouts. By using ROS communication to map virtual policies to the actual robotic arm's motion commands, simulation training results can be directly applied to actual harvesting operations, significantly improving the accuracy of policy transfer and control consistency.

[0107] This invention utilizes a self-learning process based on closed-loop feedback to dynamically correct virtual model parameters and control strategies, achieving continuous optimization and adaptive evolution. It is not only applicable to automated harvesting operations in orchards for lychees, apples, and citrus fruits, but can also be extended to other agricultural robotic scenarios, such as sorting, pruning, and fertilization, demonstrating significant potential for industrial application and economic value.

[0108] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit disclosed in the present invention should fall within the patent scope covered by the present invention.

Claims

1. A control method for a harvesting robot based on 3D reconstruction and reinforcement learning, characterized in that, Includes the following steps: By collecting multi-view images of the orchard using drones and mobile robots, and using the 3D Gaussian splash rendering method for three-dimensional reconstruction, the multi-view images of the orchard are transformed into a virtual orchard scene containing fruit trees, branches and leaves and fruits. Based on a virtual orchard scene, fruit area detection and spatial positioning are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture and maturity, and to establish a fruit picking task database. Based on the geometric, kinematic, and dynamic parameters of the real picking robot, a digital twin model corresponding to the real picking robot is established in a virtual orchard scene to simulate the fruit picking path and picking actions. By using a virtual orchard scene and a fruit picking task database, reinforcement learning training of the digital twin model is performed based on the SAC algorithm to obtain the optimal picking path and picking action of the picking robot to perform fruit picking tasks. The picking path and picking actions are mapped to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality. By using feedback data from the fruit-picking process of the picking robot, the reinforcement learning training parameters of the digital twin model are updated, and the picking robot performs fruit picking according to the optimized picking path and picking actions.

2. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 1, characterized in that, A 3D Gaussian splash rendering method was used for 3D reconstruction, transforming multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches, leaves, and fruits. Specifically: Photometric correction and distortion correction were performed on multi-view images of the orchard; Sparse point clouds are generated using the SFM algorithm; The color, depth, and density information of each point in the point cloud are mapped to a three-dimensional Gaussian distribution. Continuous volume rendering is achieved through 3D Gaussian splashing to generate a visualized virtual orchard scene.

3. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 1, characterized in that, Based on a virtual orchard scene, fruit area detection and spatial localization are performed. Fruit features are extracted through point cloud segmentation and cluster analysis to obtain fruit location, size, posture, and maturity, and a fruit picking task database is established. Specifically: Semantic segmentation was performed on the virtual orchard scene generated by 3D Gaussian splashing, and the PointNet++ network was used to divide the point cloud into fruit, branches and leaves and background. An improved DBSCAN algorithm was used to cluster the fruit point cloud to calculate the three-dimensional center coordinates, orientation, and radius of the fruit. Fruit surface color features were extracted based on the HSV color space for maturity assessment, and a fruit harvesting task database was established.

4. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 1, characterized in that, Using a virtual orchard scenario and a database of fruit picking tasks, reinforcement learning training is performed on the digital twin model based on the SAC algorithm, specifically as follows: Using a virtual orchard scenario as the background, a digital twin model of a harvesting robot as the intelligent agent, and a fruit harvesting task database as the task objective, the digital twin model is trained using reinforcement learning based on the SAC algorithm. The trained digital twin model is then output, which is used by the harvesting robot to generate corresponding control commands based on real-time status information during actual operation.

5. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 4, characterized in that, The objective function of the SAC algorithm is defined as: ； Where J(π) represents the objective function value of the policy function π; E represents the mathematical expectation; s∈S represents the state variable in the environment, where S is the state space; a∈A represents the action variable in the environment, where A is the action space; α is the entropy adjustment coefficient; Q(s,a) is the value function, representing the value of performing action a in state s; π(a|s) is the policy probability distribution function, representing the probability density of performing action a in state s.

6. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 5, characterized in that, A reinforcement learning environment is defined as: State space S = {robotic arm joint angles, end effector pose, coordinates of fruit relative to the robotic arm, obstacle position, gripper state}; Motion space A = {joint angular velocity, gripper opening / closing range}; The reinforcement learning reward function is: ； Where R represents the total reward value; R grasp To retrieve the reward item upon successful capture; R collision For collision penalty items; R smooth λ is the motion smoothness reward; β is the collision penalty weight coefficient; and β is the smoothness reward weight coefficient.

7. The harvesting robot control method based on 3D reconstruction and reinforcement learning according to claim 1, characterized in that, The harvesting path and harvesting actions are mapped to the harvesting robot, specifically as follows: The robot operating system enables data interaction between virtual simulation and real harvesting robots: the virtual end acts as the control command issuing end, outputting control commands for the harvesting robot; the real end acts as the execution end, receiving control commands through the robot operating system bridge middleware and driving the various servo motors of the harvesting robot to perform actions.

8. A harvesting robot control system based on 3D reconstruction and reinforcement learning, employing the harvesting robot control method based on 3D reconstruction and reinforcement learning as described in any one of claims 1 to 7, characterized in that, include: The 3D reconstruction module is used to collect multi-view images of the orchard through drones and mobile robots, and to perform 3D reconstruction using the 3D Gaussian splash rendering method, transforming the multi-view images of the orchard into a virtual orchard scene containing fruit trees, branches and leaves and fruits. The fruit localization module is used to detect and spatially locate fruit areas based on a virtual orchard scene. It extracts fruit features through point cloud segmentation and cluster analysis, obtains fruit location, size, posture and maturity, and establishes a fruit picking task database. The digital twin module is used to create a digital twin model of the real picking robot in a virtual orchard scene based on the geometric, kinematic and dynamic parameters of the real picking robot, and to simulate the fruit picking path and picking action. The reinforcement learning module is used to train the digital twin model using the virtual orchard scene and fruit picking task database based on the SAC algorithm, so as to obtain the optimal picking path and picking action of the picking robot to perform fruit picking tasks. The virtual-real control module is used to map the picking path and picking actions to the picking robot. The picking robot performs the fruit picking task according to the picking path and picking actions obtained by reinforcement learning training, realizing synchronous picking control from virtual to reality. The feedback optimization module is used to update the reinforcement learning training parameters of the digital twin model using feedback data from the fruit-picking robot during the fruit-picking process. The fruit-picking robot then performs fruit picking according to the optimized picking path and actions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the harvesting robot control method based on three-dimensional reconstruction and reinforcement learning as described in any one of claims 1 to 7.

10. A storage medium, a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the harvesting robot control method based on three-dimensional reconstruction and reinforcement learning as described in any one of claims 1 to 7.

Images