Intelligent orchard robot and automatic detection picking data processing method

Abstract

The invention relates to the technical field of intelligent orchards, in particular to an intelligent orchard robot and an automatic detection picking data processing method, and the intelligent orchard robot comprises a mobile operation platform, a road strength planning module, an identification module, a detection module and an upper computer; the road strength planning module autonomously plans a path in real time based on a laser radar technology and an instant positioning and mapping technology to obtain a planned path; the recognition module recognizes and positions the fruits on the planned path through a YOLOv5 target detection algorithm, and the positions of the fruits are obtained; the mobile operation platform moves in the orchard based on the planned path, and fruits are picked according to the positions of the fruits; the detection module detects environment information in an orchard in real time, the upper computer detects the working state of the robot in real time, and the robot integrates multiple sensors based on an ROS framework and integrates a deep learning algorithm, so that target detection and automatic picking of fruits, autonomous planning of a path and detection of environment data can be realized; and the pressure caused by labor shortage can be relieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent orchards, and particularly to an intelligent orchard robot and an automatic detection and picking data processing method. Background Art

[0002] Fruit production is a labor-intensive industry, especially in the harvesting process, which requires a large amount of labor. Currently, the labor market is facing a severe reality: agricultural labor is gradually flowing to other industries, and at the same time, the aging of the population has exacerbated the shortage of agricultural labor.

[0003] In recent years, robot technology has been widely applied and rapidly developed in various fields. The accuracy, flexibility, and autonomy of robots have been continuously improved, providing technical support for their application in orchards. Smart agriculture combines the latest developments in technology and artificial intelligence, providing a new direction for the transformation and upgrading of traditional agriculture. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent orchard robot and an automatic detection and picking data processing method, aiming to solve the problem of labor shortage in the fruit picking process.

[0005] To achieve the above purpose, in the first aspect, the present invention provides an intelligent orchard robot, including a mobile operation platform, a path planning module, an identification module, a detection module, and a host computer;

[0006] The mobile operation platform is used to move in the orchard and pick fruits;

[0007] The path planning module, based on lidar technology and simultaneous localization and mapping technology, real-time autonomously plans the path of the mobile operation platform to obtain a planned path;

[0008] The identification module, based on the YOLOv5 object detection algorithm, identifies and locates the fruits on the planned path to obtain the fruit positions;

[0009] The detection module is used to real-time detect the environmental information in the orchard and transmit it to the host computer;

[0010] The host computer is used to real-time detect the working state of the robot.

[0011] Among them, the mobile operation platform includes a crawler chassis, a robotic arm, a driving unit, and a positioning unit;

[0012] The crawler chassis is used to drive the robotic arm to move in the orchard;

[0013] The robotic arm picks the fruits based on the identification and positioning;

[0014] The driving unit is used to drive the crawler chassis and the driving unit to act, so as to realize the movement of the robot in the orchard to pick fruits;

[0015] The positioning unit is used to perform cruise positioning on the crawler chassis.

[0016] Among them, the detection module includes a light humidity sensor, an atmospheric pressure sensor, a carbon dioxide detector and an oxygen sensor;

[0017] The light humidity sensor is used to detect the light intensity, temperature and humidity in real time;

[0018] The atmospheric pressure sensor is used to detect the atmospheric pressure in real time;

[0019] The carbon dioxide detector is used to detect the carbon dioxide content in the environment around the robot in real time;

[0020] The oxygen sensor detects the oxygen present in the air based on the electrochemical principle.

[0021] Among them, the driving unit is TB6612, the positioning unit is ATK1218-BD, its positioning accuracy is 2.5mCEP, and the capture and tracking sensitivity is -165dBm.

[0022] Among them, the light humidity sensor is MAX4409, the atmospheric pressure sensor is BMP280, the carbon dioxide detector is SGP30, and the oxygen sensor is SC03-O2 type electrochemical oxygen module.

[0023] In a second aspect, the present invention also provides a method for processing automatically detected picking data of a smart orchard robot, which is applied to the smart orchard robot as described in the first aspect above, and includes the following steps:

[0024] The path planning module autonomously plans a path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path;

[0025] The recognition module uses the YOLOv5 object detection algorithm to identify and locate the fruits on the planned path to obtain the fruit positions;

[0026] The mobile operation platform moves in the orchard based on the planned path and picks fruits according to the fruit positions;

[0027] The detection module detects the environmental information in the orchard in real time, and the upper computer detects the working state of the robot in real time.

[0028] Among them, the specific way for the path planning module to autonomously plan a path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path:

[0029] The lidar emits laser and measures the round-trip time to real-time map the environment. Combining simultaneous localization and mapping technology, it constructs the map while self-localizing, and incorporates a path planning algorithm for real-time path planning. Figure 1 And it conducts real-time path planning.

[0030] A smart orchard robot of the present invention. The path planning module autonomously plans a path in real-time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path. The recognition module uses the YOLOv5 object detection algorithm to identify and locate fruits on the planned path to obtain the fruit positions. The mobile operation platform moves in the orchard based on the planned path and picks fruits according to the fruit positions. The detection module real-time detects the environmental information in the orchard, and the host computer real-time detects the working state of the robot. This robot, based on the ROS framework, fuses multiple sensors and incorporates deep learning algorithms, can achieve object detection and automatic picking of fruits, autonomous path planning, and detection of environmental data. This robot has functions such as fruit variety recognition, sorting and picking into boxes, automatic navigation, real-time monitoring, and collection and statistics of environmental data, can greatly improve the picking productivity of the orchard, can relieve the pressure brought by the shortage of labor, can also improve the intelligent level of agricultural production, promote the process of agricultural modernization, and solve the problem of labor shortage in the process of fruit picking. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0032] Figure 1 It is a schematic connection diagram of a smart orchard robot provided by the present invention.

[0033] Figure 2 It is a schematic diagram of the finished product of a smart orchard robot provided by the present invention.

[0034] Figure 3 It is a flowchart of a method for automatically detecting and processing picking data of a smart orchard robot provided by the present invention.

[0035] Figure 4 It is a flowchart of a method for automatically detecting and processing picking data of a smart orchard robot provided by the present invention.

[0036] Figure 5 It is a flowchart of the path planning algorithm.

[0037] Figure 6It is a schematic diagram of the basic process of object detection in the YOLOv5 model.

[0038] In the figure: 1 - Mobile operation platform, 2 - Path planning module, 3 - Recognition module, 4 - Detection module, 5 - Host computer, 6 - Crawler chassis, 7 - Manipulator, 8 - Drive unit, 9 - Positioning unit, 10 - Light and humidity sensor, 11 - Atmospheric pressure sensor, 12 - Carbon dioxide detector, 13 - Oxygen sensor. Specific implementation mode

[0039] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0040] Please refer to Figures 1 to 2 , the present invention provides a smart orchard robot, including a mobile operation platform 1, a path planning module 2, a recognition module 3, a detection module 4 and a host computer 5;

[0041] The mobile operation platform 1 is used to move in the orchard and pick fruits;

[0042] The path planning module 2 autonomously plans the path of the mobile operation platform 1 in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path;

[0043] The recognition module 3 identifies and locates the fruits on the planned path based on the YOLOv5 object detection algorithm to obtain the fruit positions;

[0044] The detection module 4 is used to detect the environmental information in the orchard in real time and transmit it to the host computer 5;

[0045] The host computer 5 is used to detect the working state of the robot in real time.

[0046] In an embodiment of the present invention, the path planning module 2 autonomously plans a path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path; the recognition module 3 uses the YOLOv5 object detection algorithm to identify and locate fruits on the planned path to obtain the fruit positions; the mobile operation platform 1 moves in the orchard based on the planned path and picks fruits according to the fruit positions; the detection module 4 detects the environmental information in the orchard in real time, and the host computer 5 detects the working state of the robot in real time. This robot integrates multiple sensors based on the ROS framework and combines deep learning algorithms, and can realize object detection and automatic picking of fruits, autonomous path planning, and detection of environmental data. This robot has functions such as fruit type recognition, sorting and picking, automatic navigation, real-time monitoring, and collection and statistics of environmental data, can greatly improve the picking productivity of the orchard, can relieve the pressure brought by the shortage of labor, can also improve the intelligent level of agricultural production, promote the process of agricultural modernization, and solve the problem of labor shortage in the fruit picking process.

[0047] Further, the mobile operation platform 1 includes a crawler chassis 6, a robotic arm 7, a drive unit 8, and a positioning unit 9;

[0048] The crawler chassis 6 is used to drive the robotic arm 7 to move in the orchard;

[0049] The robotic arm 7 picks fruits based on the recognition and positioning;

[0050] The drive unit 8 is used to drive the crawler chassis 6 and the drive unit 8 to act, so as to realize the robot moving in the orchard to pick fruits;

[0051] The positioning unit 9 is used to perform cruise positioning on the crawler chassis 6.

[0052] In an embodiment of the present invention, the crawler chassis 6 is used to drive the robotic arm 7 to move in the orchard. The crawler chassis 6 has stronger ground adaptability and can adapt to the complex and changeable road conditions and muddy land conditions in the orchard. The robotic arm 7 picks fruits based on the recognition and positioning. The robotic arm 7 is a six-degree-of-freedom robotic arm 7. To adapt to the complex and changeable growth distribution of fruits and complete complex fruit picking tasks efficiently, the drive unit 8 uses a motor drive module connected to a motor for control, which is TB6612, and is used to drive the crawler chassis 6 and the drive unit 8 to act, so as to realize the robot moving in the orchard to pick fruits; the positioning unit 9 is used to perform cruise positioning on the crawler chassis 6. The positioning unit 9 selects a GPS+Beidou dual-mode positioning ATK1218-BD module, whose positioning accuracy is 2.5mCEP, and the capture and tracking sensitivity is -165dBm.

[0053] Further, the detection module 4 includes a light and humidity sensor 10, an atmospheric pressure sensor 11, a carbon dioxide detector 12, and an oxygen sensor 13;

[0054] The light and humidity sensor 10 is used to detect the light intensity, temperature, and humidity in real time;

[0055] The atmospheric pressure sensor 11 is used to detect the atmospheric pressure in real time;

[0056] The carbon dioxide detector 12 is used to detect the carbon dioxide content in the environment around the robot in real time;

[0057] The oxygen sensor 13 detects the oxygen present in the air based on the electrochemical principle.

[0058] In the embodiment of the present invention, the light and humidity sensor 10 detects the light intensity, temperature, and humidity in real time. The light and humidity sensor 10 is a MAX4409. The atmospheric pressure sensor 11 detects the atmospheric pressure in real time. The atmospheric pressure sensor 11 is a BMP280. The carbon dioxide detector 12 detects the carbon dioxide content in the environment around the robot in real time. The carbon dioxide detector 12 is an SGP30. The oxygen sensor 13 detects the oxygen present in the air based on the electrochemical principle. The oxygen sensor 13 is an SC03-O2, which has good selectivity and stability.

[0059] Please refer to Figures 3 to 6 Second, the present invention also provides a method for processing automatically detected picking data of a smart orchard robot, which is applied to the smart orchard robot as described in the first aspect above, and includes the following steps:

[0060] S1 The path planning module 2 autonomously plans a path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path;

[0061] In the embodiment of the present invention, based on the 2D map constructed by the lidar SLAM algorithm, a path planning algorithm needs to be added to achieve the autonomous navigation and path planning of the robot, and higher automation is realized.

[0062] Common path planning algorithms applicable to this project include: A* algorithm (A-star), Dijkstra algorithm. The A* algorithm is a heuristic search algorithm that combines the ideas of the Dijkstra algorithm and greedy best-first search and adds a heuristic function. Compared with the Dijkstra algorithm, its time complexity and computational amount are smaller, and the search efficiency is higher.

[0063] The A* algorithm evaluates each node through the following two key cost functions:

[0064] Actual cost (g(n)): The actual path cost from the starting point to the current node, and heuristic cost (h(n)): The estimated path cost from the current node to the target node. This is usually calculated through a heuristic function and is used to estimate the distance from the current node to the target node.

[0065] The A* algorithm combines these two costs and defines an evaluation function:

[0066] f(n) = g(n) + h(n)

[0067] g(n) is the actual cost from the starting point to node n.

[0068] h(n) is the heuristic estimated cost from node n to the target node.

[0069] f(n) is the combined cost and is used to evaluate the priority of node n.

[0070] The lidar SLAM and path planning system achieves precise navigation in the orchard environment through multi-sensor fusion and intelligent algorithms. The system uses a Velodyne VLP-16 lidar, an Xsens MTi-30 IMU, and a dual encoder odometer to build a multi-source perception array, realizes μs-level time synchronization through the PTPv2 protocol, and uses the DBSCAN clustering algorithm to filter dynamic obstacles and specular reflection noise. In the SLAM mapping stage, the improved Gmapping algorithm improves the mapping efficiency through dynamic particle management, combines PointNet++ to detect the semantic features of tree trunks, constructs a double-layer map including a grid layer with 5 cm accuracy and a fruit tree ID annotation layer, and at the same time uses SegMatch point cloud feature matching and the spatial distribution of tree trunks to verify loop detection, and finally optimizes the six-degree-of-freedom pose through the g2o framework. The path planning adopts a hierarchical architecture to achieve global and local coordination. The global planning is based on the improved A* algorithm, incorporates the penalty for deviation from the middle line between rows (λ = 0.8) and the obstacle density weight into the standard path cost, and generates a curvature-limited B-spline smooth path; local obstacle avoidance optimizes the DWA algorithm, comprehensively evaluates the path following degree, obstacle distance, and the stability of the robotic arm 7 under a speed constraint of 0.8 m / s, and designs a trajectory prediction module for the swaying of fruit trees. The control of the robotic arm 7 integrates the MoveIt framework, sets a spherical workspace constraint of 1.2 m, adopts a master-slave dual-arm coordination strategy, dynamically compensates for the torque change caused by fruit picking through the Jacobian matrix, and combines the artificial potential field method to avoid self-collision. The system ensures the long-term stability of the system in the complex orchard environment through scenario-based modules such as slope adaptive speed reduction and slippery ground warning, as well as a maintenance mechanism for sensor calibration (checkerboard joint calibration method) and algorithm parameter self-tuning, compresses the single navigation decision cycle to within 100 ms, and meets the real-time operation requirements of high-density fruit tree scenarios.

[0071] The S2 recognition module 3 uses the YOLOv5 object detection algorithm to identify and locate the fruits on the planned path to obtain the fruit positions;

[0072] In the embodiment of the present invention, multi-spectral data fusion and dynamic enhancement are used to optimize the YOLOv5 model. By synchronously collecting multi-angle images with an RGB-D camera and a near-infrared camera, and combining adaptive data enhancement (such as random glare simulation and occlusion synthesis) to improve the generalization ability of the model; embedding the CBAM attention mechanism in the Neck layer of YOLOv5 to strengthen the feature extraction of the occluded area, and at the same time using the depth map to calculate the occlusion-aware weight to optimize the loss function. Finally, high-precision real-time detection within 100 ms is achieved through TensorRT acceleration, and JSON data containing depth information is output, with the accuracy rate (AP@0.5) reaching more than 95%, effectively coping with complex environmental interference.

[0073] The fruit maturity judgment system realizes accurate grading through multi-modal data acquisition and intelligent analysis. In the data acquisition stage, a dual-sensor array composed of a structured light module and an industrial camera is used to scan the fruits from 8 perspectives in a spiral trajectory, and high-precision three-dimensional point clouds and 24 million-pixel images are synchronously obtained within 3 seconds, and polarized supplementary lighting is used to eliminate the reflection interference. In the preprocessing link, multi-view point clouds are fused to build a three-dimensional model, color consistency is achieved through calibration with a standard color card, and surface depressions and lesion areas are preliminarily detected. Feature extraction focuses on three-dimensional texture and chromaticity analysis, calculates physical indexes such as wrinkle density and glossiness, extracts the main hue distribution in the CIELAB space, and combines the improved LBP algorithm to quantify the texture complexity. After data processing, an 18-dimensional weighted feature vector is formed and input into an integrated model composed of XGBoost and SVM, and a continuous maturity score of 0-1 is output, and abnormal fruits are synchronously identified and the picking priority is dynamically planned.

[0074] The S3 mobile operation platform 1 moves in the orchard based on the planned path and picks fruits according to the fruit positions;

[0075] The S4 detection module 4 real-time detects the environmental information in the orchard, and the upper computer 5 real-time detects the working status of the robot.

[0076] In the embodiment of the present invention, the upper computer 5 real-time detects the working status of the robot such as the working mode, position information, power information, and parameters of each sensor, etc., so as to enable the manager to realize the real-time monitoring of the efficient operation of the robot and the orchard environment.

[0077] The above-disclosed is only a preferred embodiment of a smart orchard robot and an automatic detection and picking data processing method of the present invention. Of course, the scope of the rights of the present invention cannot be limited by this. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. An intelligent orchard robot, characterized in that: It includes a mobile operation platform, a path planning module, an identification module, a detection module and a host computer; The mobile operation platform is used to move in the orchard and pick fruits; The path planning module, based on lidar technology and simultaneous localization and mapping technology, autonomously plans the path of the mobile operation platform in real time to obtain a planned path; The identification module, based on the YOLOv5 object detection algorithm, identifies and locates the fruits on the planned path to obtain the fruit positions; The detection module is used to detect the environmental information in the orchard in real time and transmit it to the host computer; The host computer is used to detect the working state of the robot in real time.

2. The intelligent orchard robot according to claim 1, Characterized in that; The mobile operation platform includes a crawler chassis, a robotic arm, a drive unit and a positioning unit; The crawler chassis is used to drive the robotic arm to move in the orchard; The robotic arm picks fruits based on the identification and positioning; The drive unit is used to drive the crawler chassis and the drive unit to act, so as to realize the robot moving and picking fruits in the orchard; The positioning unit is used to perform cruise positioning on the crawler chassis.

3. The intelligent orchard robot according to claim 1, Characterized in that; The detection module includes a light and humidity sensor, an atmospheric pressure sensor, a carbon dioxide detector and an oxygen sensor; The light and humidity sensor is used to detect the light intensity and temperature and humidity in real time; The atmospheric pressure sensor is used to detect the atmospheric pressure in real time; The carbon dioxide detector is used to detect the carbon dioxide content in the environment around the robot in real time; The oxygen sensor detects the oxygen present in the air based on the electrochemical principle.

4. The intelligent orchard robot according to claim 2, Characterized in that; The drive unit is TB6612, and the positioning unit is ATK1218-BD, and its positioning accuracy is 2.5mCEP, and the capture and tracking sensitivity is -165dBm.

5. The intelligent orchard robot according to claim 3, wherein ； The light and humidity sensor is MAX4409, the atmospheric pressure sensor is BMP280, the carbon dioxide detector is SGP30, and the oxygen sensor is an SC03-O2 type electrochemical oxygen module.

6. A method for processing automatically detected picking data of a smart orchard robot, which is applied to the smart orchard robot according to any one of claims 1-5, characterized in that, It includes the following steps: The path planning module autonomously plans the path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path; The identification module uses the YOLOv5 object detection algorithm to identify and locate the fruits on the planned path to obtain the fruit positions; The mobile operation platform moves in the orchard based on the planned path and picks fruits according to the fruit positions; The detection module detects the environmental information in the orchard in real time, and the host computer detects the working state of the robot in real time.

7. The method for processing automatically detected picking data of the intelligent orchard robot according to claim 6, Characterized in that; The specific method for the path planning module to autonomously plan the path in real time based on lidar technology and simultaneous localization and mapping technology to obtain a planned path: The lidar emits laser and measures the return time to draw the environmental map in real time. Combining with the simultaneous localization and mapping technology, it constructs the map and self-locates while adding a path planning algorithm for real-time path planning.

Images