Potato picking and harvesting method based on PLP-net

By applying the PLP-net algorithm and the grasping strategy of parallel robots in the potato picking and harvesting machine, the problem of high damage rate of potato picking and harvesting machine is solved, and efficient, precise grasping and grading of potatoes is achieved, and the harvesting efficiency and quality are improved.

CN119563462AActive Publication Date: 2025-03-07QINGDAO AGRI UNIV
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411702491.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-26
Publication Date
2025-03-07
Estimated Expiration
2044-11-26

AI Technical Summary

Technical Problem

The potato picking and harvesting machine has a high rate of damage to the potato, which leads to poor results when harvesting starch potatoes. The labor intensity and efficiency of manual picking are high, which limits the development of the potato industry.

Method used

The potato picking and harvesting method based on PLP-net is used to identify potatoes through the PLP-net algorithm, and combined with the grasping strategy of parallel robots, precise grasping is achieved, and the potato grading is performed through the grading thrust device.

Benefits of technology

It improves the lossless and precise grasping efficiency of potatoes, reduces damage and omissions during the grasping process, realizes efficient grading of potatoes, and improves harvesting efficiency and quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119563462A_ABST
    Figure CN119563462A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of potato harvesting, in particular to a PLP-net-based potato picking and harvesting method, which comprises the following steps: S1, moving a picking and harvesting robot to a non-picking area; s2, identifying the potatoes to be picked up based on a PLP-net algorithm; s3, positioning and calculating the plane position and the grabbing depth of the potatoes to be picked up; s4, making a potato grabbing strategy and sending a command to a grabbing device; s5, after grabbing is completed, whether the potato is the last potato in the area or not is judged; and S6, judging whether an unpicked area exists or not. A whole set of complete potato identification and harvesting scheme is provided, and the automation degree and the harvesting quality of the potato harvesting process can be effectively improved; the PLP-net algorithm is adopted to identify the potatoes, on one hand, a grabbing object and a grabbing strategy are provided for the parallel manipulator, the grabbing accuracy and the grabbing efficiency are guaranteed, on the other hand, a basis is provided for the grading pushing device to grade the potatoes, fruit grading is directly conducted after grabbing is completed, and the harvesting efficiency and quality are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of potato harvesting, in particular to a potato picking and harvesting method based on PLP-net. Background Art

[0002] At present, the main way to harvest potatoes is segmented harvesting, that is, first use a potato harvester to dig the potatoes out of the ground and preliminarily separate the impurities. Then, use a potato picker harvester to pick up the potatoes on the ground. Compared with the potato combine harvester, the potato picker harvester has stronger adaptability and flexibility. However, the potato picker harvester still has the problem of high potato damage rate, so the potato picker harvester is generally used to harvest starch potatoes, and the commercial potatoes on the market are still picked up manually. On the one hand, manual picking is labor-intensive and inefficient; on the other hand, with the annual decrease in the number of people engaged in agriculture, the cost of employment is increasing year by year. The above two aspects have seriously restricted the development of the potato industry since the harvesting link. For this reason, the potato industry urgently needs a solution of machine replacement. Summary of the invention

[0003] The present invention aims to solve the above problems and provides a potato picking and harvesting method based on PLP-net, and the technical solution adopted is as follows: A potato picking and harvesting method based on PLP-net comprises the following steps: S1. The picking and harvesting robot moves from the starting point of the work to the unpicked area; S2. Identify potatoes to be picked up in the unpicked area based on the PLP-net algorithm; S3. Positioning and calculating the plane position and grasping depth of the potatoes to be picked up; S4. Formulate potato grasping strategy and send commands to the grasping device; S5. After the capture is completed, it is determined whether it is the last potato in the area. If not, steps S3 and S4 are repeated. If so, the picking and harvesting robot is driven to move to the next unpicked area; S6. Determine whether there is an unpicked area. If so, repeat steps S2 to S5. If not, complete the picking task; The PLP-net algorithm described in step S2 includes a backbone extraction network and a detection head network. The backbone extraction network includes two branches, feature extraction and downsampling. The feature extraction branch uses the captured potato picture as an input signal, inputs the convolution layer, and then sequentially inputs the convolution layer and the residual block for feature extraction. The feature extraction branch includes a total of 5 convolution layers and 4 residual blocks. The result output from the first convolution layer is also used as the input of the downsampling branch for maximum pooling, and then sequentially performs feature fusion and maximum pooling with the results output from the second to fourth convolution layers. A residual block is set between the feature fusion after the fourth convolution layer and the maximum pooling. The last residual block output of the downsampling branch is input into the feature fusion branch for the last feature fusion. The calculated result is then subjected to the ECA residual block for the final feature fusion. The spatial pyramid pooling is performed and then input into the detection head network; in the detection head network, the first upsampling operation is performed first, and then the upsampling result is fused with the 6th layer of the feature extraction branch, and the fused features are input into the residual block. After the residual block is output, the second upsampling operation is performed, and the features are fused with the 4th layer of the feature extraction branch, and the fused features are input into the convolution layer. The output of the convolution layer and the output of the residual block are fused again, and the output features are obtained after the residual block is calculated again; the output features are divided into two branches, one of which is directly output by the medium target detection head, and the other branch is input into another convolution layer, whose output is fused with the output of the spatial pyramid pooling, and the fused features are output by the large target detection head.

[0004] Based on the above scheme, the picking and harvesting robot includes a body and a moving chassis, the moving chassis carries the body to move, the body is provided with a navigation depth camera for identifying unpicked areas and a satellite positioning module for positioning, the body is provided with an identification depth camera, a parallel manipulator and a conveying device, the identification depth camera is used to identify potatoes to be picked up in the area, the parallel manipulator is arranged in front of the conveying device, and is used to clamp the potatoes identified by the identification depth camera and place them on the conveying device.

[0005] On the basis of the above scheme, the potato grasping strategy includes calculation of the center point of the potato plane, calculation of the grasping depth and calculation of the grasping angle; when the PLP-net algorithm is used to identify potatoes, the center point of the selection box is used as the center point of the potato plane, and the center point is calculated as the coordinate of the potato center point. The distance between the highest point of the potato surface and the ground is identified and calculated by the recognition depth camera, and the grasping depth coordinate value of the parallel manipulator is calculated by pre-setting a specific grasping depth value or depth ratio; the angle between the edge line of the potato selection box and the coordinate axis is calculated, and the grasping direction of the parallel manipulator is made parallel to the short side of the potato selection box to obtain the grasping angle of the parallel manipulator.

[0006] Preferably, after the parallel manipulator completes the grasping action, it provides feedback to the host computer of the picking and harvesting robot, and the feedback content includes the command acceptance status and the command execution status; the command acceptance status includes the target object grasping command and the manipulator behavior command, the target object grasping command refers to whether to start the action and grasp the target potato, the manipulator behavior command includes the grasping plane coordinates, the grasping depth coordinates and the manipulator grasping angle, the feedback result of the command acceptance status includes receiving a completion signal, missing a partial signal and missing a complete signal; the command execution status refers to whether the target potato is successfully grasped.

[0007] Preferably, the conveying device includes a conveyor belt and a baffle, the number of the baffles is multiple and fixedly connected to the conveyor belt, and one potato is placed between adjacent baffles; the picking and harvesting robot also includes a grading pushing device, the grading pushing device includes a pushing cylinder and a grading collection frame, the pushing cylinder is arranged on one side of the conveyor belt, and pushes the potatoes on the conveyor belt into the grading collection frame; the number of the grading collection frames is multiple and arranged along the transmission direction of the conveyor belt, and each grading collection frame contains potatoes of different grades; after the appearance information of the potato to be grasped is recognized by the recognition depth camera, the upper computer calculates the weight of the potato and marks the grading information of the potato, and the potatoes of each grade are pushed into the grading collection frame by the pushing cylinder at the corresponding grading collection frame.

[0008] On the basis of the above scheme, the length, width and height information of the potato are extracted according to the potato pictures taken by the recognition depth camera, and the weight of the potato is estimated and marked through the deep learning algorithm. Potatoes weighing more than 150g are defined as level one, potatoes weighing more than 100g and less than or equal to 150g are defined as level two, and potatoes weighing more than 50g and less than or equal to 100g are defined as level three.

[0009] On the basis of the above scheme, the deep learning algorithm uses a PW neural network to fit the potato weight training set. The input layer of the PW neural network includes three input data, namely, length input l, width input b and height input c. The output layer includes one output signal, which is the potato weight d. Several hidden layers are set between the input layer and the output layer. The length, width, height and weight information of the potatoes in the training set are input into the PW neural network. Through multiple iterative training, the fitting relationship between the length, width and height of the potato and the weight is obtained.

[0010] Preferably, a lifting mechanism is arranged in the vehicle body, the lifting mechanism includes a weight sensor and a travel switch, the grading and collecting frame is arranged on the lifting mechanism, the lifting mechanism drives the grading and collecting frame to descend according to the weight of the potatoes therein, when the grading and collecting frame descends to the travel switch, the picking and harvesting robot stops working, and the upper computer sends a box changing command to the user.

[0011] Preferably, the picking and harvesting robot also includes a laser radar for identifying obstacles randomly encountered during the movement and driving the picking and harvesting robot to avoid them.

[0012] Preferably, the picking and harvesting robot uses the corner of the working area as the starting point of the work and moves continuously in an S shape.

[0013] The beneficial effects of the present invention are: 1. The PLP-net algorithm is used to identify potatoes. On the one hand, it provides the parallel manipulator with the position information and grasping strategy of the grasping object to ensure the grasping accuracy and grasping efficiency, and effectively avoids the problems of potato damage or omission during the grasping process. On the other hand, it provides a basis for the grading and pushing device to grade potatoes. After the grasping is completed, the fruit is directly graded, integrating multiple functions to improve the harvesting efficiency and quality; 2. The PLP-net algorithm reasonably reduces the number of detection heads and only retains the detection heads for detecting large and medium targets, thereby simplifying the model structure and reducing the computational complexity, so that the model can focus more on detecting potato targets with actual harvest value and improve the overall detection performance. The ECA attention mechanism is introduced to compensate for the accuracy loss caused by the deletion of detection heads, and the accuracy of the algorithm model in potato offal identification in potato picking scenarios is further improved. The algorithm model is lightweight, and the computing speed is greatly improved while meeting the accuracy requirements. At the same time, the model structure is simplified, so that it can avoid the defect of insufficient computing power of the host computer carried by small mobile devices. 3. Propose a picking and harvesting robot that integrates the functions of movement, positioning, potato identification, grasping, potato grading and obstacle avoidance. It can complete the non-destructive and accurate grasping and grading of potatoes in one operation, meeting the functional requirements of the potato harvesting process for mobile devices. At the same time, the device is highly integrated, reducing the size of the equipment, reducing the equipment cost, and making the device flexible and controllable; 4. Proposing a complete set of potato identification and harvesting solutions can effectively improve the automation level and harvesting quality of the potato harvesting process, significantly reduce the labor costs required, increase the harvesting speed, and contribute to the large-scale development of the potato industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 : Flow chart of the picking and harvesting method of the present invention; Figure 2 : Schematic diagram of the structure of the picking and harvesting robot of the present invention; Figure 3 : Front view of the interior of the picking and harvesting robot of the present invention; Figure 4 : Schematic diagram of the internal structure of the picking and harvesting robot of the present invention; Figure 5 : Schematic diagram of the internal structure of the picking and harvesting robot of the present invention from another perspective; Figure 6 : Schematic diagram of the lifting mechanism of the picking and harvesting robot of the present invention; Figure 7 : Control principle diagram of the operating system of the present invention; Figure 8 : Schematic diagram of the working process of the parallel manipulator of the present invention; Fig. 9 : Specific implementation diagram of the parallel manipulator workflow of the present invention; Fig.10 : PLP-net algorithm network model diagram of the present invention; Fig.11 : The residual block calculation structure model diagram in the PLP-net algorithm of the present invention; Fig.12 : The convolutional layer calculation structure model diagram in the PLP-net algorithm of the present invention; Fig.13 : The spatial pyramid pooling calculation structure model diagram in the PLP-net algorithm of the present invention; Fig.14 : The calculation structure model diagram of the residual bottleneck module in the PLP-net algorithm of the present invention; Fig.15 : ECA residual block structure model diagram in the PLP-net algorithm of the present invention; Fig.16 : ECA residual bottleneck block structure model diagram in the PLP-net algorithm of the present invention; Fig.17 : Diagram of the calculation structure model of the detection head in the PLP-net algorithm of the present invention; Fig.18 : The present invention adopts the Yolo v8n-obb algorithm test effect diagram; Fig.19 : The present invention adopts the PLP-net algorithm test effect diagram; Fig. 20 : The structure diagram of the PW neural network of the present invention; Fig.21 : Schematic diagram of the travel path of the picking and harvesting robot of the present invention.

[0015] Explanation of the accompanying drawings: 1-vehicle body, 2-motion chassis, 3-satellite positioning module, 41-navigation depth camera, 42-laser radar, 43-recognition depth camera, 44-parallel manipulator, 51-conveyor belt, 52-baffle, 53-push cylinder, 54-grading and collecting frame, 55-lifting mechanism, 56-travel switch. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0017] In the description of the present invention, it should be understood that the terms "center", "length", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0018] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0019] like Figure 1 As shown, a potato picking and harvesting method based on PLP-net comprises the following steps: S1. The picking and harvesting robot moves from the starting point of the work to the unpicked area; S2. Identify potatoes to be picked up in the unpicked area based on the PLP-net algorithm; S3. Positioning and calculating the plane position and grasping depth of the potatoes to be picked up; S4. Formulate potato grasping strategy and send commands to the grasping device; S5. After the capture is completed, it is determined whether it is the last potato in the area. If not, steps S3 and S4 are repeated. If so, the picking and harvesting robot is driven to move to the next unpicked area; S6. Determine whether there is an unpicked area. If so, repeat steps S2 to S5. If not, complete the picking task.

[0020] like Figures 10 to 16 As shown, the PLP-net algorithm described in step S2 is improved based on the Yolo v8-obb ​​algorithm model, including a backbone extraction network and a detection head network, wherein the backbone extraction network includes two branches, feature extraction and downsampling, wherein the feature extraction branch takes the captured potato picture as input, inputs the convolution layer, and then sequentially inputs the convolution layer and the residual block for feature extraction, the feature extraction branch includes a total of 5 convolution layers and 4 residual blocks, the result output from the first convolution layer is simultaneously used as the input of the downsampling branch for maximum pooling, and then sequentially performs feature fusion and maximum pooling with the results output from the second to fourth convolution layers, a residual block is set between the feature fusion after the fourth convolution layer and the maximum pooling, the last residual block output of the downsampling branch is input into the feature fusion branch for the last feature fusion, and the calculated result is subjected to the ECA residual block for space The spatial pyramid pooling is then input into the detection head network; in the detection head network, the first upsampling operation is first performed, and then the upsampling result is feature fused with the 6th layer of the feature extraction branch, and the fused features are input into the residual block. After the residual block is output, the second upsampling operation is performed, and the features are fused with the 4th layer of the feature extraction branch, and the fused features are input into the convolution layer. The output of the convolution layer and the output of the residual block are feature fused again, and after the residual block is calculated again, the output features are obtained; the output features are divided into two branches, one of which is directly output by the medium target detection head, and the other branch is input into another convolution layer, and its output is feature fused with the output of the spatial pyramid pooling, and the fused features are output by the large target detection head. The detection feature map size corresponding to the large target detection head is 20*20, which is used to detect targets larger than 32*32, and the detection feature map size corresponding to the medium target detection head is 40*40, which is used to detect targets larger than 16*16.

[0021] Using the same equipment and the same configuration, the Yolo v8obb algorithm and the PLP-net algorithm were used to test the same batch of test sets. The experimental results are as follows: Fig.18 , Fig.19 As shown in Table 1.

[0022] Table 1 Comparison test data between Yolo v8obb algorithm and PLP-net algorithm

[0023] It can be seen from the above charts that compared with Yolov8n-obb, the accuracy and average precision of the PLP-net model are slightly lower, but they are sufficient to meet the requirements of potato recognition. At the same time, the floating-point operations of the PLP-net model are reduced by 7.2GFLOPs, the BN weight is reduced by 2.1MB, and the frame rate is increased by 99.4f / s. The data shows that the PLP-net model has less operations, better real-time performance, faster running speed, and is more suitable for deployment on embedded devices with lower computing power.

[0024] like Figure 7 As shown in the figure, the picking and harvesting robot can be divided into an upper decision-making part and a lower execution part. The upper decision-making part includes the potato picking robot operating system, which receives input signals from the navigation depth camera, recognition depth camera, laser radar and satellite positioning module. The lower execution part includes a parallel mechanical arm, a conveying device and a graded push device, and an intelligent motion chassis. The lower execution part receives the command information from the upper decision-making part and performs the corresponding action and provides information feedback. It also includes a wireless control method, and the operator can use a 2.4G wireless remote control to drive the intelligent motion chassis to move.

[0025] like Figures 2 to 6 As shown, the picking and harvesting robot includes a body 1 and a moving chassis 2, the moving chassis 2 carries the body 1 to move, the body 1 is provided with a navigation depth camera 41 for identifying unpicked areas and a satellite positioning module 3 for positioning, the body 1 is provided with an identification depth camera 43, a parallel manipulator 44 and a conveying device, the identification depth camera 43 is used to identify potatoes to be picked up in the area, the parallel manipulator 44 is provided in front of the conveying device, and is used to grab the potatoes identified by the identification depth camera 43 and place them on the conveying device.

[0026] like Figure 8 and Fig. 9 As shown, the potato grasping strategy includes calculation of the center point of the potato plane, calculation of the grasping depth and calculation of the grasping angle; when the PLP-net algorithm is used to identify the potato, the center point of the selection box is used as the center point (x, y) of the potato plane, and the center point is calculated as the coordinate of the potato center point, and the distance between the highest point of the potato surface and the ground is identified and calculated by the recognition depth camera 43, and the grasping depth coordinate value z of the parallel manipulator 44 is calculated by pre-setting a specific grasping depth value or depth ratio; the angle between the edge line of the potato selection box and the coordinate axis in the plane is calculated, and the grasping direction of the parallel manipulator 44 is made parallel to the short side of the potato selection box to obtain the grasping angle of the parallel manipulator 44.

[0027] After the parallel manipulator 44 completes the grasping action, it provides feedback to the host computer of the picking and harvesting robot, and the feedback content includes the command acceptance status and the command execution status; the command acceptance status includes the target object grasping command and the manipulator behavior command, the target object grasping command refers to whether to start the action and grasp the target potato, the manipulator behavior command includes the grasping plane coordinates, the grasping depth coordinates and the manipulator grasping angle, the feedback results of the command acceptance status include receiving a completion signal, missing a partial signal and missing a complete signal; the command execution status refers to whether the target potato is successfully grasped.

[0028] The conveying device includes a conveyor belt 51 and a baffle 52. There are multiple baffles 52 and they are fixedly connected to the conveyor belt 51. One potato is placed between adjacent baffles 52. The picking and harvesting robot also includes a grading and pushing device. The grading and pushing device includes a pushing cylinder 53 and a grading and collecting frame 54. The pushing cylinder 53 is arranged on one side of the conveyor belt 51, and pushes the potatoes on the conveyor belt 51 into the grading and collecting frame 54. There are multiple grading and collecting frames 54 and they are arranged along the transmission direction of the conveyor belt 51. Each grading and collecting frame 54 contains potatoes of different grades. After the appearance information of the potato to be grasped is recognized by the recognition depth camera 43, the upper computer calculates the weight of the potato and marks the grading information of the potato. Potatoes of each grade are pushed into the grading and collecting frame 54 by the pushing cylinder 53 at the corresponding grading and collecting frame 54.

[0029] According to the potato picture taken by the recognition depth camera 43, the length, width and height information of the potato are extracted, and the weight of the potato is estimated and marked by the deep learning algorithm, and potatoes with a weight greater than 150g are defined as level one, potatoes with a weight greater than 100g and less than or equal to 150g are defined as level two, and potatoes with a weight greater than 50g and less than or equal to 100g are defined as level three. Specifically, Fig. 20 As shown, the deep learning algorithm uses a PW neural network to fit the potato weight training set. The input layer of the PW neural network includes three input data, namely, length input l, width input b and height input c. The output layer includes one output signal, which is the potato weight d. Several hidden layers are set between the input layer and the output layer. The length, width, height and weight information of the potatoes in the training set are input into the PW neural network. Through multiple iterative training, the fitting relationship between the length, width and height of the potato and the weight is obtained.

[0030] A lifting mechanism 55 is arranged in the vehicle body 1, and the lifting mechanism 55 includes a weight sensor and a travel switch 56. The grading and collecting frame 54 is arranged on the lifting mechanism 55. The lifting mechanism 55 drives the grading and collecting frame 54 to descend according to the weight of the potatoes therein. When the grading and collecting frame 54 descends to the travel switch 56, the picking and harvesting robot stops working, and the upper computer sends a box changing command to the user.

[0031] The picking and harvesting robot also includes a laser radar 42 for identifying obstacles randomly encountered during the movement and driving the picking and harvesting robot to avoid them.

[0032] The potatoes are laid out in a loose pattern and are not in a straight line. Since satellite navigation is less flexible, this solution combines camera vision navigation with satellite navigation. The main idea is to use vision navigation and satellite navigation in different scenarios based on their advantages and disadvantages. Fig.21 As shown, the picking and harvesting robot uses the corner of the working area as the starting point of the work and moves continuously in an S shape. When working in the field, it uses the navigation depth camera 41 for visual navigation, which has good flexibility. When turning and turning around in the field, it uses the satellite positioning module 3 for satellite navigation, which has high accuracy. The dotted part in the figure is the visual navigation area, and the realized part is the satellite navigation area.

[0033] The present invention is described above by way of examples, but the present invention is not limited to the above specific embodiments, and any changes or modifications made based on the present invention belong to the scope of protection claimed by the present invention.

Claims

1. A potato picking and harvesting method based on PLP-net, characterized in that: The following steps are included: S1. The picking and harvesting robot moves from the starting point of the work to the unpicked area; S2. Identify potatoes to be picked up in the unpicked area based on the PLP-net algorithm; S3. Positioning and calculating the plane position and grasping depth of the potatoes to be picked up; S4. Formulate potato grasping strategy and send commands to the grasping device; S5. After the capture is completed, it is determined whether it is the last potato in the area. If not, steps S3 and S4 are repeated. If so, the picking and harvesting robot is driven to move to the next unpicked area; S6. Determine whether there is an unpicked area. If so, repeat steps S2 to S5. If not, complete the picking task; The PLP-net algorithm described in step S2 includes a backbone extraction network and a detection head network, wherein the backbone extraction network includes two branches: feature extraction and downsampling. The feature extraction branch takes the captured potato picture as input, inputs the convolution layer, and then sequentially inputs the convolution layer and the residual block for feature extraction. The feature extraction branch includes 5 convolution layers and 4 residual blocks. The output result from the first convolution layer is also used as the input of the downsampling branch for maximum pooling, and then is sequentially combined with the output results of the second to fourth convolution layers for feature fusion and maximum pooling. A residual block is set between the feature fusion and maximum pooling after the fourth convolution layer, and the last residual block of the downsampling branch is output and input into the feature fusion branch for the last feature fusion. The calculated result is subjected to spatial pyramid pooling after the ECA residual block and then input into the detection head network. In the detection head network, first The first upsampling operation is performed, and then the upsampling result is fused with the 6th layer of the feature extraction branch, and the fused features are input into the residual block. After the residual block is output, the second upsampling operation is performed, and the features are fused with the 4th layer of the feature extraction branch, and the fused features are input into the convolution layer. The output of the convolution layer and the output of the residual block are fused again. After the residual block is calculated again, the output feature is obtained; the output feature is divided into two branches, one of which is directly output by the medium target detection head, and the other branch is input into another convolution layer, and its output is fused with the output of the spatial pyramid pooling, and the fused feature is output by the large target detection head.

2. A potato picking and harvesting method based on PLP-net according to claim 1, characterized in that: The picking and harvesting robot comprises a vehicle body (1) and a moving chassis (2), wherein the moving chassis (2) carries the vehicle body (1) for movement, and a navigation depth camera (41) for identifying an unpicked area and a satellite positioning module (3) for positioning are arranged on the vehicle body (1). An identification depth camera (43), a parallel manipulator (44) and a conveying device are arranged in the vehicle body (1), wherein the identification depth camera (43) is used to identify potatoes to be picked up in the area, and the parallel manipulator (44) is arranged in front of the conveying device and is used to clamp potatoes identified by the identification depth camera (43) and place them on the conveying device.

3. A potato picking and harvesting method based on PLP-net according to claim 2, characterized in that: The potato grasping strategy includes calculation of the center point of the potato plane, calculation of the grasping depth and calculation of the grasping angle; when the potato is identified by the PLP-net algorithm, the center point of the selection box is used as the center point of the potato plane, and the center point is calculated as the coordinate of the potato center point; the distance between the highest point of the potato surface and the ground is identified and calculated by an identification depth camera (43); the grasping depth coordinate value of the parallel manipulator (44) is calculated by pre-setting a specific grasping depth value or a depth ratio; the angle between the edge line of the potato selection box and the coordinate axis is calculated, and the grasping direction of the parallel manipulator (44) is made parallel to the short side of the potato selection box, thereby obtaining the grasping angle of the parallel manipulator (44).

4. A potato picking and harvesting method based on PLP-net according to claim 2, characterized in that: After the parallel manipulator (44) completes the grasping action, it provides feedback to the host computer of the picking and harvesting robot, and the feedback content includes command acceptance status and command execution status; the command acceptance status includes the target object grasping command and the manipulator behavior command, the target object grasping command indicates whether to start the action and grasp the target potato, the manipulator behavior command includes the grasping plane coordinates, the grasping depth coordinates and the manipulator grasping angle, the feedback result of the command acceptance status includes receiving a completion signal, missing a part of the signal and missing a whole signal; the command execution status indicates whether the target potato is successfully grasped.

5. A potato picking and harvesting method based on PLP-net according to claim 2, characterized in that: The conveying device comprises a conveyor belt (51) and a baffle (52). The baffles (52) are provided in a plurality and fixedly connected to the conveyor belt (51), and one potato is placed between adjacent baffles (52). The picking and harvesting robot further comprises a grading and pushing device, the grading and pushing device comprises a pushing cylinder (53) and a grading and collecting frame (54). The pushing cylinder (53) is arranged on one side of the conveyor belt (51) and pushes the potatoes on the conveyor belt (51) into the grading and collecting frame (54). The grading and collecting frames (54) are provided in a plurality and arranged along the transmission direction of the conveyor belt (51), and each grading and collecting frame (54) contains potatoes of different grades. After the appearance information of the potato to be grasped is recognized by the recognition depth camera (43), the weight of the potato is calculated by the upper computer, and the grading information of the potato is marked. Potatoes of each grade are pushed into the grading and collecting frame (54) by the pushing cylinder (53) at the corresponding grading and collecting frame (54).

6. A potato picking and harvesting method based on PLP-net according to claim 5, characterized in that: Based on the potato picture taken by the recognition depth camera (43), the length, width and height information of the potato are extracted, and the weight of the potato is estimated and marked by a deep learning algorithm. Potatoes weighing more than 150g are defined as level one, potatoes weighing more than 100g but less than or equal to 150g are defined as level two, and potatoes weighing more than 50g but less than or equal to 100g are defined as level three.

7. A potato picking and harvesting method based on PLP-net according to claim 6, characterized in that: The deep learning algorithm uses a PW neural network to fit a potato weight training set. The input layer of the PW neural network includes three input data, namely, a length input l, a width input b and a height input c. The output layer includes an output signal, which is the potato weight d. Several hidden layers are arranged between the input layer and the output layer. The length, width, height and weight information of the potatoes in the training set are input into the PW neural network. Through multiple iterative training, the fitting relationship between the length, width and height of the potato and the weight is obtained.

8. The method for picking and harvesting potatoes based on PLP-net according to claim 5, characterized in that: A lifting mechanism (55) is arranged in the vehicle body (1), the lifting mechanism (55) comprising a weight sensor and a travel switch (56), the grading and collecting frame (54) being arranged on the lifting mechanism (55), the lifting mechanism (55) driving the grading and collecting frame (54) to descend according to the weight of the potatoes therein, when the grading and collecting frame (54) descends to the travel switch (56), the picking and harvesting robot stops working, and the upper computer sends a box-changing command to the user.

9. The method for picking and harvesting potatoes based on PLP-net according to claim 1, characterized in that: The picking and harvesting robot also includes a laser radar (42) for identifying obstacles randomly encountered during the movement process and driving the picking and harvesting robot to avoid them.

10. The method for picking and harvesting potatoes based on PLP-net according to claim 1, characterized in that: The picking and harvesting robot uses the corners of the working area as the starting point and moves continuously in an S shape.

Citation Information

Patent Citations

  • Apple picking method based on improved deep neural network

    CN112734727A

  • Potato quality identification method and processor

    CN117253070A

  • Yolo v8-DPE-based potato impurity identification and positioning method

    CN118379726A

  • Potato picking robot and picking method for missing harvesting

    CN118511737A

  • Winter jujube detection and positioning and mechanical arm picking sequence planning method based on YOLO-MLG and YAGR methods

    CN118636150A