A fresh grape detection method applied to agricultural robot picking

Abstract

A kind of fresh grape detection method applied to agricultural robot picking, comprising the following steps: step 1, the data of the grape image data set photographed is loaded into model, image is subjected to mosaic data enhancement operation, more abundant data sample is obtained, subsequent feature extraction operation is carried out again;Step 2, the features of the preprocessed image are extracted to generate a feature vector;Step 3, the extracted features are fused in multiple scales;Step 4, the multi-scale feature maps output by the network are used to predict targets and calculate loss functions;Step 5, determine whether the training has reached the optimal training effect, perform a new round of iteration or output the model result;Step 6, use the trained model to perform target detection operations, and use the non-maximum suppression method to delete redundant prediction boxes.The present application improves the recognition accuracy and speed of fresh grapes in complex scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for detecting fresh grapes used in robot picking. Background Art

[0002] Fresh grapes are an important cash crop widely cultivated across my country. However, currently, harvesting grapes is primarily done manually, a labor-intensive and time-consuming process. Due to the increasing aging population and the loss of rural labor, intelligent harvesting robots are needed to replace manual labor. Grape-harvesting robots must be able to navigate the vineyard independently and smoothly, ensuring full mobility and operational stability. Secondly, their visual recognition system must be able to quickly and accurately identify and locate grapes, even when partially obscured or overlapping. Finally, the robotic arm must quickly and accurately reach and pick the grapes, while also being able to adapt to changes in grape position caused by environmental factors. A harvesting robot system primarily consists of a control system, a visual recognition system, and a mechanical system. The mechanical system includes a travel mechanism, a support frame, a storage bin, a robotic arm, and an end effector. The recognition and positioning accuracy and speed of the visual system are crucial to improving the precision and efficiency of the harvesting robot. Furthermore, given the vulnerability of grapes to damage, an efficient and accurate visual recognition system is essential for the performance of the harvesting robot.

[0003] Traditional object detection relies primarily on machine learning and digital image processing techniques. These methods utilize basic features such as an object's color, shape, and texture for matching. However, because their thresholds require manual determination, these methods suffer from poor robustness and generalization. They are significantly affected by lighting conditions, the detection environment, and various subjective factors, leading to problems such as low detection accuracy, incomplete detection, and low efficiency.

[0004] In recent years, with the rapid development of computer vision, deep learning-based object detection algorithms have begun to be applied in agriculture. Currently, deep learning-based object detection algorithms are mainly divided into two-stage and single-stage algorithms. Due to the poor real-time performance of the two-stage algorithm, it is difficult to apply it to actual agricultural scenarios. YOLO is a typical CNN-based single-stage object detection algorithm. Compared with traditional algorithms, it has greater advantages in speed and accuracy, better meeting the requirements of agricultural scenarios and capable of rapid target detection and location. However, when applied to embedded devices, the conventional YOLO algorithm is still limited by accuracy and speed. Summary of the Invention

[0005] In order to overcome the shortcomings of existing technologies and solve the problems of poor recognition accuracy and slow recognition speed of grape picking robots in complex scenes, the present invention provides a fresh grape detection method applied to agricultural robot picking, which improves the recognition accuracy and recognition speed of fresh grapes in complex scenes.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A method for detecting table grapes used in agricultural robot picking, the method comprising the following steps:

[0008] Step 1: Collect the data of the grape image dataset, perform mosaic data enhancement on the data to obtain richer data samples, and then perform subsequent feature extraction operations;

[0009] Step 2: Extract features from the preprocessed image and generate a feature vector. The process is as follows: the data sample is input into the feature extraction neural network as a training sample. The feature map is first obtained through convolution, pooling and activation operations. The feature map is then input into the ELSAN structure to extract the shape, color and texture surface information of the grapes.

[0010] Step 3: Multi-scale fusion of the extracted features: The feature map output by the feature extraction neural network is used to achieve feature fusion at different levels through the feature pyramid structure;

[0011] Step 4: During training,

[0012] The image is divided into n*n cells. The grapes in each cell are judged based on the features obtained by the convolution operation. The boundary information of the grapes in the cell is obtained, and the location and category of the grapes are predicted. Then, the location and category of the grapes are compared with the calibrated real grape location and category. The loss function is calculated and backpropagation is performed based on the loss function to correct the weights of each layer of the neural network and adjust the model parameters.

[0013] Step 5: Determine whether the training has reached the end condition. If not, return to step 4. If the condition is met, stop training and output the model results.

[0014] Step 6: Use the trained model to perform target detection operations and use the non-maximum suppression method to delete redundant prediction boxes.

[0015] Furthermore, in step 2, the process of ELSAN extracting information is as follows:

[0016] 2.1.1. The input feature map is divided into two parts, namely cross-level connections and stack calculation blocks. Cross-level connections are used to speed up inference, and stack calculation blocks are used to enrich gradient information and improve learning ability.

[0017] 2.1.2. Use the concat operation to stack feature tensors to reduce the speed of the algorithm accessing memory;

[0018] 2.1.3. Finally, the channel shuffling operation is performed to improve the fusion of feature information in different channels.

[0019] Furthermore, in step 2, the SPPF module is introduced to perform pooling operations on the input feature map at different scales and concatenate the results to obtain rich feature information. The SE attention mechanism is added before the SPPF module to obtain the optimal feature weights between its channels. The process is as follows:

[0020] 2.2.1. First, spatially compress the input feature map H×W×C and perform global average pooling in the spatial dimension to obtain a 1×1×C feature map.

[0021] 2.2.2. Perform full-connection layer learning on the compressed feature map to obtain a feature map with channel attention;

[0022] 2.2.3. Combine the feature map with channel attention with the original feature map, and obtain the final feature map with channel attention by multiplying the weight coefficient channel by channel.

[0023] Furthermore, the process of step 3 is as follows:

[0024] 3.1. Gradually reduce the image resolution through downsampling operations, while extracting higher-level semantic information to obtain a feature map with low resolution but more abstract and global information;

[0025] 3.2. Upsample the low-resolution feature map to a higher resolution through upsampling to obtain a more local and detailed feature map;

[0026] 3.3. The high-resolution feature map obtained by upsampling is fused with the original feature map layer by layer to obtain rich contextual information.

[0027] Furthermore, in step 4, the loss function is defined as:

[0028] Loss 总体 =Loss 置信度 +Loss 定位

[0029] Among them, Loss 置信度 The binary cross entropy loss function used is used to calculate the difference between the confidence score obtained by the prediction and the actual situation; Loss 定位 The α-IoU used introduces power loss based on IoU and analyzes the loss function and gradient weight features. The positioning loss function is defined as follows:

[0030]

[0031] Among them, α is the coefficient of the positioning loss function, which can adapt to different recognition models; IoU is the intersection over union ratio of the predicted box and the true box, and the formula is defined as follows:

[0032]

[0033] In step 5, each training cycle is set to include N training rounds, where N is an integer. After each training cycle, the evaluation index mAP of each training cycle is calculated using the following formula:

[0034]

[0035] AP=∫P(R)dR

[0036]

[0037] Among them, k is the number of grape categories, TP is the number of true positive samples, that is, samples that are grapes but are also identified as grapes, FP is the number of false positive samples, that is, samples that are not grapes but are identified as grapes, and FN is the number of false negative samples, that is, samples that are grapes but are identified as non-grapes; different confidence levels will be obtained when predicting the results, that is, the probability that the algorithm believes it is grapes. Only when the confidence level is greater than the set confidence threshold will it be recorded in TP. Therefore, different confidence thresholds will produce different precision P and recall rate R; in order to reduce the interference of the artificially set confidence threshold on the algorithm evaluation, when the confidence threshold changes from 0-1, the current precision P and recall rate R are recorded, and a curve with the horizontal axis being the recall rate R and the vertical axis being the precision P is established, and the area AP enclosed by the precision-recall curve and the coordinate axis is calculated;

[0038] The mAP is calculated once in each training cycle, and the current maximum mAP is saved. When the mAP in X training cycles is less than the maximum mAP, the training stops, and the model parameters of the last training round of the training cycle corresponding to the maximum mAP are used as the final training result.

[0039] Preferably, in step 5, if training is not stopped when the maximum number of training rounds is reached, the model parameters trained in the last round of the training cycle corresponding to the maximum mAP are used as the final training results. This solution is to avoid extreme situations where the stopping condition cannot be met, and the maximum number of training rounds is often set to a larger value.

[0040] Of course, the end condition of step 5 can also be the maximum training rounds. Once the training rounds reach the maximum training rounds, the training stops and the training model parameters of the round with the minimum loss function are used as the final training result. This solution is relatively inefficient.

[0041] Preferably, in step 6, the non-maximum suppression method is defined as follows:

[0042]

[0043] Among them, M is the bounding box with the highest confidence, b i represents the comparison bounding box in the current target, N t is the threshold, α-IoU(M,b i ) is the loss function and σ is the penalty coefficient.

[0044] More preferably, in step 1, the mosaic data enhancement operation includes random cropping, rotation, scaling, splicing, brightness adjustment and masking operations.

[0045] In step 2 of the present invention, before the data-enhanced training samples are input into the backbone network to obtain the backbone network, a new backbone network is used to replace the original backbone network of YOLO. The new backbone network deletes the residual units and transition layers in the original network, thereby improving the inference speed of the model and the ability to extract features on small sample data sets. In addition, a channel shuffling operation is introduced into the new backbone network to increase the ability to fuse features in different channel layers and improve the detection performance of the model.

[0046] In step 3, the SE attention mechanism is introduced before multi-scale feature fusion. Based on global pooling and squeeze excitation, the SE module can obtain global information and adaptively adjust feature weights. SE is integrated into CSPDarknet53 to connect YOLO, establish interdependencies between model channels, and adaptively recalibrate feature responses in the channel direction, thereby improving network performance.

[0047] In step 4, a new convolution operation is introduced into the neck network of YOLO to replace the original convolution operation, thereby reducing the extraction of redundant features in the neck network, accelerating the reasoning speed of the model, and improving the spatial feature extraction capability of the model.

[0048] In the present invention, the intelligent picking task of the grape picking robot can be decomposed into three subtasks: 1) The picking robot needs to be able to walk alone and steadily in the vineyard, maintaining the full range of passability and the stability of the full range of operation; 2) The visual recognition system can quickly and accurately identify and locate grapes, can identify grapes under partial occlusion, and can separate overlapping grapes; 3) The robotic arm needs to quickly and accurately approach and pick grapes, and can also cope with the situation where the grapes are forced to move by unknown forces in the environment. Therefore, the picking robot includes a mechanical structure, a visual recognition system 3, and a control system containing robotic arm control and visual recognition, wherein the mechanical structure includes a walking mechanism 6, a bracket 1, a storage box 5, a robotic arm 2, and an end effector 4. The main functions of each module are as follows: (1) Visual recognition system. When the grape picking robot moves to the workspace, the recognition system first identifies and locates the grape bunches. The main purpose at this time is to preliminarily locate and distinguish overlapping grape bunches and guide the subsequent picking plan. Then the robotic arm is driven to approach the grapes, identify the fruit stems, and locate the picking point. The robotic arm is controlled by providing positioning feedback on the picking point, thus reducing the damage to the grape bunches caused by the robotic arm. (2) Control system. Based on the information obtained by the recognition system, the mechanical structure of the picking robot is controlled. That is, the path planning and walking control of the walking mechanism of the picking robot are performed, and the trajectory and posture of the robotic arm are planned and controlled. Due to the limited space of the picking robot, an embedded device is used as the control center of the picking robot. The embedded system of the picking robot must perform image processing, target detection, and control the robotic arm and mobile platform, so it has high requirements for the real-time performance of the target detection algorithm.

[0049] The beneficial effects of the present invention are mainly manifested in:

[0050] 1) The present invention improves the grape feature extraction capability by designing a new backbone network, effectively assists the training of small-scale data sets and accelerates the inference speed of target detection.

[0051] 2) The present invention replaces the convolution operation of the neck network of the YOLO model, thereby reducing the redundant features of grapes and improving the ability of spatial feature extraction.

[0052] 3) The present invention has designed an optimization scheme combining the YOLO model with the attention mechanism, combining spatial and channel attention to effectively suppress irrelevant features, making the model have better performance in complex scenes and better deal with problems such as the cluttered background of grapes, the lack of prominence of the target, and easy occlusion.

[0053] 4) This paper improves the ability to segment overlapping grapes by designing a new non-maximum suppression method. This effectively increases the accuracy and efficiency of fruit detection, thus providing a reliable detection solution for automated agricultural harvesting that does not rely on manual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a design flow chart of the present invention;

[0055] Figure 2 It is a network structure diagram of the present invention;

[0056] Figure 3 This is the ELSAN network structure diagram of the present invention;

[0057] Figure 4 This is a schematic diagram of a picking robot, where 1 represents a bracket, 2 represents a robotic arm, 3 represents a visual recognition system, 4 represents an end effector, 5 represents a storage box, and 6 represents a mobile platform. DETAILED DESCRIPTION

[0058] The present invention will be further described below with reference to the accompanying drawings.

[0059] Reference Figures 1 to 3 A method for detecting table grapes used in agricultural robot picking comprises the following steps:

[0060] Step 1: Load the data of the grape image dataset into the model, perform mosaic data enhancement on the image to obtain richer data samples, and then perform subsequent feature extraction operations;

[0061] In this example, grape images captured by a camera or mobile phone are subjected to mosaic data augmentation, including random cropping, rotation, scaling, splicing, brightness adjustment, and masking. This simulates real-world scenarios, enriches sample diversity, and enhances learning capabilities from small samples.

[0062] Step 2: Extract features from the preprocessed image and generate feature vectors: Before inputting the data-augmented training samples into the backbone network, a new backbone network is used to replace the original YOLO backbone network. The new backbone network deletes the residual units and transition layers in the original network, which improves the model's inference speed and the ability to extract features on small sample data sets. In addition, a channel shuffling operation is introduced in the new backbone network to increase the ability to fuse features in different channel layers and improve the detection performance of the model.

[0063] This embodiment uses ELSAN to replace the C3 and CBS structures in the original backbone network. Figure 1 As shown in the figure, a 640×640×3 preprocessed grape image is input into the feature extraction network. First, a 32×32×64 feature map is obtained through convolution, pooling, and activation operations (CBS module). The feature map is then input into the ELSAN structure to extract surface information such as shape, color, and texture. By continuously deepening the network structure, richer semantic information is obtained.

[0064] The process of ELSAN extracting information is as follows:

[0065] 2.1.1. The input feature map is divided into two parts: cross-level connections and stacked computation blocks. Cross-level connections are used to speed up inference, while stacked computation blocks are used to enrich gradient information and improve learning capabilities.

[0066] 2.1.2. Use the concat operation to stack feature tensors to reduce the speed of the algorithm accessing memory;

[0067] 2.1.3. Finally, the channel shuffling operation is performed to improve the fusion of feature information in different channels;

[0068] The SPPF module is introduced to perform pooling operations on the input feature map at different scales, and the results are concatenated to obtain rich feature information. In order to better perform multi-scale fusion operations, the SE attention mechanism is added before the SPPF module to obtain the optimal feature weights between its channels. The process is as follows:

[0069] 2.2.1. First, spatially compress the input feature map H×W×C and perform global average pooling in the spatial dimension to obtain a 1×1×C feature map.

[0070] 2.2.2. Perform full-connection layer learning on the compressed feature map to obtain a feature map with channel attention;

[0071] 2.2.3. Combine the feature map with channel attention with the original feature map, and obtain the final feature map with channel attention by multiplying the weight coefficient channel by channel.

[0072] Step 3: Multi-scale fusion of the extracted features: The feature map output by the feature extraction network is used to achieve feature fusion at different levels through the feature pyramid structure. The process is as follows:

[0073] 3.1. Gradually reduce the image resolution through downsampling operations, while extracting higher-level semantic information to obtain a feature map with low resolution but more abstract and global information;

[0074] 3.2. Upsample the low-resolution feature map to a higher resolution through upsampling to obtain a more local and detailed feature map;

[0075] 3.3. The upsampled high-resolution feature map is fused layer by layer with the original feature map to obtain rich context information;

[0076] In this process, the C3_Flase module is replaced with the C3P module to improve the neck network, such as Figure 1Because redundant feature information will appear when multi-scale features are fused, C3P only selects one-quarter of the channels for convolution operation to improve the detection speed of the model.

[0077] Step 4: Training process

[0078] The image is divided into n*n cells. The grapes in each cell are judged based on the features obtained by the convolution operation. The boundary information of the grapes in the cell is obtained, and the location and category of the grapes are predicted. Then, the location and category of the grapes are compared with the calibrated real grape location and category. The loss function is calculated and backpropagation is performed based on the loss function to correct the weights of each layer of the neural network and adjust the model parameters.

[0079] The loss function is defined as:

[0080] Loss 总体 =Loss 置信度 +Loss 定位

[0081] Among them, Loss 置信度 The binary cross entropy loss function used is used to calculate the difference between the confidence score obtained by the prediction and the actual situation; Loss 定位 The α-IoU used introduces power loss based on IoU and analyzes the loss function and gradient weight features. The positioning loss function is defined as follows:

[0082]

[0083] Among them, α is the coefficient of the positioning loss function, which can adapt to different recognition models; IoU is the intersection over union ratio of the predicted box and the true box, and the formula is defined as follows:

[0084]

[0085] Step 5: Determine whether the training has reached the end condition. If not, return to step 4. If the condition is met, stop training and output the model results.

[0086] In step 5, each training cycle is set to include N rounds of training rounds, where N is an integer (N is 10). After each training cycle, the evaluation index mAP of each training cycle is calculated using the following formula:

[0087]

[0088] AP=∫P(R)dR

[0089]

[0090] Among them, k is the number of grape categories, TP is the number of true positive samples, that is, samples that are grapes but are also identified as grapes, FP is the number of false positive samples, that is, samples that are not grapes but are identified as grapes, and FN is the number of false negative samples, that is, samples that are grapes but are identified as non-grapes; different confidence levels will be obtained when predicting the results, that is, the probability that the algorithm believes it is grapes. Only when the confidence level is greater than the set confidence threshold will it be recorded in TP. Therefore, different confidence thresholds will produce different precision P and recall rate R; in order to reduce the interference of the artificially set confidence threshold on the algorithm evaluation, when the confidence threshold changes from 0-1, the current precision P and recall rate R are recorded, and a curve with the horizontal axis being the recall rate R and the vertical axis being the precision P is established, and the area AP enclosed by the precision-recall curve and the coordinate axis is calculated;

[0091] The mAP is calculated once in each training cycle, and the current maximum mAP is saved. When the mAP is less than the maximum mAP in X (for example, X is 8) training cycles, the training stops and the model parameters of the last round of training corresponding to the maximum mAP are used as the final training result.

[0092] In step 5, if training is not stopped after reaching the maximum number of training rounds (e.g., 600), the model parameters trained in the last round of the training cycle corresponding to the maximum mAP are used as the final training result. This solution is to avoid extreme situations where the stopping condition cannot be met, and the maximum number of training rounds is often set to a larger value.

[0093] Of course, the end condition of step 5 can also be directly the maximum training rounds (for example, 600 rounds). Once the training rounds reach the maximum training rounds, the training will stop, and the round training model parameters with the minimum loss function will be used as the final training result. This solution is relatively inefficient.

[0094] Step 6: Target detection and non-maximum suppression

[0095] Since the YOLO algorithm is based on anchor frame detection, it requires non-maximum suppression to extract the most appropriate target. This paper uses the Soft Alpha_NMS method to replace the original NMS method, improving the algorithm's ability to detect grapes in overlapping scenes.

[0096] The non-maximum suppression method is used to replace the original method. Combined with Alpha-IoU, it has a finer division for deleting and retaining prediction boxes, which helps improve the detection accuracy of the algorithm in overlapping scenes. The non-maximum suppression method is defined as follows:

[0097]

[0098] Among them, M is the bounding box with the highest confidence, b i represents the comparison bounding box in the current target, N t is the threshold, usually set to 0.5, α-IoU(M,b i ) is the loss function and σ is the penalty coefficient.

[0099] Reference Figure 4 , applying the fresh grape detection method to an agricultural harvesting robot, which includes the following harvesting process: 1) Relying on a visual system to navigate and position the mobile platform, allowing harvestable grapes to enter the harvesting robot's operating space; 2) The robot uses a high-precision visual recognition system to identify the position and size of the grapes. During the recognition process, the robot calculates the precise coordinates of the grapes and plans the optimal harvesting path; 3) When approaching the grapes, the robot adjusts the position of its robotic arm based on the coordinates and posture of the grapes' picking points, and uses a shear-and-clamp integrated end mechanism to avoid damage to the fruit; 4) After harvesting, the robot places the grapes in its carrying storage box and continues with the next round of search and harvesting tasks. Through this automated process, the mobile grape harvesting robot significantly improves harvesting efficiency and quality, reduces labor costs and time consumption, and also reduces human interference in the vineyard during operations.

[0100] The embodiments of this specification are merely examples of implementations of the invention and are provided for illustrative purposes only. The scope of protection of the present invention should not be considered limited to the specific embodiments described in these embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by a person of ordinary skill in the art based on the invention.

Claims

1. A method for detecting table grapes used in agricultural robot picking, characterized in that: The method comprises the following steps: Step 1: Collect the data of the grape image dataset, perform mosaic data enhancement on the data to obtain richer data samples, and then perform subsequent feature extraction operations; Step 2: Extract features from the preprocessed image and generate a feature vector. The process is as follows: the data sample is input into the feature extraction neural network as a training sample. The feature map is first obtained through convolution, pooling and activation operations. The feature map is then input into the ELSAN structure to extract the shape, color and texture surface information of the grapes. Step 3: Multi-scale fusion of the extracted features: The feature map output by the feature extraction neural network is used to achieve feature fusion at different levels through the feature pyramid structure; Step 4: During the training process, the image is segmented into n*n cells. The grapes in each cell are judged based on the features obtained by the convolution operation, and the boundary information of the grapes in the cell is obtained. The location and category of the grapes are predicted, and then compared with the calibrated real grape location and category. The loss function is calculated, and backpropagation is performed based on the loss function to correct the weights of each layer of the neural network and adjust the model parameters. Step 5: Determine whether the training has reached the end condition. If not, return to step 4. If the condition is met, stop training and output the model results. Step 6: Use the trained model to perform target detection and use the non-maximum suppression method to delete redundant prediction boxes. In step 2, the process of ELSAN extracting information is as follows: 2.1.

1. The input feature map is divided into two parts, namely cross-level connections and stack calculation blocks. Cross-level connections are used to speed up inference, and stack calculation blocks are used to enrich gradient information and improve learning ability. 2.1.

2. Use the concat operation to stack feature tensors to reduce the speed of the algorithm accessing memory; 2.1.

3. Finally, the channel shuffling operation is performed to improve the fusion of feature information in different channels; In step 2, the SPPF module is introduced to perform pooling operations on the input feature map at different scales, and the operation results are concatenated to obtain rich feature information. The SE attention mechanism is added before the SPPF module to obtain the optimal feature weights between its channels. The process is as follows: 2.2.

1. First, spatially compress the input feature map H×W×C and perform global average pooling in the spatial dimension to obtain a 1×1×C feature map. 2.2.

2. Perform full-connection layer learning on the compressed feature map to obtain a feature map with channel attention; 2.2.

3. Combine the feature map with channel attention with the original feature map, and obtain the final feature map with channel attention by multiplying the weight coefficient channel by channel.

2. The method for detecting table grapes used in agricultural robot picking according to claim 1, wherein: The process of step 3 is as follows: 3.

1. Gradually reduce the image resolution through downsampling operations, while extracting higher-level semantic information to obtain a feature map with low resolution but more abstract and global information; 3.

2. Upsample the low-resolution feature map to a higher resolution through upsampling to obtain a more local and detailed feature map; 3.

3. The high-resolution feature map obtained by upsampling is fused with the original feature map layer by layer to obtain rich contextual information.

3. The method for detecting table grapes used in agricultural robot picking according to claim 1, wherein: In step 4, the loss function is defined as: ； in, The binary cross entropy loss function used is used to calculate the difference between the confidence score obtained by the prediction and the actual situation; Used α - IoU , based on IoU, power loss is introduced and loss function and gradient weight features are analyzed. The positioning loss function is defined as follows: ； Among them, α is the coefficient of the positioning loss function, which can adapt to different recognition models; IoU is the intersection over union ratio of the predicted box and the true box, and the formula is defined as follows: 。 4. The method for detecting table grapes used in agricultural robot picking according to claim 1, wherein: In step 5, each training cycle is set to include N training rounds, where N is an integer. After each training cycle, the evaluation index mAP of each training cycle is calculated using the following formula: ； ； ； ； Among them, k is the number of grape categories, TP is the number of true positive samples, that is, samples that are grapes but also identified as grapes, FP is the number of false positive samples, that is, samples that are not grapes but are identified as grapes, and FN is the number of false negative samples, that is, samples that are grapes but are identified as non-grapes. When predicting results, only when the confidence is greater than the set confidence threshold will it be recorded in TP. When the confidence threshold changes from 0 to 1, the accuracy of the current round of training is recorded. P and recall R , and calculate the area enclosed by the precision-recall curve and the coordinate axis AP ; The mAP is calculated once in each training cycle, and the current maximum mAP is saved. When the mAP in X training cycles is less than the maximum mAP, the training stops, and the model parameters of the last round of training corresponding to the maximum mAP are used as the final training result.

5. The method for detecting table grapes used in agricultural robot picking according to claim 4, wherein: In step 5, if the training is not stopped when the maximum number of training rounds is reached, the model parameters of the last round of training in the training cycle corresponding to the maximum mAP are used as the final training result.

6. The method for detecting table grapes used in agricultural robot picking according to claim 1, wherein: In step 6, the non-maximum suppression method is defined as follows: ； in, M is the bounding box with the highest confidence, b i represents the comparison bounding box in the current target, N t is the threshold, α-IoU (M, b i ) is the loss function and σ is the penalty coefficient.

7. The method for detecting table grapes used in agricultural robot picking according to claim 1, wherein: In step 1, the mosaic data enhancement operation includes random cropping, rotation, scaling, splicing, brightness adjustment and masking operations.

Images