A method for identifying pollinated flowers of kiwifruit based on improved YOLOv5

Abstract

This invention belongs to the field of object detection, and its purpose is to provide a method for identifying kiwi pollinating flowers based on an improved YOLOv5 model, to solve the problem of difficulty in identifying kiwi flowers due to overlapping and tilt angles, thus achieving full pollination. The technical solution is an improved YOLOv5 method for identifying kiwi pollinating flowers, specifically including the following steps: Step S1: Acquire images of kiwi flowers; Step S2: Perform data augmentation on the acquired image data; Step S3: Label the kiwi flowers in the dataset, and then divide them into training, validation, and test sets; Step S4: Change the K-means clustering algorithm in the YOLOv5 model to the K-means++ algorithm; Step S5: Add a CBMA attention mechanism to the YOLOv5 model; Step S6: Change the GIOU-loss loss function in YOLOv5 to the CIOU-loss loss function; Step S7: Add a flower angle calculation module to the detect function; Step S8: Add a module for judging overlapping flowers and finding pollination points to the detect function; Step S9: Training and testing.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically the Ov5 method for identifying pollinated flowers of kiwifruit. Background Technology

[0002] Kiwifruit is a dioecious vine, requiring cross-pollination through pollen produced by the stamens. Under natural growing conditions, the female-to-male ratio is generally 1:8. Kiwifruit has a short flowering period of only 3-5 days, and its pollination requirements are very strict. The quality of pollination during the flowering period directly affects the viability of the pollen, which in turn directly affects pollen germination, fertilization, and fruit set.

[0003] The relative positions of the kiwi flower and its stamen change under different tilt angles. When there is no tilt angle, the stamen is located in the center of the kiwi, with petals evenly distributed around it. When there is a certain tilt angle, the kiwi flower and stamen are oval-shaped, with more petals on one side of the stamen and fewer petals on the other side due to obstruction. Analysis of the color and shape characteristics of the kiwi flower reveals that overlapping occurs, and the accuracy of flower identification is reduced due to the tilt angle.

[0004] Currently, manual pollination is widely used in my country's kiwifruit production, but this method is labor-intensive, inefficient, and costly. Electric pollinators are also used in some cases. With decreasing labor numbers and rising labor costs, manual pollination lacks stability and economic viability. Therefore, it is necessary to develop a method that can find the optimal pollination point even when kiwifruit flowers are overlapping or tilted. Combining target detection technology with pollination robots can not only reduce costs and save time but also achieve automated and intelligent pollination operations.

[0005] Currently, there are various object detection algorithms, such as YOLO v5, MobileNet, SSD, Faster-RCNN, VGG, and ResNet. Compared to YOLO v5, MobileNet is a lightweight deep neural network with fewer parameters and higher accuracy, generating the required feature maps in less computation time. However, due to its fewer convolutional layers, its feature extraction capability is very insufficient. The SSD model is fast, but its performance in recognizing small objects is poor. Faster-RCNN uses RPN to generate candidate regions and then extracts features through RoL pooling, resulting in relatively high accuracy and fast processing speed. However, its time complexity is high, making it difficult to apply in practice, and its training and inference times are long. The VGG model, obtained through the convolutional series method, has fewer parameters, is stable, and is easy to port. However, due to the large number of fully connected points and the deep network structure, its training speed is slow. Adding residual blocks to the ResNet network helps with backpropagation of gradients during training, but deep residual networks have a lot of redundancy. In summary, the YOLO v5 model can significantly separate features, enhance grid feature fusion, and ensure faster training speed. Summary of the Invention

[0006] This invention provides a method for identifying pollinated flowers of kiwifruit based on an improved YOLOv5, in order to solve the problem of difficulty in identification due to overlapping kiwifruit flowers and a certain tilt angle, thus achieving full pollination.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: an improved YOLOv5 method for identifying pollinated flowers of kiwifruit, specifically including the following steps:

[0008] Step S1: Use an RGB-D real-sensing camera to collect images of kiwi flowers in the afternoon with different numbers of flowers, different angles, and different overlap conditions to increase the diversity of the data;

[0009] Step S2: Use Python scripts to perform data augmentation on the acquired image data, including methods such as darkness, brightness, Gaussian noise, and mirroring, to highlight the local features of the image, effectively distinguish the features of different objects, increase the amount of data in the dataset, and thus enhance the robustness of the neural network model.

[0010] Step S3: Use the open-source image annotation software Labelimg to annotate the kiwi flower images in the dataset, and save the annotated results as an XML file. Use a Python script to divide all the annotated kiwi flower images into training, validation, and test sets in a 7:2:1 ratio;

[0011] Step S4: Change the K-means clustering algorithm in the YOLOv5 model to the K-means++ algorithm to reduce the impact of randomly selecting cluster centers on the anchor box selection size, which can select better initial cluster centers and improve the speed of the algorithm.

[0012] Step S5: A CBMA attention mechanism was added to the YOLOv5 model to improve the model's accuracy in extracting kiwi blossom features;

[0013] Step S6: Change the GIOU-loss function in YOLOv5 to the CIOU-loss function;

[0014] Step S7: Add a flower angle calculation module to the detect function; since the tilt angle of the flower is related to the position of the recognition box, by calculating and analyzing the flower angle calculation module, the positional relationship between the flower tilt angle, the center of the flower stamen, and the center of the flower can be obtained;

[0015] Step S8: Add a module for judging overlapping flowers and finding pollination points to the detect function. The overlapping of flowers is judged based on the intersection-union ratio of the target box and the predicted box, and the coordinates of the pollination point are determined according to different overlapping situations.

[0016] Step S9: Train and test the kiwi flower recognition model on a desktop computer using the PyCharm platform. The initial learning rate is set to 0.032, the IOU threshold is set to 0.01, and 8 samples are used as batch units during model training, with 200 training epochs. After training, the optimal training weight parameter file is obtained. The optimal weight parameter file is then used to recognize kiwi flower images, and the performance of the recognition model is evaluated based on the recognition scenario.

[0017] In step S6, the CIOU-loss loss function used by the YOLOv5 model is expressed as follows:

[0018]

[0019] Where: IOU is the intersection-union ratio of the target box and the predicted box; ρ is the Euclidean distance between the center points of the target box and the predicted box; b is the center point of the predicted box; b gt α is the center point of the ground truth bounding box; c is the diagonal distance of the smallest closure region that can simultaneously contain both the predicted and ground truth bounding boxes; α1 is the weight parameter; v is the parameter that measures aspect ratio consistency.

[0020] In step S7, the flower angle calculation module added to the detect function is:

[0021]

[0022] Where: the original length of the flower is L, α is the tilt angle of the flower itself, °; a is the thickness between the flower and the stamen, mm; b is the horizontal length of the flower after tilting, mm; Δx is the horizontal distance between the center point of the flower and the center point of the stamen, mm.

[0023] To reduce the negative impact of flower gravity on the calculation of the actual tilt angle, a regression function was used to calculate the collected images and data. The regression function is as follows:

[0024]

[0025] In step S8, the module for judging overlapping flowers and finding pollination points added to the detect function is as follows:

[0026]

[0027] Where x and y are the coordinates of each vertex of the polygon, and n is the number of sides of the polygon.

[0028] The formula for the coordinates of the centroid of a polygon is:

[0029]

[0030] Based on the output centroid coordinates and the coordinates of the center points of the flowers that make up the overlapping polygons, the target tilt angle β can be calculated. The formula for calculating β is:

[0031]

[0032] Where: x1 and x2 are the x-coordinates of the pollination point and the center of the flower stamen, respectively; H is the pollination distance, mm.

[0033] In step S9, the evaluation criterion for the feature recognition quality of the YOLOv5 model is expressed by the following formula:

[0034]

[0035] Where: P is the proportion of the true location in the image being identified; R is the proportion of the positive samples correctly identified in the test set; AP is the interpolation average accuracy of the detection algorithm; TP is the number of correctly identified kiwi flowers; FP is the number of incorrectly identified kiwi flowers; and FN is the number of kiwi flowers not detected.

[0036] The beneficial effects of this invention are:

[0037] This invention proposes an efficient kiwifruit pollination flower identification model based on an improved YOLO v5. It employs the K-means++ clustering algorithm to obtain better initial cluster centers. The CBMA mechanism is added to the model to improve the accuracy of kiwifruit flower feature extraction, effectively reducing the false negative and false positive rates. An angle-finding module and a flower overlap detection module are added to the `detect` function to improve the accuracy of flower overlap identification and flower tilt angle calculation. Results show that the improved YOLOv5 model can maintain both detection speed and accuracy, achieving prediction values ​​of 96.7% and 91.1% for flowers and stamens, respectively, with high recall values, meeting the requirements for identifying kiwifruit flower overlap and tilt angle. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the identification process using a modified YOLOv5-based kiwi pollinated flower recognition model.

[0039] Figure 2 A schematic diagram illustrating the calculation of the tilt angle for a tilted model of a kiwi flower.

[0040] Figure 3 A schematic diagram showing the angular relationship between the pollination point at the pollination end and the coordinate point of the kiwifruit pollination flower.

[0041] Figure 4 Photos of kiwi flowers taken from a kiwi orchard, showing overlapping and tilted patterns.

[0042] Figure 5 This is one of the images showing the recognition effect of kiwi pollinated flowers based on the improved YOLOv5.

[0043] Figure 6 The second image shows the results of kiwi pollinated flower recognition based on the improved YOLOv5. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The purpose is to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the concept and technical solutions of the present invention, and to facilitate its implementation.

[0045] like Figure 1As shown, this invention is a method for identifying kiwi pollinated flowers based on an improved YOLOv5 architecture. The overall framework of YOLOv5 mainly consists of four parts: the input layer, the backbone network, the neck network, and the output layer. First, an image of a kiwi flower is input to the input layer. Then, a Focus structure and a Spatial Pyramid Pooling (SPP) structure are added to the backbone network. The Focus structure uses slicing to break down a high-resolution feature map into multiple low-resolution feature maps. The SPP structure, also known as Spatial Pyramid Pooling, converts feature maps of arbitrary size into feature vectors of fixed size. The addition of these two structures enhances the network's feature fusion capability. Furthermore, a feature pyramid (top-down feature convolution) and a path aggregation structure (bottom-up path enhancement) are added to the neck network. The complete structure consists of a top-down FPN layer (a method of feature extraction) that conveys strong semantic features, plus a bottom-up PAN layer that conveys strong localization features (making it easier for lower-level information to be transmitted to higher-level layers). The synergistic effect of these two layers enhances the feature extraction effect and improves the accuracy of network training. The `detect` function incorporates modules for flower angle detection and overlap assessment, and uses K-means++ clustering to generate new anchor boxes. Figure 5 and Figure 6 It can be seen that the old anchor frames have been replaced by new ones. The flowers outlined in the image are all framed by the new anchor frames. The value in the upper right corner of the anchor frame represents the confidence level, i.e., the accuracy rate, in identifying the flowers (kiwi flowers) and stamens (stamens) in the actual captured image. Based on different overlap situations, the pollination point is determined, along with the angle at which the nozzle tilts towards the overlapping flowers in the X and Y directions. The results are then printed, and the corresponding pollination point coordinates are plotted.

[0046] Example 1

[0047] like Figures 1 to 6 As shown, the method for identifying pollinated flowers of kiwifruit based on the improved YOLOv5 includes the following steps:

[0048] Step S1: In the afternoon, use an RGB-D real-sensor camera to collect images of kiwi pollinated flowers under different overlap conditions and tilt angles in the kiwi orchard to increase the diversity of the data;

[0049] Step S2: Use Python scripts to perform data augmentation on the collected kiwi flower image data, including methods such as darkness, brightness, Gaussian noise, and mirroring, to highlight the local features of the image, effectively distinguish the features of different objects, increase the amount of data, and thus enhance the robustness of the neural network model.

[0050] Step S3: Use the open-source image annotation software Labelimg to annotate the kiwi pollinating flowers in the dataset, and save the annotated results as an XML file. Use a Python script to divide all the annotated kiwi flower images into training, validation, and test sets in a 7:2:1 ratio;

[0051] Step S4: The K-means clustering algorithm in the YOLOv5 model was changed to the K-means++ algorithm, which reduced the impact of randomly selecting cluster centers on the anchor box selection size, was able to select better initial cluster centers, and improved the speed of the algorithm.

[0052] Step S5: The CBMA attention mechanism was added to the YOLOv5 model;

[0053] Step S6: Change the GIOU-loss function in YOLOv5 to the CIOU-loss function;

[0054] Step S7: Add a pollinating flower angle calculation module to the detect function; since the tilt angle of the pollinating flower is related to the position of the recognition box, and the pollinating flower angle calculation module contains data of the flower and the stamen, by calculating and analyzing the pollinating flower angle calculation module, the tilt angle of the flower, the positional relationship between the center of the stamen and the center of the flower can be obtained. Figure 2 A schematic diagram for calculating the tilt angle of a kiwi flower model;

[0055] Step S8: Add a module for judging overlapping flowers and finding pollination points to the detect function. The overlap of flowers is determined based on the intersection-union ratio (IU), and the coordinates of the pollination points are determined according to different overlap conditions. Figure 3 A schematic diagram showing the angular relationship between the pollination point at the pollination end and the coordinates of the pollinated flower of kiwifruit is presented; x1, y1 and x2, y2 are the coordinates of the pollination point and the center point of the flower, respectively; H is the pollination distance in mm.

[0056] Step S9: Train and test the kiwi pollination flower recognition model on a desktop computer using the PyCharm platform. Input the collected dataset images into the network. Images of kiwi flowers under overlapping and tilted conditions are shown below. Figure 4 As shown in the attached image. The initial learning rate was set to 0.032, the IOU threshold was set to 0.01, and 8 samples were used as batch units during model training, with 200 training epochs. The optimal training weight parameter file was obtained after model training. The optimal weight file was then used to recognize kiwi blossom images, and the performance of the recognition model was evaluated based on the recognition scenario. Figure 5 and Figure 6As shown, the improved YOLOv5 model was used to identify kiwi flowers under overlapping and tilted conditions. The model distinguished the kiwi flowers and stamens by different anchor boxes and labeled them with corresponding flower and stamen information. Figure 5 and Figure 6 In the algorithm, K-means++ clustering generates new anchor boxes that replace the old ones. The flowers in the image are all framed with these new anchor boxes. The outer anchor box marking the entire kiwi flower is labeled "flowers," representing a single kiwi flower. Inside the outer anchor box is a smaller anchor box marking the stamens of the kiwi flower and labeled "stamens," representing the stamens of the kiwi flower. These new anchor boxes select better initial cluster centers for kiwi flower clusters, improving the algorithm's speed. Simultaneously, the `detect` function module is optimized by adding a flower angle calculation module and modules for overlapping flower detection and pollination point finding. This allows the improved YOLOv5 to detect the tilt angle and overlap of kiwi pollinated flowers. Figure 5 As shown, flowers2 has an accuracy rate of 88% in the upper right corner, while stamens2 has an accuracy rate of 94%. The x-coordinate of the flower is 0.189, and the y-coordinate is 6.361. The stamens of the kiwi flower cluster form a polygon (as shown). Figure 5 , Figure 6 The centroid coordinates of the triangles and polygons in the model are calculated using a formula, and the centroid is obtained after training and testing with the improved YOLOv5.

[0057] Calculations revealed that under the operating conditions of a flower tilt angle of 12°, a pollination air pressure of 0.150MPa, and a pollination hydraulic pressure of 0.070MPa, the optimal pollination distance for overlapping kiwi flowers was found to be 40cm, the required nozzle tilt angle was 6°, and the theoretical pollination time was 3.64s.

[0058] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above. Any implementation that adopts the inventive concept, uses the inventive technology and various non-innovative improvements, or directly applies the inventive concept and technology to other situations is within the protection scope of the present invention.

Claims

1. An improved YOLOv5 method for identifying pollinated flowers of kiwifruit, specifically including the following steps: Step S1: Use a camera to collect images of kiwi flowers in the afternoon with different numbers of flowers, different angles, and different overlap conditions to increase the diversity of the data; Step S2: Use a Python script to perform data augmentation on the acquired image data, including darkness, brightness, Gaussian noise, and mirroring, to highlight the local features of the image, effectively distinguish the features of different objects, increase the amount of data in the dataset, and thus enhance the robustness of the neural network model. Step S3: Use the open-source image annotation software Labelimg to annotate the kiwi flowers in the dataset and save the annotated results as an XML file; use a Python script to divide all the annotated kiwi flower images into training set, validation set and test set; Step S4: Change the K-means clustering algorithm in the YOLOv5 model to the K-means++ algorithm in order to select better initial cluster centers and improve the speed of the algorithm; Step S5: A CBMA attention mechanism was added to the YOLOv5 model to improve the model's accuracy in extracting kiwi blossom features; Step S6: Change the GIOU-loss function in YOLOv5 to the CIOU-loss function; Step S7: Add a flower angle calculation module to the detect function; by calculating and analyzing the flower angle calculation module, the flower tilt angle, the positional relationship between the center of the flower and the center of the flower can be obtained. Step S8: Add a module for judging overlapping flowers and finding pollination points to the detect function. The overlapping of flowers is judged based on the intersection-union ratio of the target box and the predicted box, and the coordinates of the pollination point are determined according to different overlapping situations. Step S9: Use the PyCharm platform to train and test the kiwi flower recognition model on a desktop computer; In step S7, the flower angle calculation module added to the detect function is: ； Where: the original length of the flower is L, α is the tilt angle of the flower itself, a is the thickness between the flower and the stamen, b is the horizontal length of the flower after tilting, and ∆x is the horizontal distance between the center point of the flower and the center point of the stamen; To reduce the negative impact of flower gravity on the calculation of the actual flower tilt angle, regression functions were used to calculate the collected images and data, resulting in the optimized flower angle calculation module: ； Where: α' is the calculated value of the flower's own tilt angle after optimization; In step S8, the module for judging overlapping flowers and finding pollination points added to the detect function is as follows: ； Where: x and y are the coordinates of each vertex of the polygon, and n is the number of sides of the polygon; The formula for the coordinates of the centroid of a polygon is: ； The target tilt angle is calculated based on the output centroid coordinates and the center point coordinates of the flowers that make up the overlapping polygons.

2. The improved YOLOv5 method for identifying pollinated flowers of kiwifruit according to claim 1, characterized in that: In step S6, the CIOU-loss loss function used by the YOLOv5 model is: ； Where: IOU is the intersection-union ratio of the target box and the predicted box; ρ is the Euclidean distance between the two center points; b is the center point of the predicted box; b gt α is the center point of the ground truth bounding box; c is the diagonal distance of the smallest closure region that can simultaneously contain both the predicted and ground truth bounding boxes; α1 is the weight parameter; v is the parameter that measures aspect ratio consistency.

3. The improved YOLOv5 method for identifying pollinated flowers of kiwifruit according to claim 2, characterized in that: In step S9, the initial learning rate is set to 0.032, the IOU threshold is set to 0.01, 8 samples are used as batch processing units during model training, and the number of training rounds is 200; after model training, the optimal training weight parameter file is obtained. The optimal weight parameter file was used to test the recognition of kiwi blossom images, and the performance of the recognition model was evaluated based on the recognition scenario.

4. The improved YOLOv5 method for identifying pollinated flowers of kiwifruit according to claim 3, characterized in that: In step S9, the evaluation criterion for the feature recognition quality of the YOLOv5 model is expressed by the following formula: ； ； ； Where: P is the proportion of the true location in the image being identified; R is the proportion of the positive samples correctly identified in the test set; AP is the interpolation average accuracy of the detection algorithm; TP is the number of correctly identified kiwi flowers; FP is the number of incorrectly identified kiwi flowers; and FN is the number of kiwi flowers not detected.

5. The improved YOLOv5 method for identifying pollinated flowers of kiwifruit according to claim 4, characterized in that: In step S3, the ratio of the training set, validation set, and test set is 7:2:1.

Images