A method for identifying red bayberry fruits based on an efficient aggregation network

Abstract

The application provides a kind of based on efficient aggregation network's Yangmei fruit identification method, belong to image recognition field, propose C3ELAN-YOLO model, on the basis of YOLOV5 C3 module is combined with ELAN design architecture and is integrated into backbone network, improve the feature extraction capability of network model and the context semantic information representation capability while reducing class confusion.Average pooling is added in the SPPF module, so that the network model not only focuses on the texture and contour features of the target fruit, but also pays attention to the overall data features of the fruit and the surrounding environment.Finally, the Yangmei fruit image is data enhanced to improve the robustness of C3ELAN-YOLO model.The application can improve the feature extraction capability of Yolov5 backbone network, improve the detection accuracy while considering the detection speed, and can accurately identify Yangmei in complex environment.

Description

Technical Field

[0001] This invention belongs to the field of deep learning image processing and relates to a method for recognizing bayberry fruits based on an efficient aggregation network. Background Technology

[0002] With the continuous development of agricultural technology in my country, traditional fruit harvesting operations are gradually facing problems such as rising labor costs and labor shortages. The previous manual harvesting methods are no longer sufficient to meet the needs of modern agriculture. Meanwhile, with the rapid development and popularization of robotics technology, high-performance robotic arms are designed to replace manual labor in heavy and monotonous tasks, and their research and application will greatly improve fruit harvesting efficiency. However, due to the dense foliage of fruit trees and the severe shading of fruits, robots cannot accurately identify and locate fruits, increasing the difficulty of robot operations and thus affecting their efficiency. Currently, agricultural harvesting robots in my country still face the challenges of low accuracy and slow speed in fruit identification and positioning. Improving the identification accuracy and speed of harvesting robots is of great significance to the development of my country's fruit industry.

[0003] In recent years, the rapid development of convolutional neural network technology has provided a new approach for fruit detection algorithms. Compared with traditional feature extraction methods, this algorithm has a stronger self-learning ability, can extract more abstract features, and has a stronger adaptability to complex environments. Although SSD and Faster RCNN networks have achieved good recognition results, their neural network structures are relatively complex, their real-time detection speed is relatively slow, and they are not very practical. In addition, the working efficiency of harvesting robots is related to the quality of the target detection network. At present, there are the following difficulties in the identification of fruits by agricultural harvesting robots: (1) The natural environment has complex lighting, and the image quality varies greatly under different lighting angles; (2) Under natural conditions, the fruit growth state is diverse, randomly distributed, and overlaps or is blocked by branches and leaves. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for identifying bayberry fruit based on an efficient aggregation network, which solves the problems of traditional target detection models being easily affected by complex backgrounds and missing or falsely detecting fruits with severe occlusion and overlap. The method uses C3ELAN-YOLO, which is an improvement based on the YOLOv5 network, to improve detection accuracy while taking into account detection speed, and can accurately identify bayberry fruit in complex environments.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for identifying bayberries based on an efficient aggregation network includes the following steps:

[0007] Step 1: Use the Labeling module to label the acquired bayberry images and generate a txt label file;

[0008] Step 2: Use the Imgaug module to perform data augmentation on the labeled dataset. For each generated image, there is a 50% probability of random translation, Gaussian blur, random contrast, random image rotation, motion blur, scaling blur, and splashing—seven image transformation methods. This randomness in image transformation will generate tens of thousands of possible image transformations.

[0009] Step 3: Use the improved YOLOv5 as the main algorithm for target detection of bayberry fruit, and replace the C3 module of the YOLOv5 backbone network with the C3ELAN module.

[0010] Step 4: Add the SPPF module in YOLOv5 to the average pooling layers of 5×5, 9×9, and 13×13;

[0011] Step 5: Adjust the parameters in the YOLOv5 related files: image input size is 640 pixels × 640 pixels, batch size is 8, maximum number of iterations is 300, initial learning rate is 0.01, cosine annealing hyperparameter is 0.2, and weight decay coefficient is 0.0005.

[0012] Step 6: Run the train file to train the fruit dataset. After training is complete, stop training and obtain the weights of the network training.

[0013] Step 7: The harvested fruit can be identified using the trained C3ELAN-YOLO network model.

[0014] In step 2, the random translation used can be specifically represented as:

[0015] Y1=(2x1-1)×64

[0016] Where Y1 is the pixel size of the image translation; x1 is a random number following Beta(0.5,0.5).

[0017] The random rotation angle of the image follows a normal distribution. Motion blur, scaling blur, and splashing are simulations of the blurring of the image produced by the camera during movement, as well as the occlusion of water droplets and other stains on the lens.

[0018] In step 3, the improved Yolov5 is used as the main algorithm for the detection of bayberry fruit. Its backbone network includes Conv, C3 and SPPF modules. The ELAN module structure is combined with the C3 module to form the C3ELAN module.

[0019] In step 4, 5×5, 9×9, and 13×13 average pooling layers are added to the SPPF module, and the average pooling, the original feature map, and the 5×5, 9×9, and 13×13 max pooling layers are spliced ​​together to preserve the overall data features of the target fruit.

[0020] Compared with existing technologies, the beneficial effects of the above technical solution are as follows: This invention solves the problem that traditional target detection models are easily affected by complex backgrounds in complex environments, resulting in missed detections and false detections of fruits with severe occlusion and overlap. It proposes a C3ELAN-YOLO network model based on YOLOv5, improving the backbone network by replacing the C3 module with a C3ELAN module, which has a high-efficiency aggregation network, thus enhancing the feature extraction and contextual semantic information representation capabilities of the C3ELAN-YOLO network model. Furthermore, the SPP module in the YOLOv5 network model is replaced with an SPPF module, and 5×5, 9×9, and 13×13 average pooling layers are added to preserve the overall data features of the target fruit. The C3ELAN-YOLO network model can maintain the detection speed of the traditional YOLOv5 network while improving the detection accuracy, making it more practical. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0022] Figure 1 The flowchart of the bayberry fruit identification method based on efficient aggregation network proposed in this invention is shown below;

[0023] Figure 2 The present invention proposes a C3ELAN-YOLO network model structure diagram;

[0024] Figure 3 This is a schematic diagram of the C3 module and the improved C3ELAN structure of the present invention;

[0025] Figure 4 This is a schematic diagram of the SPP module and the improved SPPF structure of the present invention;

[0026] Figure 5 These are schematic diagrams of the original bayberry image and the bayberry image after data enhancement, as presented in this invention.

[0027] Figure 6 , 7 This is a comparison diagram of the backbone network feature extraction of the C3ELAN-YOLO network model and the YOLOv5 network model of this invention. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content described in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] like Figure 1 , 4 As shown, this invention proposes a method for identifying bayberry fruits based on an efficient aggregation network, comprising the following steps:

[0030] Step 1: Use the Labeling module to label the acquired bayberry images and generate a txt label file;

[0031] Step 2: Use the Imgaug module to perform data augmentation on the labeled dataset. For each generated image, there is a 50% probability of random translation, Gaussian blur, random contrast, random image rotation, motion blur, scaling blur, and splashing—seven image transformation methods. This randomness in image transformation will generate tens of thousands of possible image transformations.

[0032] Step 3: Use the improved YOLOv5 as the main algorithm for target detection of bayberry fruit, and replace the C3 module of the YOLOv5 backbone network with the C3ELAN module.

[0033] Step 4: Replace the SPP module in YOLOv5 with the SPPF module and add average pooling layers of 5×5, 9×9, and 13×13.

[0034] Step 5: Adjust the parameters in the YOLOv5 related files: image input size is 640 pixels × 640 pixels, batch size is 8, maximum number of iterations is 300, initial learning rate is 0.01, cosine annealing hyperparameter is 0.2, and weight decay coefficient is 0.0005.

[0035] Step 6: Run the train file to train the fruit dataset. After training is complete, stop training and obtain the weights of the network training.

[0036] Step 7: The trained C3ELAN-YOLO network model can be used to identify bayberry fruits.

[0037] Step 8: In the `detect` file, replace the pre-trained weights in `yolov5s.pt` with the best training weights obtained after training in `best.pt`. Input the images from another dataset into the network to test the performance of the C3ELAN-YOLO network model. Extract the feature maps and compare the feature maps of layers 8 and 9 in the YOLOv5 network and the C3ELAN-YOLO network, as shown in the attached diagram. Figure 4 As shown.

[0038] like Figure 2 , 3 As shown in Figure 4, the C3ELAN-YOLO network model is based on the YOLOv5 network model. The backbone feature extraction network consists of CBS (Conv, BN, Mish), SPPF, and C3ELAN. C3ELAN is composed of three CBSs and two Resunit residual components. Its main advantage lies in using an efficient aggregation network architecture to increase gradient propagation efficiency, thereby balancing the network's learning ability and addressing the issue of the convergence of deep models gradually deteriorating during model scaling. Furthermore, SPPF employs 5×5, 9×9, and 13×13 max pooling and average pooling, combined with two CBSs. Its main advantage is that the network model not only preserves the edge data features of the target fruit but also the overall data features, enabling the C3ELAN-YOLO network model to pay attention to the environment of the bayberry fruit and improve the accuracy of its identification.

[0039] like Figure 5 As shown, the Imgaug module is used to augment the labeled dataset, and seven image transformations are randomly applied to the images, which greatly improves the robustness of the C3ELAN-YOLO network model.

[0040] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for identifying bayberry fruits based on an efficient aggregation network, characterized in that, Includes the following steps: Step 1: Use the Labeling image annotation tool to annotate the collected bayberry images and generate a txt label file; Step 2: Use the Imgaug image enhancement library in Python to perform data augmentation on the labeled dataset. Each generated image has a 50% probability of random translation, Gaussian blur, random contrast, random image rotation, motion blur, scaling blur, and rain occlusion, which improves the diversity of the dataset. Step 3: Use the improved YOLOv5 as the main algorithm for target detection of bayberry fruit, and replace the C3 module of the YOLOv5 backbone network with the C3ELAN module. The YOLOv5 network model consists of a backbone feature extraction network composed of Conv, BN, Mish, CBS, SPPF, and C3ELAN. C3ELAN is composed of three CBS and two Resunit residual components. SPPF uses 5×5, 9×9, and 13×13 max pooling and average pooling and is composed of two CBS. Step 4: Replace the SPP module in YOLOv5 with the SPPF module and add average pooling layers of 5×5, 9×9, and 13×13. Step 5: Adjust the parameters in the YOLOv5 related files: image input size is 640 pixels × 640 pixels, batch size is 8, maximum number of iterations is 300, initial learning rate is 0.01, cosine annealing hyperparameter is 0.2, and weight decay coefficient is 0.0005. Step 6: Run the train file to train the fruit dataset. After training is complete, stop training and obtain the weights of the network training. Step 7: The trained C3ELAN-YOLO network model can be used to identify the harvested fruit.

2. The method for identifying bayberry fruit according to claim 1, characterized in that: In step 2, the random translation used can be specifically represented as: Y1 = (2x1-1)×64, where Y1 is the pixel size of the image translation; x1 is a random number following Beta(0.5,0.5); The random rotation angle of the image follows a normal distribution N = (0, 30). The image contrast is selected from 50%, 150%, and 200% with a probability of 40%, 30%, and 30%, respectively. Motion blur, scaling blur, and rain occlusion in the image simulate the blurring of the image during camera movement and the occlusion of water droplets and dirt on the lens.

3. The method for identifying bayberry fruit according to claim 1, characterized in that: In step 3, the improved Yolov5 is used as the main algorithm for bayberry target detection. Its backbone network includes Conv, C3 and SPPF modules. The ELAN module structure is combined with the C3 module to form the C3ELAN module.

4. The method for identifying bayberry fruit according to claim 1, characterized in that: In step 4, 5×5, 9×9, and 13×13 average pooling layers are added to the SPPF module, and the average pooling, the original feature map, and the 5×5, 9×9, and 13×13 max pooling layers are spliced ​​together to preserve the overall data features of the target fruit.

Images