Cherry tomato maturity detection method based on YOLOv8 improved algorithm

Abstract

The invention discloses a cherry tomato maturity detection method based on a YOLOv8 improved algorithm. Relates to the field of image recognition, in particular to the technical field of maturity detection of cherry tomatoes in a complex environment based on a YOLOv8 model. The problems of insufficient detection precision and insufficient dense target processing capability in the prior art are solved. The method comprises the following steps: preprocessing a data set; optimizing the YOLOv8n model: in a trunk part of the YOLOv8n model, replacing a Conv module with an ADown module; in the neck part of the YOLOv8n model, a C2F module is replaced by a VoVGSCSP module, a Conv module is replaced by a GSConv module, and an EMA mechanism is added; and evaluating the performance of the ASE-YOLOv8n model by adopting the test set, and determining that the test is qualified when the threshold value of the confidence coefficient is greater than 0.6.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to the technical field of maturity detection of cherry tomatoes in complex environments based on a YOLOv8 model. Background Art

[0002] Cherry tomatoes are a fruit and vegetable rich in nutrients such as lycopene, organic acids, and multiple vitamins. They are deeply loved by consumers and occupy an important position in people's daily diet. In the production process of cherry tomatoes, picking is crucial, but it still mainly relies on manual operation. Although manual picking can ensure a certain degree of precision, it has the problems of high labor intensity and high cost.

[0003] The growth characteristics of cherry tomatoes are extremely complex, with different fruit sizes, maturity levels, and positions, often overlapping. Coupled with the dense growth of branches and leaves, and the complex and changeable lighting conditions, this poses a great challenge to fruit picking. How to effectively organize labor, pick at the right time, and quickly and accurately obtain the precise positioning information of cherry tomatoes and grasp the distribution details of different maturity levels have become urgent issues to be solved. This can not only significantly reduce labor costs and improve picking efficiency, but also increase fruit farmers' income and market competitiveness.

[0004] In related studies, Gai et al. combined the densely connected convolutional network (DenseNet) to propose an improved YOLOv4 deep learning algorithm that can detect the three ripening stages of cherry fruits; Zhang et al. used the improved Yolov4-tiny model to perform multi-category detection on cherry tomatoes. This method can accurately detect fruits as pickable objects in complex environments such as no occlusion during the day, branch occlusion, fruit occlusion, and leaf occlusion; Wang et al. proposed an improved target detection algorithm based on YOLOv5n, and the improved model has improved accuracy and recall; Gu et al. built a citrus detection model in complex environments based on the YOLOv7-tiny model. The model has a good detection effect on citrus in scenes with occlusion, light changes, and motion changes.

[0005] Although the above methods have achieved certain results in the field of agricultural fruit target detection, they still have limitations. For example, the detection effect of small objects is poor, the positioning accuracy is not high enough, the adaptability to complex backgrounds and occlusions is poor, the ability to handle dense targets is insufficient, and there are also problems such as large consumption of computing resources. To overcome these shortcomings, it is necessary for the agricultural field to continue to improve existing models, or combine other technologies such as image segmentation, deep learning transfer learning, etc., to enhance detection accuracy and promote the overall level of agricultural intelligent applications. Summary of the invention

[0006] In view of the problems that the existing technology has insufficient detection accuracy, insufficient ability to handle dense targets in complex environments, high model complexity, and requires a large number of parameters, the purpose of the present invention is to propose a cherry tomato maturity detection method based on the improved YOLOv8 algorithm. The method introduces the ADown module, Slim-Neck structure and EMA attention mechanism module to obtain the ASE-YOLOv8n model, which can improve the maturity detection accuracy of cherry tomatoes in complex environments, while maintaining a lower computational overhead than the original YOLOv8 model, providing strong technical support for promoting smart agriculture.

[0007] The method comprises the following steps: S1. Establish and preprocess the data set, and randomly divide the preprocessed data set into training set, validation set and test set in a ratio of 8:1:1; S2. Build the basic YOLOv8n model; S3. Optimize the YOLOv8n model to obtain the ASE-YOLOv8n model, specifically: S31. In the backbone of the YOLOv8n model, replace the Conv module with the ADown module. S32. In the neck part of the YOLOv8n model, replace the C2F module with the VoVGSCSP module, replace the Conv module with the GSConv module, and add the EMA mechanism; S4. Use the training set and the validation set to iteratively train the ASE-YOLOv8n model to obtain the trained ASE-YOLOv8n model; S5. Use the test set to evaluate the performance of the trained ASE-YOLOv8n model. When the confidence threshold is greater than 0.6, a qualified ASE-YOLOv8n model is obtained. S6. Use the qualified ASE-YOLOv8n model to detect maturity.

[0008] Furthermore, the preprocessing is to perform data enhancement on the data set and manually label the enhanced data set.

[0009] Furthermore, the YOLOv8n model is a version of the YOLOv8 model; the construction of the basic YOLOv8n model is specifically: adjusting the YOLOv8n model and configuring corresponding training parameters, including setting the number of training rounds to 200.

[0010] Furthermore, in the backbone of the YOLOv8n model, replacing the Conv module with the ADown module specifically includes replacing the first Conv module, the second Conv module, and the third Conv module from the input end with the ADown module.

[0011] Furthermore, in the neck part of the YOLOv8n model, the replacing of the C2F module with the VoVGSCSP module is specifically: the first C2F module of the bottom-up branch and the first C2F module, the second C2F module and the third C2F module of the top-down branch are all replaced with VoVGSCSP modules; the replacing of the Conv module with the GSConv module is specifically: the first Conv module and the second Conv module of the top-down branch are both replaced with GSConv modules; the adding of the EMA mechanism is specifically: adding the EMA mechanism after the last VoVGSCSP module of the top-down branch.

[0012] Furthermore, the iterative training is as follows: inputting the training set and the validation set into the optimized ASE-YOLOv8n model and setting parameters for training. During the training process, the network parameters are optimized by the stochastic gradient descent method until the number of training rounds reaches 200 and the training is stopped to obtain the optimized ASE-YOLOv8n model after training.

[0013] Furthermore, the setting parameters are specifically as follows: in each round of training, the model input size and the number of samples are set to 640×640 and 32 respectively; the momentum, the initial learning rate of the weights and the weight decay of the stochastic gradient descent method are set to 0.937, 0.001 and 0.0005 respectively.

[0014] Furthermore, the processor implements the steps of the above method when executing the computer program.

[0015] Furthermore, a computer-readable storage medium is provided for storing computer instructions, wherein the computer instructions implement the steps of the above method when executed by a processor.

[0016] The beneficial effects of the method of the present invention are: (1) The present invention replaces the Conv module of the backbone part (Backbone) of the YOLOv8n model with the ADown module, so that the ASE-YOLOv8n model can capture the features of the image at a higher level, so as to facilitate more accurate target detection, and at the same time reduce the number of parameters, thereby reducing the complexity of the model and helping to improve the operating efficiency of the model.

[0017] (2) The present invention adds an EMA attention mechanism to the neck of the YOLOv8n model, so that the output feature map is input into the detection head after passing through the EMA attention mechanism, so as to enhance the inter-channel and global feature expression capabilities, realize more efficient and effective feature representation, and provide important technical support for improving the performance of computer vision tasks.

[0018] (3) The present invention accelerates the prediction speed of the ASE-YOLOv8n model while retaining the hidden connections, thereby reducing the resistance caused by the depth of the model by replacing the C2F module and the Conv module of the neck part (Neck) of the YOLOv8n model with the VoVGSCSP (Vision of Vision Grouped Spatial and Channel-wise Cross StagePartia) module and the GSConv module respectively.

[0019] (4) The invention can support efficient operation on edge devices, making it suitable for deployment in complex field conditions where real-time analysis of important status is required. This capability improves the accuracy and reliability of crop monitoring and management, making a significant contribution to the development of smart agriculture. The invention provides strong technical support for the advancement of smart agriculture by promoting more accurate and efficient farm management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the method described in Example 1 of the present invention; Figure 2 Schematic diagram of the structure of the Conv module in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the structure of the ADown module in Example 1 of the present invention; Figure 4 This is a schematic diagram of the EMA mechanism structure in Example 1 of the present invention; Figure 5 Schematic diagram of the GSConv module structure in Example 1 of the present invention; Figure 6 This is a schematic diagram of the structure of the GS bottleneck module in Example 1 of the present invention; Figure 7 Schematic diagram of the VoVGSCSP module structure in Example 1 of the present invention; Figure 8 Schematic diagram of the optimized YOLOv8n model structure in Example 1 of the present invention; Fig. 9 It is a schematic diagram of comparative experimental results of the YOLOv8n standard model and the qualified ASE-YOLOv8n model in Example 1 of the present invention; Fig.10 It is a schematic diagram of comparative experimental results of the ASE-YOLOv8n model and the YOLOv3-tiny, YOLOv5n, YOLO6, YOLOv7-tiny, YOLOv8n, YOLOv9t, YOLOv10n and YOLOv11 models in Example 2 of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Embodiment 1, This embodiment provides a method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm. Figure 1 As shown, the method comprises the following steps: S1. Establish and preprocess the data set, and randomly divide the preprocessed data set into training set, validation set and test set in the ratio of 8:1:1.

[0023] The relevant operations of step S1 are introduced with a specific example: The present invention uses a self-built cherry tomato dataset, which contains images of ripe cherry tomatoes and unripe cherry tomatoes. The cherry tomato image dataset is shot in a plantation. The image dataset contains cherry tomato fruits of different maturity, which are shot from 5 directions: left side, right side, overhead, upward and front. Two lighting conditions are considered, representing different light intensities. These images also contain various growth postures of cherry tomatoes in real life, such as overlapping, occluding and adhesion fruit images, and a total of 870 valid pictures of original cherry tomatoes are obtained. The dataset is subjected to data enhancement such as random angle rotation, vertical flipping, random brightness adjustment, noise addition, mosaic, etc., and the image attributes are comprehensively modified by combining geometric transformation and content transformation to enhance the robustness and generalization ability of the model. 9360 clear and complete images are left to ensure the quality of the dataset. The cherry tomato images were manually annotated and saved in YOLO format, with 55,342 labels for ripe cherry tomatoes and 51,974 labels for unripe cherry tomatoes. After annotation, they were divided into training sets (80%), validation sets (10%), and test sets (10%).

[0024] S2. Build a basic YOLOv8n model.

[0025] The relevant operations of step S2 are introduced with a specific example: The present invention compares different model configuration files, selects the YOLOv8n model in the YOLOv8 model, adjusts and configures the corresponding training parameters, including setting the number of training rounds to 200, so that it is more suitable for detection tasks.

[0026] S3. Optimize the YOLOv8n model, specifically: S31. In the backbone of the YOLOv8n model, replace the Conv module with the ADown module. S32. In the neck part of the YOLOv8n model, replace the C2F module with the VoVGSCSP module, replace the Conv module with the GSConv module, and add the EMA mechanism; The relevant operations of step S3 are introduced with a specific example: like Figure 8 As shown in the figure, in the trunk of the YOLOv8n model, the first Conv module, the second Conv module and the third Conv module from the input end are replaced by ADown modules; in the neck of the YOLOv8n model, the first C2F module of the bottom-up branch and the first C2F module, the second C2F module and the third C2F module of the top-down branch are replaced by VoVGSCSP modules; the first Conv module and the second Conv module of the top-down branch are replaced by GSConv modules, and the EMA mechanism is added after the last VoVGSCSP module of the top-down branch.

[0027] like Figure 2 As shown in the figure, in the convolution (Conv) module, the input feature map passes through the standard convolution layer (Conv), the batch normalization module (BatchNorm2d), and the nonlinear activation function (SiLU) in sequence to obtain the output of the convolution module; as shown in the figure Figure 3 As shown in the figure, after the input feature map in the ADown module passes through the average pooling layer (AvgPool2d) and the separation layer (Split) in sequence, it is divided into two sub-feature maps X1 and X2 in the channel dimension. The output of X1 after the convolution layer and the output of X2 after the maximum pooling layer (MaxPool2d) and the convolution layer are concatenated (Concat), and the concatenated feature map is used as the output of the ADown module.

[0028] In the ADown module, the average pooling layer can retain global information, and the maximum pooling layer can emphasize significant features. By performing convolution operations on the outputs of the two different pooling methods, multi-level information is effectively retained, which helps prevent potential information loss during the downsampling process and improves the model's sensitivity to detail features.

[0029] like Figure 4 As shown, in the EMA mechanism, the input feature map size is: ,in represents the number of feature channels, Represents the height and width of the feature map respectively. The input feature map first passes through the grouping module (Groups) and ( ) is split into the height of the feature map: ( ), the width of the feature map: ( ) and the number of feature channels of the feature map: ( ),in, Represents the feature dimension; ( )and( ) are respectively subjected to one-dimensional horizontal global pooling (X Avg Pool) and one-dimensional vertical global pooling (YAvg Pool) and then fed into the splicing + 1x1 convolution layer. The output of the splicing + 1x1 convolution layer is then subjected to the activation function (Sigmoid) and ( ) is imported into the first Re-weight module; ( ) After passing through the 3x3 convolutional layer and the output of the first Re-weight module, they pass through the two branches of the cross-spatial learning module. The output of the cross-spatial learning module passes through the activation function (Sigmoid) and ( ) is fed into the second Re-weight module to obtain the output of the EMA mechanism.

[0030] The cross-space learning module includes two branches. The first branch receives the output of the first Re-weight module, and inputs the output of the first Re-weight module into the group normalization module (GroupNorm), average pooling (Avgpool), normalization index (Softmax) and matrix multiplication (Matmul) in sequence to obtain the output of the first branch; the second branch receives the output of the 3x3 convolution layer, and inputs the output of the 3x3 convolution layer into the average pooling (Avg Pool), normalization index (Softmax) and matrix multiplication (Matmul) in sequence to obtain the output of the second branch; the output of the group normalization module and the output of the second branch, the output of the 3x3 convolution layer and the output of the first branch are weighted and summed to obtain the output of the cross-space learning module.

[0031] The three-way branch parallel structure enables the EMA mechanism to not only encode cross-channel feature information, but also retain the precise spatial structure information in the channel, and then input it into the cross-space learning module to improve the response speed, thereby reducing the computational overhead and effectively improving the performance of the ASE-YOLOv8n model. The ASE-YOLOv8n model has stronger feature extraction capabilities, thereby improving the detection capabilities of subsequent recognition.

[0032] The EMA mechanism can dynamically learn the weights of cherry tomato images. For small cherry tomato image targets, the weights can also be automatically adjusted according to the current input, so that the model can pay more attention to the target information, thereby improving the model's detection performance for cherry tomato targets in complex situations. The EMA mechanism does not change the input image size and feature dimension, so it has strong versatility.

[0033] like Figure 5 As shown in the figure, in the GSconv module, the input feature map passes through the convolution layer and the DWConv module in sequence, the output of the convolution layer and the output of the DWConv module are respectively input into the splicing module, and the output of the splicing module passes through the Shffule module to obtain the output of the GSconv module.

[0034] In the DWConv module, the input of the DWConv module is respectively input into a 1x1 convolution layer after N nxn convolutions to obtain the output of the DWConv module, where N represents the processing batch size of the DWConv module input, and n represents the number of channels of the DWConv module input.

[0035] In the latest progress of deep learning, the GSConv module is a new lightweight convolution method. The GSConv module has outstanding advantages in improving model accuracy, accelerating model convergence and detection process. Its core design is to optimize the feature extraction mechanism. By reducing the computational complexity and the number of parameters and improving the effectiveness of the attention module, the GSConv module enables the model to pay more attention to the key areas of the image. Although the GSConv module can replace the traditional convolution in the model, the use of the backbone part deepens the network level, increases the data flow resistance and reasoning time. Therefore, the present invention mainly deploys the GSConv module in the neck part where the feature map is already compact enough to avoid further conversion. The computational cost of the GSConv module to process the feature map is about 60% to 70% of the standard convolution method, which effectively reduces redundancy and eliminates duplicate information.

[0036] like Figure 7 As shown in FIG. 1 , in the VoVGSCSP module, the input of the VoVGSCSP module passes through two branches and then enters the convolution layer, and the output of the VoVGSCSP module is obtained after passing through the convolution layer; in the first branch, from input to output, only the convolution layer is passed; in the second branch, from input to output, the convolution layer, the GS bottleneck module (GS bottleneek) module and the splicing module are passed in sequence; as shown in FIG. Figure 6 As shown, the GS bottleneck module passes through two GS convolution modules (GSConv) from input to output, and the input and output of the GS bottleneck module are residually connected through the convolution layer.

[0037] The GSconv module and the VoVGSCSP module form a Slim-Neck structure.

[0038] S4. Use the training set and the validation set to iteratively train the optimized YOLOv8n model to obtain the trained YOLOv8n model.

[0039] The relevant operations of step S4 are introduced with a specific example: First, the training environment is configured: the GPU and CPU used in this embodiment are NVIDIA GeForce RTX4060ti 16G and 12th Gen Intel Core i7-12700KF@3.60 GHz respectively. The present invention is based on Windows 11 system equipped with Pytorch2.1.0 framework and Python 3.8 programming language to complete the operation.

[0040] The training set and validation set are input into ASE-YOLOv8n and the parameters are set for training. During the training process, the stochastic gradient descent (SGD) method is used to optimize the network parameters until the training number reaches 200 rounds and the training is stopped to obtain the trained ASE-YOLOv8n model.

[0041] In each round of training, the model input size and the number of samples are set to 640×640 and 32 respectively; the momentum, initial learning rate of weights and weight decay of the stochastic gradient descent method are set to 0.937, 0.001 and 0.0005 respectively.

[0042] like Figure 8 As shown in the figure, the backbone part (Backbone) of the optimized YOLOv8n model processes the input cherry tomato image. The backbone part gradually extracts the feature information of different levels of the input image from the bottom layer to the top layer, and finally outputs the cherry tomato multi-level feature map. These feature maps include features of different scales and depths extracted from the cherry tomato image and feature information of different levels. The neck part (Neck) receives the cherry tomato multi-level feature map from the trunk part, performs upsampling, downsampling and feature fusion processing on the cherry tomato multi-level feature map, and obtains a feature map that fuses multi-level cherry tomato features; The head part (Head) receives the feature map of the fused multi-level cherry tomato features from the neck part, performs recognition detection on the feature map of the fused multi-level cherry tomato features, and finally outputs the detection result of the cherry tomato maturity.

[0043] S5. Use the test set to evaluate the performance of the trained ASE-YOLOv8n model. When the confidence threshold is greater than 0.6, a qualified ASE-YOLOv8n model is obtained.

[0044] The relevant operations of step S5 are introduced with a specific example: The confidence threshold, often referred to as a "confidence score" or "confidence probability", is used to filter out low-quality bounding boxes in the detection results. The confidence threshold determines which detection results will be considered valid. The confidence score is the probability that the bounding box predicted by the model contains an object, usually between 0 and 1.

[0045] S6. Use the qualified ASE-YOLOv8n model to detect maturity.

[0046] The relevant operations of step S6 are introduced with a specific example: like Fig. 9 As shown in the figure, we selected six special cherry tomato dataset samples and conducted comparative experiments between the YOLOv8n standard model and the qualified ASE-YOLOv8n model. The qualified ASE-YOLOv8n model has a better recognition rate of cherry tomatoes in various environments than the YOLOv8n model. This not only reflects the progress of the qualified ASE-YOLOv8 model compared to the YOLOv8n model, but also shows that the qualified ASE-YOLOv8 model is more adaptable in real scenarios and can effectively cope with complex detection tasks.

[0047] Embodiment 2, To demonstrate the effectiveness of our proposed ASE-YOLOv8n model in detecting the maturity of cherry tomatoes, we compared it with several state-of-the-art object detection models in an experimental environment. Among them, we selected YOLOv3-tiny, YOLOv5n, YOLO6, YOLOv7-tiny, YOLOv8n, YOLOv9t, YOLOv10n, and YOLOv11 for comparative experiments. Fig.10 As shown in the figure, the comparative experiment evaluates the performance of each model on five key performance indicators, including precision, recall, F1 score, mAP50 and mAP50-95, where mAP5 represents the mean average precision of all categories when the IoU (intersection over union) threshold is 0.5, and mAP50-95 represents the mean average precision of all categories with a step size of 0.05 within the range of IoU thresholds of 0.5 to 0.95. Fig.10As shown in the figure, Model represents the model, Parameters represents the number of parameters, and Weight represents the weight. Compared with the comparison model, the ASE-YOLOv8n model performs well in five key performance indicators, and is also better than the average values ​​of other models in terms of parameter quantity and weight. It can be seen that the ASE-YOLOv8n model has high feasibility in practical applications.

Claims

1. A method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm, characterized in that: The method comprises the following steps: S1. Establish and preprocess the data set, and randomly divide the preprocessed data set into training set, validation set and test set in a ratio of 8:1:1; S2. Build the basic YOLOv8n model; S3. Optimize the YOLOv8n model to obtain the ASE-YOLOv8n model, specifically: S31. In the backbone of the YOLOv8n model, replace the Conv module with the ADown module. S32. In the neck part of the YOLOv8n model, replace the C2F module with the VoVGSCSP module, replace the Conv module with the GSConv module, and add the EMA mechanism; S4. Use the training set and the validation set to iteratively train the ASE-YOLOv8n model to obtain the trained ASE-YOLOv8n model; S5. Use the test set to evaluate the performance of the trained ASE-YOLOv8n model. When the confidence threshold is greater than 0.6, a qualified ASE-YOLOv8n model is obtained. S6. Use the qualified ASE-YOLOv8n model to detect maturity.

2. The cherry tomato maturity detection method based on the improved YOLOv8 algorithm according to claim 1 is characterized in that: The preprocessing is to perform data enhancement on the data set and manually label the enhanced data set.

3. The cherry tomato maturity detection method based on the improved YOLOv8 algorithm according to claim 1 is characterized in that: The YOLOv8n model is a version of the YOLOv8 model; the construction of the basic YOLOv8n model is specifically: adjusting the YOLOv8n model and configuring the corresponding training parameters, including setting the number of training rounds to 200.

4. The method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm according to claim 1, characterized in that: In the backbone part of the YOLOv8n model, the step of replacing the Conv module with the ADown module is as follows: replacing the first Conv module, the second Conv module, and the third Conv module from the input end with the ADown module.

5. The method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm according to claim 1, characterized in that: In the neck part of the YOLOv8n model, the C2F module is replaced with the VoVGSCSP module specifically as follows: the first C2F module of the bottom-up branch and the first C2F module, the second C2F module and the third C2F module of the top-down branch are all replaced with VoVGSCSP modules; the Conv module is replaced with the GSConv module specifically as follows: the first Conv module and the second Conv module of the top-down branch are all replaced with GSConv modules; the EMA mechanism is added specifically as follows: the EMA mechanism is added after the last VoVGSCSP module of the top-down branch.

6. The method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm according to claim 1, characterized in that: The iterative training is as follows: the training set and the validation set are input into the optimized ASE-YOLOv8n model and the parameters are set for training. During the training process, the network parameters are optimized by the stochastic gradient descent method until the training number reaches 200 rounds and the training is stopped to obtain the optimized ASE-YOLOv8n model after training.

7. The method for detecting the maturity of cherry tomatoes based on the improved YOLOv8 algorithm according to claim 6, characterized in that: The setting parameters are specifically as follows: in each round of training, the model input size and the number of samples are set to 640×640 and 32 respectively; the momentum, initial learning rate of weights and weight decay of the stochastic gradient descent method are set to 0.937, 0.001 and 0.0005 respectively.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images