Cherry tomato maturity identification method based on improved YOLOv8

Abstract

The invention discloses a cherry tomato maturity identification method based on improved YOLOv8. The method comprises the following steps: step 1, acquiring an original image data set of cherry tomatoes by using image acquisition equipment; step 2, classifying and marking anchor frames for the collected original data set based on fruit maturity, and performing data enhancement on the data set marked with the anchor frames; 3, a traditional YOLOv8 network model is improved, an improved YOLOv8-CCZ model is obtained, a training environment is configured, environment parameters are set, and the best weight file best.pt in a training result is stored; 4, performing identification test on the pre-divided test set data by using the best.pt weight file in the step 3; and 5, carrying out an ablation experiment, and carrying out a comparison experiment on the YOLOv8-CCZ network model and a plurality of advanced target detection network models. According to the method, the extraction capability of the maturity characteristics of the cherry tomatoes is improved, and the identification precision of the maturity of the cherry tomatoes is improved while the parameter quantity is compressed.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a method for recognizing the maturity of cherry tomatoes based on improved YOLOv8, which is suitable for cherry tomato image recognition scenarios with complex background interference, branches and leaves blocking targets, and adhesion and overlapping targets. Technical Background

[0002] In recent years, cherry tomatoes have been widely planted due to their high nutritional value and unique flavor. However, since cherry tomatoes are densely packed and mature at different times, there are often fruits of different maturity in a cluster. If the maturity of cherry tomatoes can be accurately monitored, cherry tomato supply chain managers can better plan the transportation, storage and sales strategies of cherry tomatoes, avoid surplus or shortage, and thus improve economic benefits. At the same time, it also allows farmers to harvest at the best maturity period, ensuring that the taste, nutrition and appearance of cherry tomatoes are in the best condition. This is very important for improving consumer satisfaction and market competitiveness. Therefore, it is crucial to propose a method that can quickly and accurately identify the maturity of cherry tomatoes.

[0003] People have conducted various studies on fruit maturity detection, such as the article entitled "Oil-tea fruit maturity detection based on improved YOLOv7" in the Journal of Agricultural Engineering, Vol. 40, No. 5, 2024, pp. 177-184; this article improves the original YOLOv7 network model by introducing the cross-attention mechanism CCA and using the non-maximum suppression DIoU-NMS algorithm based on distance and intersection-over-union ratio. Although the improved YOLOv7 network model can improve the ability to extract the maturity features of fruits obscured by branches and leaves and the ability to detect mutually obscured fruits, tea-oil fruits are not as dense as cherry tomatoes. If this method is applied to the maturity recognition of cherry tomatoes, missed detection and false detection will occur due to the dense occlusion of cherry tomatoes. At the same time, the improved YOLOv7 network model has a large number of parameters, and it is difficult to deploy it on embedded mechanical equipment.

[0004] Another example is Chinese patent CN1117593740A, published on February 13, 2024, which discloses a tomato maturity detection method, device, equipment and storage medium. The patent improves the original YOLOv8 network model by adding a CPCA attention mechanism, adding an upsampling layer, replacing part of the upsampling layer with a CARAFE sampling operator, and replacing the three C2f modules after the neck with a V0VGSCSP module. Although the improved YOLOv8 network model has a higher accuracy in tomato maturity recognition and reduces the model parameters, the patent divides tomatoes into three categories according to maturity, namely fully mature, semi-mature, and immature, and the prediction of tomato maturity is not accurate enough.

[0005] From the above, it can be seen that although the existing technology also uses the YOLO target detection algorithm to classify and detect the maturity of plant fruits, there are still problems such as unclear distinction of fruit maturity, reduced recognition accuracy when encountering situations such as changes in lighting, overlapping fruits, occlusion by branches and leaves, and backgrounds with high similarity, as well as a large number of model parameters. Summary of the invention

[0006] In order to overcome the limitations of various technical solutions given in the background technology, the present invention provides a cherry tomato maturity recognition method based on improved YOLOv8, aiming at the problems that the existing technology has unclear distinction between fruit maturity, reduced recognition accuracy when encountering conditions such as changed illumination, overlapping fruits, occlusion by branches and leaves, and backgrounds with high similarity, as well as a large number of model parameters.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] 1. A method for identifying the maturity of cherry tomatoes based on improved YOLOv8, characterized in that it comprises the following steps:

[0009] Step 1: Use an image acquisition device to collect a data set of cherry tomato original images;

[0010] Step 2: Anchor frames are annotated on the collected original data set based on the fruit maturity classification, and the data set with the annotated anchor frames is data enhanced. The data set after data enhancement is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0011] Step 3: Improve the traditional YOLOv8 network model to obtain the improved YOLOv8-CCZ model, configure the training environment and set the environment parameters, input the data set after data enhancement in step 2 into the YOLOv8-CCZ model for training, and save the best weight file best.pt in the training result;

[0012] Step 4: Use the best.pt weight file in step 3 to perform recognition test on the pre-divided test set data;

[0013] Step 5: Use the same data set and training environment parameters to conduct ablation experiments and compare the YOLOv8-CCZ network model with the traditional YOLOv8, YOLOv7, YOLOv5, YOLOv4, YOLOv3, and SSD target recognition network models.

[0014] Preferably, in step 1, the cherry tomato original image dataset is collected under natural light during the day. During the collection, a high-definition SLR camera is used to shoot the cherry tomatoes at a distance of 30 to 90 cm. The shooting angles are 6 directions, namely left, right, front, back, overhead and upward. The types of the collected cherry tomato image datasets cover various cherry tomato images such as front light, back light, densely overlapping occlusion of fruits, adhesion of fruits, and occlusion of branches and leaves.

[0015] Preferably, in step 2, the collected original image data set is classified and labeled using the labeling tool LabelImg, wherein the cherry tomatoes in the data set can be divided into 6 categories based on the maturity of the fruit, namely, unripe period, green ripe period, color change period, early red ripe period, middle red ripe period and late red ripe period. When labeling the anchor frame, the labeling is based on the minimum circumscribed rectangle of the outline of each cherry tomato.

[0016] Preferably, in step 2, the data enhancement method adopts CutMix data enhancement technology, which specifically includes:

[0017] In the CutMix method, a pair of images in the training set are randomly selected and defined as image A and image B. A cropping frame is randomly generated, the corresponding position of image A is cropped, and then the pixel map of the corresponding position of image B is placed in the cropped area of ​​image A to form a new image.

[0018] The CutMix data enhancement method has the following principle formula:

[0019] X ～ =M⊙X A +(1-M)⊙X B (1)

[0020] Y ～ =λY A +(1-λ)Y B (2)

[0021] Among them, X A With X B are two different training samples, Y A With Y B are their corresponding label values ​​respectively. ～ With Y ～ are the generated new samples and corresponding labels, M∈{0,1} w*h is a binary mask for subtracting some areas and filling, ⊙ is the pixel-by-pixel multiplication symbol, 1 is a binary mask with all elements equal to 1, and λ belongs to the Beta distribution: λ~Beta(a, b), where a and b are adjustable parameters;

[0022] The rectangular mask M is sampled, and the bounding box sampling formula of the clipping area is:

[0023]

[0024]

[0025] Among them, the bounding box of the clipping area is B = (γ x , γ y , γ w , γ h ), the bounding box of the entire sample area is A = (X, Y, W, H), Unif (0, W) represents γ x Uniform sampling on 0~W, Unif(0,H) represents γ x Sampling uniformly from 0 to H, Indicates the scale of the clipping area.

[0026] Preferably, in step 3, the traditional YOLOv8 network model is improved, and the improvement step specifically includes:

[0027] S31. Construct all C2f modules in the backbone network Backbone through the attention mechanism module CA, named C2f-CA;

[0028] S32. Construct all C2f modules in the neck network Neck through the variable convolution module DCNv3, named C2f-DCNv3;

[0029] S33. Use the NWD loss function instead of the original CIoU loss function.

[0030] Preferably, in step S31, the C2f-CA is combined with the attention mechanism module CA through the C2f module, specifically comprising: a CBL module, a Split module, n parallel connected Bottleneck modules, two parallel connected attention mechanism modules CA and a CBL module are sequentially connected, the Split module divides the number of input channels into two halves, and half of the channels are connected to the other half of the channels through the residual Connect after passing through the n parallel connected Bottleneck modules, the CBL module includes a standard convolution Conv module, a BN module and a Leaky_relu module, and the calculation process of the attention mechanism CA module on the input image specifically includes:

[0031] (1) Perform global average pooling twice on the input feature map in the width direction and the height direction to obtain two feature maps, which capture the global features in the width direction and the height direction respectively. The specific formula is:

[0032]

[0033] Among them, zc is the global feature, x c (i, j) is the input of the original feature.

[0034] (2) After channel-level splicing operations and convolution smoothing processing, a global receptive field and more accurate information representation are generated. The specific formula is:

[0035] f=δ(F 1 ([z h , z w ])) (6)

[0036] Among them, F 1 is the feature map after batch normalization; f is the feature map obtained by Sigmoid activation function; δ is the nonlinear activation function; z h and z w is the aggregate feature map.

[0037] (3) Through batch normalization and nonlinear mapping, it is transformed into a tensor with the same number of channels as the input data. The specific formula is:

[0038] g h =δ(F h (f h )) (7)

[0039] g w =δ(F w (f w )) (8)

[0040] Among them, f h and f w The feature map f is divided into two independent tensors F along the spatial dimension h and F w To respectively h and f w Transformed into a tensor with the same number of channels as the input; g h and g w are the attention weights of the input feature map in the height and width directions respectively after the above calculation; δ is the Sigmoid function.

[0041] (4) Through multiplication weighted calculation on the original feature map, the final feature map with attention weights in the width and height directions is obtained. The specific formula is:

[0042]

[0043] Among them, y c (i, j) is the final output feature, x c (i, j) is the original feature, and The features of the original number of channels are restored in the height and width dimensions after aggregation.

[0044] Preferably, in step S32, the C2f-DCNv3 specifically includes: a CBL module, a Split module, n parallel-connected DCNv3-Bottleneck modules and a CBL module are sequentially connected, the Split module divides the number of input channels into two halves, and half of the number of channels is connected to the other half of the number of channels through the residual Connect after passing through the n parallel-connected DCNv3-Bottleneck modules, the DCNv3-Bottleneck module includes three deformable convolution modules DCNv3 connected in sequence, and the deformable convolution module DCNv3 calculates the input image as follows:

[0045]

[0046] Where G represents the number of convolution groups; w g represents the shared projection weight within each group; m gk represents the normalized modulation factor of the kth sampling point in the gth group; x g Represents the feature map of the slice input; Δp gk represents the offset corresponding to the grid sampling position (g, k) in the g-th group, p 0 Represents each position on the input feature map, p k Represents each position of the convolution kernel.

[0047] Preferably, in step S33, the NWD loss function calculation process specifically includes:

[0048] (1) The second-order Wasserstein distance formula is:

[0049]

[0050] Among them, N a , N b Respectively represent the bounding box A=(cx a ,cy a , w a ,h a ) and bounding box B = (cx b ,cy b , w b ,h b ), ||·|| F represents the Frobenius norm, [] T Represents vector transpose;

[0051] (2) The normalized Wasserstein distance formula is:

[0052]

[0053] Where C is a constant related to the data set.

[0054] (3) The NWD loss function calculation formula is:

[0055]

[0056] Preferably, in step 3, the YOLOv8-CCZ network model includes an input terminal Input, a backbone network backbone, a neck network Neck and a prediction terminal Head, and the backbone network backbone includes in sequence: a first layer of standard convolution Conv module, a second layer of standard convolution Conv module, a third layer of C2f-CA module, a fourth layer of standard convolution Conv module, a fifth layer of C2f-CA module, a sixth layer of standard convolution Conv module, a seventh layer of C2f-CA module, an eighth layer of standard convolution Conv module, a ninth layer of standard convolution C2f-CA module, and a tenth layer of fast spatial pyramid pooling layer S PPF; the neck network Neck includes in sequence: an eleventh upsampling Upsample module, a twelfth residual connection Concat layer, a fourteenth C2f-DCNv3 module, a fifteenth upsampling Upsample module, a sixteenth C2f-DCNv3 module, a seventeenth standard convolution Conv module, an eighteenth residual connection Concat layer, a nineteenth C2f-DCNv3 module, a twentieth standard convolution Conv module, a twenty-first residual connection Concat layer, and a twenty-second C2f-DCNv3 module; the input end Input includes a twenty-third Detect layer.

[0057] Preferably, in step 3, the training environment configuration includes: the training platform operating system is Windows 11, the CPU is Inter (R) Core (TM) i7-13700KF @ 3.4GHz, and the memory is 64GiB; the graphics card is Nvidia GeForce RTX3090, 24GiB, the programming language is python 3.8, the deep learning framework is Pytorch 1.7.0, and the CUDA version is 10.1; the training parameter settings include: the image input size is 640×640, the batch processing size is set to 100, the initial learning rate is set to 0.003, the optimizer is SGD, the training rounds are 100, and other parameter values ​​​​all use the official default parameter values ​​of YOLOv8.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] The present invention provides a method for identifying the maturity of cherry tomatoes based on improved YOLOv8. The method enhances the data set by using the CutMix data enhancement technology, enriches the data set, and improves the generalization ability of the detection model. The attention mechanism module CA is embedded in all C2f modules in the backbone network Backbone, and the channel and spatial information are taken into consideration for feature enhancement, thereby effectively improving the model's attention to the ripening characteristics of cherry tomatoes, effectively reducing the cases of missed detection and false detection in cherry tomato image detection, and effectively solving the problem in the prior art that the recognition accuracy rate will be reduced when encountering situations such as illumination changes, fruit adhesion and overlap, branch and leaf occlusion, and backgrounds with high similarity. The variable convolution module DCNv3 All C2f modules in the neck network Neck are constructed, which have better feature extraction capabilities for irregular shapes in feature maps, so that the backbone network can better adapt to irregular spatial structures and focus on important targets more accurately, thereby improving the model's detection ability for occluded and overlapping targets, and further improving detection accuracy while compressing the model parameters; by using the NWD loss function instead of the original CIoU loss function, cherry tomatoes are dense small targets in the image. The NWD loss function has scale invariance, smoothness of position deviation, and the ability to measure the similarity between non-overlapping or mutually contained bounding boxes in detecting small targets, which effectively improves the recognition accuracy of cherry tomatoes' maturity; refer to the national standard GH / T 1193-2021 divides the tomato maturity level into 6 levels according to the color changes of the peel and flesh, namely, unripe stage, green ripe stage, color change stage, early red ripe stage, middle red ripe stage and late red ripe stage. Therefore, the present invention divides cherry tomatoes into 6 categories, namely, unripe stage, green ripe stage, color change stage, early red ripe stage, middle red ripe stage and late red ripe stage, which solves the problem of unclear distinction of fruit maturity in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of an embodiment of a method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention.

[0061] Figure 2 This is a schematic diagram of the improved process of the YOLOv8 network model according to an embodiment of a method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention:

[0062] Figure 3 It is a structural schematic diagram of the C2f-CA module in an embodiment of the method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention.

[0063] Figure 4 This is a structural diagram of the attention mechanism module CA in an embodiment of the method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention.

[0064] Figure 5 It is a structural schematic diagram of the C2f-DCNv3 module in an embodiment of the method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention.

[0065] Figure 6 It is a structural schematic diagram of the YOLOv8-CCZ network model in an embodiment of the method for identifying the maturity of cherry tomatoes based on improved YOLOv8 of the present invention.

[0066] Figure 7 A comparison curve of the recognition average precision mAP@0.5 of the YOLOv8-CCZ network model and the original YOLOv8 network model in the embodiment of the cherry tomato maturity recognition method based on the improved YOLOv8 of the present invention;

[0067] Figure 8 Schematic diagram of the detection results of the maturity of 6 types of cherry tomatoes in the cherry tomato data set by the YOLOv8-CCZ network model and the original YOLOv8 network model according to the embodiment of the cherry tomato maturity recognition method based on the improved YOLOv8 of the present invention DETAILED DESCRIPTION

[0068] The following specific implementation will be further described in conjunction with the above drawings, as follows:

[0069] See also Figure 1 The present invention proposes a method for identifying the maturity of cherry tomatoes based on improved YOLOv8, comprising the following steps:

[0070] Step 1: Use an image acquisition device to collect a data set of cherry tomato original images;

[0071] As a preferred implementation of step 1, step 1 specifically includes: the original image dataset of cherry tomatoes is collected under natural light during the day. During the collection, a high-definition SLR camera is used to shoot at a distance of 30 to 90 cm from the cherry tomatoes. The shooting angles are left, right, front, back, overhead, and overhead, a total of 6 directions. The types of cherry tomato image datasets collected cover various cherry tomato images such as front light, back light, densely overlapping occlusion of fruits, fruit adhesion, and occlusion of branches and leaves.

[0072] Step 2: Anchor frames are annotated on the collected original data set based on the fruit maturity classification, and the data set with the annotated anchor frames is data enhanced. The data set after data enhancement is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0073] As a preferred implementation of step 2, step 2 specifically includes: classifying and annotating the collected original image data set using the labeling tool LabelImg, wherein the cherry tomatoes in the data set can be divided into 6 categories based on the maturity of the fruit, namely, unripe period, green ripe period, color change period, early red ripe period, middle red ripe period and late red ripe period. When annotating the anchor frame, the annotation is performed based on the minimum circumscribed rectangle of the outline of each cherry tomato.

[0074] As a preferred implementation of step 2, step 2 also includes: the data enhancement method adopts CutMix data enhancement technology, specifically including:

[0075] In the CutMix method, a pair of images in the training set are randomly selected and defined as image A and image B. A cropping frame is randomly generated, the corresponding position of image A is cropped, and then the pixel map of the corresponding position of image B is placed in the cropped area of ​​image A to form a new image.

[0076] The CutMix data enhancement method has the following principle formula:

[0077] X ～ =M⊙X A +(1-M)⊙X B (1)

[0078] Y ～ =λY A +(1-λ)Y B (2)

[0079] Among them, X A With X B are two different training samples, Y A With Y B are their corresponding label values ​​respectively. ～ With Y ～ are the generated new samples and corresponding labels, M∈{0,1} w*h is a binary mask for subtracting some areas and filling, ⊙ is the pixel-by-pixel multiplication symbol, 1 is a binary mask with all elements equal to 1, and λ belongs to the Beta distribution: λ~Beta(a, b), where a and b are adjustable parameters;

[0080] The rectangular mask M is sampled, and the bounding box sampling formula of the clipping area is:

[0081]

[0082]

[0083] Among them, the bounding box of the clipping area is B = (γ x , γ y , γw , γ h ), the bounding box of the entire sample area is A = (X, Y, W, H), Unif (0, W) represents γ x Uniform sampling on 0~W, Unif(0,H) represents γ x Sampling uniformly from 0 to H, Indicates the scale of the clipping area.

[0084] Step 3: Improve the traditional YOLOv8 network model to obtain the improved YOLOv8-CCZ model, configure the training environment and set the environment parameters, input the data set after data enhancement in step 2 into the YOLOv8-CCZ model for training, and save the best weight file best.pt in the training result;

[0085] As a preferred implementation of step 3, Figure 2 As shown, step 3 specifically includes:

[0086] S31. Construct all C2f modules in the backbone network Backbone through the attention mechanism module CA, named C2f-CA;

[0087] S32. Construct all C2f modules in the neck network Neck through the variable convolution module DCNv3, named C2f-DCNv3

[0088] S33. Use the NWD loss function instead of the original CIoU loss function.

[0089] As a preferred implementation of S31, Figure 3 As shown, S31 specifically includes: C2f-CA is combined with the attention mechanism module CA through the C2f module, specifically including: a CBL module, a Split module, n parallel connected Bottleneck modules, two parallel connected attention mechanism modules CA and a CBL module are sequentially connected, the Split module divides the number of input channels into two halves, and half of the channels are connected to the other half of the channels through the residual Connect after passing through the n parallel connected Bottleneck modules. The CBL module includes a standard convolution Conv module, a BN module and a Leaky_relu module. The calculation process of the attention mechanism CA module on the input image specifically includes:

[0090] (1) Perform global average pooling twice on the input feature map in the width direction and the height direction to obtain two feature maps, which capture the global features in the width direction and the height direction respectively. The specific formula is:

[0091]

[0092] Among them, z c is the global feature, x c (i, j) is the input of the original feature.

[0093] (2) After channel-level splicing operations and convolution smoothing processing, a global receptive field and more accurate information representation are generated. The specific formula is:

[0094] f=δ(F 1 ([z h , z w ])) (6)

[0095] Among them, F 1 is the feature map after batch normalization; f is the feature map obtained by Sigmoid activation function; δ is the nonlinear activation function; z h and z w is the aggregate feature map.

[0096] (3) Through batch normalization and nonlinear mapping, it is transformed into a tensor with the same number of channels as the input data. The specific formula is:

[0097] g h =δ(F h (f h )) (7)

[0098] g w =δ(F w (f w )) (8)

[0099] Among them, f h and f w The feature map f is divided into two independent tensors F along the spatial dimension h and F w To respectively h and f w Transformed into a tensor with the same number of channels as the input; g h and g w are the attention weights of the input feature map in the height and width directions respectively after the above calculation; δ is the Sigmoid function.

[0100] (4) Through multiplication weighted calculation on the original feature map, the final feature map with attention weights in the width and height directions is obtained. The specific formula is:

[0101]

[0102] Among them, y c (i, j) is the final output feature, x c (i, j) is the original feature, and The features of the original number of channels are restored in the height and width dimensions after aggregation.

[0103] As a preferred implementation of S32, Figure 5 As shown, C2f-DCNv3 specifically includes: a CBL module, a Split module, n parallel-connected DCNv3-Bottleneck modules and a CBL module are connected in sequence, the Split module divides the number of input channels into two halves, and half of the channels are connected to the other half of the channels through the residual Connect after passing through n parallel-connected DCNv3-Bottleneck modules. The DCNv3-Bottleneck module includes three deformable convolution modules DCNv3 connected in sequence, and the deformable convolution module DCNv3 calculates the input image as follows:

[0104]

[0105] Where G represents the number of convolution groups; w g represents the shared projection weight within each group; m gk represents the normalized modulation factor of the kth sampling point in the gth group; x g Represents the feature map of the slice input; Δp gk represents the offset corresponding to the grid sampling position (g, k) in the g-th group, p0 represents each position on the input feature map, and p k Represents each position of the convolution kernel.

[0106] Preferably, in step S33, the NWD loss function calculation process specifically includes:

[0107] (1) The second-order Wasserstein distance formula is:

[0108]

[0109] Among them, N a , N b Respectively represent the bounding box A=(cx a ,cy a , w a ,h a ) and bounding box B = (cx b ,cy b , w b ,h b ), ||·|| F represents the Frobenius norm, [] T Represents vector transpose;

[0110] (2) The normalized Wasserstein distance formula is:

[0111]

[0112] Where C is a constant related to the data set.

[0113] (3) The NWD loss function calculation formula is:

[0114]

[0115] As a preferred implementation of step 3, Figure 6 As shown, step 3 also includes: the YOLOv8-CCZ network model includes an input terminal Input, a backbone network backbone, a neck network Neck and a prediction terminal Head, and the backbone network backbone includes in sequence: a first layer of standard convolution Conv module, a second layer of standard convolution Conv module, a third layer of C2f-CA module, a fourth layer of standard convolution Conv module, a fifth layer of C2f-CA module, a sixth layer of standard convolution Conv module, a seventh layer of C2f-CA module, an eighth layer of standard convolution Conv module, a ninth layer of standard convolution C2f-CA module, and a tenth layer of fast spatial pyramid pooling layer S PPF; the neck network Neck includes: the eleventh upsampling Upsample module, the twelfth residual connection Concat layer, the fourteenth C2f-DCNv3 module, the fifteenth upsampling Upsample module, the sixteenth C2f-DCNv3 module, the seventeenth standard convolution Conv module, the eighteenth residual connection Concat layer, the nineteenth C2f-DCNv3 module, the twentieth standard convolution Conv module, the twenty-first residual connection Concat layer, and the twenty-second C2f-DCNv3 module; the prediction end Head includes the twenty-third Detect layer.

[0116] As a preferred implementation of step 3, step 3 also includes: the training environment configuration includes: the training platform operating system is Windows 11, the CPU is Inter (R) Core (TM) i7-13700KF @ 3.4GHz, and the memory is 64GiB; the graphics card is Nvidia GeForce RTX 3090, 24GiB, the programming language is python 3.8, the deep learning framework is Pytorch 1.7.0, and the CUDA version is 10.1; the training parameter settings include: the image input size is 640×640, the batch processing size is set to 100, the initial learning rate is set to 0.003, the optimizer is SGD, the training rounds are 100, and other parameter values ​​​​all use the official default parameter values ​​of YOLOv8.

[0117] Step 4: Use the best.pt weight file in step 3 to perform recognition test on the pre-divided test set data;

[0118] Step 5: Use the same data set and training environment parameters to conduct ablation experiments and compare the YOLOv8-CCZ network model with the traditional YOLOv8, YOLOv7, YOLOv5, YOLOv4, YOLOv3, and SSD target recognition network models.

[0119] Example

[0120] In order to verify the superior performance of the cherry tomato maturity recognition method based on the improved YOLOv8 proposed in the present invention, two groups of experiments were set up, namely, an ablation experiment was set up based on the three improved methods (CA, DCNv3, NWD) of the present invention, and a comparative experiment was set up between YOLOv8-CCZ and the traditional YOLOv8, YOLOv7, YOLOv5, YOLOv4, YOLOv3, and SSD target recognition network models. The details are as follows:

[0121] First, configure the training environment and set relevant parameters. In this embodiment, the training environment configuration and relevant parameter settings of the cherry tomato maturity recognition method based on the improved YOLOv8 of the present invention are as follows:

[0122] The training environment configuration includes: the training platform operating system is Windows 11, the CPU is Inter(R) Core(TM) i7-13700KF@3.4GHz, and the memory is 64GiB; the graphics card is Nvidia GeForce RTX 3090, 24GiB, the programming language is python 3.8, the deep learning framework is Pytorch 1.7.0, and the CUDA version is 10.1; the relevant parameter settings include: the image input size is 640×640, the batch processing size is set to 100, the initial learning rate is set to 0.003, the optimizer is SGD, the training rounds are 100, and other parameter values ​​​​use the official default parameter values ​​of YOLOv8.

[0123] Next, the experimental evaluation criteria are clarified. In this embodiment, the experimental evaluation criteria of the cherry tomato maturity recognition method based on the improved YOLOv8 of the present invention are as follows:

[0124] This paper uses mean average precision (mAP) mAP@0.5, mAP0.5:0.95 as the evaluation index of the model, and its value is positively correlated with the model performance. The calculation method is as follows:

[0125]

[0126]

[0127]

[0128] Among them, TP represents the number of correctly detected targets, FP represents the number of incorrectly detected targets, FN represents the number of missed detections, P represents precision, R represents recall, AP is the area under the P and R curves, AP represents the sum of AP values ​​of all categories, and N represents the total number of categories. mAP@0.5 means that when the intersection over Union (IoU) of the predicted box and the true bounding box is set to 0.50, the mean AP of each category is calculated. mAP@0.5:0.95 means the mean AP when the IoU is from 0.50 to 0.95 with an interval of 0.05.

[0129] Then, an experiment was conducted, in which two groups of experiments were set up as follows:

[0130] Experiment 1: ablation experiments were performed based on the three improved methods (C2f-CA, C2f-DCNv3, NWD) of the present invention. Five working conditions were set in Experiment 1, based on the following:

[0131] Experimental condition 1 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the original YOLOv8 network model for the experiment.

[0132] Experimental condition 2 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv8+C2f-CA network model for the experiment.

[0133] Experimental condition 3 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv8+C2f-DCNv3 network model for the experiment.

[0134] Experimental condition 4 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv8+NWD network model for the experiment.

[0135] Experimental condition 5 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv8+C2f-DCNv3+NWD network model for the experiment.

[0136] Experimental condition 6 is set as follows: ensure that the configured training environment and the relevant parameters of the settings remain unchanged, and use the YOLOv8+C2f-.CA+C2f-DCNv3+NWD (YOLOv8-CCZ) network model for the experiment.

[0137] Among them, the experimental results of the above six working conditions on the cherry tomato dataset are shown in Table 1.

[0138] Table 1 Ablation experiment results (the bold value is the highest value in the column)

[0139] Working conditions Precision P / % Recall rate R / % mAP@0.5 / % Condition 1 80.23％ 81.56％ 88.32％ Condition 2 82.65％ 83.77％ 88.12％ Condition 3 86.58％ 83.21％ 88.39％ Condition 4 86.33％ 85.19％ 89.84％ Condition 5 88.36％ 87.25％ 90.01％ Condition 6 90.37％ 89.66％ 92.08％

[0140] As can be seen from Table 1, by comparing the experimental results of working conditions 1, 2, 3, and 4, it can be seen that the three improvement methods have effectively improved the detection performance of the network model. Among them, using the NWD loss function to replace the original CIOU loss function has the most obvious effect on the improved model, with the precision P%, recall R%, and mAP@0.5 / % increased by 8.1%, 3.63%, and 2.52%, respectively. By comparing working conditions 5 and 6, it can be found that the C2f-CA module is finally introduced on the basis of the YOLOv8+C2f-DCNv3+NWD network model, and the network model The precision P / %, recall R / %, and mAP@0.5 / % of the model increased by 2.01%, 2.41%, and 2.07% respectively, indicating that the model has enhanced its ability to extract the maturity characteristics of cherry tomatoes. Finally, by comparing working condition 1 with working condition 6, it can be seen that compared with the original YOLOv8 network model, the precision P%, recall R%, and mAP@0.5 / % of the YOLOv8-CCZ network model increased by 10.14%, 8.1%, and 3.76% respectively. It can be seen that the performance of the YOLOv8-CCZ network model is better than that of the original YOLOv8 network model.

[0141] Experiment 2: Compare YOLOv8-CCZ with traditional YOLOv8, YOLOv7, YOLOv5, YOLOv4, YOLOv3, and SSD target recognition network models. Experiment 2 sets 7 working conditions, as follows:

[0142] Experimental condition 1 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the SSD network model for the experiment.

[0143] Experimental condition 2 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv3 network model for the experiment.

[0144] Experimental condition 3 is set as follows: ensure that the configured training environment and the relevant parameters of the settings remain unchanged, and use the YOLOv4 network model for the experiment.

[0145] Experimental condition 4 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the YOLOv5 network model for the experiment.

[0146] Experimental condition 5 is set as follows: ensure that the configured training environment and the relevant parameters of the settings remain unchanged, and use the YOLOv7 network model for the experiment.

[0147] Experimental condition 6 is set as follows: ensure that the configured training environment and related parameters remain unchanged, and use the original YOLOv8 network model for the experiment.

[0148] Experimental condition 7 is set as follows: ensure that the configured training environment and the relevant parameters of the settings remain unchanged, and use the YOLOv8-CCZ network model for the experiment.

[0149] Among them, the experimental results of the above 7 working conditions on the cherry tomato dataset are shown in Table 2.

[0150] Table 2 Comparative experimental results of different network models (the bold value is the highest value in the column)

[0151] Working conditions Precision P / % Recall rate R / % mAP@0.5 / % Condition 1 77.23％ 71.51％ 73.89％ Condition 2 78.57％ 80.79％ 78.56％ Condition 3 73.52％ 83.74％ 84.77％ Condition 4 83.63％ 83.58％ 84.69％ Condition 5 91.23％ 78.12％ 80.36％ Condition 6 80.23％ 81.56％ 88.32％ Condition 7 90.37％ 89.66％ 92.08％

[0152] It can be found from Table 2 that the mAP@0.5 / % value of the YOLOv8-CCZ network model is 18.19% higher than SSD, 13.52% higher than YOLOv3, 7.31% higher than YOLOv4, 7.39% higher than YOLOv5, 11.72% higher than YOLOv7, and 3.76% higher than the original YOLOv8. Among them, the YOLOv7 network model has the highest recall rate improvement, but its recall rate R / % and mAP@0.5 / % are reduced by 11.54% and 11.72% respectively compared with the YOLOv8-CCZ network model. It can be seen that the YOLOv8-CCZ network model has better comprehensive performance than the current mainstream target detection network models.

[0153] Finally, in order to further demonstrate the superiority of the YOLOv8-CCZ network model over the original YOLOv8 network model, the mAP@0.5 data of the YOLOv8-CCZ network model and the YOLOv8 network model are compared, and the detection results of the six types of maturity of the YOLOv8-CCZ network model and the YOLOv8 network model in the cherry tomato dataset are compared. The details are as follows:

[0154] like Figure 7 As shown in the figure, the mAP@0.5 value of the YOLOv8-CCZ network model is larger than the mAP@0.5 value of the YOLOv8 network model in each round of training, and the changes in the mAP@0.5 values ​​of both fluctuate. However, it can be seen that the mAP@0.5 of the YOLOv8-CCZ network model converges faster and is more stable than the mAP@0.5 of the YOLOv8 network model.

[0155] Figure 8As shown in the figure, compared with the original YOLOv8 network model, the YOLOv8-CCZ network model has improved the detection accuracy of cherry tomatoes of various maturity levels. For the unripe class, the mAP@0.5 value increased from 87.5% to 91.2%, for the green-ripe class, the mAP@0.5 value increased from 89.6% to 93.2%, for the color-changing class, the mAP@0.5 value increased from 86.8% to 92.3%, for the early red-ripe class, the mAP@0.5 value increased from 89.7% to 90.7%, for the middle red-ripe class, the mAP@0.5 value increased from 88.2% to 91.5%, and for the late red-ripe class, the mAP@0.5 value increased from 90.2% to 93.6%.

[0156] In summary, the method for recognizing the maturity of cherry tomatoes based on improved YOLOv8 provided by the present invention performs data enhancement on the data set through the CutMix data enhancement technology, enriches the data set, and improves the generalization ability of the detection model; by embedding the attention mechanism module CA into all C2f modules in the backbone network Backbone, the channel and spatial information are taken into account for feature enhancement, which effectively improves the model's attention to the ripening characteristics of cherry tomatoes, effectively reduces the cases of missed detection and false detection in cherry tomato fruit image detection, and effectively solves the problem in the prior art that the recognition accuracy rate will be reduced when encountering situations such as illumination changes, fruit adhesion and overlap, branch and leaf occlusion, and backgrounds with high similarity; by using the variable convolution module DC Nv3 builds all C2f modules in the neck network Neck, and has better feature extraction capabilities for irregular shapes in feature maps, so that the backbone network can better adapt to irregular spatial structures and focus more accurately on important targets, thereby improving the model's ability to detect occluded and overlapping targets, and further improving detection accuracy while compressing the model parameters; by using the NWD loss function instead of the original CIoU loss function, cherry tomatoes are dense small targets in the image. The NWD loss function has scale invariance, smoothness of position deviation, and the ability to measure the similarity between non-overlapping or mutually contained bounding boxes in detecting small targets, which effectively improves the recognition accuracy of the maturity of cherry tomatoes; refer to the national standard GH / T 1193-2021 divides the tomato maturity level into 6 levels according to the color changes of the peel and flesh, namely, unripe stage, green ripe stage, color change stage, early red ripe stage, middle red ripe stage and late red ripe stage. Therefore, the present invention divides cherry tomatoes into 6 categories, namely, unripe stage, green ripe stage, color change stage, early red ripe stage, middle red ripe stage and late red ripe stage, which solves the problem of unclear distinction of fruit maturity in the prior art.

[0157] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for identifying the maturity of cherry tomatoes based on improved YOLOv8. It is characterized in that The following steps are involved: Step 1: Use an image acquisition device to collect a cherry tomato original image dataset; Step 2: Anchor frames are annotated on the collected original data set based on the fruit maturity classification, and the data set with the annotated anchor frames is data enhanced. The data set after data enhancement is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1; Step 3: Improve the traditional YOLOv8 network model to obtain the improved YOLOv8-CCZ model, configure the training environment and set the environment parameters, input the data set after data enhancement in step 2 into the YOLOv8-CCZ model for training, and save the best weight file best.pt in the training result; Step 4: Use the best.pt weight file in step 3 to perform recognition test on the pre-divided test set data; Step 5: Use the same data set and training environment parameters to conduct ablation experiments and compare the YOLOv8-CCZ network model with the traditional YOLOv8, YOLOv7, YOLOv5, YOLOv4, YOLOv3, and SSD target recognition network models.

2. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 1, the cherry tomato original image dataset is collected under natural light during the day. During the collection, a high-definition SLR camera is used to shoot the cherry tomatoes at a distance of 30 to 90 cm. The shooting angles are left, right, front, back, overhead, and upward, a total of 6 directions. The types of cherry tomato image datasets collected cover various cherry tomato images such as front light, back light, densely overlapping occlusion of fruits, fruit adhesion, and occlusion of branches and leaves.

3. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 2, the collected original image dataset is classified and annotated using the labeling tool LabelImg. The cherry tomatoes in the dataset can be divided into six categories based on the maturity of the fruit, namely, unripe, green-ripe, color-changing, early-red-ripe, mid-red-ripe, and late-red-ripe. When annotating the anchor frame, the annotation is based on the minimum circumscribed rectangle of the outline of each cherry tomato.

4. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 2, the data enhancement method adopts CutMix data enhancement technology, which specifically includes: In the CutMix method, a pair of images in the training set are randomly selected and defined as image A and image B. A cropping frame is randomly generated, the corresponding position of image A is cropped, and then the pixel map of the corresponding position of image B is placed in the cropped area of ​​image A to form a new image. The CutMix data enhancement method has the following principle formula: X ～ =M⊙X A +(1-M)⊙X B (1) AND ～ =LY A +(1-λ)Y B (2) Among them, X A With X B are two different training samples, Y A With Y B are their corresponding label values. ～ With Y ～ are the generated new samples and corresponding labels, M∈{0,1} w*h is a binary mask for subtracting some areas and filling, ⊙ is the pixel-by-pixel multiplication symbol, 1 is a binary mask with all elements equal to 1, and λ belongs to the Beta distribution: λ~Beta(a, b), where a and b are adjustable parameters; The rectangular mask M is sampled, and the bounding box sampling formula of the clipping area is: Among them, the bounding box of the clipping area is B = (Υ x , Υ y , Υ w , Υ h ), the bounding box of the entire sample area is A = (X, Y, W, H), Unif (0, W) represents Υ x Uniform sampling on 0~W, Unif(0,H) represents Υ x Sampling uniformly from 0 to H, Indicates the scale of the clipping area.

5. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 3, the traditional YOLOv8 network model is improved, and the improvement steps specifically include: S31. Construct all C2f modules in the backbone network Backbone through the attention mechanism module CA, named C2f-CA; S32. Construct all C2f modules in the neck network Neck through the variable convolution module DCNv3, named C2f-DCNv3; S33. Use the NWD loss function instead of the original CIoU loss function.

6. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 5, It is characterized in that In step S31, the C2f-CA is combined with the attention mechanism module CA through the C2f module, specifically including: a CBL module, a Split module, n parallel connected Bottleneck modules, two parallel connected attention mechanism modules CA and a CBL module are sequentially connected, the Split module divides the number of input channels into two halves, and half of the channels are connected to the other half of the channels through the residual Connect after passing through the n parallel connected Bottleneck modules, the CBL module includes a standard convolution Conv module, a BN module and a SiLU module, and the calculation process of the attention mechanism CA module on the input image specifically includes: (1) Perform global average pooling twice on the input feature map in the width direction and the height direction to obtain two feature maps, which capture the global features in the width direction and the height direction respectively. The specific formula is: Among them, z c is the global feature, x c (i, j) is the input of the original feature. (2) After channel-level splicing operations and convolution smoothing processing, a global receptive field and more accurate information representation are generated. The specific formula is: f=δ(F 1 ([z h ,z w ])) (6) Among them, F 1 is the feature map after batch normalization; f is the feature map obtained by Sigmoid activation function; δ is the nonlinear activation function; z h and z w is the aggregate feature map. (3) Through batch normalization and nonlinear mapping, it is transformed into a tensor with the same number of channels as the input data. The specific formula is: g h =δ(F h (f h )) (7) g w =δ(F w (f w )) (8) Among them, f h and f w The feature map f is divided into two independent tensors F along the spatial dimension h and F w To respectively h and f w Transformed into a tensor with the same number of channels as the input; g h and g w are the attention weights of the input feature map in the height and width directions respectively after the above calculation; δ is the Sigmoid function. (4) Through multiplication weighted calculation on the original feature map, the final feature map with attention weights in the width and height directions is obtained. The specific formula is: Among them, y c (i, j) is the final output feature, x c (i, j) is the original feature, and The features of the original number of channels are restored in the height and width dimensions after aggregation.

7. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 5, It is characterized in that In step S32, the C2f-DCNv3 specifically includes: a CBL module, a Split module, n parallel-connected DCNv3-Bottleneck modules and a CBL module are connected in sequence, the Split module divides the number of input channels into two halves, and half of the channels are connected to the other half of the channels through the residual Connect after passing through the n parallel-connected DCNv3-Bottleneck modules, the DCNv3-Bottleneck module includes three deformable convolution modules DCNv3 connected in sequence, and the deformable convolution module DCNv3 calculates the input image as follows: Where G represents the number of convolution groups; w g represents the shared projection weight within each group; m gk represents the normalized modulation factor of the kth sampling point in the gth group; x g Represents the feature map of the slice input; Δp gk represents the offset corresponding to the grid sampling position (g, k) in the g-th group, p 0 Represents each position on the input feature map, p k Represents each position of the convolution kernel.

8. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 5, It is characterized in that In step S33, the NWD loss function calculation process specifically includes: (1) The second-order Wasserstein distance formula is: Among them, N a , N b Respectively represent the bounding box A=(cx a ,cy a , w a ,h a ) and bounding box B = (cx b ,cy b , w b ,h b ), ||·|| F represents the Frobenius norm, [] T Represents vector transpose; (2) The normalized Wasserstein distance formula is: Where C is a constant related to the data set. (3) The NWD loss function calculation formula is: 。 9. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 3, the YOLOv8-CCZ network model includes an input terminal Input, a backbone network backbone, a neck network Neck and a prediction terminal Head, and the backbone network backbone includes in sequence: a first layer of standard convolution Conv module, a second layer of standard convolution Conv module, a third layer of C2f-CA module, a fourth layer of standard convolution Conv module, a fifth layer of C2f-CA module, a sixth layer of standard convolution Conv module, a seventh layer of C2f-CA module, an eighth layer of standard convolution Conv module, a ninth layer of standard convolution C2f-CA module, and a tenth layer of fast spatial pyramid pooling layer SPP. F; the neck network Neck includes in sequence: an eleventh upsampling Upsample module, a twelfth residual connection Concat layer, a fourteenth C2f-DCNv3 module, a fifteenth upsampling Upsample module, a sixteenth C2f-DCNv3 module, a seventeenth standard convolution Conv module, an eighteenth residual connection Concat layer, a nineteenth C2f-DCNv3 module, a twentieth standard convolution Conv module, a twenty-first residual connection Concat layer, and a twenty-second C2f-DCNv3 module; the input end Input includes a twenty-third Detect layer.

10. The cherry tomato maturity recognition method based on improved YOLOv8 according to claim 1, It is characterized in that In step 3, the training environment configuration includes: the training platform operating system is Windows 11, the CPU is Inter(R) Core(TM) i7-13700KF@3.4GHz, and the memory is 64GiB; the graphics card is Nvidia GeForce RTX 3090, 24GiB, the programming language is python 3.8, the deep learning framework is Pytorch 1.7.0, and the CUDA version is 10.1; the training parameter settings include: the image input size is 640×640, the batch processing size is set to 100, the initial learning rate is set to 0.003, the optimizer is SGD, the training rounds are 100, and the other parameter values ​​​​all use the official default parameter values ​​of YOLOv8.

Images