Fruit variety intelligent identification system and method based on improved YOLO model

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to an intelligent fruit variety identification system and method based on an improved YOLO model, and the system comprises a data marking module which is used for marking the image of each fruit variety in an existing fruit data set, and obtaining a marked data set; the initial model construction module is used for introducing a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; the identification model construction module is used for training the initial model based on the labeled data set to obtain a fruit variety identification model; and the fruit variety identification module is used for identifying the fruit variety to be identified based on the fruit variety identification model to obtain a fruit variety identification result. By improving the YOLO model, the fruit variety can be accurately identified.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a fruit variety intelligent recognition system and method based on an improved YOLO model. Background Art

[0002] With the rapid development of artificial intelligence and computer vision technology, the intelligent fruit variety recognition system is gradually being applied to agriculture, food testing and other fields. Traditional fruit variety recognition methods usually rely on manual observation and classification, which is time-consuming and labor-intensive, and prone to errors. With the continuous advancement of deep learning technology, the fruit variety intelligent recognition system based on the YOLO algorithm is becoming an efficient and accurate solution.

[0003] Traditional fruit variety recognition systems usually use CNN and R-CNN based methods for image classification and recognition. However, these methods have some shortcomings in practical applications, which limits their performance in efficient and real-time recognition.

[0004] R-CNN is a deep learning method based on region extraction. It generates candidate regions for image segmentation and classifies each candidate region using a convolutional neural network to achieve target recognition. Although R-CNN can accurately identify fruit varieties, it has a huge amount of computation and a slow processing speed because each candidate region requires separate CNN feature extraction and classification, which makes it perform poorly in real-time recognition tasks.

[0005] CNN is a basic network architecture in the field of deep learning and is widely used in tasks such as image classification and feature extraction. Traditional CNN methods usually require pre-extraction of image features and input of these features into the fully connected layer for classification. Although CNN performs well in recognition accuracy, it lacks the ability to accurately predict the location and number of targets when dealing with complex scenes or multi-target detection. Therefore, CNN usually requires additional algorithms to handle multi-target detection tasks, which increases the amount of computation and complexity.

[0006] Compared with R-CNN and CNN, YOLO has obvious advantages in the intelligent recognition system of fruit varieties. Its high efficiency and real-time characteristics enable it to quickly identify fruit varieties without sacrificing accuracy and adapt to various dynamic environments. Although R-CNN and CNN perform well in static image recognition, they are usually difficult to meet the needs of real-time recognition due to their high computational complexity. Therefore, YOLO has become a more ideal solution in the field of fruit variety recognition.

[0007] The yolov11 used in this system is developed based on yolov8. On the basis of yolov8, the C3k2 module is proposed, which can greatly improve the accuracy. In addition, two DWConvs are inserted in the classification detection head, which can greatly reduce the number of parameters and calculations. Overall, yolov11 is better than previous versions. Summary of the invention

[0008] In order to solve the problems existing in the prior art, the present invention provides a fruit variety intelligent identification system and method based on an improved YOLO model to achieve accurate identification of fruit varieties.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A fruit variety intelligent identification system based on an improved YOLO model, the system comprising:

[0011] A data annotation module is used to annotate the images of various varieties of fruits in the existing fruit dataset to obtain annotated datasets;

[0012] An initial model building module, used to introduce a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; wherein the wavelet pooling layer is used to replace the preset convolution layer in the YOLOv11 model, and the CoTAttention attention mechanism layer is introduced after the 11th layer of the YOLOv11 model;

[0013] A recognition model building module, used for training the initial model based on the annotated data set to obtain a fruit variety recognition model;

[0014] The fruit variety identification module is used to identify the fruit variety to be identified based on the fruit variety identification model to obtain a fruit variety identification result.

[0015] Preferably, the data annotation module includes:

[0016] A data labeling unit is used to mark the fruit images of various varieties in the existing fruit data set with boxes to obtain labeled fruit images;

[0017] The data processing unit is used to flip and crop the marked fruit image, complete the preprocessing of the marked fruit image, and obtain the annotated data set.

[0018] Preferably, the initial model building module includes a wavelet pooling layer introduction unit for replacing the 5th, 7th and 9th Conv convolutional layers in the Backbone network layer of the YOLOv11 model with wavelet pooling layers.

[0019] Preferably, the recognition model building module includes:

[0020] A wavelet decomposition unit is used to perform wavelet pooling on the fruit images in the labeled data set based on the wavelet pooling layer in the initial model to obtain a feature map of the fruit images; wherein the feature map includes a local feature map and a global feature map;

[0021] A context extraction unit, configured to mine context information on the feature map based on the CoTAttention attention mechanism layer in the initial model;

[0022] The variety identification unit is used to identify the fruit variety based on the mined context information to complete the construction of the fruit variety identification model.

[0023] Preferably, the process of performing wavelet pooling on the fruit images in the labeled data set includes:

[0024] Decomposing the fruit images in the labeled data set into sub-bands; wherein the sub-bands include a low-frequency sub-band, a low-high-frequency sub-band, a high-low-frequency sub-band, and a high-high sub-band;

[0025] Subbands that do not meet preset requirements are discarded to obtain a feature map of the fruit image.

[0026] The present invention also provides a method for intelligently identifying fruit varieties based on an improved YOLO model. The system is applied, and the method comprises:

[0027] Label the images of various varieties of fruits in the existing fruit dataset to obtain a labeled dataset;

[0028] Introducing a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; wherein the wavelet pooling layer is used to replace the preset convolution layer in the YOLOv11 model, and the CoTAttention attention mechanism layer is introduced after the 11th layer of the YOLOv11 model;

[0029] Training the initial model based on the labeled data set to obtain a fruit variety recognition model;

[0030] The fruit variety to be identified is identified based on the fruit variety identification model to obtain a fruit variety identification result.

[0031] Preferably, the Conv convolution layers of the 4th, 6th and 8th layers in the Backbone network layer of the YOLOv11 model are replaced with wavelet pooling layers.

[0032] Preferably, the method for constructing the fruit variety identification model comprises:

[0033] Based on the wavelet pooling layer in the initial model, wavelet pooling is performed on the fruit images in the labeled data set to obtain a feature map of the fruit images; wherein the feature map includes a local feature map and a global feature map;

[0034] Performing context information mining on the feature map based on the CoTAttention attention mechanism layer in the initial model;

[0035] The fruit variety is identified based on the mined context information to complete the construction of the fruit variety identification model.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention is improved on the basis of yolov11, the latest version of yolo end-to-end detection, and can have higher recognition accuracy than other target detections, and still has the ability of rapid detection of the yolo series. In addition, a wavelet pooling module is added, which uses wavelet transform to better denoise the feature map, remove high-frequency information, retain low-frequency information, save the main information of the feature map, and extract recognizable features. The robustness of the model can be effectively improved. In addition, the CoTAttention attention mechanism is added to capture contextual information, enhance the visual expression ability, and pay more attention to the capture of important information. Therefore, the added module can further improve the extraction and capture of feature information by yolov11 and improve the robustness of the entire model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 Schematic diagram of the structure of a fruit variety intelligent identification system based on an improved YOLO model in an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the improved network structure of YOLOv11 in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of the structure of the original module of YOLOv11 in an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the CoTAttention attention mechanism in an embodiment of the present invention;

[0042] Figure 5Schematic diagram of the Wavelet Pooling algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Embodiment 1

[0046] like Figure 1 As shown, a fruit variety intelligent identification system based on an improved YOLO model includes: a data labeling module, an initial model building module, a recognition model building module and a fruit variety identification module.

[0047] The data annotation module is used to annotate the images of various varieties of fruits in the existing fruit dataset to obtain annotated datasets. In this embodiment, 14 categories of fruit images, including apples, avocados, bananas, cherries, grapes, kiwis, mangoes, oranges, peaches, pineapples, pomegranates, raspberries, strawberries, and watermelons, which are open source on the kaggle website, are used as datasets; the Labelimg image annotation tool is used to annotate various categories of fruit targets, and the annotated datasets are divided into training sets, test sets, and validation sets, which are distributed in a ratio of 7:2:1.

[0048] A further implementation method is that the data annotation module includes:

[0049] The data labeling unit is used to mark the fruit images of various varieties in the existing fruit data set with boxes to obtain labeled fruit images. In this embodiment, when labelimg is used to mark the images of a batch-size training set, a box is used for marking, the coordinates of the center point of the fruit are (x, y), and the width of the marked box is w and the height is h.

[0050] The data processing unit is used to flip, crop and rotate the marked fruit image, complete the preprocessing of the marked fruit image, and obtain the labeled data set.

[0051] The initial model building module is used to introduce a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; wherein the preset convolution layer in the YOLOv11 model is replaced by the wavelet pooling layer (WaveletPool layer), and the CoTAttention attention mechanism layer is introduced after the YOLOv11 model 11 layer; a further implementation method is, as Figure 2 As shown in the figure, the initial model building module includes a wavelet pooling layer introduction unit, which is used to replace the 4th, 6th and 8th Conv convolution layers in the Backbone network layer of the YOLOv11 model with wavelet pooling layers, which can greatly improve the aliasing detection of small targets. Especially for markets with a large number of fruits, the effect of detection ability on fruit trees is more obvious. Introducing the CoTAttention attention mechanism layer after the 10th layer of the YOLOv11 model can enhance the reasoning ability of the model, make up for the ability to process information, and ignore unimportant information.

[0052] The recognition model building module, as shown in a further embodiment of the figure, is that the recognition model building module includes:

[0053] The wavelet decomposition unit is used to perform wavelet pooling on the fruit images in the labeled data set based on the wavelet pooling layer in the initial model to obtain a feature map of the fruit image; wherein the feature map includes a local feature map and a global feature map; a further implementation method is that the process of performing wavelet pooling on the fruit images in the labeled data set includes:

[0054] The fruit images in the labeled dataset are decomposed into sub-bands; wherein the sub-bands include low-frequency sub-bands, low-high-frequency sub-bands, high-low-frequency sub-bands and high-high sub-bands.

[0055] Discard non-low-frequency sub-bands and obtain the feature map of the fruit image.

[0056] In this embodiment, the wavelet transform used by Wavelet Pooling can minimize the artifacts generated by the neighborhood. During the recognition process, the image can easily use noise as part of the recognition, resulting in detection errors. Eliminating artifacts can restore the image to a state closest to the original reality, which can enhance the recognition ability of the image. In addition, for the commonly used maximum pooling and average pooling, they only downsample in space. Maximum pooling selects the maximum value in each pooling area, and average pooling selects the average value in each pooling area. These two types of pooling usually only retain the most significant features in a specific area. Wavelet pooling decomposes the image into sub-bands of different frequencies, which can capture the regional and global features of the image. This makes wavelet pooling more effective in processing multi-dimensional and multi-frequency information, and can extract more detailed local and global information.

[0057] Wavelet Pooling implements the wavelet transform of images. Figure 5 As shown in the figure, LL is the low-frequency subband, LH is the low-high frequency subband, HL is the high-low frequency subband, and HH is the high-high subband. After the first wavelet transform, the non-low-frequency subband is discarded, and then the wavelet transform is performed again on the low-frequency subband. Finally, a new feature map is generated to complete the pooling process.

[0058] The context extraction unit is used to mine context information on the feature map based on the CoTAttention mechanism layer in the initial model; specifically, Figure 4 As shown in the figure, after extracting important information based on the feature information of the previous wavelet pooling process, an attention enhancement mechanism is implemented through a series of convolutions, fusions, segmentations, etc., which improves the model's attention to important channels and improves the model's performance in processing complex reasoning tasks. The traditional self-attention mechanism triggers feature interactions at different spatial locations well based on the input itself. However, in the traditional self-attention mechanism, all paired query key relationships are isolated and learned independently on the query key pairs without exploring the rich context between the two. This severely limits the self-learning ability on 2D feature maps. To alleviate this problem, CoTAttention integrates context information mining and self-attention learning into a unified architecture, which can fully mine the context between adjacent keys.

[0059] The variety recognition unit is used to identify the fruit varieties based on the mined context information and complete the construction of the fruit variety recognition model. The test set is input into the fruit variety recognition model to complete the recognition of the fruit species and obtain a series of evaluation indicators such as the corresponding precision rate and recall rate.

[0060] In summary, the present invention inputs the image to be detected into the network structure for forward propagation, first enters the BackBone layer, which is the key layer for feature extraction, and extracts the important information of the image through a series of CBS, C3K2, WaveletPooling, SPPF, CoTAttention, C2PSA layers, and undergoes a series of convolution and activation functions. The specific convolution process of each layer is as follows: Figure 3 As shown. Then it enters the Neck layer, which is an important layer for feature fusion in the image recognition process. After downsampling (Upsample layer) and a series of convolution fusion processes, the feature information of BackBone is fused with the feature information of the Neck layer. This part of the process can re-acquire the previously ignored information. Finally, it enters the Head layer, predicts the feature information obtained from the Neck layer, and obtains the score of whether the corresponding input image belongs to this class and whether there is a corresponding object. Finally, the classification and location of the fruit type in the image are completed.

[0061] Based on the existing yolov11, the present invention adds Wavelet Pooling, and the map50 (the average accuracy of the model when the intersection-over-union ratio threshold of the real box and the predicted box is 0.5) value is increased from 84.2% to 85.4%, and the map50-95 (the average accuracy of the model in the range of the intersection-over-union ratio threshold of the real box and the predicted box from 0.5 to 0.95) value is increased from 62.2% to 63.5%. After that, the CoTAttention attention mechanism is added, and the map50 value is increased to 86.2%, and the map50-95 is increased to 64.1%. It can better identify the variety of fruits based on the picture.

[0062] The fruit variety recognition module is used to identify the fruit variety to be identified based on the fruit variety recognition model to obtain the fruit variety recognition result.

[0063] Embodiment 2

[0064] The present invention also provides a method for intelligently identifying fruit varieties based on an improved YOLO model, using the system described in the first embodiment, the method comprising:

[0065] Label the images of various varieties of fruits in the existing fruit dataset to obtain a labeled dataset;

[0066] Introduce wavelet pooling layer and CoTAttention mechanism layer into the YOLOv11 model to obtain the initial model; wherein, the preset convolution layer in the YOLOv11 model is replaced by the wavelet pooling layer, and the CoTAttention mechanism layer is introduced in the 10th layer of the YOLOv11 model;

[0067] Train the initial model based on the labeled data set to obtain a fruit variety recognition model;

[0068] The fruit variety to be identified is identified based on the fruit variety recognition model to obtain the fruit variety recognition result.

[0069] A further implementation method is to replace the 4th, 6th and 8th Conv convolution layers in the Backbone network layer of the YOLOv11 model with wavelet pooling layers.

[0070] A further embodiment is that the method for constructing a fruit variety identification model comprises:

[0071] Based on the wavelet pooling layer in the initial model, wavelet pooling is performed on the fruit images in the labeled data set to obtain feature maps of the fruit images; wherein the feature maps include local feature maps and global feature maps;

[0072] Mining contextual information on feature maps based on the CoTAttention attention mechanism layer in the initial model;

[0073] The fruit varieties are identified based on the mined context information, completing the construction of the fruit variety identification model.

[0074] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A fruit variety intelligent identification system based on an improved YOLO model, characterized in that: The system comprises: A data annotation module is used to annotate images of fruits of various varieties in the existing fruit dataset to obtain annotated datasets; An initial model building module, used to introduce a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; wherein the wavelet pooling layer is used to replace the preset convolution layer in the YOLOv11 model, and the CoTAttention attention mechanism layer is introduced after the 11th layer of the YOLOv11 model; A recognition model building module, used for training the initial model based on the labeled data set to obtain a fruit variety recognition model; The fruit variety identification module is used to identify the fruit variety to be identified based on the fruit variety identification model to obtain a fruit variety identification result.

2. The system according to claim 1, characterized in that The data annotation module includes: A data labeling unit is used to mark the fruit images of various varieties in the existing fruit data set with boxes to obtain labeled fruit images; The data processing unit is used to flip and crop the marked fruit image, complete the preprocessing of the marked fruit image, and obtain the annotated data set.

3. The system according to claim 1, characterized in that The initial model building module includes a wavelet pooling layer introduction unit, which is used to replace the 5th, 7th and 9th Conv convolution layers in the Backbone network layer of the YOLOv11 model with wavelet pooling layers.

4. The system according to claim 3, characterized in that The recognition model building module includes: A wavelet decomposition unit is used to perform wavelet pooling on the fruit images in the labeled data set based on the wavelet pooling layer in the initial model to obtain a feature map of the fruit images; wherein the feature map includes a local feature map and a global feature map; A context extraction unit, configured to mine context information on the feature map based on the CoTAttention attention mechanism layer in the initial model; The variety identification unit is used to identify the fruit variety based on the mined context information to complete the construction of the fruit variety identification model.

5. The system according to claim 4, characterized in that The process of performing wavelet pooling on the fruit images in the labeled dataset includes: Decomposing the fruit images in the labeled data set into sub-bands; wherein the sub-bands include a low-frequency sub-band, a low-high-frequency sub-band, a high-low-frequency sub-band, and a high-high sub-band; Subbands that do not meet preset requirements are discarded to obtain a feature map of the fruit image.

6. A method for intelligent identification of fruit varieties based on an improved YOLO model, using the system described in any one of claims 1 to 5, characterized in that: The method comprises: Label the images of various varieties of fruits in the existing fruit dataset to obtain a labeled dataset; Introducing a wavelet pooling layer and a CoTAttention attention mechanism layer into the YOLOv11 model to obtain an initial model; wherein the wavelet pooling layer is used to replace the preset convolution layer in the YOLOv11 model, and the CoTAttention attention mechanism layer is introduced after the 11th layer of the YOLOv11 model; Training the initial model based on the labeled data set to obtain a fruit variety recognition model; The fruit variety to be identified is identified based on the fruit variety identification model to obtain a fruit variety identification result.

7. The method according to claim 6, characterized in that Replace the 5th, 7th, and 9th Conv convolution layers in the Backbone network layer of the YOLOv11 model with wavelet pooling layers.

8. The method according to claim 6, characterized in that The method for constructing the fruit variety identification model comprises: Based on the wavelet pooling layer in the initial model, wavelet pooling is performed on the fruit images in the labeled data set to obtain a feature map of the fruit images; wherein the feature map includes a local feature map and a global feature map; Performing context information mining on the feature map based on the CoTAttention attention mechanism layer in the initial model; The fruit variety is identified based on the mined context information to complete the construction of the fruit variety identification model.

Images