Citrus fruit recognition method based on improved YOLOv8 and knowledge distillation

Abstract

The application discloses a citrus fruit recognition method based on an improved YOLOv8 and knowledge distillation, and specifically comprises the following steps: collecting initial citrus images, and obtaining a citrus fruit recognition dataset; training a teacher network model by using the citrus fruit recognition dataset, and obtaining a teacher network model weight, wherein the teacher network model is constructed by using an improved YOLOv8x model; inputting the citrus fruit recognition dataset into a student network model and the teacher network model after fixed weights are inputted, obtaining a loss between student network output and teacher network output, iteratively updating the weight of the student network model based on the loss between the student network output and the teacher network output, and obtaining a student network model weight, wherein the student network model is constructed by using an improved YOLOv8n model; and recognizing a target citrus image based on the student network model after the fixed weights are inputted, and obtaining a citrus fruit recognition result.

Description

Technical Field

[0001] This invention belongs to the field of agriculture, and in particular relates to a method for identifying citrus fruits based on improved YOLOv8 and knowledge distillation. Background Technology

[0002] Traditional fruit recognition algorithms primarily utilize the characteristics of different fruits, designing different feature operators to distinguish fruits from the background and thus obtain the coordinates of the corresponding fruit regions. For example, Qin Lei et al. compared the similarity between the color of each pixel in the image and the main color of strawberries, and used a threshold classification method to obtain the strawberry fruit regions. Feng Junhui et al. improved the Hough circle transform algorithm, and based on the algorithm results, performed cluster analysis on regions with similar centers and radii to achieve the recognition of ripe tomatoes, achieving an average recognition rate of 86.3% in complex backgrounds. Cui Yongjie et al. used Otsu for threshold segmentation, extracted boundary information through the Canny operator, and finally performed Hough transform on the boundary information to achieve the recognition of kiwifruit, with a recognition accuracy of 89.1%. Ferhat Kurtulmus et al. utilized the differences in shape, color, and texture features between fruits and the background, and used a sliding window method to detect and recognize green citrus fruits, achieving a recognition rate of 75.3%.

[0003] However, these traditional image processing algorithms all require manually designed features. Therefore, different feature descriptors need to be designed for different field environments, making it difficult for traditional image processing algorithms to meet the needs of actual production. With the rapid development of artificial intelligence, deep learning has become an important technology for target recognition. Compared with traditional target detection algorithms, deep learning has significant advantages in image classification, target detection, and recognition.

[0004] Furthermore, current citrus fruit recognition algorithms have a large number of parameters, requiring significant computing resources and making them difficult to deploy on mobile devices. Lightweight citrus fruit recognition algorithms have a small number of parameters and are easy to deploy, but their lower recognition accuracy makes them difficult to apply in real-world environments. Therefore, there is an urgent need to design a citrus fruit recognition method with a small number of parameters and high recognition accuracy to address the shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to propose a citrus fruit recognition method based on improved YOLOv8 and knowledge distillation. By using knowledge distillation, this method solves the problems of large number of parameters, slow calculation speed, and difficulty in deployment on mobile devices in current citrus fruit recognition algorithms.

[0006] To achieve the above objectives, this invention provides a citrus fruit identification method based on improved YOLOv8 and knowledge distillation, specifically including the following steps:

[0007] Collect initial citrus images to obtain a citrus fruit recognition dataset;

[0008] The teacher network model was trained using the citrus fruit recognition dataset to obtain the weights of the teacher network model, wherein the teacher network model was constructed using an improved YOLOv8x model;

[0009] The citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights, respectively. The loss between the student network output and the teacher network output is obtained. The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the weights of the student network model. The student network model is constructed using an improved YOLOv8n model.

[0010] The student network model with fixed weights is used to identify the target citrus image and obtain the citrus fruit identification result.

[0011] Optionally, acquiring the initial citrus image and obtaining the citrus fruit recognition dataset includes:

[0012] Acquire the initial citrus image;

[0013] The initial citrus image is labeled to obtain the location information of the initial citrus fruit;

[0014] The initial location information of the citrus fruits is converted into a new format, normalized, and augmented to obtain the citrus fruit recognition dataset.

[0015] Optionally, labeling the initial citrus image to obtain the location information of the initial citrus fruit includes:

[0016] The initial citrus image is labeled using image data annotation software to obtain the location information of the initial citrus fruit;

[0017] The image data annotation software includes Labelme and LabelImg.

[0018] Optionally, the initial location information of the citrus fruits is converted into a new format, normalized, and augmented to obtain the citrus fruit recognition dataset, including:

[0019] The initial location information of the citrus fruits was converted from JSON format to TXT format, and then normalized and augmented to obtain the citrus fruit recognition dataset.

[0020] Optionally, the data augmentation includes random cropping, random rotation, radial transformation, chromaticity variation, and Mosaic enhancement.

[0021] Optionally, the teacher network model is trained using the citrus fruit recognition dataset, and the weights of the teacher network model are obtained by:

[0022] Obtain the training results of the teacher network model on the COCO public dataset and use them as pre-training weights;

[0023] The pre-trained weights are loaded into the teacher network model, and the teacher network model is trained using the citrus fruit recognition dataset for several epochs to obtain the weights of the teacher network model.

[0024] Optionally, the citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights, respectively. The loss between the student network output and the teacher network output is obtained. The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the weights of the student network model, including:

[0025] Fix the weights of the teacher network model to obtain the teacher network model after fixing the weights;

[0026] The citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights to obtain the student network output and the teacher network output.

[0027] Calculate the loss between the student network output and the teacher network output;

[0028] The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the weights of the student network model.

[0029] Optionally, the target citrus image is identified based on the student network model with fixed weights, and the citrus fruit identification result is obtained by:

[0030] The weights of the student network model are loaded into the student network model and the weights are fixed to obtain the student network model with fixed weights.

[0031] The target citrus image is input into the student network model with fixed weights to obtain the location information of the target citrus fruit, and image post-processing is used to generate the corresponding matrix box;

[0032] Using the corresponding matrix frame and the initial citrus image, the citrus fruit recognition result is obtained.

[0033] The present invention has the following beneficial effects:

[0034] This invention is based on an improved YOLOv8 network model and combines it with knowledge distillation. It selects the YOLOv8x model (which has the largest number of parameters, the highest recognition accuracy, and the slowest computation speed) as the teacher network in knowledge distillation, and the YOLOv8n model (which has the smallest number of parameters, the lowest recognition accuracy, and the fastest computation speed) as the student network. After knowledge distillation, the recognition accuracy of the YOLOv8n model in citrus fruit identification scenarios is effectively improved, enabling rapid detection of citrus fruits. This method not only helps in rapid yield estimation but also provides a technological foundation for future automated citrus fruit harvesting. Attached Figure Description

[0035] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a flowchart of a citrus fruit identification method based on an improved YOLOv8 and knowledge distillation, according to an embodiment of the present invention.

[0037] Figure 2 This is an image showing the improved YOLOv8n (student network) recognition results proposed in an embodiment of the present invention;

[0038] Figure 3 This is a structural diagram of the improved YOLOv8 model proposed in an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] like Figure 1 As shown, this embodiment provides a citrus fruit recognition method based on improved YOLOv8 and knowledge distillation. Knowledge distillation is performed offline, meaning a network model with a large number of parameters and high accuracy is selected as the teacher network (improved YOLOv8x), and a network model with a small number of parameters and lower accuracy is selected as the student network (improved YOLOv8n). The overall process is divided into four steps: dataset creation, teacher network training, knowledge distillation, and fruit recognition. Specifically, the following steps are included:

[0042] (I) Dataset Creation:

[0043] The process of creating the dataset is as follows: Figure 1 As shown in the outline of the dataset creation, the first step is to collect citrus images. Citrus fruit images should be collected under different times, angles, and lighting conditions, with a minimum of 500 images to ensure dataset quality. Next, data annotation is performed using image data annotation software such as Labelme and LabelImg to annotate the collected citrus images, obtaining the position coordinates (x, y, w, h) of the citrus fruits. Normalization is then applied, dividing (x, w) by the image width and (y, h) by the image height. Finally, data augmentation is performed using random cropping, random rotation, radiometric transformation, chromaticity changes, and Mosaic enhancement methods to increase the number of training samples, resulting in the citrus fruit recognition dataset. The specific process is as follows:

[0044] First, image data was collected by taking 500 images of citrus fruits from the citrus orchard;

[0045] Then, the image is annotated using the Labelme image annotation tool;

[0046] Next, the labeled data is converted from JSON format to TXT format and then normalized.

[0047] Using random cropping, random rotation, radial transformation, chromaticity change, and Mosaic enhancement algorithms, the citrus fruit identification dataset was augmented to obtain the citrus fruit identification dataset.

[0048] (II) Online Training for Teachers:

[0049] The online training process for teachers is as follows Figure 1 As shown in the teacher network training framework, firstly, a transfer learning method is used to load the training results of the teacher network model on a public dataset as pre-training weights into the teacher network model to improve the convergence speed of the network; then, the citrus fruit dataset is used to train the teacher model, and stochastic gradient descent is used to train for several epochs to obtain the weights of the citrus fruit recognition teacher network model. The specific process is as follows:

[0050] First, the training results of the improved YOLOv8x on the COCO public dataset are used as pre-training weights and loaded into the improved YOLOv8x network.

[0051] The citrus fruit recognition dataset was used to improve the training of YOLOv8x (teacher network) for 100 epochs to obtain the network weights of the teacher network.

[0052] (III) Knowledge Distillation Process:

[0053] Knowledge distillation process such as Figure 1 As shown in the knowledge distillation framework, firstly, the weights of the teacher network model are fixed. Then, the citrus fruit dataset is input into both the teacher and student networks, respectively, to obtain the outputs of the teacher and student networks. Next, the loss between the student and teacher network outputs is calculated, and the loss between the student network output and the actual fruit location is added to obtain the total loss of knowledge distillation. Finally, based on the total loss, the weights of the student network are updated using stochastic gradient descent. After training for several epochs, the weights of the citrus fruit recognition student network model are obtained. The specific process is as follows:

[0054] First, the citrus fruit recognition dataset was input into the improved YOLOv8x (teacher network) and the improved YOLOv8n (student network) respectively, and the outputs of the two networks were obtained respectively.

[0055] Then, the loss between the student network output and the teacher network output is calculated, and the loss between the student network output and the actual value is added to get the total loss of the knowledge distillation process.

[0056] Finally, the total loss is minimized using stochastic gradient descent for 100 epochs to obtain the network weights of the improved YOLOv8n (student network).

[0057] (iv) Fruit identification process:

[0058] Fruit identification process as follows Figure 1 As shown in the fruit recognition outline, firstly, the weights of the student network obtained from knowledge distillation are fixed, then the citrus image is input into the student network, and finally the citrus fruit recognition result is obtained. The specific process is as follows:

[0059] First, the network weights obtained from knowledge distillation are loaded into the improved YOLOv8n (student network), and the weights are fixed.

[0060] Then, the image of the target citrus fruit is input into the student network to obtain the location information of the citrus fruit;

[0061] Finally, image post-processing is performed. Based on the fruit's location information, corresponding bounding boxes are generated, and the initial citrus images are merged to obtain the final recognition result, such as... Figure 2 As shown.

[0062] To address the problems of large model parameters and low recognition accuracy in current citrus fruit recognition methods, this embodiment proposes a citrus fruit recognition method based on improved YOLOv8 and knowledge distillation. The overall structure of the improved YOLOv8 network model is as follows: Figure 3 As shown in the figure, the traditional convolution Conv in the YOLOv8 model is replaced with the full-dimensional dynamic convolution ODConv. Figure 3 (As shown by the dashed line), the full-dimensional dynamic convolution ODConv generates four attention vectors of different dimensions for different input feature maps. After multiplication and weighting, a dynamic convolution kernel that changes with the input feature map is obtained, making this structure more robust than traditional convolution and therefore more suitable for complex and variable field environments. The YOLOv8 series of models includes five types: YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x. Figure 3 The model shown is obtained by scaling it down with different scaling factors, namely [depth scaling factor, width scaling factor, maximum number of channels]. The scaling factors for YOLOv8n are [0.33, 0.25, 1024], for YOLOv8s are [0.33, 0.50, 1024], for YOLOv8m are [0.67, 0.75, 768], for YOLOv8l are [1.00, 1.00, 512], and for YOLOv8x are [1.00, 1.25, 512].

[0063] like Figure 3 As shown, the input image of the original YOLOv8 model first undergoes feature extraction and encoding in the Backbone part, and then the features are decoded in the Head part, resulting in three output feature maps of different sizes, representing the location information of the citrus fruit in different image regions. After the improvement, as shown... Figure 3 As shown in the dashed box, the traditional convolution Conv is replaced with the full-dimensional dynamic convolution ODConv, which enables the backbone and head parts of the YOLOv8 network to dynamically adjust the convolution kernel according to different input images, thereby improving its adaptability in complex field environments.

[0064] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A citrus fruit identification method based on improved YOLOv8 and knowledge distillation, characterized in that, Specifically, the following steps are included: Collect initial citrus images to obtain a citrus fruit recognition dataset; The teacher network model was trained using the citrus fruit recognition dataset to obtain the weights of the teacher network model, wherein the teacher network model was constructed using an improved YOLOv8x model; The citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights, respectively. The loss between the student network output and the teacher network output is obtained. The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the weights of the student network model. The student network model is constructed using an improved YOLOv8n model. The improved YOLOv8 model replaces the traditional convolution Conv in the YOLOv8 model with a full-dimensional dynamic convolution ODConv. The full-dimensional dynamic convolution ODConv obtains attention vectors of different dimensions for different input feature maps. After multiplication and weighting, a dynamic convolution kernel that can change with the input feature map is obtained. The improved YOLOv8 model is scaled down by different scaling factors to obtain the improved YOLOv8x model and the improved YOLOv8n model, respectively. The student network model with fixed weights is used to identify the target citrus image and obtain the citrus fruit identification result.

2. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 1, characterized in that, The initial citrus images were collected, and the citrus fruit recognition dataset was obtained, including: Acquire the initial citrus image; The initial citrus image is labeled to obtain the location information of the initial citrus fruit; The initial location information of the citrus fruits is converted into a new format, normalized, and augmented to obtain the citrus fruit recognition dataset.

3. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 2, characterized in that, Labeling the initial citrus image to obtain the location information of the initial citrus fruit includes: The initial citrus image is labeled using image data annotation software to obtain the location information of the initial citrus fruit; The image data annotation software includes Labelme and LabelImg.

4. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 2, characterized in that, The initial location information of the citrus fruits is converted into a new format, normalized, and augmented to obtain the citrus fruit recognition dataset, which includes: The initial location information of the citrus fruits was converted from JSON format to TXT format, and then normalized and augmented to obtain the citrus fruit recognition dataset.

5. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 4, characterized in that, The data augmentation includes random cropping, random rotation, radial transformation, chromaticity variation, and Mosaic enhancement.

6. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 1, characterized in that, The teacher network model is trained using the citrus fruit recognition dataset, and the weights of the teacher network model are obtained as follows: Obtain the training results of the teacher network model on the COCO public dataset and use them as pre-training weights; The pre-trained weights are loaded into the teacher network model, and the teacher network model is trained using the citrus fruit recognition dataset for several epochs to obtain the weights of the teacher network model.

7. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 1, characterized in that, The citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights, respectively. The loss between the student network output and the teacher network output is obtained. The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the student network model weights, including: Fix the weights of the teacher network model to obtain the teacher network model after fixing the weights; The citrus fruit recognition dataset is input into the student network model and the teacher network model with fixed weights to obtain the student network output and the teacher network output. Calculate the loss between the student network output and the teacher network output; The weights of the student network model are iteratively updated based on the loss between the student network output and the teacher network output to obtain the weights of the student network model.

8. The citrus fruit identification method based on improved YOLOv8 and knowledge distillation as described in claim 1, characterized in that, The target citrus image is identified based on the student network model with fixed weights, and the citrus fruit identification result is obtained as follows: The weights of the student network model are loaded into the student network model and the weights are fixed to obtain the student network model with fixed weights. The target citrus image is input into the student network model with fixed weights to obtain the location information of the target citrus fruit, and image post-processing is used to generate the corresponding matrix box; Using the corresponding matrix frame and the initial citrus image, the citrus fruit recognition result is obtained.

Images