Lightweight fruit target detection method based on characteristic distillation and improved YOLOv8

Abstract

The invention discloses a lightweight fruit target detection method based on characteristic distillation and improved YOLOv8, and belongs to the field of computer vision and target detection. The method comprises the following steps: 1, obtaining a fruit public data set based on a real scene; 2, a lightweight student model based on YOLOv8-nano is constructed; 3, inputting the data set into a student model for training and verification; 4, constructing a high-precision teacher model based on YOLOv8-x; 5, inputting the data set into a teacher model for training and verification; 6, constructing a distillation model, and minimizing the difference between the student model and the teacher model; and 7, inputting the test set into the distillation model to realize fruit target detection. Compared with the prior art, the method has the advantages that the calculation complexity and the model size are greatly reduced under the condition that relatively high detection precision is kept aiming at the condition that actual detection environment calculation resources are limited, so that the requirements of an actual agricultural detection scene are more easily met.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, specifically to the field of fruit target detection in agricultural scenes, and in particular to a lightweight fruit target detection method based on feature distillation and improved YOLOv8. Background Art

[0002] In recent years, with the development of artificial intelligence and computer vision technology, the precision agriculture industry has made significant progress. The counting, picking and management of fruits in traditional orchards are mainly done manually, which is costly, inefficient and prone to errors. Therefore, the research work on automated object detection is increasing.

[0003] Traditional target detection methods are based on computer vision and digital image processing technology and rely on manual feature design. These methods match objects by using basic features such as intensity, color, shape, and texture. However, the detection results of this method are easily affected by lighting, background, and human subjective factors in terms of accuracy and speed. In recent years, detection algorithms based on deep learning have been widely used in many fields and have shown superior performance. Convolutional Neural Network (CNN) is a deep learning algorithm that extracts features from images through multi-layer convolution, pooling, and fully connected operations, and can effectively complete tasks such as target detection and classification. Compared with traditional manual methods, CNN can learn the representation of complex visual patterns and can perform excellent performance in such detection tasks.

[0004] Nevertheless, deep learning-based detection models usually require a lot of computing resources and are usually deployed on high-end GPUs, which makes them not feasible in many real-world scenarios. Therefore, the study of lightweight target detection models is necessary. Lightweighting aims to achieve a balance between detection accuracy and computational efficiency, and is very suitable for resource-constrained environments (such as mobile devices or edge computing units used in parks). YOLO is a typical CNN-based single-stage target detection algorithm. Compared with traditional algorithms, it has greater advantages in speed and accuracy. In particular, lightweight variants such as YOLOv5-nano and YOLOv8-nano are given, which significantly reduce the demand for computing resources while maintaining relatively high accuracy, which is crucial for practical applications. Summary of the invention

[0005] Technical problem: The purpose of the present invention is to provide a lightweight target detection model for fruit detection tasks in resource-constrained agricultural production environments, so as to achieve accurate detection of fruits while maintaining low computational complexity and high detection speed.

[0006] Technical solution: The present invention provides a lightweight fruit target detection method based on feature distillation and improved YOLOv8, comprising:

[0007] Step 1: Obtain a public fruit dataset based on real scenarios and process the label format;

[0008] Step 2: Build a lightweight student model based on YOLOv8-nano;

[0009] Step 3: Input the data set into the student model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained student model.

[0010] Step 4: Build a high-precision teacher model based on YOLOv8-x;

[0011] Step 5: Input the data set into the teacher model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained teacher model;

[0012] Step 6: Build a distillation model, minimize the difference between the student model and the teacher model through KL divergence calculation, and obtain a distillation model that has learned the feature representation ability of the teacher model;

[0013] Step 7: Input the test set into the distillation model to obtain the detection results and realize fruit target detection.

[0014] Furthermore, the lightweight student model detection model includes a backbone network, a neck network, and a detection head network. The backbone network includes a first Conv module, a second Conv module, a first DFC-LightConv module, a first DSC module, a second DFC-LightConv module, a second DSC module, a third DFC-LightConv module, a third DSC module, a fourth DFC-LightConv module, and an SPPF module arranged in sequence. The neck network includes a first Unsample module, a first Concat layer, a fifth DFC-LightConv module, a second Unsample module, a second Concat layer, a sixth DFC-LightConv module, a third Conv module, a third Concat layer, a seventh DFC-LightConv module, a fourth Conv module, a fourth Concat layer, and an eighth DFC-LightConv module. The detection head network includes a first Detect module, a second Detect module, and a third Detect module.

[0015] Furthermore, the high-precision teacher model detection model includes a backbone network, a weighted fusion neck network, and a detection head network. The backbone network includes a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a third C2f module, a fifth Conv module, a fourth C2f module, and an SPPF module, which are arranged in sequence. The weighted fusion neck network includes a first Unsample module, a first weighted connection module, a fifth C2f module, a second Unsample module, a second weighted connection module, a sixth C2f module, a sixth Conv module, a third weighted connection module, a seventh C2f module, a seventh Conv module, a fourth weighted connection module, and an eighth C2f module. The detection head network includes a first Detect module, a second Detect module, and a third Detect module.

[0016] Furthermore, the DFC-LightConv module in the lightweight student model includes two operation branches, the first branch includes the first DFC attention module, the first deep convolution module, the first point-by-point convolution module, and the GhostConv convolution module, and the second branch includes the second deep convolution module, the second point-by-point convolution module, and the Conv module. The input feature map passes through the two operation branches respectively, and the results are added as the output of the DFC-LightConv module.

[0017] The DFC attention is an attention mechanism built on a fully connected layer, which can be quickly executed on ordinary hardware and effectively capture long-distance dependencies. First, pixel information is aggregated in the horizontal and vertical directions of the feature map:

[0018]

[0019] A=A h +A v

[0020] Among them, F is the learnable weight, x is the original input, ⊙ represents element-level multiplication, w and h are used to indicate the current pixel position, and A is the result of aggregation in two directions. After the generated feature map is normalized, the final output feature is obtained through the SiLU activation function:

[0021] F enhanced =SiLU(BatchNorm(A⊙F))

[0022] By decoupling the fully connected operations, the DFC mechanism enables the model to consider both local and global information at the same time, and by sharing part of the weight information, it eliminates the delay caused by tensor reshaping and transposition operations, thereby better maintaining the lightweight characteristics of the model.

[0023] Furthermore, multiple standard convolution modules in the lightweight student model are replaced by DSC modules, which include depthwise convolution operations, pointwise convolution operations, Batchnorm layers, and SiLU function layers. When inputting H×W feature maps, the computational complexity of the standard convolution block and the DSC module is compared as follows:

[0024] P Conv =H×W×K 2 ×M×N

[0025] P DSC =H×W×(K 2 ×M+M×N)

[0026] Among them, P Conv and P DSC They represent the parameters of standard convolution and depth-wise separable convolution respectively, H and W are the width and height of the feature map, M is the number of feature map channels, and K 2 Indicates the size of the convolution kernel. The present invention adopts the DSC module to replace some standard convolution modules of the basic YOLO framework backbone network, which significantly reduces the amount of calculation, while the first Conv module and the second Conv module are retained to ensure that the model fully extracts important shallow features such as texture and edge, so that the subsequent lightweight structure can effectively process the features.

[0027] Furthermore, the weighted fusion neck network in the high-precision teacher model includes four weighted connection modules, the first weighted module connects the first Unsample module and the third C2f module, the second weighted module connects the second Unsample module and the second C2f module, the third weighted module connects the seventh Conv module and the fifth C2f module, and the fourth weighted module connects the eighth Conv module and the SPFF module. Each module is assigned two learnable weight parameters, and the weight parameters are dynamically changed during the model training process to adjust the contribution of feature maps at different levels to achieve the best detection performance.

[0028] Furthermore, the distillation model measures the difference between the teacher model and the student model by aligning the activation distributions of the three output layers of the teacher model and the student model at the channel level, and then summing the KL divergence of the teacher model and the student model at all spatial positions in each channel. KL divergence is an asymmetric metric with different sensitivities to the difference between two probability distributions. It pays more attention to high-probability areas (foreground targets). If the probability distribution calculated in the teacher model is large, in order to minimize the KL divergence, the corresponding value of the student model needs to be close to the teacher model, thereby minimizing the difference to learn channel-level information similar to the teacher model and realize knowledge transfer.

[0029] Beneficial effects of the invention: The present invention proposes a lightweight fruit target detection method based on feature distillation and improved YOLOv8, which realizes efficient and innovative feature extraction, lightweight design and knowledge distillation process, significantly improves the performance and practicality of target detection tasks under resource-constrained conditions, and has broad application potential and market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is an overall flow chart of the method of the present invention;

[0031] Figure 2 It is a framework diagram of a lightweight student model of the method of the present invention;

[0032] Figure 3 A high-precision teacher model framework diagram for the method of the present invention;

[0033] Figure 4 It is a distillation process diagram of the method of the present invention;

[0034] Figure 5 It is a schematic diagram of the detection results of the method of the present invention. DETAILED DESCRIPTION

[0035] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific drawings and specific embodiments. It should be understood that the specific embodiments described below are intended to explain the present invention and are not intended to limit the present invention.

[0036] The present invention provides a lightweight fruit target detection method based on feature distillation and improved YOLOv8, referring to Figure 1 As shown, including:

[0037] Step 1: Obtain a public fruit dataset based on real scenarios and process the label format:

[0038] Specifically, the present invention conducts experiments based on the public dataset WGISD. The dataset contains 300 photos of grape fruits of five varieties, which are taken in real scenes of wineries and divided into training set and test set according to 8:2. Each image file is attached with a corresponding label file to provide annotations for the location of grape clusters.

[0039] Step 2: Build a lightweight student model based on YOLOv8-nano:

[0040] Specifically, refer to Figure 2As shown, the lightweight student model detection model includes a backbone network, a neck network, and a detection head network. The backbone network includes a first Conv module, a second Conv module, a first DFC-LightConv module, a first DSC module, a second DFC-LightConv module, a second DSC module, a third DFC-LightConv module, a third DSC module, a fourth DFC-LightConv module, and an SPPF module arranged in sequence. The neck network includes a first Unsample module, a first Concat layer, a fifth DFC-LightConv module, a second Unsample module, a second Concat layer, a sixth DFC-LightConv module, a third Conv module, a third Concat layer, a seventh DFC-LightConv module, a fourth Conv module, a fourth Concat layer, and an eighth DFC-LightConv module. The detection head network includes a first Detect module, a second Detect module, and a third Detect module.

[0041] Specifically, multiple standard convolution modules in the lightweight student model are replaced by DSC modules, which include depthwise convolution operations, pointwise convolution operations, Batchnorm layers, and SiLU function layers. When inputting H×W feature maps, the computational complexity of the standard convolution block and the DSC module is compared as follows:

[0042] P Conv =H×W×K 2 ×M×N

[0043] P DSC =H×W×(K 2 ×M+M×N)

[0044] Among them, P Conv and P DSC They represent the parameters of standard convolution and depth-wise separable convolution respectively, H and W are the width and height of the feature map, M is the number of feature map channels, and K 2 Indicates the size of the convolution kernel. The present invention adopts the DSC module to replace some standard convolution modules of the basic YOLO framework backbone network, which significantly reduces the amount of calculation, while the first Conv module and the second Conv module are retained to ensure that the model fully extracts important shallow features such as texture and edge, so that the subsequent lightweight structure can effectively process the features.

[0045] Specifically, the DFC-LightConv module in the lightweight student model includes two operation branches, the first branch includes the first DFC attention module, the first deep convolution module, the first point-by-point convolution module, and the GhostConv convolution module, and the second branch includes the second deep convolution module, the second point-by-point convolution module, and the Conv module. The input feature map passes through the two operation branches respectively, and the results are added as the output of the DFC-LightConv module. The DFC attention is an attention mechanism built on a fully connected layer, which can be quickly executed on ordinary hardware and effectively capture long-distance dependencies. It aggregates pixel information in the horizontal and vertical directions of the feature map, normalizes it, and then obtains the final output features through the SiLU activation function. By decoupling the fully connected operation, the DFC mechanism enables the model to consider local and global information at the same time, and by sharing part of the weight information, it eliminates the delay caused by tensor shaping and transposition operations, thereby better maintaining the lightweight characteristics of the model.

[0046] Step 3: Input the data set into the student model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained student model.

[0047] Step 4: Build a high-precision teacher model based on YOLOv8-x:

[0048] Specifically, refer to Figure 3 As shown, the high-precision teacher model detection model includes a backbone network, a weighted fusion neck network, and a detection head network. The backbone network includes a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a third C2f module, a fifth Conv module, a fourth C2f module, and an SPPF module arranged in sequence. The weighted fusion neck network includes a first Unsample module, a first weighted connection module, a fifth C2f module, a second Unsample module, a second weighted connection module, a sixth C2f module, a sixth Conv module, a third weighted connection module, a seventh C2f module, a seventh Conv module, a fourth weighted connection module, and an eighth C2f module. The detection head network includes a first Detect module, a second Detect module, and a third Detect module.

[0049] Specifically, the weighted fusion neck network in the high-precision teacher model includes four weighted connection modules, the first weighted module connects the first Unsample module and the third C2f module, the second weighted module connects the second Unsample module and the second C2f module, the third weighted module connects the seventh Conv module and the fifth C2f module, and the fourth weighted module connects the eighth Conv module and the SPFF module. Each module is assigned two learnable weight parameters (ω ij ,i=1,2,3,4;j=1,2), the weight parameters are dynamically changed during model training to adjust the contribution of feature maps at different levels to achieve the best detection performance.

[0050] Step 5: Input the data set into the teacher model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained teacher model;

[0051] Step 6: Build a distillation model, minimize the difference between the student model and the teacher model through KL divergence calculation, and obtain a distillation model that has learned the feature representation ability of the teacher model:

[0052] Specifically, refer to Figure 4 As shown in the figure, the distillation model measures the difference between the teacher model and the student model by aligning the activation distributions of the three output layers of the teacher model and the student model at the channel level, and then summing the KL divergence of the teacher model and the student model at all spatial positions on each channel. KL divergence is an asymmetric metric with different sensitivities to the difference between two probability distributions. It pays more attention to high-probability areas (foreground targets). If the probability distribution calculated in the teacher model is large, in order to minimize the KL divergence, the corresponding value of the student model needs to be close to the teacher model, thereby minimizing the difference to learn channel-level information similar to the teacher model and realize knowledge transfer.

[0053] Step 7: Input the test set into the distillation model to obtain the test results. Figure 5 As shown, fruit target detection is achieved.

[0054] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent methods or changes that do not deviate from the technical creation of the present invention should be included in the scope of protection of the present invention.

Claims

1. A lightweight fruit target detection method based on feature distillation and improved YOLOv8, characterized in that: The following steps are involved: Step 1: Obtain a public fruit dataset based on real scenarios and process the label format; Step 2: Build a lightweight student model based on YOLOv8-nano; The lightweight student model detection model includes a backbone network, a neck network, and a detection head network. The lightweight student model backbone network includes a first Conv module, a second Conv module, a first DFC-LightConv module, a first DSC module, a second DFC-LightConv module, a second DSC module, a third DFC-LightConv module, a third DSC module, a fourth DFC-LightConv module, and an SPPF module, which are arranged in sequence; The lightweight student model neck network includes a first Unsample module, a first Concat layer, a fifth DFC-LightConv module, a second Unsample module, a second Concat layer, a sixth DFC-LightConv module, a third Conv module, a third Concat layer, a seventh DFC-LightConv module, a fourth Conv module, a fourth Concat layer, and an eighth DFC-LightConv module; The lightweight student model detection head network includes a first Detect module, a second Detect module, and a third Detect module; Step 3: Input the data set into the student model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained student model; Step 4: Build a high-precision teacher model based on YOLOv8-x; The high-precision teacher model detection model includes a backbone network, a weighted fusion neck network, and a detection head network; The precision teacher model backbone network includes a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a third C2f module, a fifth Conv module, a fourth C2f module, and an SPPF module, which are arranged in sequence; The precision teacher model weighted fusion neck network includes a first Unsample module, a first weighted connection module, a fifth C2f module, a second Unsample module, a second weighted connection module, a sixth C2f module, a sixth Conv module, a third weighted connection module, a seventh C2f module, a seventh Conv module, a fourth weighted connection module, and an eighth C2f module; The precision teacher model detection head network includes a first Detect module, a second Detect module, and a third Detect module; Step 5: Input the data set into the teacher model for training and verification, and iteratively update the training weight file through forward and backward propagation to obtain the trained teacher model; Step 6: Build a distillation model, minimize the difference between the student model and the teacher model through KL divergence calculation, and obtain a distillation model that has learned the feature representation ability of the teacher model; Step 7: Input the test set into the distillation model to obtain the detection results and realize fruit target detection.

2. According to claim 1, a lightweight fruit target detection method based on feature distillation and improved YOLOv8 is characterized in that: The DFC-LightConv module in the lightweight student model includes two operation branches, the first branch includes the first DFC attention module, the first deep convolution module, the first point-by-point convolution module, and the GhostConv convolution module, and the second branch includes the second deep convolution module, the second point-by-point convolution module, and the Conv module. The input feature map passes through the two operation branches respectively, and the results are added as the output of the DFC-LightConv module; The DFC attention is an attention mechanism built on a fully connected layer, which can be quickly executed on ordinary hardware and effectively capture long-distance dependencies. First, pixel information is aggregated in the horizontal and vertical directions of the feature map: A=A h +A v Among them, F is the learnable weight, x is the original input, ⊙ represents element-level multiplication, w and h are used to indicate the current pixel position, and A is the result of aggregation in two directions. After the generated feature map is normalized, the final output feature is obtained through the SiLU activation function: F enhanced =SiLU(BatchNorm(A⊙F)) By decoupling the fully connected operations, the DFC mechanism enables the model to consider both local and global information at the same time, and by sharing part of the weight information, it eliminates the delay caused by tensor reshaping and transposition operations, thereby better maintaining the lightweight characteristics of the model.

3. A lightweight fruit target detection method based on feature distillation and improved YOLOv8 according to claim 1, characterized in that: In the lightweight student model, multiple standard convolution modules are replaced by DSC modules, which include depthwise convolution operations, pointwise convolution operations, Batchnorm layers, and SiLU function layers. When inputting H×W feature maps, the computational complexity of the standard convolution block and the DSC module is compared as follows: P Conv =H×W×K 2 ×M×N P DSC =H×W×(K 2 ×M+M×N) Among them, P Conv and P DSC They represent the parameters of standard convolution and depth-wise separable convolution respectively, H and W are the width and height of the feature map, M is the number of feature map channels, and K 2 Indicates the size of the convolution kernel. The present invention adopts the DSC module to replace some standard convolution modules of the basic YOLO framework backbone network, which significantly reduces the amount of calculation, while the first Conv module and the second Conv module are retained to ensure that the model fully extracts important shallow features such as texture and edge, so that the subsequent lightweight structure can effectively process the features.

4. A lightweight fruit target detection method based on feature distillation and improved YOLOv8 according to claim 1, characterized in that: The weighted fusion neck network in the high-precision teacher model includes four weighted connection modules, the first weighted module connects the first Unsample module and the third C2f module, the second weighted module connects the second Unsample module and the second C2f module, the third weighted module connects the seventh Conv module and the fifth C2f module, and the fourth weighted module connects the eighth Conv module and the SPFF module. Each module is assigned two learnable weight parameters, and the weight parameters are dynamically changed during the model training process to adjust the contribution of feature maps at different levels to achieve optimal detection performance.

5. A lightweight fruit target detection method based on feature distillation and improved YOLOv8 according to claim 1, characterized in that: The distillation model measures the difference between the teacher model and the student model by aligning the activation distributions of the three output layers of the teacher model and the student model at the channel level, and then summing the KL divergence of the teacher model and the student model at all spatial positions on each channel. KL divergence is a measure of asymmetry and has different sensitivities to the difference between two probability distributions. It pays more attention to high-probability areas (foreground targets). If the probability distribution calculated in the teacher model is large, in order to minimize the KL divergence, the corresponding value of the student model needs to be close to the teacher model, thereby minimizing the difference to learn channel-level information similar to the teacher model and achieve knowledge transfer.

Images