A Grain Flow Detection Method Based on a Lightweight Improved YOLOv8 Model Using SCConv

Abstract

This application discloses a grain flow detection method based on a lightweight improved YOLOv8 model using SCConv, which can reduce the probability of missed and false detections of small objects and improve the detection accuracy of small objects. The method includes: extracting spatial and semantic information from the preprocessed image input to the backbone network; aggregating and fusing feature maps of different scales using a feature pyramid network and a path aggregation network; processing high-resolution feature maps using a newly added small object detection head to obtain more local image information and capture small objects; predicting the bounding box position, size, and category of the object using each detection head in the model based on the received feature map; performing non-maximum suppression on the detection results from each detection head; combining the detection results contributed by the newly added small object detection head with the results of the original detection heads to output the final detection result; and using the ByteTrack algorithm to track and count the detected grain particles to calculate the grain flow.

Description

Technical Field

[0001] This application relates to the field of image detection technology based on a specific computing system, and in particular to a grain flow detection method based on a lightweight improved YOLOv8 model using SCConv. Background Technology

[0002] YOLOv8 is a deep learning and computer vision-based object detection algorithm launched by Ultralytics in early 2023. It is a major update to the YOLO series, building on the success of previous YOLO versions and introducing new features and improvements to further enhance the performance and flexibility of object detection. It achieves a good balance between accuracy and real-time performance and is suitable for various application scenarios.

[0003] For small objects, such as grains, they occupy fewer pixels in the image and contain limited feature information. The YOLOv8 model has difficulty accurately extracting and learning effective features, which can easily lead to missed detections or false detections, thereby reducing detection accuracy. Summary of the Invention

[0004] To overcome the aforementioned technical deficiencies, reduce the probability of missed or false detections of small target objects, and improve the detection accuracy of small target objects, this application provides a grain flow detection method based on a lightweight improved YOLOv8 model using SCConv. The specific scheme is as follows:

[0005] The lightweight YOLOv8 model based on SCConv is obtained by replacing the original c2f convolution module with the SCConv convolution module on the original YOLOv8 model, and adding a small target detection head to the original detection head to handle the detection of small objects.

[0006] The grain flow detection method specifically includes the following steps:

[0007] S1. Image preprocessing: Perform image resizing, normalization, and data augmentation operations on the input image;

[0008] S2. Backbone network feature extraction: The preprocessed image is input into the backbone network to extract the spatial and semantic information of the image;

[0009] S3. Aggregation and Fusion Operations: The Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) are used to aggregate and fuse feature maps of different scales.

[0010] S4. Small Object Detection: The newly added small object detection head processes high-resolution feature maps to obtain more local image information and capture small objects.

[0011] S5. Multi-scale bounding box prediction and classification: Utilize each detector head in the model to predict the bounding box location, size, and category of the object based on the received feature map;

[0012] S6. Non-maximum suppression operation: Perform non-maximum suppression operation on the detection results from each detection head;

[0013] S7. Detection result output: The detection results contributed by the newly added small target detection head will be combined with the results of the original detection head to output the final detection result;

[0014] S8. Grain Flow Calculation: The ByteTrack algorithm is used to track and count detected grain particles, thereby calculating the grain flow.

[0015] Furthermore, in the YOLOv8 model improved by SCConv lightweighting, the MPDIoU loss function is used to calculate the model loss and directly predict the distance between the top left and bottom right corners of the bounding box and the actual labeled box.

[0016] The specific calculation formula is as follows:

[0017]

[0018]

[0019]

[0020] Furthermore, the SCConv module includes a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU), which are set sequentially. Spatial refinement features are obtained through SRU operations, and channel refinement features are obtained through CRU operations.

[0021] Furthermore, the small target object includes grain particles.

[0022] Furthermore, the final detection results include the category, bounding box coordinates, and confidence level for each target.

[0023] The technical effects achieved by employing the above-described technical solution in the embodiments of this application are as follows:

[0024] 1. The YOLOv8 network is optimized by adding a more refined detection head specifically for handling small targets. Small targets typically occupy a small pixel area in an image and are easily obscured by large features on high-level feature maps. By adding an additional detection head on the shallow feature maps, more local information can be utilized, thereby improving the detection accuracy of small targets. This detection head will perform detection on a finer feature map (i.e., a higher-resolution feature map) to better capture small targets. This prediction head will allow the network to focus more on grain particle instances with smaller pixel values, thereby improving the accuracy of small target grain particle recognition;

[0025] 2. Using the MPDIOU loss function can effectively solve the problem of detection box distortion caused by overlapping of small target objects such as grain particles, and effectively reduce the false detection and false negative detection of small target objects such as grain particles. Attached Figure Description

[0026] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0027] Figure 1 This is a schematic diagram of a YOLOv8 model structure;

[0028] Figure 2 This is a schematic diagram of the improved YOLOv8 model based on SCConv in this application.

[0029] Figure 3 This is a schematic diagram of the SCConv convolutional module;

[0030] Figure 4 This is a flowchart illustrating the grain flow detection method based on the lightweight improved YOLOv8 model using SCConv in this application.

[0031] Figure 5 This is a schematic diagram illustrating the object tracking principle of the combination of YOLOv8 and ByteTrack algorithms in this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0033] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0034] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0035] Please provide an explanation of the terms used in this application:

[0036] YOLOv8 is the latest algorithm in the YOLO series and currently the highest-performing single-stage object detection algorithm. YOLOv8 employs a deep convolutional neural network, transforming the object detection task into an end-to-end regression problem, simultaneously predicting the class and bounding boxes of the input image. The structure of YOLOv8 is as follows: Figure 1 As shown, it can be roughly divided into two parts: Backbone and Head.

[0037] Backbone section:

[0038] A. Convolutional layer P1: This is a basic convolutional layer used for preliminary feature extraction from the input image.

[0039] The B.c2f structure corresponds to "CSP Bottleneck with 2 convolutions," providing intermediate feature extraction capabilities for the backbone. Figure 1 As can be seen, the c2f structure appears in each stage from P1 to P5, indicating that feature extraction and reshaping occur at each scale. It consists of convolutional layers (conv), splitting layers (split), and Bottleneck×N (N applications of the Bottleneck structure).

[0040] C. Convolutional layers P2, P3, P4, P5: These convolutional layers further extract features in specific steps. These layers may correspond to different feature map sizes and extract features at different levels.

[0041] D.SPPF: SPPF is a spatial pyramid pooling structure, which is typically used in convolutional networks to capture multi-scale information.

[0042] Head section:

[0043] a. c2f and convolutional operations: In the Head section, you can see that the c2f structure is combined with other convolutional layers to generate richer and more detailed feature maps.

[0044] b. Concat and Upsample: These two operations are used to fuse feature maps from different levels and adjust their size, which is very important in multi-scale object detection.

[0045] c. Detect: The final detection operation, responsible for generating bounding boxes and classification scores for the targets. It includes BoxLoss and ClsLoss, where BoxLoss handles the bounding box regression problem, and ClsLoss handles the classification problem.

[0046] Given that the detection target is a small object such as grain particles, and considering the structure and function of YOLOv8, the shortcomings of this application can be summarized as follows:

[0047] Small-scale target detection: Although YOLOv8 performs excellently in multi-scale target detection, its ability to detect small targets such as grain particles remains limited. In practical applications, the detection of small targets like grain particles is crucial.

[0048] In many practical applications, grain particles often overlap during harvesting operations or test bench operations. This situation often poses a challenge to traditional target detection models because they are susceptible to interference, leading to reduced detection accuracy. Given that grain flow detection typically needs to be completed in real-time at the edge, the model's lightweight design not only facilitates its convenient deployment at the edge but also improves computational efficiency, ensuring the real-time performance and accuracy of the detection work.

[0049] To facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below:

[0050] To address the aforementioned shortcomings, this application provides a grain flow detection method based on a lightweight improved YOLOv8 model using SCConv, such as... Figure 2 As shown, it includes:

[0051] Improvements to the YOLOv8 model based on SCConv: On the basis of the original YOLOv8 model, the original c2f convolution module is replaced with the SCConv convolution module, and a small target detection head is added to the original detection head to handle the detection of small objects.

[0052] like Figure 2As shown, based on the original YOLOv8 model, some structural improvements were made to the model to suit the actual situation. The SCConv lightweight model was used, and a small target detection head was added to improve the detection accuracy of small targets such as grain particles.

[0053] like Figure 3 The diagram shows the structure of SCConv, which consists of two units: a Spatial Reconstruction Unit (SRU) and a Channel Reconstruction Unit (CRU), arranged sequentially. Specifically, for the intermediate input feature X in the bottleneck residual block, we first obtain the spatially refined feature Xw through SRU operations, and then obtain the channel refined feature Y using CRU operations. We utilize spatial and channel redundancy between features in the SCConv module, allowing for seamless integration into any CNN architecture to reduce redundancy between intermediate feature maps and enhance the feature representation of the CNN.

[0054] like Figure 4 As shown, the grain flow detection method based on the SCConv lightweight improved YOLOv8 model in this application includes:

[0055] S1. Image Preprocessing: Perform the following preprocessing operations on the input image: image resizing, normalization, and data augmentation.

[0056] Specifically, when the input is a feature map of size 640×640, after receiving the image input, the model first performs a series of preprocessing steps on the image, including adjusting the image size, normalization (e.g., scaling the pixel values ​​to between 0 and 1), and data augmentation (e.g., flipping, rotating, scaling, etc.) to adapt to the input dimension of the model and improve the model's generalization ability.

[0057] S2. Backbone network feature extraction: The preprocessed image is input into the backbone network to extract the spatial and semantic information of the image.

[0058] The improved YOLOv8 model based on SCConv utilizes deep convolutional networks to extract basic image features. These features gradually extract the spatial and semantic information of the image, while reducing the dimensionality of the data, enabling the model to extract multi-level image features.

[0059] S3. Aggregation and Fusion Operations: The Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) are used to aggregate and fuse feature maps of different scales.

[0060] FPN aggregates features from deep to shallow levels, while PAN aggregates features from shallow to deep levels, ensuring that feature maps at different levels can capture details of multi-scale objects in the image. The difference before and after the improvement: After adding a small object detection head, the designs of FPN and PAN place greater emphasis on high-resolution feature maps, as these feature maps are more suitable for small object detection. Compared to the previous YOLOv8 model without a small object detection head, this multi-scale feature map aggregation method provides more fine-grained feature information specifically for small objects.

[0061] S4. Small Object Detection: The newly added small object detection head processes high-resolution feature maps to obtain more local image information and capture small objects.

[0062] In addition to the existing detection heads, a small object detection head is added, specifically for processing high-resolution feature maps. This small object detection head is directly connected to the high-resolution feature map (e.g., a high-resolution feature layer from PAN) to better identify the feature information of small objects. The difference before and after the improvement: YOLOv8 without the small object detection head typically uses three detection heads, one for large, one for medium, and one for small-scale object detection. However, the "small" scale detection head often fails to capture even smaller object features. With the addition of the small object detection head, the model can utilize a higher-resolution feature layer, focusing more on detecting small objects and enhancing its ability to recognize small objects in complex backgrounds.

[0063] S5. Multi-scale bounding box prediction and classification: Utilize each detector head in the model to predict the bounding box location, size, and category of the object based on the received feature map.

[0064] The small object detection head pays special attention to object information on small-scale feature maps, generating more detailed predictions. The difference between the improved and unimproved models lies in the enhanced predictive ability of the new small object detection head compared to the previous YOLOv8 model, particularly in object classification and bounding box regression. This improvement is primarily reflected in the small object detection head's specialized handling of small-scale objects, significantly reducing false negatives and false positives.

[0065] S6. Non-Maximum Suppression Operation: Perform non-maximum suppression (NMS) operation on the detection results from each detection head to remove overlapping or redundant detection boxes, and finally retain the boxes with higher confidence as the final result.

[0066] NMS comprehensively considers the detection results of large, medium, and small targets to select the optimal predicted bounding box. The difference before and after the improvement is that the addition of a small target detection head generates more small target detection boxes. Therefore, further optimization is needed when performing NMS to avoid redundant detections between small targets. This improvement can reduce the number of false positives and overlapping detection boxes caused by the density of small targets.

[0067] S7. Detection Result Output: The detection results contributed by the newly added small target detection head will be combined with the results of the original detection head to output the final detection result, including the category, bounding box coordinates and confidence score of each target.

[0068] After NMS, the model outputs the final detection results, including the category, bounding box coordinates, and confidence score for each target. The detection results contributed by the newly added small target detection head are combined with the results of the original detection head to ensure a balance in the detection of large and small targets. The difference before and after the improvement: In the YOLOv8 model without the small target detection head, if there are many small targets in the image, the model may not be able to accurately detect them. With the small target detection head added, the model can better cover small targets in its output, especially in complex backgrounds or situations with a high density of small targets, providing more comprehensive detection results.

[0069] S8. Grain Flow Calculation: The ByteTrack algorithm is used to track and count detected grain particles, thereby calculating the grain flow.

[0070] ByteTrack is an innovative Multiple Object Tracking (MOT) system that demonstrates significant advantages in video tracking due to its efficient algorithm optimization and concise design. ByteTrack employs the Tracking By Detection (TBD) method, where detection is performed in each frame, and then a Kalman filter is used to predict the position of each trajectory in the current frame. This method combines the advantages of detection and tracking, improving tracking accuracy and robustness. ByteTrack uses a Kalman filter to predict the possible positions of trajectories in the current frame. Position prediction is performed using the `predict` function, and the `update` function adjusts the Kalman parameters based on the predicted positions and paired detection boxes. Through its unique detection box association technique, ByteTrack achieves faster and more accurate object tracking. It can correctly associate detection boxes of the same target across different frames, maintaining stable tracking performance even in complex and changing environments.

[0071] The specific workflow is as follows: First, the position of the current target detection frame and the depth features of each image patch in the detection frame are read. Then, frames are filtered based on their confidence level, removing frames and features with low confidence. Next, non-extreme suppression is applied to the detection frames to eliminate multiple frames containing a single target. Finally, Kalman filtering is used to predict the target's position in the current frame.

[0072] Trackers are typically combined with object detection networks to achieve multi-object tracking capabilities. For example... Figure 5 The diagram illustrates the principle behind object tracking achieved by combining a target detector and ByteTrack. By tracking and counting detected grain particles, the flow rate of the grain can be calculated.

[0073] In this embodiment, YOLOv8 inherits the characteristics of the YOLO series, possessing efficient feature extraction capabilities and multi-scale feature fusion. Typically, YOLOv8 performs detection at different scales, usually employing three detection heads to handle targets of large, medium, and small scales. However, for very small targets, such as grain particle image recognition, it faces challenges due to the grain particle diameter being in the range of 4–7 mm and low pixel coverage. Furthermore, the input image size also interferes with the recognition of small grain particles, as excessively large images lead to scaling, making small target recognition even more difficult. Therefore, these three scales may still be insufficient for effective capture. To address these challenges, the YOLOv8 network is optimized by adding a more refined detection head specifically for small targets. Small targets typically occupy a small pixel area in an image and are easily obscured by large features on high-level feature maps. By adding an additional detection head on the shallow feature map, more local information can be utilized, thereby improving the detection accuracy of small targets. This detection head will perform detection on a finer feature map (i.e., a higher-resolution feature map) to better capture small targets. This prediction head will allow the network to pay more attention to grain particle instances with smaller pixel values, thereby improving the accuracy of small target grain particle recognition.

[0074] To address the issue of overlapping grains causing distorted model detection performance, this application employs an improved MPDIOU method. MPDIOU (Minimum Point Distance based IoU) is an innovative bounding box regression loss function primarily used in computer vision tasks such as object detection and instance segmentation. MPDIOU directly predicts the distance between the top-left and bottom-right corners of the bounding box and the actual labeled box. The specific calculation formula is as follows:

[0075]

[0076]

[0077]

[0078] In the formula, A and B represent two arbitrary convex shapes.

[0079] w, h — width and height of the input image

[0080] — These represent the coordinates of the upper left and lower right points of A, respectively.

[0081] — These represent the coordinates of the upper left and lower right points of B, respectively.

[0082] MPDIOU, which calculates the squares of the Euclidean distances between the top-left and bottom-right corners of points A and B respectively, simplifies the similarity comparison between two bounding boxes, helping the algorithm select the most suitable bounding box for accurate target localization. Using MPDIOU can effectively solve the detection box distortion caused by overlapping grain particles, significantly reducing the problem of missed grain particle detections.

[0083] To facilitate the smooth deployment of the model at the edge and ensure the real-time execution of the detection task, thereby laying the foundation for subsequent applications on harvesters, this application uses SCConv to lightweight the YOLOv8 model. SCConv (Spatial and channel reconstruction convolution) is a lightweight convolution consisting of two concatenated parts: SRU (Spatial reconstruction unit) and CRU (Channel reconstruction unit). It reduces model parameters and floating-point operations (FLOPs) by reducing the redundancy of spatial and channel features in standard convolution.

[0084] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0085] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A method for grain flow detection based on a lightweight improved YOLOv8 model using SCConv, characterized in that, The lightweight YOLOv8 model based on SCConv is obtained by replacing the original c2f convolution module with the SCConv convolution module on the original YOLOv8 model, and adding a small target detection head to the original detection head to handle the detection of small objects. The grain flow detection method specifically includes the following steps: S1. Image preprocessing: Perform image resizing, normalization, and data augmentation operations on the input image; S2. Backbone network feature extraction: Extract spatial and semantic information of the image from the preprocessed image input into the backbone network. S3. Aggregation and Fusion Operations: The Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) are used to aggregate and fuse feature maps of different scales. S4. Small Object Detection: The newly added small object detection head processes high-resolution feature maps to obtain more local image information and capture small objects, including grain particles. S5. Multi-scale bounding box prediction and classification: Utilize each detector head in the model to predict the bounding box location, size, and category of the object based on the received feature map; S6. Non-maximum suppression operation: Perform non-maximum suppression operation on the detection results from each detection head; S7. Detection result output: The detection results contributed by the newly added small target detection head will be combined with the results of the original detection head to output the final detection result; S8. Grain Flow Calculation: The ByteTrack algorithm is used to track and count detected grain particles, thereby calculating the grain flow.

2. The grain flow detection method according to claim 1, characterized in that, In the YOLOv8 model improved by SCConv lightweight, the MPDIoU loss function is used to calculate the model loss and directly predict the distance between the top left and bottom right corners of the bounding box and the actual labeled box. The specific calculation formula is as follows:

3. The grain flow detection method according to claim 1, characterized in that, The SCConv module includes a Spatial Reconstruction Unit (SRU) and a Channel Reconstruction Unit (CRU), which are set sequentially. Spatial refinement features are obtained through SRU operations, and channel refinement features are obtained through CRU operations.

4. The grain flow detection method according to claim 1, characterized in that, The final detection results include the category, bounding box coordinates, and confidence level for each target.

Images