Grain flow detection method based on SCConv lightweight improved YOLOv8 model

Abstract

The invention discloses a grain flow detection method based on an SCConv lightweight improved YOLOv8 model, which can reduce the missing detection and false detection probability of small target objects and improve the detection precision of the small target objects. The method comprises the following steps: inputting a preprocessed image into a backbone network to extract space and semantic information of the image; using a feature pyramid network and a path aggregation network to aggregate and fuse the feature maps of different scales; a newly added small target detection head is used for processing the high-resolution feature map, more local image information is obtained, and a small target object is captured; utilizing each detection head in the model to predict the position, size and category of the bounding box of the object according to the received feature map; performing a non-maximum suppression operation on a detection result from each detection head; and combining the detection result contributed by the newly added small target detection head with the result of the original detection head to output a final detection result. And tracking and counting the detected grain particles by using a ByteTrack algorithm, and calculating the grain flow.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of image detection based on a specific computing system, and more particularly to a grain flow detection method based on a SCConv lightweight improved YOLOv8 model. Background Art

[0002] YOLOv8 is a target detection algorithm based on deep learning and computer vision launched by ultralytics in early 2023. It is a major updated version of the YOLO series. It builds on the success of previous YOLO versions, introduces new features and improvements, and further improves the performance and flexibility of target detection. It achieves a good balance between accuracy and real-time performance and is suitable for various application scenarios.

[0003] For small target objects, such as grains, they occupy fewer pixels in the image and contain limited feature information. It is difficult for the YOLOv8 model to accurately extract and learn effective features, which can easily lead to missed detections or false detections, thereby reducing detection accuracy. Summary of the invention

[0004] In order to overcome the above technical defects, reduce the probability of missed detection and false detection of small target objects, and improve the detection accuracy of small target objects, the embodiment of the present application provides a grain flow detection method based on the SCConv lightweight improved YOLOv8 model, and its specific scheme is as follows:

[0005] The SCConv-based lightweight improved YOLOv8 model is based on the original YOLOv8 model. The SCConv convolution module is used to replace the original c2f convolution module. At the same time, a small target detection head is added to the original detection head to specialize in detecting small target objects.

[0006] The grain flow detection method specifically includes the following steps:

[0007] S1, image preprocessing: perform image resizing, normalization and data augmentation operations on the input image;

[0008] S2, backbone network feature extraction: the preprocessed image is input into the backbone network to extract the spatial and semantic information of the image;

[0009] S3, aggregation and fusion operation: use the feature pyramid network FPN and path aggregation network PAN to aggregate and fuse feature maps of different scales;

[0010] S4, small target object detection: Use the newly added small target detection head to process high-resolution feature maps, obtain more local image information, and capture small target objects;

[0011] S5. Multi-scale bounding box prediction and classification: Use each detection head in the model to predict the bounding box position, size and category of the object based on the received feature map;

[0012] S6, non-maximum suppression operation: performing a non-maximum suppression operation on the detection results from each detection head;

[0013] S7, detection result output: the detection result contributed by the newly added small target detection head is combined with the result of the original detection head to output the final detection result;

[0014] S8. Grain flow calculation: The detected grain particles are tracked and counted using the ByteTrack algorithm to calculate the grain flow.

[0015] Furthermore, in the YOLOv8 model based on SCConv lightweight improvement, the MPDIoU loss function is used to calculate the model loss, directly predicting the distance between the upper left corner and the lower right corner of the bounding box and the actual annotation box;

[0016] The specific calculation formula is as follows:

[0017]

[0018]

[0019]

[0020] Furthermore, the SCConv module includes a spatial reconstruction unit SRU and a channel reconstruction unit CRU, which are arranged sequentially, and the spatial refinement feature is obtained by the SRU operation, and the channel refinement feature is obtained by the CRU operation.

[0021] Furthermore, the small target objects include grains.

[0022] Furthermore, the final detection result includes the category, bounding box coordinates and confidence of each object.

[0023] The embodiments of the present application adopt the above technical solution to achieve the following technical effects:

[0024] 1. Optimize the YOLOv8 network and add a more refined detection head specifically for small objects. Small objects usually occupy a small pixel area in the image and are easily overwhelmed by large-scale features on high-level feature maps. By adding an additional detection head on shallow feature maps, more local information can be utilized, thereby improving the detection accuracy of small objects. This detection head will perform detection on finer feature maps (i.e., feature maps with higher resolution) to better capture small objects. This prediction head will allow the network to pay more attention to grain instances with smaller pixel values, thereby improving the accuracy of small target grain recognition;

[0025] 2. The use of the MPDIOU loss function can effectively solve the detection frame distortion caused by the overlap of small target objects such as grain grains, and effectively reduce the problems of false detection and missed detection of small target objects such as grain grains. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0027] Figure 1 This is a schematic diagram of the model structure of YOLOv8;

[0028] Figure 2 This is a schematic diagram of the model structure after the YOLOv8 model is improved based on SCConv in this application;

[0029] Figure 3 It is a schematic diagram of the structure of the SCConv convolution module;

[0030] Figure 4 A flowchart of a grain flow detection method based on the SCConv lightweight improved YOLOv8 model in this application;

[0031] Figure 5 This is a schematic diagram of the object tracking principle of the combination of YOLOv8 and ByteTrack algorithms in this application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0033] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0034] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0035] Provide explanations of terms involved in this application:

[0036] YOLOv8 is the latest algorithm in the YOLO series and is also the best performing algorithm among the current single-stage object detection algorithms. YOLOv8 uses a deep convolutional neural network to convert the object detection task into an end-to-end regression problem, while predicting the category and bounding box of the input image. The structure of YOLOv8 is as follows: Figure 1 As shown, it can be roughly divided into two parts: Backbone and Head.

[0037] Backbone part:

[0038] A. Convolutional layer P1: It is a basic convolutional layer used to perform preliminary feature extraction on the input image.

[0039] B.c2f structure: corresponds to "CSP Bottleneck with 2convolutions", which provides the intermediate feature extraction capability for Backbone. Figure 1 As can be seen in Figure 1, the c2f structure appears in each stage from P1 to P5, indicating that features are extracted and reorganized at each scale. It consists of a convolutional layer (conv), a separation layer (spilt), and a Bottleneck×N (N applications of the Bottleneck structure).

[0040] C. Convolutional layers P2, P3, P4, P5: These convolutional layers further extract features at specific steps. These layers may correspond to different feature map sizes and extract features at different levels.

[0041] D.SPPF: SPPF is a spatial pyramid pooling structure, which is usually used in convolutional networks to capture multi-scale information.

[0042] Head part:

[0043] a.c2f and convolution operation: In the Head part, you can see that the c2f structure is combined with other convolutional layers to generate richer and more refined feature maps.

[0044] b.Concat and Upsample: These two operations are used to fuse feature maps of different levels and adjust their sizes, which is very important in multi-scale object detection.

[0045] c.Detect: The final detection operation, responsible for generating the target's bounding box, classification score, etc. It includes BoxLoss & Cls Loss, where Box Loss handles the regression problem of the bounding box and Cls Loss handles the classification problem.

[0046] Considering that the detection target is small target objects such as grains, combined with the structure and function of YOLOv8, this application can summarize its shortcomings as follows:

[0047] Small-scale object detection: Although YOLOv8 performs well in multi-scale object detection, its detection capability is still limited for small objects such as grains. In practical applications, the detection of small objects such as grains is crucial.

[0048] In many practical application scenarios, when harvesters are harvesting or test benches are operating, grains often overlap. This situation often poses a challenge to traditional target detection models because they are susceptible to interference, resulting in reduced detection accuracy. Given that grain flow detection usually needs to be completed instantly at the edge, the lightweight design of the model not only facilitates its convenient deployment at the edge, but also improves computing efficiency, ensuring the real-time and accuracy of detection work.

[0049] To facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below, as follows:

[0050] In view of the above defects, a grain flow detection method based on the SCConv lightweight improved YOLOv8 model is provided in the embodiment of the present application. Figure 2 As shown, including:

[0051] Improve the YOLOv8 model based on SCConv: Based on the original YOLOv8 model, use the SCConv convolution module to replace the original c2f convolution module, and add a small target detection head specializing in detecting small target objects on the basis of the original detection head.

[0052] like Figure 2As shown in the figure, based on the original YOLOv8 model, some structural improvements were made to the model according to the actual situation. The SCConv lightweight model was used and a small target detection head was added to improve the detection accuracy of small targets such as grain particles.

[0053] like Figure 3 The figure shows the structure of SCConv, which consists of two units, the spatial reconstruction unit (SRU) and the channel reconstruction unit (CRU), placed in a sequential manner. Specifically, for the intermediate input feature X in the bottleneck residual block, we first obtain the spatially refined feature Xw through the SRU operation, and then use the CRU operation to obtain the channel-refined feature Y. We exploit the spatial redundancy and channel redundancy between features in the SCConv module, which can be seamlessly integrated into any CNN architecture to reduce the redundancy between intermediate feature maps and enhance the feature representation of CNN.

[0054] like Figure 4 As shown, this application adopts a grain flow detection method based on the SCConv lightweight improved YOLOv8 model, including:

[0055] S1. Image preprocessing: Perform the following preprocessing operations on the input image: image resizing, normalization, and data augmentation.

[0056] Specifically, when the input is a feature map of size 640×640, after the model receives the image input, it first performs a series of preprocessing steps on the image, including resizing the image, normalizing (for example, scaling the pixel values ​​to between 0 and 1), and data augmentation (such as flipping, rotating, scaling, etc.) to adapt to the input dimension of the model and improve the generalization ability of the model.

[0057] S2. Backbone network feature extraction: The preprocessed image is input into the backbone network to extract the spatial and semantic information of the image.

[0058] The improved YOLOv8 model based on SCConv uses a deep convolutional network to extract the basic features of the image. These features gradually extract the spatial and semantic information of the image, while reducing the dimension of the data, enabling the model to extract multi-level image features.

[0059] S3, aggregation and fusion operations: Use the feature pyramid network FPN and path aggregation network PAN to aggregate and fuse feature maps of different scales.

[0060] FPN aggregates features from deep to shallow layers, while PAN aggregates features from shallow to deep layers, ensuring that feature maps at different levels can capture the details of multi-scale objects in the image. Differences before and after the improvement: After adding the small target detection head, the design of FPN and PAN will pay more attention to high-resolution feature maps, because these feature maps are more suitable for small target detection. Compared with the previous YOLOv8 model without the small target detection head, this aggregation method of multi-scale feature maps will provide more fine-grained feature information for small targets.

[0061] S4, Small target object detection: Use the newly added small target detection head to process high-resolution feature maps to obtain more local image information and capture small target objects.

[0062] In addition to the original detection head, a small target detection head is added, which is specifically used to process high-resolution feature maps. This small target detection head is directly connected to the high-resolution feature map (for example, the high-resolution feature layer from PAN) to better identify the feature information of small targets. The difference before and after the improvement: YOLOv8 without the small target detection head usually uses three detection heads for large, medium and small scale target detection. However, the "small" scale detection head here usually cannot capture smaller object features. After adding the small target detection head, the model can use higher resolution feature layers and focus more on detecting small objects, enhancing the model's ability to recognize small objects in complex backgrounds.

[0063] S5. Multi-scale bounding box prediction and classification: Use each detection head in the model to predict the bounding box position, size and category of the object based on the received feature map.

[0064] The small object detection head pays special attention to the object information on the small-scale feature map and generates more detailed predictions. The difference before and after the improvement: Compared with the previous YOLOv8 model, the newly added small object detection head enhances the model's prediction ability in small object detection, especially in object classification and bounding box regression. This improvement is mainly reflected in the small-scale objects that the small object detection head specifically handles, which greatly reduces missed detections and false detections.

[0065] S6, non-maximum suppression operation: perform non-maximum suppression (NMS) operation on the detection results from each detection head to remove overlapping or redundant detection frames, and finally retain the frames with higher confidence as the final results.

[0066] NMS will comprehensively consider the detection results of large targets, medium targets, and small targets to select the optimal prediction box. The difference before and after the improvement: Due to the addition of the small target detection head, the model will generate more small target detection boxes. Therefore, when performing NMS, further optimization is required to avoid redundant detection between small targets. This improvement can reduce the number of false detections and overlapping detection boxes caused by dense small targets.

[0067] S7, detection result output: The detection results contributed by the newly added small target detection head will be combined with the results of the original detection head to output the final detection results, including the category, bounding box coordinates and confidence of each target.

[0068] After NMS, the model will output the final detection results, including the category, bounding box coordinates, confidence, and other information of each target. The detection results contributed by the newly added small target detection head will be combined with the results of the original detection head to ensure balanced detection of large and small targets. The difference before and after the improvement: In the YOLOv8 model without adding a small target detection head, if there are a large number of small targets in the image, the model may not be able to accurately detect these small targets. After adding the small target detection head, the model can better cover small targets when outputting detection results, especially in complex backgrounds or when small targets are dense, providing more comprehensive detection results.

[0069] S8. Grain flow calculation: The detected grain particles are tracked and counted using the ByteTrack algorithm to calculate the grain flow.

[0070] The ByteTrack algorithm is an innovative Multiple Object Tracking (MOT) system that has shown significant advantages in the field of video tracking with its efficient algorithm optimization and concise design. ByteTrack uses the TBD (Tracking By Detection) method, that is, each frame needs to be detected, and then the position of each track in the current frame is predicted by the Kalman filter. This method combines the advantages of detection and tracking to improve the accuracy and robustness of tracking. ByteTrack uses a Kalman filter, which is responsible for predicting the possible position of the track in the current frame. The position is predicted by the predict function, and the Kalman parameters are adjusted according to the predicted position and the paired detection frame by the update function. ByteTrack achieves faster and more accurate object tracking through a unique associated detection frame technology. It can correctly associate the detection frames of the same target between different frames, and maintain a stable tracking effect even in complex and changing environments.

[0071] The specific workflow is: read the position of the current target detection frame and the depth features of each detection frame image block. Filter the frame according to the confidence of the frame, and remove the frames and features with low confidence. Then perform non-extreme suppression on the detection frame to eliminate multiple frames of a target. Use Kalman filtering to predict the position of the target in the current frame.

[0072] Trackers are usually combined with target detection networks to achieve multi-target tracking. Figure 5 The figure shows the principle of object tracking by combining the target detector and ByteTrack. By tracking and counting the detected grain particles, the grain flow rate can be calculated.

[0073] In the embodiment of the present application, YOLOv8 inherits the characteristics of the YOLO series, and has efficient feature extraction capabilities and multi-scale feature fusion. Normally, YOLOv8 will detect at different scales, and usually there will be three detection heads for processing targets of different scales, large, medium and small. But for very small targets, for example, when dealing with the problem of small target grain grain image recognition, the grain grain diameter is in the range of 4 to 7 mm and low pixel coverage. In addition, the input image size also interferes with the recognition of small target grain grains, because too large images will cause scaling, making the recognition of small targets more difficult. Therefore, these three scales may still not be enough to capture effectively. In order to meet these challenges, the YOLOv8 network is optimized and a more sophisticated detection head is added to deal with small targets. Small targets usually occupy a small pixel area in the image and are easily overwhelmed by large-scale features on high-level feature maps. By adding an additional detection head to the shallow feature map, more local information can be used to improve the detection accuracy of small targets. This detection head will detect on a finer feature map (i.e., a feature map with higher resolution) to better capture small targets. This prediction head will allow the network to pay more attention to grain instances with smaller pixel values, thereby improving the accuracy of small target grain recognition.

[0074] To address the problem of grain overlap causing distortion in model detection, this application uses MPDIOU improvement. MPDIoU (Minimum Point Distance based IoU) is an innovative bounding box regression loss function, which is mainly used in computer vision tasks such as object detection and instance segmentation. Using MPDIOU, you can directly predict the distance between the upper left and lower right corners of the bounding box and the actual annotation box. The specific calculation formula is:

[0075]

[0076]

[0077]

[0078] In the formula, A and B represent two arbitrary convex shapes respectively.

[0079] w, h - width and height of the input image

[0080] ——represents the coordinates of the upper left and lower right points of A respectively

[0081] ——represents the coordinates of the upper left and lower right points of B respectively

[0082] ——The square of the Euclidean distance between the upper left corners of A and B and the Euclidean distance between the lower right corners. MPDIOU simplifies the similarity comparison between the two bounding boxes and helps the algorithm select the most appropriate bounding box to accurately locate the target. Using MPDIOU can effectively solve the detection frame distortion caused by grain overlap and effectively reduce the problem of missed grain detection.

[0083] In order to promote the smooth deployment of the model at the edge and ensure the real-time execution of the detection task, and thus lay the foundation for subsequent applications on the harvester, this application uses SCConv to lightweight the YOLOv8 model. SCConv (Spatial and channel reconstruction convolution) is a lightweight convolution consisting of two parts, SRU (Spatial reconstruction unit) and CRU (Channel reconstruction unit), which reduces model parameters and floating-point operations (FLOPs) by reducing the redundancy of spatial and channel features in standard convolution.

[0084] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0085] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A grain flow detection method based on SCConv lightweight improved YOLOv8 model, characterized in that: The SCConv-based lightweight improved YOLOv8 model is based on the original YOLOv8 model. The SCConv convolution module is used to replace the original c2f convolution module. At the same time, a small target detection head is added to the original detection head to specialize in detecting small target objects. The grain flow detection method specifically includes the following steps: S1, image preprocessing: perform image resizing, normalization and data augmentation operations on the input image; S2, backbone network feature extraction: the preprocessed image is input into the backbone network to extract the spatial and semantic information of the image; S3, aggregation and fusion operation: use the feature pyramid network FPN and path aggregation network PAN to aggregate and fuse feature maps of different scales; S4, small target object detection: Use the newly added small target detection head to process high-resolution feature maps, obtain more local image information, and capture small target objects, such as grains; S5. Multi-scale bounding box prediction and classification: Use each detection head in the model to predict the bounding box position, size and category of the object based on the received feature map; S6, non-maximum suppression operation: performing a non-maximum suppression operation on the detection results from each detection head; S7, detection result output: the detection result contributed by the newly added small target detection head is combined with the result of the original detection head to output the final detection result; S8. Grain flow calculation: The detected grain particles are tracked and counted using the ByteTrack algorithm to calculate the grain flow.

2. The grain flow detection method according to claim 1, characterized in that: In the YOLOv8 model based on SCConv lightweight improvement, the MPDIoU loss function is used to calculate the model loss, directly predicting the distance between the upper left corner and the lower right corner of the bounding box and the actual annotation box; The specific calculation formula is as follows:

3. The grain flow detection method according to claim 1, characterized in that: The SCConv module includes a spatial reconstruction unit SRU and a channel reconstruction unit CRU, which are arranged sequentially. The spatial refinement feature is obtained by the SRU operation, and the channel refinement feature is obtained by the CRU operation.

4. The grain flow detection method according to claim 1, characterized in that: The final detection results include the category, bounding box coordinates and confidence of each object.

Images