A method for detecting and counting imperfect grains based on improved YOLOX

Abstract

This invention discloses a method for detecting and counting imperfect grains based on an improved YOLOX algorithm, comprising the following steps: S1, acquiring original RGB images of imperfect grains randomly arranged in a single layer at different densities, and classifying them according to insect-eaten grains, diseased grains, broken grains, sprouted grains, moldy grains, and perfect grains, establishing an original image dataset; S2, inputting the original image dataset into the improved YOLOX network model to determine the learning parameters of the improved YOLOX network model; S3, correcting the total count of grains with overlapping features to obtain a detection model for imperfect grains; S4, detecting the imperfect grains based on actual images or videos of the grains to be detected, and obtaining the counting results of various types of imperfect grains. This invention is suitable for deep learning network structures for fine-grained feature recognition and high-throughput detection and counting of imperfect grains to improve the accuracy of detection and counting.

Description

Technical Field

[0001] This invention relates to the field of online grain detection technology, and in particular to a method for detecting and counting imperfect grains based on an improved YOLOX. Background Technology

[0002] Grain plays a vital role in agricultural production, distribution, and food processing. During storage, production, and transportation, imperfect grains can occur. For example, in GB 1351-2008 "Wheat," imperfect grains refer to wheat kernels that are damaged but still usable. These include insect-damaged kernels, diseased kernels, broken kernels, sprouted kernels, and moldy kernels. Imperfect grains are a crucial indicator of quality inspection during grain procurement and distribution, significantly impacting grain storage and processing and easily leading to grain quality and safety issues. Currently, the main methods for detecting imperfect grains are manual inspection and machine vision identification. Manual inspection relies heavily on the subjective judgment of the inspector, making the process time-consuming, labor-intensive, and lacking repeatability, thus failing to meet the requirements for rapid and accurate large-scale wheat kernel detection.

[0003] In recent years, the use of machine vision for automatic wheat grain identification has attracted widespread attention. One type of machine vision detection method is based on manual feature extraction. This method uses a camera to capture images of the grain, and then analyzes and processes these images to calculate various feature parameters such as color, texture, and shape. This method overcomes the shortcomings of manual inspection and ensures the integrity of the grain grains during the identification process. However, this method requires manual extraction of feature parameters, continuous testing and selection, and is relatively complex. Furthermore, it is susceptible to the influence of mixed grain varieties, the presence of imperfect grains (such as a single broken wheat grain having a lesion in a certain part), and lighting conditions. As a result, it is difficult to find accurate and stable features in practical applications, and the identification accuracy is relatively unstable.

[0004] Another type is the deep learning-based feature extraction method, an end-to-end approach where convolutional neural networks automatically extract defect features. The feature extraction capability of deep learning algorithms allows the network to automatically learn the image features representing imperfect grains. Its understanding of imperfect grain targets, from local to global (e.g., from structural to semantic information), is distributed across different layers of the network, ultimately forming a holistic perception of the imperfect grain targets. However, the differences in morphology and color among imperfect grains are subtle, and the local features of different imperfect grains are very similar, which poses challenges for identification and counting. Therefore, there is an urgent need to design deep learning network structures suitable for fine-grained feature recognition and high-throughput detection and counting of imperfect grains to improve the accuracy of detection and counting.

[0005] This invention uses YOLOX as the framework base and adds a CBAM attention module to it. The CBAM attention module allows the network model to focus more on the feature parts in the image that distinguish different imperfect granules, thereby ignoring some useless information. The improved YOLOX structure is as follows: Figure 3 YOLOX is an improvement on YOLOv3-SSP, and uses many modules such as... Figure 3 As shown, YOLOX employs various techniques, such as Focus, CSP (Cross-Stage Partial Connections), SPP (Special Pyramid Pooling), FPN (Feature Pyramid Network), and PAN (Pixel Aggregation Network). YOLOX also utilizes several techniques, including Anchor-Free architecture, which eliminates the need for cluster analysis to select appropriate prior box sizes for small samples of imperfect wheat grains, enhancing the model's generalization ability while reducing the number of parameters. SimOTA is used to dynamically match positive samples for targets of different sizes. A Decoupled Head structure is used, separating classification and regression into two branches within the Head. In conclusion, adding an appropriate number of CBAM modules at suitable locations in YOLOX provides better recognition of fine-grained features of imperfect grains and is more suitable for detecting small samples of grains. Summary of the Invention

[0006] This invention proposes a method for detecting and counting imperfect grains based on an improved YOLOX, which uses a deep learning network structure suitable for fine-grain size feature recognition and high-throughput detection and counting of imperfect grains to improve the accuracy of detection and counting.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for detecting and counting imperfect grains based on improved YOLOX includes the following steps:

[0009] S1. Collect original RGB images of imperfect grains randomly arranged in a single layer with different densities, and classify and label them according to insect-eaten grains, diseased grains, broken grains, sprouted grains, moldy grains and perfect grains to establish an original image dataset.

[0010] S2. Construct an improved YOLOX network model framework that integrates attention modules, input the original image dataset into the improved YOLOX network model, and determine the learning parameters of the improved YOLOX network model;

[0011] S3. Based on the learning parameters of the improved YOLOX network model, the total count of particles with interlacing features is corrected to obtain a detection model for imperfect grains.

[0012] S4. Based on the aforementioned grain imperfect grain detection model, the imperfect grain to be detected is detected according to the actual picture or video of the grain imperfect grain to be detected, and the counting results of various imperfect grains are obtained.

[0013] Preferably, the improved YOLOX network model framework for constructing the fused attention module in S2 includes:

[0014] The YOLOX model is decomposed into a Backbone network, an FPN network, a PAN network, and a Head network.

[0015] Replace Darknet53 in the Backbone network with CSPDarknet;

[0016] Add CBAM attention modules to the three feature layers Feat1, Feat2 and Feat3 obtained from the Backbone network respectively;

[0017] The CBAM attention module is added after the upsampled feature layer in the FPN network.

[0018] Preferably, in step S3, the total count of particles with interlacing characteristics is corrected, and the correction method is as follows:

[0019] The Iou (Intersection over Union) of the particle detection results obtained by the improved YOLOX network model is calculated for each particle. Iou represents the degree of overlap between two bounding boxes. If Iou > 0.8, it is considered that an interlacing feature phenomenon has occurred, and the total number of detections is reduced by one.

[0020]

[0021] In the formula, the numerator represents the area of ​​the intersection between the predicted bounding box and the ground truth bounding box, and the denominator represents the area occupied by the predicted bounding box plus the ground truth bounding box.

[0022] Compared with existing technologies, this invention provides an online machine vision recognition technology for imperfect grains. Grain grains do not require special placement; they can all be identified in a single photograph using this method, providing an efficient grain quality inspection method for grain storage and distribution. This invention improves the YOLOX model framework by adding attention modules, resulting in an improved YOLOX model framework. This enhances the high-throughput detection and counting accuracy of fine-grained features such as multi-scale or interleaved features of imperfect grains. Constructing a YOLOX network model for fine-grained feature recognition of imperfect grains requires determining the location and number of attention modules suitable for the feature scale of imperfect grains. The improved YOLOX model framework is trained, and the total count of grains with interleaved features is corrected to obtain a grain imperfect grain detection model. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of a method for detecting and counting imperfect grains based on an improved YOLOX according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the attention mechanism module structure provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the improved YOLOX network structure provided in an embodiment of the present invention;

[0026] Figure 4 Here is an example image of a defective wheat grain to be detected according to an embodiment of the present invention;

[0027] Figure 5 This is an example image showing the identification and counting results of the improved YOLOX model in an embodiment of the present invention for the imperfect grains of wheat to be detected. Detailed Implementation

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0029] Reference Figure 1 This invention proposes a method for detecting and counting imperfect grains based on an improved YOLOX.

[0030] S1. Collect original RGB images of imperfect grains randomly arranged in a single layer with different densities, and classify and label them according to insect-eaten grains, diseased grains, broken grains, sprouted grains, moldy grains and perfect grains to establish an original image dataset.

[0031] The method for collecting grain images involves placing a random number of grains on a black background and taking images of the grains from above. To ensure sufficient training samples, 1500 images containing grains were taken, with each image containing 5-30 grains. To ensure the independence of the test set, another sample of grains was taken, with each image containing 20-50 grains, and 100 images were taken. Each image was manually labeled using Labelimg software. All images in the dataset are 1920×1080 pixels in size. The 1500 images used for training were divided into training and validation sets in a 9:1 ratio. The test set used 100 images. (Refer to...) Figure 4 , Figure 5 , Figure 4 Here is an example image of a defective wheat grain to be detected according to an embodiment of the present invention; Figure 5 This is an example image showing the identification and counting results of the improved YOLOX model in an embodiment of the present invention for the imperfect grains of wheat to be detected.

[0032] S2. Construct an improved YOLOX network model framework that integrates attention modules, input the original image dataset into the improved YOLOX network model, and determine the learning parameters of the improved YOLOX network model;

[0033] Please refer to the network model constructed in this embodiment. Figure 3 The network is divided into four parts according to its structure: Backbone network, FPN network, PAN network and Head network;

[0034] The first layer of the Backbone network is the data input layer, which receives the training image and consists of a feature layer of size (640, 640, 3). This is followed by a Focus network, which uses a 2×2 convolutional kernel to extract values ​​from the feature layer at pixel intervals. Next, it passes through two CBS (Convolution + Batch Normalization + SiLU) activation functions. Finally, it passes through a CSP network, which divides the input layer into two branches. One branch passes through one CBS, and the other branch passes through n CBS structures before concatenating the input and output. The results from these two branches are then concatenated before passing through another CBS.

[0035] The first feature extraction layer, Feature1, is obtained through the first CBS and CSP network. The second feature extraction layer, Feature2, is obtained through the second CBS and CSP network. The third feature extraction layer, Feature3, is obtained through the third CBS, SPP, and CSP network. The SPP network operates by first passing the input through a CBS network, followed by max pooling with different strides of 1, 5, 9, and 13. The results are then concatenated and passed through a CBS network once more.

[0036] The FPN network structure involves passing the three feature extraction layers (Feature1, Feature2, and Feature3) obtained from the Backbone through a CBAM module to obtain new Feature1, Feature2, and Feature3. For the specific CBAM network model, please refer to [link to CBAM]. Figure 2 Feature3 is processed through a convolution, plus sampling and CBAM. The upsampled feature layer is then concatenated with the CBAM-processed feature layer. The output is then concatenated with Feature2, followed by a CSP convolution, plus sampling and CBAM. The upsampled feature layer is then concatenated with the CBAM-processed feature layer, and the output is then concatenated with Feature3.

[0037] The PAN network structure involves passing the FPN output through one CSP to obtain the first Feature1. After one downsampling and CSP, the second Feature2 is obtained. Then, after another downsampling and CSP, the third Feature3 is obtained.

[0038] The Head network structure divides the obtained three features into two branches after passing them through a single CBS (Classification by Baseline) process. Each branch then passes through two separate CBS processes, and the results from these two branches are used to predict classification and regression coordinates, respectively.

[0039] This embodiment also employs transfer learning in the training of the convolutional neural network model, using pre-trained network weights as initial weights. The specific training process is as follows:

[0040] The training process involved 150 iterations. For the first 50 iterations, the backbone network was frozen, meaning its weights remained unchanged to accelerate convergence. After 50 iterations, the backbone network was unfrozen, at which point the weights of the entire network were adjusted. The learning rate was set to 0.0001 during training.

[0041] S3. Based on the learning parameters of the improved YOLOX network model, the total count of particles with interlacing features is corrected to obtain a detection model for imperfect grains.

[0042] In step S3, the total count of particles with interlacing characteristics is corrected. The correction method is as follows:

[0043] The Iou (Intersection over Union) of the particle detection results obtained by the improved YOLOX network model is calculated for each particle. Iou represents the degree of overlap between two bounding boxes. If Iou > 0.8, it is considered that an interlacing feature phenomenon has occurred, and the total number of detections is reduced by one.

[0044]

[0045] In the formula, the numerator represents the area of ​​the intersection between the predicted bounding box and the ground truth bounding box, and the denominator represents the area occupied by the predicted bounding box plus the ground truth bounding box.

[0046] Counting methods typically involve calculating the number of bounding boxes in a single detected image. However, considering that a single wheat grain may have multiple imperfect grain features, this means that a single grain might have multiple bounding boxes during detection. If this method is used to detect the number of imperfect grains, the total count will be inaccurate. Therefore, the algorithm described above was designed to eliminate redundant calculations. In practice, the coordinates of all detected bounding boxes are obtained, the IoU (Interchange of Value) is calculated for each, and an IoU threshold of 0.8 is set. When the IoU between two bounding boxes is greater than 0.8, it is considered that a single kernel has multiple bounding boxes, and this case is removed from the total count.

[0047] S4. Based on the aforementioned grain imperfect grain detection model, the imperfect grain to be detected is detected according to the actual picture or video of the grain imperfect grain to be detected, and the counting results of various imperfect grains are obtained.

[0048] During training, the 1500 images were divided into training and validation sets in a 9:1 ratio. An additional 100 images were used as the test set. The test results are as follows:

[0049]

[0050] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting and counting imperfect grains based on improved YOLOX, characterized in that, Includes the following steps: S1. Collect original RGB images of imperfect grains randomly arranged in a single layer with different densities, and classify and label them according to insect-eaten grains, diseased grains, broken grains, sprouted grains, moldy grains and perfect grains to establish an original image dataset. S2. Construct an improved YOLOX network model framework that integrates attention modules, input the original image dataset into the improved YOLOX network model, and determine the learning parameters of the improved YOLOX network model; S3. Based on the learning parameters of the improved YOLOX network model, the total count of particles with interlacing features is corrected to obtain a detection model for imperfect grains. S4. Based on the aforementioned grain imperfect grain detection model, the imperfect grain to be detected is detected according to the actual picture or video of the grain imperfect grain to be detected, and the counting results of various imperfect grains are obtained.

2. The method for detecting and counting imperfect grains based on improved YOLOX according to claim 1, characterized in that, The improved YOLOX network model framework for constructing the fused attention module in S2 includes: The YOLOX model is decomposed into a Backbone network, an FPN network, a PAN network, and a Head network. Replace Darknet53 in the Backbone network with CSPDarknet; Add CBAM attention modules to the three feature layers Feat1, Feat2 and Feat3 obtained from the Backbone network respectively; The CBAM attention module is added after the upsampled feature layer in the FPN network.

3. The method for detecting and counting imperfect grains based on improved YOLOX according to claim 1, characterized in that, In step S3, the total count of particles with interlacing characteristics is corrected. The correction method is as follows: The Iou (Intersection over Union) of the particle detection results obtained by the improved YOLOX network model is calculated for each particle. Iou represents the degree of overlap between two bounding boxes. If Iou > 0.8, it is considered that an interlacing feature phenomenon has occurred, and the total number of detections is reduced by one. In the formula, the numerator represents the area of ​​the intersection between the predicted bounding box and the ground truth bounding box, and the denominator represents the area occupied by the predicted bounding box plus the ground truth bounding box.

Images