Highland barley grain sorting method based on improved YOLO recognition algorithm

Abstract

The invention discloses a highland barley grain sorting method based on an improved YOLO recognition algorithm. The highland barley grain sorting method specifically comprises the steps that collected highland barley images are preprocessed; constructing a convolutional neural network by fusing a SimAM mechanism to perform feature extraction on the highland barley image, and obtaining feature maps of four different levels from the network; constructing a bidirectional feature pyramid network by using PANet, performing multi-scale fusion and feature enhancement processing on feature maps from four different scales, and extracting multi-scale feature maps of three different levels from the network; and performing target bounding box and category label prediction on the three multi-scale feature maps by using DyHead, and finally outputting specific position information and category of the highland barley. The method solves the problems of low detection efficiency and poor precision of manual detection and physical detection methods, improves the detection efficiency and accuracy of the highland barley variety, reduces the labor cost, can quickly adapt to the detection of the highland barley variety of a novel product, shortens the development period, and improves the flexibility.

Description

Technical Field

[0001] The present invention belongs to the field of detection technology, and in particular relates to a method for sorting hulless barley grains based on an improved YOLO recognition algorithm. Background Art

[0002] Currently, breeders have developed a large number of hulless barley varieties. Although these varieties can meet market demands, they also increase the risk of seed variety mixing. In the processes of seed treatment, sowing, harvesting, threshing, drying, and shipping, due to non-compliance with breeding technical regulations or operational negligence, the seed purity may decrease. This mixing problem will significantly reduce the germination rate, emergence rate, and disease resistance of hulless barley, directly affecting crop yields and farmers' incomes. Therefore, there is an urgent need to develop a non-destructive, efficient, and environmentally friendly identification method to identify high-quality hulless barley seed varieties and provide higher-purity hulless barley seeds for the market and consumers.

[0003] Currently, the methods for detecting hulless barley varieties mainly include manual detection methods, instrument-based physical detection methods, and machine vision detection methods. The problems are as follows:

[0004] Manual detection method: Relying on manual visual observation of the appearance of hulless barley varieties and combining empirical knowledge to judge the hulless barley category, this method has the problems of strong subjectivity, time-consuming and laborious, and low detection accuracy.

[0005] Instrument-based physical detection methods: Including ultrasonic detection and X-ray detection, etc. Although these methods can meet the detection requirements, their instrument equipment is large in volume, expensive in price, complex in maintenance, and has low detection efficiency and insufficient accuracy.

[0006] Machine vision detection method: It has the advantages of being non-destructive, fast, accurate, and reliable, can avoid human subjective differences and visual fatigue, reduce labor costs, and at the same time improve detection efficiency and accuracy. However, traditional machine vision methods require manual feature extraction, have poor product adaptability, and long development cycles. Summary of the Invention

[0007] In view of this, the present invention aims to propose a method for sorting hulless barley grains based on an improved YOLO recognition algorithm, relying on machine vision technology for non-destructive detection of hulless barley seeds. It can not only eliminate human subjective differences and visual fatigue, reduce labor costs, but also improve detection efficiency and accuracy, and reduce detection classification errors. Compared with traditional machine vision methods, deep learning methods can directly learn features from underlying data, have higher complex structure expression capabilities, thus completely replacing the artificial design of features with an automatic learning process, can be applicable to different products, shorten the development cycle, improve flexibility, and quickly adapt to the detection of new hulless barley varieties of new products.

[0008] To achieve the above object, the technical solution of the present invention is realized as follows:

[0009] A sorting method for hulless barley grains based on an improved YOLO recognition algorithm, comprising:

[0010] Collect hulless barley images under different backgrounds, lighting conditions, and multiple perspectives, and preprocess the collected hulless barley images;

[0011] Construct a convolutional neural network by fusing the SimAM mechanism and obtain four feature maps at different levels to improve the accuracy of hulless barley image feature representation;

[0012] Use PANet to construct a bidirectional feature pyramid network, perform multi-scale fusion and feature enhancement processing on feature maps from four different scales, and extract three multi-scale feature maps at different levels;

[0013] Use DyHead to predict the target bounding box and class label for the multi-scale feature map, and enhance the detection ability of the model under different lighting conditions through dynamic aggregation of features;

[0014] Utilize MPDIoU to optimize the bounding box regression accuracy;

[0015] Output the position information and category of the hulless barley.

[0016] Furthermore, collecting hulless barley images under different backgrounds, lighting conditions, and multiple perspectives specifically includes:

[0017] Use a CCD industrial camera to take pictures, including single background, complex background, dark, bright, soft light, and overlapping environments;

[0018] The camera is located directly above, diagonally above, or on the right side, with a viewing angle of 9 - 11 cm from the hulless barley.

[0019] Furthermore, preprocessing the collected hulless barley images specifically includes:

[0020] Perform optical transformation and geometric transformation on the hulless barley images;

[0021] Use a labeling tool to generate labeling information including position information and category;

[0022] Divide the data set into a training set, a validation set, and a test set according to a preset ratio.

[0023] Furthermore, constructing a convolutional neural network by fusing the SimAM mechanism and obtaining four feature maps at different levels includes the following steps:

[0024] Use the SimAM mechanism to generate a weight matrix and enhance the feature map;

[0025] Use CSPDarkNet as the basic network to extract multi-level high-semantic features through convolutional layers, C3 layers, and pooling layers;

[0026] Output four feature maps with sizes of 160×160, 80×80, 40×40, and 20×20 respectively.

[0027] Furthermore, use PANet to construct a bidirectional feature pyramid network for multi-scale fusion and feature enhancement of the feature maps, including:

[0028] Top-down path, generating feature maps through upsampling and feature fusion;

[0029] Bottom-up path, generating the final feature map through convolutional enhancement and feature fusion;

[0030] Lateral connection, fixing the number of channels through 1×1 convolution and then performing feature addition and enhancement processing.

[0031] Furthermore, use DyHead to predict the object bounding box and class label for the multi-scale feature maps, including:

[0032] Dynamic feature aggregation module, generating feature weights through scale-aware, space-aware, and task-aware attention mechanisms;

[0033] Use MPDIoU to optimize the bounding box regression accuracy.

[0034] Furthermore, the described DyHead generates the final bounding box and class prediction results through dynamic feature aggregation and classification convolutional layers.

[0035] Furthermore, this solution discloses an electronic device, including a processor and a memory communicatively connected to the processor and used to store instructions executable by the processor, and the processor is used to execute a sorting method for hulless barley grains based on an improved YOLO recognition algorithm.

[0036] Furthermore, this solution discloses a server, including at least one processor and a memory communicatively connected to the processor, where the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to enable the at least one processor to execute a sorting method for hulless barley grains based on an improved YOLO recognition algorithm.

[0037] Furthermore, this solution discloses a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements a sorting method for hulless barley grains based on an improved YOLO recognition algorithm.

[0038] Compared with the prior art, the method for sorting highland barley grains based on an improved YOLO recognition algorithm of the present invention has the following advantages:

[0039] (1) For the method for sorting highland barley grains based on an improved YOLO recognition algorithm of the present invention, aiming at the complex and changeable image background of highland barley in the industrial scenario, this method uses CSPDarkNet as the basic network structure, and constructs a convolutional neural network by integrating the SimAM mechanism to obtain four different hierarchical feature maps. This network pays more attention to the feature extraction of target information in the image, reduces the dependence on background information at the same time, and improves the accuracy of the network's feature representation of highland barley images;

[0040] (2) For the method for sorting highland barley grains based on an improved YOLO recognition algorithm of the present invention, in order to better capture the characteristic information of various sizes and poses presented by highland barley from different perspectives, PANet (Path Aggregation Network) is used to construct a bidirectional feature pyramid network. This network can perform multi-scale fusion and feature enhancement processing on the feature maps from four different scales, and extract three different hierarchical multi-scale feature maps from them. This can ensure that the model has stronger robustness and higher detection accuracy when processing targets of different sizes and poses;

[0041] (3) For the method for sorting highland barley grains based on an improved YOLO recognition algorithm of the present invention, in order to cope with the differences in the texture and image contrast of highland barley seeds caused by changes in illumination, DyHead is used to predict the target bounding box and class label for the three multi-scale feature maps. By dynamically selecting and aggregating multi-scale features, DyHead can effectively enhance the model's detection ability for targets under different illumination conditions, ensuring that the model can stably identify and locate highland barley seeds under various illumination conditions;

[0042] (4) For the method for sorting highland barley grains based on an improved YOLO recognition algorithm of the present invention, MPDIoU is used to optimize the regression accuracy of the bounding box, solve the problems of slow convergence speed and low accuracy of the traditional loss function for bounding box regression, and improve the accuracy recall rate of model detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 It is the overall flowchart of the method for sorting highland barley grains based on an improved YOLO recognition algorithm described in the embodiment of the present invention;

[0045] Figure 2This is a detailed step diagram of a method for sorting naked barley grains based on an improved YOLO recognition algorithm described in the embodiments of the present invention. Specific embodiments

[0046] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0047] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0048] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0049] The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0050] As Figure 1As shown in the figure, the present invention discloses a method for sorting naked barley grains based on an improved YOLO recognition algorithm, specifically as follows: collecting naked barley images under different backgrounds, lighting conditions, and various perspectives; preprocessing the collected naked barley images; constructing a convolutional neural network through integrating the SimAM mechanism to extract features from the naked barley images, and obtaining four feature maps at different levels from this network; using PANet (Path Aggregation Network) to construct a bidirectional feature pyramid network, performing multi-scale fusion and feature enhancement processing on the feature maps from four different scales, and extracting three multi-scale feature maps at different levels from this network; using DyHead to predict the target bounding boxes and class labels for the three multi-scale feature maps, and optimizing the bounding box regression accuracy by using MPDIoU during this process, and finally outputting the specific position information and its category of the naked barley.

[0051] As Figure 2 shown, the method for detecting naked barley varieties based on deep learning of the present invention is specifically as follows:

[0052] Step 1: Acquisition of naked barley images

[0053] Using a CCD industrial camera to take single-grain naked barley images under different backgrounds, lighting conditions, and various perspectives, including single backgrounds, complex backgrounds, dark, bright, soft light, and overlapping complex environments, as well as perspectives where the camera is directly above, diagonally above, and on the right side, about 9 - 11 cm away from the naked barley.

[0054] Step 2: Image preprocessing

[0055] First, perform data augmentation on the naked barley images obtained in Step 1. Data augmentation mainly includes optical transformation and geometric transformation. Among them, optical transformation includes random adjustment of brightness, random adjustment of contrast, and random adjustment of channels, and geometric transformation includes rotation, stretching, translation, horizontal flipping, vertical flipping, random cropping of pictures, and random scaling.

[0056] Then, use a labeling tool to generate corresponding labeling information for the expanded image dataset. The labeling information includes the position information and category of the naked barley in the sample, and the categories are new strain 0628, 0349 - 1, Ganqing No. 4, Ganqing No. 6, Ganqing No. 8, and Ganqing No. 9.

[0057] Finally, divide the dataset into a training set, a validation set, and a test set; among them, the training set is 80% of the total sample size, the validation set is 10%, and the test set is 10%.

[0058] Step 3: Construct a convolutional neural network for feature extraction

[0059] Construct a convolutional neural network by integrating the SimAM mechanism to extract features from hulless barley images, and obtain feature maps of four different levels from this network

[0060] (1)The SimAM structure consists of three parts: feature mapping, weight matrix generation, and feature enhancement; first, for the input feature map Calculate the mean value at each position , for the input feature map Calculate the variance at each position ; then, use the calculated mean and variance to generate a weight matrix , and generate weight values ranging from 0 to 1 through the Siamoid function; finally, multiply the generated weight matrix element-wise with the input feature map to obtain the enhanced feature map .

[0061] (2)Use CSPDarkNet to construct the basic network structure to extract high-semantic features of the image; first, pass through 5 convolutional layers and 3 C3 layers, where the kernel size of the first convolutional layer is changed from 6×6 to 3x3, add the Split operation in the C3 structure, and replace the Bottleneck with Fasterblock. In addition, in each convolutional layer, perform the SiLU function and the normalization Batch Norm2d operation; second, after the features of the important regions are weighted by the SimAM mechanism, input them into the pooling layer, which uses the modified SPPF and replaces the traditional MaxPooling operation with a 3×3 shared convolution; finally, output four different levels of feature maps with sizes of 160×160, 80×80, 40×40, and 20×20 respectively.

[0062] Step 4: Construct a bidirectional feature pyramid network for multi-scale feature fusion and enhancement

[0063] Use PANet to construct a bidirectional feature pyramid network to perform multi-scale fusion and feature enhancement on the feature maps from four different scales, and extract three different levels of multi-scale feature maps from this network

[0064] The overall architecture of this network includes three parts: a top-down path, a bottom-up path, and lateral connections;

[0065] Top-down path: After the modified SPPF output, the feature map F1 is obtained. After performing a nearest neighbor upsampling operation with a factor of 2 on the modified SPPF output feature map of 20×20 and fusing it with the CSP1_Conv3 feature map, the feature map F2 is obtained after being processed by the CSP2_Conv module. F2 is fused with the CSP1_Conv2 feature map after an upsampling operation with a factor of 2, and the feature map F3 is obtained after being processed by the CSP2_Conv module. The feature map F4 is obtained after the output of CSP1_Conv1;

[0066] Bottom-up path: F4 is fused with F3 after 3×3 convolutional feature enhancement, and then the feature map T1 is obtained through CSP2_Conv1. T1 is fused with F2 after 3×3 convolutional feature enhancement, and then the feature map T2 is obtained through CSP2_Conv2. T2 is fused with F1 after 3×3 convolutional feature enhancement, and then the feature map T3 is obtained through CSP2_Conv1.

[0067] Lateral connection: A 1×1 convolutional operation is performed on the feature map to make the fixed number of channels 512 or 256, and it is added element-wise to the high-semantic features of upsampling or 3×3 convolutional feature enhancement. After the number of channels is fixed by CSP2_Conv, the feature maps T1, T2, and T3 are obtained.

[0068] Step 5: Prediction of object bounding boxes and classes

[0069] DyHead is used to predict the object bounding boxes and class labels for three multi-scale feature maps

[0070] The DyHead detection head mainly includes four parts: a dynamic feature aggregation module, bounding box regression, class prediction, and forward propagation;

[0071] Dynamic feature aggregation module: Scale-aware, space-aware, and task-aware attention mechanisms are respectively applied to the feature tensors in each dimension to dynamically adjust the importance weights in the feature map. The calculation formulas for the attention weights in the three dimensions are as follows:

[0072]

[0073] In the formula: is the attention weight matrix of the feature layer M; M is the feature layer; is the scale-aware attention operation; is the space-aware attention operation; is the task-aware attention operation.

[0074] Bounding box regression: A regression convolutional layer is used to predict the coordinates of the object bounding box, and MPDIoU is used to optimize the boundary regression loss.

[0075] Category prediction: A classification convolutional layer is used to perform classification prediction on the target, and the cross-entropy loss is calculated to optimize the category prediction.

[0076] Forward propagation: After the three input multi-scale feature maps are processed sequentially through multiple dynamic feature aggregation modules, the final bounding box and category prediction results are generated through the regression convolutional layer and the classification convolutional layer.

[0077] Step 6: Use MPDIoU to optimize the regression accuracy of the bounding box

[0078] A new bounding box similarity comparison metric based on the minimum point distance directly minimizes the distances between the upper-left and lower-right points of the predicted bounding box and the actual annotated bounding box. The calculation formula is as follows:

[0079]

[0080]

[0081]

[0082] In the formula, is the coordinate of the upper-left point of the actual bounding box, is the coordinate of the lower-right point of the actual bounding box, is the coordinate of the upper-left point of the predicted bounding box, is the coordinate of the lower-right point of the predicted bounding box, w is the width of the image, and h is the height of the image.

[0083] Step 7: Output the position information and category of the highland barley

[0084] In the present invention, Fasterblock is used to replace the Bottleneck of the C3 module in the CSPDarkNet network, and in the subsequent pooling process, a 3×3 shared convolution is used to replace the traditional MaxPooling operation, which retains more feature information while improving the calculation efficiency, effectively improving the detection efficiency.

[0085] The present invention uses the basic network structure of CSPDarkNet for feature extraction, and at the same time introduces the SimAM mechanism to enhance the network's attention to target information. When dealing with the complex and changeable image background in the industrial environment, this network can effectively focus on the target information in the image, reduce the dependence on background information, and thus improve the accuracy of the feature representation of highland barley images.

[0086] The present invention uses PAN-FPN for multi-scale feature fusion prediction, extracts higher semantic information, improves the detection accuracy, enhances the generalization ability of the model, and improves the flexibility.

[0087] The present invention uses PANet (Path Aggregation Network) to construct a bidirectional feature pyramid network, and performs multi-scale fusion and feature enhancement processing on feature maps of four different scales. This network can better capture the feature information of various sizes and poses of highland barley presented from different perspectives, ensuring that the model has stronger robustness and higher detection accuracy when dealing with highland barley of different sizes and poses.

[0088] The present invention uses DyHead to predict the object bounding boxes and class labels for three multi-scale feature maps. By dynamically selecting and aggregating multi-scale features, DyHead can effectively enhance the model's detection ability for objects under different lighting conditions, ensuring that the model can stably identify and locate highland barley seeds under various lighting conditions.

[0089] The present invention uses MPDIoU to optimize the bounding box regression accuracy, solves the problems of slow convergence speed and low accuracy of the traditional loss function for bounding box regression, and improves the accuracy and recall rate of model detection.

[0090] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A highland barley grain sorting method based on an improved YOLO recognition algorithm, characterized in that: include: Collect highland barley images under different backgrounds, lighting conditions and multiple viewing angles, and pre-process the collected highland barley images; By integrating the SimAM mechanism, a convolutional neural network is constructed and feature maps of four different levels are obtained to improve the accuracy of highland barley image feature representation; PANet is used to construct a bidirectional feature pyramid network, which performs multi-scale fusion and feature enhancement processing on feature maps from four different scales, and extracts multi-scale feature maps at three different levels; Use DyHead to predict target bounding boxes and category labels for multi-scale feature maps, and enhance the model's detection capabilities under different lighting conditions by dynamically aggregating features; Use MPDIoU to optimize bounding box regression accuracy; Output the location information and category of highland barley.

2. A highland barley grain sorting method based on an improved YOLO recognition algorithm according to claim 1, characterized in that: The barley images collected under different backgrounds, lighting conditions and multiple viewing angles are as follows: Use CCD industrial camera to shoot, including single background, complex background, dark, bright, soft light and overlapping environment; The camera is located directly above, diagonally above or to the right, with a viewing angle of 9-11 cm from the barley.

3. A highland barley grain sorting method based on an improved YOLO recognition algorithm according to claim 1, characterized in that: The preprocessing of the collected highland barley images is as follows: Perform optical transformation and geometric transformation on highland barley images; Use annotation tools to generate annotation information including location information and categories; The dataset is divided into training set, validation set and test set according to preset proportions.

4. A highland barley grain sorting method based on an improved YOLO recognition algorithm according to claim 1, characterized in that: By integrating the SimAM mechanism, a convolutional neural network is constructed and feature maps of four different levels are obtained, including the following steps: Use SimAM mechanism to generate weight matrix and enhance feature map; CSPDarkNet is used as the basic network to extract multi-level high-level semantic features through convolutional layers, C3 layers, and pooling layers; The output sizes are 160×160, 80×80, 40×40, and 20×20.

5. A method for sorting highland barley grains based on an improved YOLO recognition algorithm according to claim 1, characterized in that: PANet is used to build a bidirectional feature pyramid network to perform multi-scale fusion and feature enhancement processing on feature maps, including: The top-down path generates feature maps through upsampling and feature fusion; The bottom-up path generates the final feature map through convolution enhancement and feature fusion; Connect horizontally, fix the number of channels through 1×1 convolution, and then perform feature addition and enhancement.

6. A highland barley grain sorting method based on an improved YOLO recognition algorithm according to claim 1, characterized in that: Use DyHead to predict object bounding boxes and category labels for multi-scale feature maps, including: Dynamic feature aggregation module, which generates feature weights through scale-aware, space-aware, and task-aware attention mechanisms; Use MPDIoU to optimize bounding box regression accuracy.

7. A highland barley grain sorting method based on an improved YOLO recognition algorithm according to claim 1, characterized in that: The DyHead generates the final bounding box and category prediction results through dynamic feature aggregation and classification convolutional layers.

8. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute a highland barley grain sorting method based on an improved YOLO recognition algorithm as described in any one of claims 1 to 7.

9. A server, characterized in that: It includes at least one processor and a memory connected to the processor in communication, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes a highland barley grain sorting method based on an improved YOLO recognition algorithm as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a method for sorting highland barley grains based on an improved YOLO recognition algorithm as described in any one of claims 1 to 7 is implemented.

Images