A wheat seed screening method based on CS-DNet lightweight model

Abstract

The present invention discloses a wheat seed screening method based on a CS-DNet lightweight model, which collects wheat seed image data required for the experiment in advance, and classifies the pictures into a data set by a manual annotation method; constructs a wheat screening network based on the CS-DNet lightweight model, including a feature extraction module FEM and a feature fusion module FFM; wherein the feature extraction module FEM adopts channel segmentation, depth-separable convolution of different scales, and residual connection operations; the feature fusion module FFM includes a hole convolution channel shuffling module C-DBlock and an enhanced inverted residual structure EIRS; C-DBlock realizes the sharing of channel information and spatial information; EIRS is a dual-branch network structure, which realizes the fusion of two branch feature vectors; obtains the model parameters with the best effect through multiple iterations; and classifies wheat seeds using a pre-trained network model. The present invention can realize accurate screening of wheat seeds in a short time, and has great prospects in the field of agricultural crop seed screening.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision image processing and crop seed screening, and specifically relates to a wheat seed screening method based on a CS-DNet lightweight model. Background Art

[0002] Wheat is one of China's main grain crops, and wheat planting areas are widely distributed throughout the country, including the Northeast, North China, Northwest and other regions. Its production volume and quality directly affect China's food supply and farmers' livelihoods. Yield and quality are also the two key factors affecting the economic benefits of wheat. Insect-damaged grains, germinated grains, moldy grains, damaged grains, and diseased grains in wheat directly affect yield and quality. Therefore, wheat screening is crucial to ensure wheat quality and yield. Screening unqualified grains in wheat can not only ensure the safety of subsequent food production, but also ensure that there are no unqualified wheat seeds containing pathogens or pests during the planting process, which will cause pollution and damage to the soil and environment.

[0003] The traditional manual screening method of wheat in my country is one of the most common methods. Workers classify and screen wheat seeds through visual inspection and manual operation, and divide seeds into different grades or categories according to the size, color, shape, integrity and other characteristics of seeds. However, traditional manual screening has disadvantages such as high labor intensity, low efficiency, low accuracy and high cost. It is difficult to meet the needs of modern agricultural production and is easily affected by subjective factors. Therefore, an automated, efficient and accurate screening method is needed to help improve the quality and quality of wheat, thereby improving economic benefits. The use of image processing technology for wheat seed screening can realize automated detection and sorting, greatly improving the screening efficiency. Compared with traditional manual screening methods, image processing technology can quickly and accurately screen a large number of seeds, saving labor costs and time costs. Summary of the invention

[0004] Purpose of the invention: The present invention proposes a wheat seed screening method based on the CS-DNet lightweight model to solve the problem of low recognition rate of unqualified wheat seeds, realize lightweight deployment of wheat seed screening, and improve recognition accuracy and speed.

[0005] Technical solution: The wheat seed screening method based on the CS-DNet lightweight model described in the present invention comprises the following steps:

[0006] (1) The pre-acquired wheat seed data images are classified according to grain type and made into a dataset, and the dataset is expanded through data enhancement;

[0007] (2) Divide the data set obtained in step (1) into a training set and a validation set, and use different strategies to perform image preprocessing;

[0008] (3) Construct a wheat screening network based on the CS-DNet lightweight model, including a feature extraction module FEM and a feature fusion module FFM; the feature extraction module FEM adopts channel segmentation, depth-separable convolution of different scales and residual connection operations; the feature fusion module FFM includes a hole convolution channel shuffle module C-DBlock and an enhanced inverted residual structure EIRS; the C-DBlock realizes the sharing of channel information and spatial information; the EIRS is a dual-branch network structure to realize the fusion of two branch feature vectors;

[0009] (4) The wheat screening network based on the CS-DNet lightweight model built in the training step (3) is first input into the built model through model training after image preprocessing in step (2) for feature extraction. Through continuous iteration, forward and backward propagation processes, the model parameters are then updated through the stochastic gradient descent optimizer to optimize the network parameters. Finally, the trained results are output through the SoftMax classifier.

[0010] Furthermore, the types of grains in step (1) include germinated grains, moldy grains, worm-eaten grains, diseased grains and broken grains.

[0011] Furthermore, in step (2), the training set is processed by random cropping and random horizontal flipping; and the verification set is processed by center cropping and short edge scaling.

[0012] Furthermore, the feature extraction module FEM implementation process in step (3) is as follows:

[0013] The input features are first passed through a 3×3 convolution, and then the feature vector is divided into two parts through channel segmentation. The first part uses 5×5 convolution and 3×3 depth-wise separable convolution, and the second part uses 5×5 convolution and 5×5 depth-wise separable convolution. The two feature vectors are then concatenated along the channel dimension to obtain a new feature vector; then the first residual connection is to connect the feature vector obtained after the dual-branch module with the original feature vector; the second residual connection is to connect the feature vector just obtained with the original feature vector whose channel is expanded and size is reduced after a 3×3 convolution.

[0014] Furthermore, the feature fusion module FFM in step (3) is stacked by two feature fusion modules FFMA and one feature fusion module FFMB, wherein FFMA includes a C-DBlock and two layers of EIRS, and FFMB includes a C-DBlock and an EIRS.

[0015] Furthermore, the C-DBlock in step (3) is a dual-branch network structure. The main branch first uses a 1×1 convolution, followed by a channel shuffle operation, and then a 3×3 depth-separable convolution. A 1×1 convolution is used at the end of the main branch. The branch is composed of a hole convolution and a 1×1 convolution in series, and the dilation rate and padding of the hole convolution are (1, 2).

[0016] Furthermore, in step (3), the EIRS first separates the input into two parts by channel separation. The first part first passes through a 1×1 convolution, then uses a 5×5 depthwise separable convolution, and finally uses a 1×1 convolution again. The feature vector of the second part after channel separation is channel-joined with the feature vector of the first part after the above operation through a channel attention mechanism.

[0017] Furthermore, the first feature fusion FFMA implementation process is as follows:

[0018] The main branch of C-DBlock uses 44 1×1 ordinary convolutions, batch normalization, and ReLU activation functions on the input feature vector; then it undergoes a channel shuffle operation; then a depth-wise separable convolution of 3×3 and a stride of 2 is used, followed by batch normalization; the last step uses 58 1×1 ordinary convolutions, batch normalization, and ReLU activation functions again; the feature vector obtained by the main branch operation is (28, 28, 88); the branch branch uses 58 Dilated rata=[1,2],padding=(1,2), 3×3 dilated convolution; then 58 1×1 ordinary convolutions, batch normalization and ReLU activation functions are used; EIRS first splits the input feature vector into channels, followed by two branches. The main branch uses 44 1×1 ordinary convolutions, batch normalization and Hswish activation functions, followed by 5×5 depth-separable convolutions, 44 1×1 ordinary convolutions, batch normalization and Hswish activation functions, and the branch branch only uses SE modules; the feature vector passes through EIRS again to obtain a feature vector size of (28, 28, 88).

[0019] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. The feature extraction module uses a channel segmentation operation. By segmenting the input data into different channels, the network can learn multiple feature representations at the same time, which helps improve the network's ability to understand and express data, while reducing the number of parameters in the network. The module also adds residual connections to alleviate the gradient vanishing problem, simplify the network model, and improve network performance;

[0021] 2. In the C-DBlock structure, both dilated convolution and channel shuffling are used, which can ensure the sharing of spatial information as well as channel information, enhance the network's receptive field, improve the model's perception ability and parameter utilization, and increase the network's nonlinearity. This operation ensures the model's accuracy while making the model lighter.

[0022] 3. The EIRS module adds channel separation operation and channel attention module on the basis of inverted residual, and expands the scale of the deep separable convolution; the expansion of the scale of the deep separable convolution improves the network's perception of local features. The use of channel segmentation and SE modules can not only improve the generalization ability of the network but also help to extract rich feature representations. The local perception of the deep separable convolution can help the network better capture subtle changes in features.

[0023] 4. The network model proposed in the present invention can achieve a recognition efficiency of 99.18%, which is higher in accuracy and has fewer parameters than traditional classification models. The reduction in parameters also makes the model easier to deploy on mobile devices. It has strong practical application value and great prospects in the field of crop seed screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of the present invention;

[0025] Figure 2 The network diagram of wheat screening method based on CS-DNet lightweight model;

[0026] Figure 3 Schematic diagram of the enhanced inverted residual structure EIRS module structure;

[0027] Figure 4 It is a structural diagram of feature extraction module;

[0028] Figure 5 Schematic diagram of the C-DBlock structure of the dilated convolution channel shuffle module. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0030] like Figure 1 As shown, the present invention proposes a wheat seed screening method based on the CS-DNet lightweight model, which specifically includes the following steps:

[0031] Step 1: Classify and preprocess the acquired wheat seed data images according to disease types, and expand the data set through data enhancement. Finally, divide the expanded data set into training set and validation set in a ratio of 8:2.

[0032] The dataset contains five kinds of wheat seeds, with a total of 2104 original data, including sprouted, moldy, diseased, worm-eaten, and damaged wheat seeds. The dataset expanded through data augmentation has a total of 5000 data images.

[0033] The dataset was preprocessed, and the sample data was divided into a training set and a validation set in a ratio of 8:2. Different data augmentation operations were performed on the training set and the validation set. For the data in the training set, all images were first randomly cropped to a size of 224×224, and then the images were randomly flipped horizontally. For the images in the validation set, the short side of the image was first scaled to 256 pixels, and then the image was cropped from the center to 224×224 pixels.

[0034] Step 2: If Figure 2 As shown in the figure, a wheat screening network based on the CS-DNet lightweight model is constructed, including a feature extraction module FEM and a feature fusion module FFM; FFM includes a hole convolution channel shuffle module (C-DBlock) and an enhanced inverted residual structure (EIRS).

[0035] like Figure 4 As shown in the figure, the feature extraction module uses channel segmentation, depthwise separable convolutions of different scales, and residual connection operations. The feature fusion module FFM is composed of two feature fusion modules FFMA and one feature fusion module FFMB stacked together, where the feature fusion module FFMA consists of a C-DBlock and two layers of EIRS, and the feature fusion module B consists of a C-DBlock and EIRS. C-DBlock is a dual-branch network structure, where the main branch is channel shuffle, and the branch branch uses hole convolution to realize the sharing of channel information and spatial information respectively. Figure 3 As shown in the figure, EIRS is also a dual-branch network structure. First, the feature vector is split by channel. The main branch uses a 5×5 depth-separable convolution, and the branch branch does not perform any operation. Finally, the feature vectors of the two branches are fused. The entire feature fusion module contains three C-DBlocks and five EIRS modules.

[0036] Constructing feature extraction module: This module first uses 3×3 convolution, and then divides the feature vector into two parts through channel segmentation. The first part uses 5×5 convolution and 3×3 depthwise separable convolution, and the second part uses 5×5 convolution and 5×5 depthwise separable convolution. Then the two outputs are spliced ​​along the channel dimension to obtain a new feature vector. Then the first residual connection is to connect the feature vector obtained after the double-branch module with the original feature vector. The second residual connection is to connect the output just obtained with the original feature vector whose channel is expanded and size is reduced after a 3×3 convolution. The input feature vector first passes through a convolution layer, in which a total of 24 convolution kernels are used, the convolution kernel size is 3×3, and the stride is 2, and then batch normalization and ReLU function are used. At this time, the feature vector is converted from the input (224, 224, 3) to (112, 112, 24). Next, channel segmentation is used to split the feature vector into two feature vectors of size (112, 112, 14) and (112, 112, 10), which enter different branches and use convolution kernels of different sizes. The first layer of the main branch uses 5×5 convolution, batch normalization and Relu activation function, and the second layer uses 3×3 depth-separable convolution, batch normalization and HSwish activation function. The first layer of the branch branch uses 5×5 convolution, batch normalization and Relu activation function, and the second layer uses 5×5 depth-separable convolution, batch normalization and HSwish activation function. Then the feature vectors obtained from the two branches are spliced ​​along the channel dimension. Compared with ordinary multi-branch modules, channel separation can effectively reduce the amount of calculation, reduce the number of parameters in the network, and improve the computational efficiency of the network. It also helps to reduce the risk of overfitting and improve the generalization ability of the model. The concatenated feature vector is residually connected with the initial feature vector after the first convolution layer, and then the new feature vector is subjected to maximum pooling. The size of the feature vector after pooling is (56, 56, 24). Finally, the obtained feature vector is residually connected with the initial feature vector. Due to the mismatch in size, the initial feature vector uses ordinary convolution to expand the channel and reduce the feature size of the feature vector before the residual connection. Compared with ordinary convolution for feature extraction, the depth-separable convolution has lower parameters and computational complexity. The use of residual connection helps to retain important information of the original input. This helps to reduce the loss of information when it is transmitted in the network, improves the network's utilization efficiency of input information, and thus improves network performance. Finally, the size of the feature vector obtained by the feature extraction module is (56, 56, 24).

[0037] The first feature fusion module FFMA, that is, the first stacking of C-DBlock and EIRS modules, is as follows: Figure 5As shown in the figure, C-DBlock is a dual-branch. The main branch first uses a 1×1 convolution, followed by a channel shuffle operation, and then a 3×3 depth-separable convolution. At the end of the main branch, a 1×1 convolution is used. The branch is composed of a dilated convolution and a 1×1 convolution in series. The dilation rate and padding of the dilated convolution are (1, 2). EIRS is an improved inverted residual structure. In the first step, the input is channel-separated into two parts. The first part first passes through a 1×1 convolution, followed by a 5×5 depth-separable convolution, and finally a 1×1 convolution is used again. The feature vector of the second part that has been channel-separated is channel-joined with the feature vector of the first part that has been operated above through the channel attention mechanism. The first stacking includes one C-DBlock and two EIRS modules. C-DBlock contains two branches. The main branch uses 44 1×1 ordinary convolutions, batch normalization, and ReLU activation functions on the input feature vector; then it undergoes channel shuffle operation; then it uses a 3×3 depthwise separable convolution with a stride of 2, followed by batch normalization; the last step uses 58 1×1 ordinary convolutions, batch normalization, and ReLU activation functions again. The feature vector obtained by the main branch operation is (28,28,88). The branch branch uses 58 dilated rata=[1,2], padding=(1,2), and 3×3 dilated convolutions; followed by 58 1×1 ordinary convolutions, batch normalization, and ReLU activation functions; EIRS first splits the input feature vector into channels, followed by two branches. The main branch uses 44 1×1 ordinary convolutions, batch normalization, and Hswish activation functions, followed by 5×5 depth-separable convolutions, 44 1×1 ordinary convolutions, batch normalization, and Hswish activation functions. The branch branch only uses SE modules. The feature vector passes through the EIRS module again to obtain a feature vector size of (28, 28, 88).

[0038] The second feature fusion module FFMA is the second stacking of C-DBlock and EIRS modules: the obtained feature vector first passes through C-DBlock, and the main branch uses 88 1×1 ordinary convolutions, batch normalization, and ReLU activation functions on the input feature vector; then it goes through the channel shuffle operation; then uses a depth-separable convolution of size 3×3 and stride 2, and batch normalization operation; the last step uses 176 1×1 ordinary convolutions, batch normalization, and ReLU activation functions again. The feature vector obtained by the main branch operation is (14, 14, 176). The branch branch uses 58 dilated rata = [1, 2], padding = (1, 2), and a 3×3 dilated convolution; then 116 ordinary convolutions of size 1×1, batch normalization, and ReLU activation function are used; then the EIRS module is used for the first time to split the input feature vector into channels, followed by two branches. The main branch uses ordinary convolutions of size 1×1, batch normalization, and Hswish activation function, followed by a 3×3 depth-separable convolution, 88 ordinary convolutions of size 1×1, batch normalization, and Hswish activation function, and the branch branch uses the SE module. Finally, the feature vector (14, 14, 232) is obtained by the EIRS module for the second time.

[0039] Feature fusion module FFMB, that is, the third stacking of C-DBlock and EIRS modules: the obtained feature vector first passes through C-DBlock, and the main branch uses 352 1×1 ordinary convolutions, batch normalization, and ReLU activation functions on the input feature vector; then it goes through a channel shuffle operation; then a depth-separable convolution of size 3×3 and stride 2 is used, followed by batch normalization; the last step uses 352 1×1 ordinary convolutions, batch normalization, and ReLU activation functions again. The feature vector obtained by the main branch operation is (7, 7, 352). The branch branch uses 352 dilated rata = [1, 2], padding = (1, 2), and 3×3 dilated convolutions; then uses 176 1×1 ordinary convolutions, batch normalization, and ReLU activation functions; the input feature vector is channel-segmented for the first time through the EIRS module, followed by two branches. The main branch uses 1×1 ordinary convolutions, batch normalization, and Hswish activation functions, followed by 3×3 depth-separable convolutions, 176 1×1 ordinary convolutions, batch normalization, and Hswish activation functions, and the branch branch uses the SE module. Finally, the feature vector (7, 7, 352) is obtained.

[0040] Step 3: The feature vector obtained by the feature extraction module and the feature extraction module is input into the ordinary convolution layer with kernel_size=1 and stride=1, the global average pooling and the fully connected layer. Finally, the output of the model is converted into the category probability distribution by constructing a Softmax classifier. These raw scores are processed and converted into the probability distribution of each category.

[0041] Step 4: Train the wheat screening network based on the CS-DNet lightweight model. First, input the training set after image preprocessing into the built network model through model training for feature extraction. Through continuous iteration of the forward and backward propagation process, the model parameters are updated through the stochastic gradient descent algorithm to optimize the network parameters. Finally, the trained results are output through the SoftMax classifier.

[0042] The network model is trained for multiple iterations, and the training parameters of the model are adjusted: learning rate (or lr), batch size (batch_size), number of iterations (epoch), and optimizer until the best model parameters are obtained.

[0043] The best training model weight is used as the pre-training model to verify the data images in the validation set, and the performance of the model is evaluated by four indicators: accuracy, precision, recall, and F1-Measure.

[0044] The calculation formulas for each indicator are as follows:

[0045]

[0046]

[0047]

[0048]

[0049] Among them, TP (True Positives): true positive examples, predicted as positive examples and actually positive examples; FP (False Positives): false positive examples, predicted as positive examples but actually negative examples; FN (false Negatives): false negative examples, predicted as negative examples but actually positive examples; TN (True Negatives): true negative examples, predicted as negative examples but actually negative examples.

[0050] Table 1 Comparison of the experimental results of the present invention with those of the traditional model and the lightweight model

[0051] Modelname Accuracy Model size Vgg16 97.30％ 525.54MB GoogleNet 97.21％ 26.70MB Mobilenet-V2 88.08％ 13.37MB CS-DNet 99.18％ 6.29MB

[0052] The experimental results of the wheat screening network model based on the CS-DNet lightweight model proposed in the present invention are shown in Table 1. It can be seen that the recognition efficiency of the model proposed in the present invention reaches 99.18%, and the model size is only 6.29M. Compared with the traditional classification model, the model has fewer parameters, higher accuracy, stronger generalization ability and practical application value, and has great prospects in the field of crop seed screening.

Claims

1. A wheat seed screening method based on the CS-DNet lightweight model, characterized in that: The following steps are involved: (1) The pre-acquired wheat seed data images are classified according to grain type and made into a dataset, and the dataset is expanded through data enhancement; (2) Divide the data set obtained in step (1) into a training set and a validation set, and use different strategies to perform image preprocessing; (3) Construct a wheat screening network based on the CS-DNet lightweight model, including a feature extraction module FEM and a feature fusion module FFM; The feature extraction module FEM adopts channel segmentation, depth-separable convolution of different scales and residual connection operations; the feature fusion module FFM includes a hole convolution channel shuffle module C-DBlock and an enhanced inverted residual structure EIRS; the C-DBlock realizes the sharing of channel information and spatial information; the EIRS is a dual-branch network structure to realize the fusion of two branch feature vectors; (4) Training the wheat screening network based on the CS-DNet lightweight model built in step (3), first input the training set after image preprocessing in step (2) into the built model through model training for feature extraction, and then update the model parameters through continuous iteration, forward and backward propagation processes through the stochastic gradient descent optimizer to optimize the parameters of the network, and finally output the trained results through the SoftMax classifier; The feature extraction module FEM implementation process in step (3) is as follows: The input features are first passed through a 3×3 convolution, and then the feature vector is divided into two parts through channel segmentation. The first part uses 5×5 convolution and 3×3 depth-separable convolution, and the second part uses 5×5 convolution and 5×5 depth-separable convolution. The two feature vectors are then concatenated along the channel dimension to obtain a new feature vector; then the first residual connection is to connect the feature vector obtained after the dual-branch module with the original feature vector; the second residual connection is to connect the feature vector just obtained with the original feature vector whose channel is expanded and size is reduced after a 3×3 convolution; Step (3) The feature fusion module FFM is composed of two feature fusion modules FFMA and one feature fusion module FFMB stacked together, wherein FFMA includes a C-DBlock and two layers of EIRS, and FFMB includes a C-DBlock and an EIRS; Step (3) The C-DBlock is a dual-branch network structure. The main branch first uses a 1×1 convolution, followed by a channel shuffle operation, and then a 3×3 depth-separable convolution. A 1×1 convolution is used at the end of the main branch. The branch branch is composed of a hole convolution and a 1×1 convolution in series, and the dilation rate and padding of the hole convolution are (1, 2). In step (3), the EIRS first separates the input into two parts by channel separation. The first part first passes through a 1×1 convolution, then uses a 5×5 depthwise separable convolution, and finally uses a 1×1 convolution again. The feature vector of the second part after channel separation is channel-joined with the feature vector of the first part after the above operation through a channel attention mechanism; The implementation process of the first feature fusion module FFMA is as follows: The main branch of C-DBlock uses 44 1×1 ordinary convolutions, batch normalization, and ReLU activation functions on the input feature vector; then it undergoes a channel shuffle operation; then a depth-wise separable convolution of 3×3 and a stride of 2 is used, followed by batch normalization; the last step uses 58 1×1 ordinary convolutions, batch normalization, and ReLU activation functions again; the feature vector obtained by the main branch operation is (28, 28, 88); the branch branch uses 58 Dilated rata=[1,2],padding=(1,2), 3×3 dilated convolution; then 58 1×1 ordinary convolutions, batch normalization and ReLU activation functions are used; EIRS first splits the input feature vector into channels, followed by two branches. The main branch uses 44 1×1 ordinary convolutions, batch normalization and Hswish activation functions, followed by 5×5 depth-separable convolutions, 44 1×1 ordinary convolutions, batch normalization and Hswish activation functions, and the branch branch only uses SE modules; the feature vector passes through EIRS again to obtain a feature vector size of (28, 28, 88).

2. A wheat seed screening method based on the CS-DNet lightweight model according to claim 1, characterized in that: The types of grains in step (1) include germinated grains, moldy grains, worm-eaten grains, diseased grains and broken grains.

3. A wheat seed screening method based on the CS-DNet lightweight model according to claim 1, characterized in that: In step (2), the training set is processed by random cropping and random horizontal flipping; the verification set is processed by center cropping and short edge scaling.

Images