A method and device for classifying and identifying varieties of japonica rice grains

Abstract

A method and device for classifying and identifying japonica rice grain varieties, belonging to deep learning neural networks, and particularly relating to classifying and identifying japonica rice grain varieties; the method solves the problems that the existing image classification processing model has high parameter complexity, low classification accuracy, cannot avoid overfitting, slow convergence speed, and cannot meet the requirements of japonica rice grain classification in terms of performance, resulting in the current use of traditional classification methods such as manual methods, chemical methods and spectral technology to evaluate or classify japonica rice grains, which is expensive, time-consuming, inefficient, and insufficient in precision; the method comprises: a step of inputting an RGB image of japonica rice grains to be classified after data preprocessing into a trained J‑Rice‑ResNeXt model to obtain a classification and identification result of the japonica rice grains; the method and device for classifying and identifying japonica rice grain varieties are suitable for classifying and identifying japonica rice grain varieties.

Description

Technical Field

[0001] The present invention relates to a deep learning neural network, and in particular to variety classification and identification of japonica rice grains. Background Art

[0002] As the staple food of nearly half of the world's population, rice provides people with rich nutrients; and japonica rice is deeply loved by consumers for its excellent rice quality and unique taste, making it one of the important crops for food supply.

[0003] Different varieties of japonica rice have significant differences in growth cycle, disease and pest resistance, yield potential, rice quality and other qualities; these differences have important impacts on subsequent yield and economic benefit analysis as well as food processing and production.

[0004] Therefore, it is of great practical significance to accurately identify japonica rice varieties suitable for the climate conditions, soil characteristics and market demand of specific regions through scientific classification, thereby optimizing the planting structure and improving agricultural production efficiency.

[0005] However, the current methods for classifying and identifying japonica rice varieties are mainly traditional classification methods such as manual methods, chemical methods and spectral technology.

[0006] Traditional classification methods such as manual methods, chemical methods and spectral technology are used to evaluate or classify japonica rice grains, which is not only expensive, time-consuming, inefficient and lacks precision.

[0007] In recent years, classification technology based on image processing and computer vision has gradually been widely used in agriculture. Compared with traditional classification methods such as manual methods, chemical methods and spectral technology, classification technology based on image processing and computer vision has the advantages of non-destructive evaluation and low cost, and has more advantages in large-scale application.

[0008] Currently, many digital image features have been used for food crop classification and quality evaluation, such as geometric parameters (length, circumference, etc.), breakage rate, whiteness and cracks.

[0009] However, compared with other grain crops, different varieties of japonica rice have high similarities in grains, which poses a great challenge to the classification performance of the model.

[0010] The existing image classification processing models have high parameter complexity, low classification accuracy, unavoidable overfitting, and slow convergence speed. In terms of performance, they cannot meet the requirements of japonica rice grain classification. Therefore, although classification technologies based on image processing and computer vision have been widely used in other crop fields, there is a lack of relevant research on how to apply classification technologies based on image processing and computer vision to the classification of japonica rice grains. As a result, traditional classification methods such as manual methods, chemical methods, and spectral technologies are still used to evaluate or classify japonica rice grains, which are expensive, time-consuming, inefficient, and lack precision. Summary of the invention

[0011] The present invention proposes a method and device for classifying and identifying varieties of japonica rice grains, which solve the problems that the existing image classification processing model has high parameter complexity, low classification accuracy, unavoidable overfitting, slow convergence speed, and cannot meet the requirements of japonica rice grain classification in terms of performance, resulting in a lack of relevant research on applying classification technology based on image processing and computer vision to the classification of japonica rice grains, which in turn leads to the current use of traditional classification methods such as manual methods, chemical methods and spectral technologies to evaluate or classify japonica rice grains, which is expensive, time-consuming, inefficient, and lacks precision.

[0012] The method for classifying and identifying varieties of japonica rice grains described in the present invention has the following technical solutions:

[0013] The method comprises the following steps:

[0014] A step for obtaining an RGB image of the collected japonica rice grains to be classified;

[0015] A step for performing data preprocessing on the RGB image of the japonica rice grains to be classified;

[0016] Steps for obtaining a trained J-Rice-ResNeXt model; where:

[0017] The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include:

[0018] Keep the backbone network framework of the ResNeXt50 model;

[0019] Replace the 1×1Conv used to increase the dimension with 1×1GConv;

[0020] Replace the 1×1Conv in downsampling with 3×3DWConv;

[0021] Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model;

[0022] The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features;

[0023] The step is used to input the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains.

[0024] Further, a preferred embodiment is provided, wherein the data preprocessing comprises:

[0025] Remove useless images from the collected RGB images of japonica rice grains to be classified;

[0026] The target area of ​​japonica rice grains in RGB images was extracted using the threshold segmentation method.

[0027] Further, a preferred embodiment is provided, wherein the threshold segmentation method is used to extract the target area of ​​japonica rice grains in the RGB image, including:

[0028] Obtain its grayscale image according to the RGB image;

[0029] According to the given threshold, the pixels in the grayscale image are divided into two categories: japonica rice grain target pixels and background pixels:

[0030] The grayscale values ​​of pixels whose grayscale values ​​are lower than a given threshold are set to 0, and the grayscale values ​​of pixels whose grayscale values ​​are higher than a given threshold are set to 1, so as to obtain a binary image; wherein, the pixels whose grayscale values ​​are set to 0 are black, representing background pixels; and the pixels whose grayscale values ​​are set to 1 are white, representing target pixels of japonica rice grains;

[0031] The binary image is used as a mask image and multiplied pixel by pixel with the RGB image, so that the pixels in the RGB image multiplied with the pixels with a gray value of 1 in the mask image are retained, and the pixels in the RGB image multiplied with the pixels with a gray value of 0 in the mask image are set to black, and the RGB image with the background area removed is obtained;

[0032] The contour extraction algorithm is used to extract the edge of the target pixel distribution area of ​​japonica rice grains in the RGB image with the background area removed, and the target area of ​​japonica rice grains is obtained.

[0033] Furthermore, a preferred implementation is provided, wherein the given threshold is 0.4.

[0034] Further, a preferred implementation is provided, in the training process of the J-Rice-ResNeXt model:

[0035] The image size of the training set input to the J-Rice-ResNeXt model is set to 224 × 224;

[0036] The number of training rounds is 100;

[0037] Batch Size is set to 32;

[0038] The base learning rate is set to 0.01;

[0039] The optimizer uses SGD.

[0040] Further, a preferred implementation is provided, in the training process of the J-Rice-ResNeXt model:

[0041] Adopt 86-Gradient learning rate dynamic adjustment strategy;

[0042] The formula for 86-Gradient is:

[0043] ;

[0044] is the learning rate of the nth round, is the initial learning rate, is the decay factor, n is the current round, and p represents how many rounds it decays. Representatives Round down.

[0045] The present invention also proposes a device for classifying and identifying varieties of japonica rice grains, and its technical solution is as follows:

[0046] The device comprises the following modules:

[0047] A module for acquiring RGB images of collected japonica rice grains to be classified;

[0048] A module for data preprocessing of RGB images of japonica rice grains to be classified;

[0049] Module for obtaining the trained J-Rice-ResNeXt model; where:

[0050] The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include:

[0051] Keep the backbone network framework of the ResNeXt50 model;

[0052] Replace the 1×1Conv used to increase the dimension with 1×1GConv;

[0053] Replace the 1×1Conv in downsampling with 3×3DWConv;

[0054] Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model;

[0055] The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features;

[0056] A module for inputting the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains.

[0057] The present invention also proposes a computer device, and its technical solution is as follows:

[0058] A computer device comprises: a processor and a memory, wherein the memory is used to store executable instructions of the processor, and the processor is configured to execute the above-mentioned method for classifying and identifying varieties of japonica rice grains by executing the executable instructions.

[0059] The present invention also proposes a computer storage medium, and its technical solution is as follows:

[0060] A computer storage medium stores a computer program, and when the computer program is run, the above-mentioned method for classifying and identifying varieties of japonica rice grains is executed.

[0061] The present invention also proposes a computer program product, and its technical solution is as follows:

[0062] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for classifying and identifying varieties of japonica rice grains.

[0063] The present invention has the following beneficial effects:

[0064] The method for classifying and identifying varieties of japonica rice grains disclosed in the present invention deeply explores the application of classic deep learning network models in the classification of japonica rice grains, improves ResNeXt50 to make it have better comprehensive performance, obtains the J-Rice-ResNeXt model by replacing the activation function, replacing the convolution, replacing the downsampling and introducing the ECA attention mechanism, and introduces the learning rate dynamic adjustment strategy "86-Gradient" during the model training process; the accuracy of the J-Rice-ResNeXt model is as high as 98.81%, which is 7.82% higher than the basic model ResNeXt50 model, while the parameter amount and the calculation amount are greatly reduced by 10.279M and 1.573G respectively; the J-Rice-ResNeXt model can accurately classify japonica rice grains of different varieties, and can also be extended to the classification and quality inspection of other grains and agricultural products, providing efficient and accurate technical support for agricultural production, and providing new solutions and possibilities for automation and intelligence in the agricultural production process.

[0065] The method and device for classifying and identifying varieties of japonica rice grains described in the present invention are suitable for classifying and identifying varieties of japonica rice grains. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A schematic diagram of a method for classifying and identifying varieties of japonica rice grains;

[0068] Figure 2 is a schematic diagram of the shooting system;

[0069] Figure 3 Schematic diagram of the image threshold segmentation process; (a) is the original image; (b) is the image after the original image is grayed; (c) is the image after the grayed image is binarized; (d) is the binary image after the original binary image is dilated and eroded to eliminate noise; (e) is the binary image after the background is removed from the original image; (f) is the image after the target area is extracted;

[0070] Figure 4Schematic diagram showing the effect of image threshold segmentation; among them, (a) is Jijing 809; (b) is Jijing 812; (c) is Jijing 816; (d) is Jijing 821; (e) is Jijing 825; (f) is Jijing 826; (g) is Jijing 827; (h) is Jijing 830; (i) is Jijing 851;

[0071] Figure 5 Schematic diagram showing the effect of data enhancement; (a) is Jijing 809; (b) is Jijing 812; (c) is Jijing 816; (d) is Jijing 821; (e) is Jijing 825; (f) is Jijing 826; (g) is Jijing 827; (h) is Jijing 830; (i) is Jijing 851;

[0072] Figure 6 Schematic diagram of standard convolution (Conv), grouped convolution (GConv) and depthwise convolution (DWConv);

[0073] Figure 7 Schematic diagram of the structure of ResNeXt50 and ECA attention mechanism; (a) is the residual structure diagram of ResNeXt50; (b) is the downsampling structure diagram of ResNeXt50; (c) is the structural diagram of ECA attention mechanism; BN: batch normalization layer, ReLU: ReLU activation function, H is height, W is width, and C is channel;

[0074] Figure 8 Comparison diagram of ReLU function and Mish function; (a) is the curve of ReLU activation function and its derivative; (b) is the curve of Mish activation function and its derivative;

[0075] Fig. 9 The figure is a schematic diagram of the structure of the J-Rice-ResNeXt model; (a) is a schematic diagram of the backbone network structure of the J-Rice-ResNeXt model; (b) is a schematic diagram of the downsampling structure; (c) is a schematic diagram of the residual structure; Conv represents standard convolution, GConv represents grouped convolution, DWConv represents deep convolution, ECA block represents ECA attention mechanism, Pooling represents pooling layer, and FC represents fully connected layer;

[0076] Fig.10 Comparison chart of accuracy and loss of 9 network models;

[0077] Fig.11 and Fig.12 This is a schematic diagram of the confusion matrix of 9 network models on the test set; Fig.11 (a) is a schematic diagram of the confusion matrix of FasterNet_T0 on the test set; Fig.11(b) is a schematic diagram of the confusion matrix of GhostNet on the test set; Fig.11 (c) is a schematic diagram of the confusion matrix of MobileNetV3_S on the test set; Fig.11 (d) is a schematic diagram of the confusion matrix of MobileNetV4_S on the test set; Fig.12 (a) is a schematic diagram of the confusion matrix of Res2Net50 on the test set; Fig.12 (b) is a schematic diagram of the confusion matrix of ResNet50 on the test set; Fig.12 (c) is a schematic diagram of the confusion matrix of ResNeXt50 on the test set; Fig.12 (d) is a schematic diagram of the confusion matrix of ShuffleNetV2 on the test set; Fig.12 (e) is a schematic diagram of the confusion matrix of SqueezeNet on the test set;

[0078] Fig.13 Schematic diagram of the effect of using 86-Gradient to optimize the learning rate during model training; (a) is the learning rate change curve after adding '86-Gradient'; (b) is the accuracy and loss comparison result of '86-Gradient' and other commonly used learning rates;

[0079] Fig.14 This is a schematic diagram of the accuracy and loss comparison results of 8 commonly used activation functions;

[0080] Fig.15 The following is a schematic diagram of the comparison results of five attention mechanisms. (a) is a schematic diagram of the comparison of accuracy and loss; (b) is a schematic diagram of the comparison of the number of parameters and the amount of calculation.

[0081] Figures 16 to 18 The figure is a comparison diagram of the J-Rice-ResNeXt model and the basic model ResNeXt50; Fig.16 (a) is the accuracy curve and loss curve on the validation set before and after the model improvement; Fig.16 (b) is the confusion matrix diagram of the J-Rice-ResNeXt model on the test set; Fig.17 It is a comparison chart of the precision, recall and F1 score on the test set before and after the model improvement; Fig.18 It is a heat map of the recognition effect on each layer before and after the model is improved. DETAILED DESCRIPTION

[0082] In order to make the technical solutions and advantages of the present invention more clearly described, the specific implementation methods of the present invention will be further described in detail and completely in conjunction with the accompanying drawings. The various implementation methods described below are only part of the preferred solutions of the present invention, rather than all implementation methods; the various implementation methods described below are intended to explain the present invention and cannot be understood as limitations on the present invention; the reasonable combination of technical features defined in the various implementation methods of the present invention, and all other implementation methods obtained by ordinary technicians in the field without creative work based on the implementation methods of the present invention, are within the scope of protection of the present invention.

[0083] In one embodiment, a method for classifying and identifying varieties of japonica rice grains is provided:

[0084] The method comprises the following steps:

[0085] A step for obtaining an RGB image of the collected japonica rice grains to be classified;

[0086] A step for performing data preprocessing on the RGB image of the japonica rice grains to be classified;

[0087] Steps for obtaining a trained J-Rice-ResNeXt model; where:

[0088] The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include:

[0089] Keep the backbone network framework of the ResNeXt50 model;

[0090] Replace the 1×1Conv used to increase the dimension with 1×1GConv;

[0091] Replace the 1×1Conv in downsampling with 3×3DWConv;

[0092] Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model;

[0093] The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features;

[0094] The step is used to input the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains.

[0095] In this implementation, the trained J-Rice-ResNeXt model is obtained after training the J-Rice-ResNeXt model.

[0096] In this embodiment, in order to solve the problem that the existing image classification processing model cannot meet the requirements of japonica rice grain classification in terms of parameter complexity, classification accuracy, avoiding overfitting and convergence speed, it is necessary to improve the existing deep learning network model.

[0097] It should be noted that the process of obtaining the J-Rice-ResNeXt model from the ResNeXt50 model is not a structural transformation that can be achieved through logical analysis, reasoning or limited experiments.

[0098] Whether it is the base model for improvement (i.e., the ResNeXt50 model), or the convolutional blocks, activation functions, attention mechanisms, and learning rate training methods, there are thousands of choices that cannot be obtained through logical analysis, reasoning, or limited experiments.

[0099] Although we can understand the performance of individual basic models, convolution blocks, activation functions, attention mechanisms, and learning rate training methods through certain logical analysis, the changes that will occur when these structures are combined with each other cannot be obtained through simple logical analysis, reasoning, or limited experiments.

[0100] Selecting the ResNeXt50 model as the basic model and improving it to obtain the J-Rice-ResNeXt model is an original design, and after the improvement, comparative experiments can prove that the improvement is effective.

[0101] In this implementation, the 1×1Conv used to increase the dimension is replaced by 1×1GConv, which greatly reduces the number of parameters and calculations, thereby improving the training efficiency of the model.

[0102] In this implementation, the 1×1Conv in the downsampling is replaced with a 3×3DWConv, which increases the receptive field and improves the training efficiency of the model.

[0103] In this implementation, the ReLU activation function is replaced by the Mish activation function, which can prevent overfitting of the model and improve the convergence speed of the model.

[0104] In this implementation, Conv is a standard convolution.

[0105] In this implementation, GConv is a group convolution.

[0106] In this implementation, DWConv is a depthwise convolution.

[0107] In this implementation, the structure of the J-Rice-ResNeXt model is as follows: Fig. 9 shown.

[0108] In addition, in one embodiment, after the ResNeXt50 model is improved to the J-Rice-ResNeXt model, in order to verify the advantage of using the ResNeXt50 model as the basic model, a comparative experiment is conducted to compare the performance of 9 network models:

[0109] Among them, the 9 network models include FasterNet_T0, GhostNet, MobileNetV3_S, MobileNetV4_S, Res2Net50, ResNet50, ResNeXt50, ShuffleNetV2 and SqueezeNet.

[0110] Through comparative experiments, it was determined that the model with the best classification and recognition performance among the 9 network models is the ResNeXt50 model, and the process is as follows:

[0111] Obtain 9 different varieties of japonica rice grains and obtain their RGB images through a camera; then process the RGB images of japonica rice grains through threshold segmentation and image enhancement technology; select 9 network models for comparison in the pre-training stage and select the optimal network model ResNeXt50;

[0112] Specifically, during image acquisition and preprocessing:

[0113] Nine different varieties of japonica rice grains were selected, with about 200 grains of each variety.

[0114] Schematic diagram of the imaging system Figure 2 As shown, it is equipped with a Nikon D7100 camera and an 18-105mm lens, two lights controlled by a computer, a stage is placed at the bottom to hold the japonica rice grains, and the camera is used for vertical shooting.

[0115] During the image acquisition process, 200 japonica rice grains of the same variety were first randomly selected and arranged in a 10 × 20 black counting plate. Without any seeds overlapping or adhering, they were inverted on a black light-absorbing cloth and their RGB images were acquired by the camera.

[0116] The data preprocessing process is as follows Figure 3 As shown, a threshold segmentation method is adopted.

[0117] First, obtain its grayscale image based on the original image, and then select a suitable threshold to divide the pixels in the image into two categories (target and background). Specifically, the threshold can be set to 0.4, all pixels below 0.4 are set to 0 (black), and all pixels above 0.4 are set to 1 (white), thus obtaining a binary image.

[0118] The binary image is multiplied pixel by pixel with the original image, so that only pixels with a value of 1 (representing the target area) in the mask image will be retained in the original image, and pixels with a value of 0 (representing the background area) will be set to black.

[0119] Finally, according to the pixel distribution of the image with the background removed, the contour extraction algorithm is used to extract the edge of the rice grain, thereby extracting the target area.

[0120] The effect of each japonica rice variety after splitting is shown as follows Figure 4 As shown, it can be seen that the grains of different varieties of japonica rice have high similarity, which means that it poses a great challenge to the classification performance of the model.

[0121] In this comparative experiment, useless images need to be eliminated after photographing japonica rice grains. For example, those that are blurred, damaged, or completely do not meet the experimental requirements may interfere with the training and evaluation process of the model and cause the model to learn incorrect or irrelevant features. These images not only fail to provide valuable information for the model, but may also reduce the performance of the model.

[0122] Data augmentation is a very effective technique that can generate more training samples by performing a series of transformations on the original images.

[0123] Data augmentation can not only increase the diversity of the dataset, but also help the model learn more different features, thereby improving its generalization ability.

[0124] For deep learning models, more complex training samples mean more learning opportunities, which helps improve the accuracy and stability of the model.

[0125] In this comparative experiment, the original images were enhanced by increasing contrast, adding color, mirroring and randomly rotating the images, injecting noise, and increasing brightness to obtain an enhanced data set.

[0126] The enhanced dataset is divided into training set, validation set and test set in a ratio of 6:2:2. The training set is used to train the 9 network models, and the test set is used to compare and analyze the performance of the 9 network models.

[0127] Comparison of accuracy and loss of 9 network models Fig.10 As shown in the figure, ResNeXt50 achieved the highest accuracy (ACC%) and the lowest loss (Loss), reaching 88.61% and 0.32 respectively.

[0128] In order to further evaluate the performance of model classification, the confusion matrices of the 9 network model tests were obtained, such as Fig.11 and Fig.12 shown.

[0129] In the field of machine learning, confusion matrices are often used to compare the results of model classification in supervised learning. Each column of the matrix represents the predicted class, and each row represents the actual class. Taking the binary classification problem as an example, if the actual result is positive, the predicted result is positive, recorded as TP; if the actual result is negative, the predicted result is positive, recorded as FP; if the actual result is positive, the predicted result is negative, recorded as FN; if the actual result is negative, the predicted result is negative, recorded as TN.

[0130] The specific structure of the confusion matrix is ​​shown in the following table:

[0131]

[0132] Accuracy (Acc), precision (P), recall (R), and F1 score (F1) can be calculated from the data in the confusion matrix and used as evaluation indicators to evaluate the classification performance of the model. The formulas and brief descriptions of these evaluation indicators are shown in the table below:

[0133]

[0134] From the confusion matrix of the 9 network model tests, it can be found that each model performs differently in the classification task.

[0135] Among them, the values ​​on the diagonal of the confusion matrix of ResNeXt50 are relatively high, which means that the model performs well in correctly classifying japonica rice grains. The higher the value on the diagonal, the higher the prediction accuracy of the model for the corresponding category. At the same time, the values ​​on the off-diagonal are relatively low, which means that the model rarely misclassifies samples into other categories. In contrast, the confusion matrices of the other eight models show different degrees of difference.

[0136] The values ​​on the diagonal of the confusion matrix of other models are lower than that of ResNeXt50, indicating that their accuracy in classification tasks is not as good as ResNeXt50. Most models performed poorly on specific categories, among which FasterNet_T0 predicted a large number of the fourth category as the eighth category, MobileNetV4_S predicted a large number of the first category as the third and sixth categories, and SqueezeNet predicted a large number of the eighth category as the fourth category.

[0137] To further evaluate the classification performance of the 9 models,

[0138] According to the confusion matrix, the precision, recall and F1 score of each network model on 9 categories are obtained, as shown in the following table:

[0139] Comparison of the precision, recall and F1 scores of the 9 pre-trained models on each category (bold font best value)

[0140]

[0141] The results show that the number of optimal values ​​for each category of the ResNeXt50 model exceeds that of other models.

[0142] It should be noted that standard convolution (Conv), grouped convolution (GConv), and deep convolution (DWConv) are different in structure, calculation method, and application scenarios, and each has unique advantages and disadvantages:

[0143] Conv can flexibly adjust the size, step size, padding and other parameters of the convolution kernel according to task requirements. By adjusting the number and parameters of the convolution kernel, the complexity of the model and the feature extraction capability can be controlled.

[0144] GConv can effectively reduce the number of parameters and the amount of calculation. By dividing the input feature map into multiple groups and using an independent convolution kernel for each group, the number of parameters and the computational complexity of the model can be significantly reduced. At the same time, GConv can also better extract different aspects of the input features, thereby obtaining richer feature information.

[0145] DWConv has extremely low parameter and computational complexity. It greatly reduces the number of model parameters and computational complexity by performing independent convolution operations on each input channel, so it is more suitable for lightweight models. Since each channel performs convolution operations independently, DWConv can better maintain the spatial information of the input data.

[0146] DWConv performs independent convolution operations on each input channel, while Conv and GConv perform convolution operations on the entire input feature map or grouped feature maps.

[0147] The number of convolution kernels of DWConv is the same as the number of input channels, and each convolution kernel is only responsible for the convolution operation of one channel, while the number of convolution kernels of ordinary convolution and grouped convolution is usually more than the number of input channels, and each convolution kernel can process data from multiple channels.

[0148] The comparison of the calculation formulas for the parameters and calculation amounts of Conv, GConv, and DWConv is shown in the following table.

[0149]

[0150] In the above table, C_in is the number of input channels, C_out The number of output channels ， is the convolution kernel size, G is the size of the group, and H×W is the size of the image.

[0151] It should be noted that ResNeXt is an upgraded version of ResNet. It introduces the concept of cardinality based on ResNet. Here, cardinality refers to the number of identical branches in a block. By increasing the cardinality, ResNeXt50 can improve the performance of the network without increasing the depth and width of the network. This strategy avoids the complexity of network design and the increase of computational overhead.

[0152] like Figure 7 As shown in (a), ResNeXt50 uses group convolution technology to divide feature maps into different groups, and then convolve each group of feature maps separately. This operation can effectively reduce the amount of calculation and improve the efficiency of the network. Therefore, the ResNeXt50 network simply uses the aggregated residual structure and local connection structure based on the ResNet50 structure, and introduces data enhancement and regularization methods such as Random Erasing and Mixup.

[0153] like Figure 7 As shown in (b), the downsampling module of ResNeXt is located at the beginning of each layer. In each residual block, if the size of the input feature map does not match the output size of the residual block (for example, between stages spanning different feature map sizes), a downsampling module is used to reduce the size of the input feature map and increase the number of channels so that it can be added to the output of the residual block. In simple terms, it is Figure 7 1×1 Conv and batch normalization layers are added to the residual channels of (a).

[0154] It should be noted that the attention mechanism allows the model to dynamically focus on the most important parts when processing input data, thereby improving the performance and efficiency of the model. However, when the attention mechanism is too large, the amount of information that the model needs to process will increase significantly, resulting in increased consumption of computing resources, which may slow down the training and reasoning of the model, and even lead to insufficient resources in some cases. Therefore, it is not possible to improve the performance of the model by simply selecting an attention mechanism and adding it to the model. Choosing the right attention mechanism is crucial to improving the performance of the model.

[0155] In this implementation, an attention mechanism called ECA (Efficient Channel Attention) is introduced to effectively and accurately filter out valuable information from massive data, which is very beneficial for image processing tasks.

[0156] The ECA module can adaptively adjust the weights of channel features so that the network can better focus on the basic features of japonica rice grains. Most varieties of japonica rice grains have similar shapes and textures, which makes it difficult to extract detailed features from the network. The ECA module helps improve the distinguishing ability of features and suppress unimportant features, thereby reducing the risk of overfitting. Ultimately, the feature representation is enhanced and the generalization ability of the model is improved without significantly increasing the computational cost.

[0157] The structure of the ECA module is as follows Figure 7 (c) shown.

[0158] In the process of forward propagation, the ECA attention mechanism first performs H×W×C The input feature map is globally averaged and pooled to obtain feature information. Then, new weights are generated through a one-dimensional convolution of size K and a Sigmoid activation function. , complete the information interaction between channels, as shown in the following formula:

[0159] ;

[0160] in, represents a one-dimensional convolution with a kernel size of k, Represents the Sigmoid activation function;

[0161] As shown in the following formula, the number of channels C is proportional to the one-dimensional convolution with kernel k:

[0162] ;

[0163] The final kernel size is obtained as shown in the following formula k :

[0164] ;

[0165] in, is the odd number closest to t, is equal to 2, b is equal to 1.

[0166] In addition, in one embodiment, after the ResNeXt50 model is improved to the J-Rice-ResNeXt model, a comparative experiment is conducted to verify the improvement effect of the ECA attention mechanism on the ResNeXt50 model. The ResNeXt50 model is used as the basic model to compare five common attention mechanisms, such as Fig.15 (a) and Fig.15 (b) As shown:

[0167] The SE attention mechanism achieved the highest accuracy of 97.82%; the EMA attention mechanism achieved the smallest loss of 0.07; and the ECA attention mechanism achieved the lowest parameters and FLOPs, which were 22.989M and 4.292G respectively.

[0168] In contrast, adding the SE attention mechanism to the ResNeXt50 model increases the number of parameters by 10.088M compared to adding the ECA attention mechanism, while the accuracy is only improved by 1.63%. Subsequent analysis shows that the ECA attention mechanism uses one-dimensional convolution operations to replace the fully connected layer in SE, reducing the number of parameters and computational complexity. One-dimensional convolution can efficiently capture the correlation between channels while maintaining a low computational cost. Using the ECA attention mechanism to optimize the model has better overall performance than using the SE attention mechanism.

[0169] It should be noted that choosing different activation functions will have a significant impact on the performance of the model, which may improve the performance of the model or reduce the performance of the model. Choosing the right activation function is crucial to improving the performance of the model.

[0170] In this implementation, the formulas of the ReLU function and the Mish function are as follows:

[0171] The formula of the ReLU function is as follows:

[0172] ;

[0173] Among them, f(x) represents the output of the ReLU function, x represents the input value, and max(0,x) represents the maximum value between the input value and 0.

[0174] The formula for the Mish function is as follows:

[0175] ;

[0176] ;

[0177] .

[0178] It should be noted that the comparison between the ReLU function and the Mish function is Figure 8 As shown:

[0179] like Figure 8As shown in (a), the ReLU function is a piecewise linear function. When the input is greater than 0, the output is equal to the input; when the input is less than or equal to 0, the output is 0. The ReLU function maintains linear growth when x>0, which helps to speed up the training of the neural network; however, when x≤0, the output of the ReLU function is 0, which may lead to the problem of gradient disappearance. The derivative of the ReLU function is 1 when x>0, which means that the gradient can propagate unimpeded in this interval; however, when x≤0, the derivative of the ReLU function is 0, which may lead to the problem of gradient disappearance. The derivative curve of the ReLU function has a mutation point at x=0, which may cause the network to jitter or be unstable during training.

[0180] like Figure 8 As shown in (b), the Mish function is a smooth, continuous and non-monotonic activation function. The Mish function has the property of having a lower bound (≈-0.31) but no upper bound, which helps to avoid the problem of gradient vanishing and allows the network to better propagate gradients during training. At the same time, the derivative curve of the Mish function shows good smoothness, which helps the network maintain a stable gradient flow during training.

[0181] In addition, in one embodiment, after the ResNeXt50 model is improved to the J-Rice-ResNeXt model, a comparative experiment is conducted to verify the improvement effect of the ResNeXt50 model using the Mish activation function. Eight common activation functions are compared with the ResNeXt50 model as the basic model, such as Fig.14 As shown:

[0182] The results show that the Mish activation function achieved the highest accuracy and the lowest loss. Compared with the activation function ReLU used in the ResNeXt50 model, the accuracy was improved by 6.19% and the loss was reduced by 0.17.

[0183] Subsequent analysis showed that the Mish activation function, by introducing nonlinear elements, enables the neural network model to learn and represent more complex functional relationships, thereby enhancing the model's expressiveness and fitting capabilities to maximize the differences between categories, such as the color, shape, and texture of japonica rice grains, thereby enabling the extraction of challenging detailed features.

[0184] This advantage of the Mish activation function is used to obtain better generalization performance and recognition results of the model, thereby significantly improving the classification performance of the model for japonica rice grains.

[0185] In addition, in one embodiment, after the ResNeXt50 model is improved to the J-Rice-ResNeXt model, an ablation experiment is performed to further verify the improvement effect of the ResNeXt50 model, as shown in the following table and Figures 16 to 18 As shown:

[0186] Some results of ablation experiments are shown

[0187]

[0188] As shown in the table above, when the ECA attention mechanism is added alone, the accuracy of the model is greatly improved, while the parameters do not change and GFLOPs only increases by 0.006G.

[0189] When the Mish activation function is added alone, the accuracy is greatly improved, while the parameters and GFLOPs remain unchanged.

[0190] When GConv is added alone, the parameters and GFLOPs are reduced by 7.544M and 1.233G respectively, and the accuracy is also improved by 2.07%.

[0191] When DWConv is added alone, the parameters and GFLOPs are reduced by 2.734M and 0.346G respectively, and the accuracy is also slightly improved.

[0192] Finally, all the improvements were integrated together, which significantly improved the accuracy by 7.82%, and reduced the number of parameters and calculation by 10.279M and 1.573G respectively.

[0193] This result further demonstrates the advantages of the model improvement strategy and its effect on improving the efficiency and accuracy of japonica rice variety classification and identification.

[0194] Comparison of model before and after improvement:

[0195] like Fig.16 (a) shows the accuracy and loss curves of the J-Rice-ResNeXt and base model ResNeXt50 models on the validation set:

[0196] The accuracy curve of ResNeXt50 still fluctuates in the later stage and has not completed convergence, which may be at risk of overfitting overall. In contrast, the accuracy curve of J-Rice-ResNeXt shows a more significant upward trend. In the early stage of training, the accuracy rate increases rapidly, and then continues to maintain a stable growth trend, finally reaching the highest accuracy level and stabilizing.

[0197] Although the loss curve of ResNeXt50 shows a stable downward trend, it still fluctuates in the later stage of training and has not completed convergence. This shows that the model encountered optimization problems during the training process and it is difficult to further reduce the loss, resulting in a relatively high final value of the loss curve. This shows that the model's fitting ability is limited and there may be problems of overfitting or underfitting. The loss curve of J-Rice-ResNeXt shows a more ideal downward trend, completes convergence in the later stage of training, and has less fluctuation. This shows that J-Rice-ResNeXt is more stable during the optimization process and can reduce losses more effectively. In the end, the value of the loss curve is low, indicating that the model's fitting ability has been significantly improved and can better adapt to the training data.

[0198] In order to analyze the results before and after the model improvement more intuitively, Fig.16 (b) The precision, recall, and F1 score of the J-Rice-ResNeXt model are obtained in the confusion matrix, and compared with the base model ResNeXt50 model and visualized.

[0199] like Fig.17 As shown, it can be seen more clearly:

[0200] Before the improvement, the accuracy of the model categories varied greatly, and the model's recognition ability was poor. After the model was improved, the precision and recall of each category reached a relatively high level, which led to a significant improvement in the F1 score and a more balanced distribution. The recognition accuracy of all categories has improved compared to before the improvement, and the difference in recognition accuracy between different categories has decreased, indicating that the overall performance of the model has been significantly improved.

[0201] The adaptability and generalization ability of the model in various categories have been enhanced. For some categories with low recognition accuracy before the improvement (such as category 4 and category 8), the model has given more attention and optimization, so that their recognition accuracy has been significantly improved after the improvement. Therefore, the overall performance of the model has been greatly improved.

[0202] In order to more intuitively analyze the effect of each layer on japonica rice grain recognition before and after model improvement, the visualization tool Grad-CAM was used.

[0203] Grad-CAM computes the gradients of the target class feature map, multiplies these gradients with the feature map to obtain weights, and finally generates a heat map to visualize the image regions that the model focuses on during prediction, with a spectrum from blue to red indicating the degree of contribution.

[0204] The experimental results are as follows Fig.18As shown in the figure, before the model was improved, the model paid more attention to the local features of japonica rice grains because the model did not have sufficient ability to extract the features of japonica rice grains. In the third layer, the model paid more attention to the features that were not related to japonica rice grains, which greatly reduced the recognition effect of the model. However, after the model was improved, the model paid more attention to the texture information of japonica rice grains, enhanced its ability to grasp the global features of japonica rice grains, and accurately avoided some useless features. This enables the model to better focus on the features of the entire target during the prediction process.

[0205] Therefore, the comprehensive improvement of the ResNeXt50 model enhanced the feature extraction capability of the japonica rice grain classification model, enabling it to more accurately locate valuable areas in japonica rice grain images.

[0206] In addition, in one embodiment, the data preprocessing includes:

[0207] Remove useless images from the collected RGB images of japonica rice grains to be classified;

[0208] The target area of ​​japonica rice grains in RGB images was extracted using the threshold segmentation method.

[0209] In this embodiment, after taking the RGB images of japonica rice grains, it is necessary to eliminate useless images, including blurred, damaged, incompletely photographed, and other images that do not meet the requirements of the model classification task at all.

[0210] These useless images will interfere with the model's recognition and evaluation process, causing the model to learn incorrect or irrelevant features. These images not only fail to provide valuable information to the model, but may also reduce the performance of the model.

[0211] In this embodiment, the japonica rice grains (ie, the target) and the background (region) are distinguished in the image by extracting the target region (japonica rice grains).

[0212] In addition, in one embodiment, the method of extracting the target region of japonica rice grains in the RGB image by using a threshold segmentation method includes:

[0213] Obtain its grayscale image according to the RGB image;

[0214] According to the given threshold, the pixels in the grayscale image are divided into two categories: japonica rice grain target pixels and background pixels:

[0215] The grayscale values ​​of pixels whose grayscale values ​​are lower than a given threshold are set to 0, and the grayscale values ​​of pixels whose grayscale values ​​are higher than a given threshold are set to 1, so as to obtain a binary image; wherein, the pixels whose grayscale values ​​are set to 0 are black, representing background pixels; and the pixels whose grayscale values ​​are set to 1 are white, representing target pixels of japonica rice grains;

[0216] The binary image is used as a mask image and multiplied pixel by pixel with the RGB image, so that the pixels in the RGB image multiplied with the pixels with a gray value of 1 in the mask image are retained, and the pixels in the RGB image multiplied with the pixels with a gray value of 0 in the mask image are set to black, and the RGB image with the background area removed is obtained;

[0217] The contour extraction algorithm is used to extract the edge of the target pixel distribution area of ​​japonica rice grains in the RGB image with the background area removed, and the target area of ​​japonica rice grains is obtained.

[0218] In addition, in one embodiment, the given threshold is 0.4.

[0219] Additionally, in one embodiment, during the training of the J-Rice-ResNeXt model:

[0220] The image size of the training set input to the J-Rice-ResNeXt model is set to 224 × 224;

[0221] The number of training rounds is 100;

[0222] Batch Size is set to 32;

[0223] The base learning rate is set to 0.01;

[0224] The optimizer uses SGD.

[0225] In this implementation, the trained J-Rice-ResNeXt model is obtained after training the J-Rice-ResNeXt model.

[0226] In this implementation, the training environment of the J-Rice-ResNeXt model is deployed on a computer, and all algorithms are run in the same environment. The specific configuration details of the computer are shown in the following table:

[0227]

[0228] Additionally, in one embodiment, during the training of the J-Rice-ResNeXt model:

[0229] Adopt 86-Gradient learning rate dynamic adjustment strategy;

[0230] The formula for 86-Gradient is:

[0231] ;

[0232] is the learning rate of the nth round, is the initial learning rate, is the decay factor, n is the current round, and p represents how many rounds it decays. Representatives Round down to the integer not greater than The maximum integer.

[0233] In this embodiment, , p=5.

[0234] In this implementation, the 86-Gradient driving learning rate is used to improve the accuracy of model classification and recognition and reduce the loss.

[0235] In addition, in one embodiment, after the ResNeXt50 model is improved to the J-Rice-ResNeXt model, a comparative experiment is conducted to verify the improvement effect of using 86-Gradient on the training process of the ResNeXt50 model. The ResNeXt50 model is used as the basic model to compare several other commonly used learning rates, such as Fig.13 As shown:

[0236] The results show that the best effect is achieved by optimizing the learning rate during model training using 86-Gradient.

[0237] like Fig.13 (a) shows the trajectory of the learning rate driven by "86-Gradient" during model training:

[0238] The learning rate is reduced every five rounds with a decay rate of 0.86.

[0239] like Fig.13 (b) shows the comparison of “86-Gradient” with other representative learning rates:

[0240] The results show that “86-Gradient” has the lowest loss and the highest accuracy, which are 0.26 and 90.99% respectively, ahead of other learning rates.

[0241] In one embodiment, a device for classifying and identifying varieties of japonica rice grains is provided:

[0242] The device comprises the following modules:

[0243] A module for acquiring RGB images of collected japonica rice grains to be classified;

[0244] A module for data preprocessing of RGB images of japonica rice grains to be classified;

[0245] Module for obtaining the trained J-Rice-ResNeXt model; where:

[0246] The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include:

[0247] Keep the backbone network framework of the ResNeXt50 model;

[0248] Replace the 1×1Conv used to increase the dimension with 1×1GConv;

[0249] Replace the 1×1Conv in downsampling with 3×3DWConv;

[0250] Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model;

[0251] The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features;

[0252] A module for inputting the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains.

[0253] The technical solution provided by the present invention is further described in detail above through several specific implementation modes in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the several specific implementation modes described above are not intended to be used as limitations on the present invention. Any reasonable changes and improvements to the present invention, reasonable combinations of implementation modes and equivalent substitutions based on the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for classifying and identifying varieties of japonica rice grains, characterized in that: The method comprises the following steps: A step for obtaining an RGB image of the collected japonica rice grains to be classified; A step for performing data preprocessing on the RGB image of the japonica rice grains to be classified; Steps for obtaining a trained J-Rice-ResNeXt model; where: The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include: Keep the backbone network framework of the ResNeXt50 model; Replace the 1×1Conv used to increase the dimension with 1×1GConv; Replace the 1×1Conv in downsampling with 3×3DWConv; Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model; The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features; A step for inputting the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains; Conv is a standard convolution; GConv is group convolution; DWConv is a depth-wise convolution.

2. The method for classifying and identifying varieties of japonica rice grains according to claim 1, characterized in that: The data preprocessing includes: Remove useless images from the collected RGB images of japonica rice grains to be classified; The target area of ​​japonica rice grains in RGB images was extracted using the threshold segmentation method.

3. The method for classifying and identifying varieties of japonica rice grains according to claim 2, characterized in that: The target area of ​​japonica rice grains extracted from the RGB image by using the threshold segmentation method includes: Obtain its grayscale image according to the RGB image; According to the given threshold, the pixels in the grayscale image are divided into two categories: japonica rice grain target pixels and background pixels: The grayscale values ​​of pixels whose grayscale values ​​are lower than a given threshold are set to 0, and the grayscale values ​​of pixels whose grayscale values ​​are higher than a given threshold are set to 1, so as to obtain a binary image; wherein, the pixels whose grayscale values ​​are set to 0 are black, representing background pixels; and the pixels whose grayscale values ​​are set to 1 are white, representing target pixels of japonica rice grains; The binary image is used as a mask image and multiplied pixel by pixel with the RGB image, so that the pixels in the RGB image multiplied with the pixels with a gray value of 1 in the mask image are retained, and the pixels in the RGB image multiplied with the pixels with a gray value of 0 in the mask image are set to black, and the RGB image with the background area removed is obtained; The contour extraction algorithm is used to extract the edge of the target pixel distribution area of ​​japonica rice grains in the RGB image with the background area removed, and the target area of ​​japonica rice grains is obtained.

4. The method for classifying and identifying varieties of japonica rice grains according to claim 3, characterized in that: The given threshold is 0.

4.

5. The method for classifying and identifying varieties of japonica rice grains according to claim 1, characterized in that: During the training of the J-Rice-ResNeXt model: The image size of the training set input to the J-Rice-ResNeXt model is set to 224 × 224; The number of training rounds is 100; Batch Size is set to 32; The base learning rate is set to 0.01; The optimizer uses SGD.

6. The method for classifying and identifying varieties of japonica rice grains according to claim 1, characterized in that: During the training of the J-Rice-ResNeXt model: Adopt 86-Gradient learning rate dynamic adjustment strategy; The formula for 86-Gradient is: ； is the learning rate of the nth round, is the initial learning rate, is the decay factor, n is the current round, and p represents how many rounds it decays. Representatives Round down.

7. A device for classifying and identifying varieties of japonica rice grains, characterized in that: The device comprises the following modules: A module for acquiring RGB images of collected japonica rice grains to be classified; A module for data preprocessing of RGB images of japonica rice grains to be classified; Module for obtaining the trained J-Rice-ResNeXt model; where: The J-Rice-ResNeXt model is an improvement on the ResNeXt50 model. The improvements include: Keep the backbone network framework of the ResNeXt50 model; Replace the 1×1Conv used to increase the dimension with 1×1GConv; Replace the 1×1Conv in downsampling with 3×3DWConv; Replace the ReLU activation function with the Mish activation function to prevent overfitting of the model and speed up the convergence of the model; The ECA attention mechanism is introduced to make the model focus on the basic features of japonica rice grains and ignore irrelevant features; A module for inputting the RGB image of the japonica rice grains to be classified after data preprocessing into the trained J-Rice-ResNeXt model to obtain the classification and identification results of the japonica rice grains; Conv is a standard convolution; GConv is group convolution; DWConv is a depth-wise convolution.

8. A computer device comprising: A processor and a memory, characterized in that the memory is used to store executable instructions of the processor, and the processor is configured to execute the method for classifying and identifying varieties of japonica rice grains as described in any one of claims 1 to 6 by executing the executable instructions.

9. A computer storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is run, the method for classifying and identifying varieties of japonica rice grains according to any one of claims 1 to 6 is executed.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of a method for classifying and identifying varieties of japonica rice grains as described in any one of claims 1 to 6 are implemented.

Images