A method and system for identifying imperfect grains of cereal based on transmitted polarized light images

Abstract

The application discloses a kind of grain imperfect grain identification method and system based on transmission polarized light image, this method constructs the deep learning network classification model for identifying grain category, and then respectively constructs the imperfect grain detection model based on InceptionV3 network, YOLOv5 network, LeNet-5 network, AlexNet network for rice, wheat, corn, soybean;The technical scheme of the application selects network category according to the correlation between the transmission polarized light image characteristics of each type of grain and the characteristics of network model, thereby improving the imperfect grain detection accuracy of each type of grain, and the method is suitable for multi-class grain imperfect grain detection.Using the method described in the technical solution, a complete detection process can be formed, i.e.from grain warehousing to obtaining imperfect grain detection results, which can eliminate manual classification and other intervention measures, and lay a foundation for subsequent grain warehousing, warehouse management and other unmanned and fully automated operations.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of food safety and processing technology, and particularly relates to a grain imperfect kernel identification method and system based on transmission polarized light images. BACKGROUND

[0002] Grains are an important part of the diet and an important food source of nutrients such as dietary fiber, B vitamins, and minerals. Grains in China are generally divided into cereal and legume, and cereal includes rice, wheat, corn, and sorghum, and legume includes soybeans and broad beans. At present, the demand for grains has shifted from quantity to quality, and people pay more attention to the appearance quality and taste of grains, so the problem of grain safety needs to be solved urgently in China.

[0003] The national standard of China regards imperfect kernels as a quality limitation standard for grains, which is an important basis for judging the quality grade during grain purchase. The phenomenon of imperfect kernels is essentially damaged kernels that still have edible value, including sprouted kernels, insect damaged kernels, diseased kernels, moldy kernels, and broken kernels. Imperfect kernels can reduce the commercial value and use value of grains, for example, moldy grains may cause poisoning, and insect damaged kernels can affect the taste and smell of food. Therefore, the detection of imperfect kernels has important value and significance for grain purchase and food processing.

[0004] However, in actual applications, the grains are usually observed and identified by the experience of the inspectors, and the perfect kernels and imperfect kernels are selected manually, weighed, and calculated, and finally the test results are obtained according to the calculation data. The results are not objective and accurate, and the repeatability is poor. There are differences between different inspectors, which leads to inconsistent test results and cannot meet the rapid detection needs of grain quality.

[0005] With the development of machine vision technology, the application of machine vision technology to grain classification and identification can greatly improve the detection efficiency. Among them, the transmission polarized light technology fuses the spatial, spectral and polarization information of the target object, which not only improves the amount of information obtained by the target object, but also enhances the detection and identification ability of the target object. Further, the image is converted into digital information, and the shape, size, color, surface defect and maturity of the sample are detected. The patent CN202111181604.7 discloses a grain vomit toxin detection device and method based on polarized light and ultraviolet light imaging, which adopts polarized light and ultraviolet light conditions to collect grain images, extracts image feature information, inputs it into a computer controller after processing and fusion, and uses a partial least squares model to determine the vomit toxin content interval. However, such detection is only for grain vomit toxin, and in actual situations, there are other adverse conditions such as disease spots and insect damage, which cannot meet the detection needs of various grains and various diseases. Especially, the existing related technologies are all for identifying and detecting a certain type of grain or a certain type of disease, but there are many types of grains, and the characteristics of imperfect grains of various grains are different, and there is a lack of effective technical means to realize the detection of multiple types of grains. SUMMARY

[0006] The purpose of the present application is to solve the problem of imperfect grain detection of multiple types of grains in the prior art, and to provide a grain imperfect grain identification method and system based on transmission polarized light image. The method of the present application selects the best network model for each of the four types of grains, rice, wheat, corn and soybeans, to construct a detection model for imperfect grain detection. Among them, the network structure is set / selected according to the correlation between the transmission polarized light image features of each type of grain and the characteristics of the network model, which improves the detection accuracy of the imperfect grain detection model of each type of grain, and truly realizes a universal method suitable for imperfect grain detection of multiple types of grains. In addition, the method of the present application also adds grain type detection, i.e. detecting the grain type first, and then selecting the corresponding imperfect grain detection model for detection according to the detected grain type. On the one hand, it ensures the matching degree of the grain type and the imperfect grain detection model; on the other hand, using the method of the present technical solution, a complete detection process can be formed, i.e. from grain storage to obtaining imperfect grain detection results, which can eliminate manual classification and other intervention measures, and lay a foundation for subsequent grain storage, warehouse management and other unmanned and fully automated operations. On the one hand, the present technical solution provides a grain imperfect grain identification method based on transmission polarized light image, which comprises the following steps:

[0007] S1: Obtain a grain sample and collect a transmission polarized light image, the type of the grain being two or more combinations of rice, wheat, corn and soybeans;

[0008] S2: Preprocess the transmission polarized light image of step S1, and then make a data set;

[0009] S3: Construct a deep learning network classification model suitable for grain classification, wherein the input data of the deep learning network classification model is a transmission polarized light image of the grain, and the output data is a grain category;

[0010] S4: According to the transmission polarized light image features of each type of grain, a detection model suitable for imperfect grain detection is constructed for each type of grain, and the input data of the detection model is a transmission polarized light image of the grain, and the output data is an imperfect grain detection result;

[0011] Among them, the detection model of rice, wheat, corn and soybean is respectively constructed based on Inception V3 network, YOLOv5 network, LeNet-5 network and AlexNet network.

[0012] S5: Use the trained deep learning network classification model to classify and identify the grain to be identified, and then select a matched detection model according to the identified grain category to detect imperfect grains.

[0013] Among them, the imperfect grains of rice include immature grains, disease spot grains, moldy grains, germinated grains and insect damaged grains; the imperfect grains of wheat include broken grains, moldy grains, insect damaged grains, disease spot grains and germinated grains; the imperfect grains of corn include disease spot grains, insect damaged grains, broken grains, moldy grains and heat damaged grains; the imperfect grains of soybean include immature grains, disease spot grains, insect damaged grains, broken grains, germinated grains and moldy grains. As can be seen, the defect categories of imperfect grains faced by different categories of grains are quite different, so the transmission polarized light images obtained under transmission polarized light will also be significantly different. Therefore, the technical scheme of the present application explores the transmission polarized light image features of each type of grain, selects a suitable network model to construct a detection model suitable for each type of grain, and improves the detection accuracy of imperfect grains. The imperfect grain detection result is the classification result of perfect grains and imperfect grains. In some examples, the inspection result is classified into perfect grains and each type of imperfect grain classification result; in some embodiments, the inspection result can also be regarded as two categories of perfect grains and imperfect grains, and the present application does not make specific limitation thereon, and the data sample category used in network model training is used to determine it in actual application.

[0014] Further optionally, the rice detection model based on Inception V3 network is constructed by adding a dropout layer before the full connection layer of the original Inception V3 network, introducing a Cot module between the convolution layer and the pooling layer, and adding a convolution attention module in part of the inception modules in the original Inception V3 network.

[0015] The transmission polarized light image is mainly caused by the birefringence of starch crystals in the endosperm of the grain. The starch crystal structure of the rice is type A, and under the irradiation of polarized light, the image presents the surface hull morphology characteristics and endosperm texture characteristics. The immature grains are dry and small, and the polarized light cannot penetrate the endosperm, so the grain image is dark; the diseased spot grains and the moldy grains present corresponding colors. The hull color is redundant information, which has a certain influence on classification. InceptionV3 is suitable for feature extraction and classification of complex data sets, wherein the inception module can increase the width and depth of the network, the 1x1 convolution kernel reduces the network parameters, and the hull color, crystal structure and other information in the polarized light image of the rice can be fused, the adaptability of the network to the scale is increased, and the utilization rate of the internal computing resources of the network is improved. Therefore, the Inception V3 network is selected to construct the imperfect grain detection model of the rice.

[0016] In order to be more suitable for the characteristics of rice, the network is also optimized. The addition of the dropout layer can randomly stop some neurons without changing the input and output, thereby reducing the calculation amount and preventing overfitting; the introduction of the CoT module between the convolution layer and the pooling layer can improve the recognition of the nearest neighbor space features; the introduction of the convolution attention module can extract the features in the channel number dimension and the spatial dimension, and improve the precision.

[0017] Further optionally, the convolution attention module is to perform global MaxPooling and AvgPooling on the pixel values of the same position on different feature maps in the axis direction, and then obtain two spatial attention maps, and connect them, and then obtain a spatial attention matrix through convolution and an activation function on the connected feature map;

[0018] M S (F)=σ(f([AvgPool](F);MaxPool(F)]

[0019] In the formula, M S (F) represents the output of the convolution attention module, that is, the spatial attention matrix, F is the feature map input into the convolution attention module, sigma represents an activation function, and f represents convolution.

[0020] The Cot module is to obtain local features of an image by convolution of input data through a kxk convolution kernel, then splice and fuse the local information with the input data of the Cot module, then obtain an attention matrix through two convolution kernels on the spliced and fused data, perform 1x1 convolution kernel operation on the input data of the Cot module, and perform matrix multiplication operation on the operation result and the attention matrix to obtain global features; finally, add and fuse the local features and the global features to obtain output features.

[0021] Further optionally, based on the wheat detection model constructed by the YOLOv5 network, the input image is first processed by a homomorphic filtering image enhancement algorithm in the input layer; then the Ghost module is used to replace the general convolution in the original YOLOv5 network, and the efficient channel attention network ECA-Net is used to replace the channel attention network in the original YOLOv5 network;

[0022] The Ghost module first performs convolution processing on the input features to obtain a part of the original feature map A; then performs convolution on the feature map A to obtain another part of the feature map B; and finally, the feature map A and the feature map B are spliced to output the result.

[0023] The efficient channel attention network ECA-Net uses one-dimensional convolution to replace the bottleneck structure composed of two fully connected layers in the original channel attention network.

[0024] The starch in wheat is divided into A-type starch and B-type starch, wherein the A-type starch is mainly in the shape of flat sphere, circle and ellipse. The transmission polarized light image undergoes birefringence through the starch crystal in the endosperm, and the wheat starch crystal type is complex and the endosperm color is deep. In addition, in addition to the disease spot on the surface of the diseased spot grain, there is a unique sclerotium, and the grain is wrinkled, pale or pink mold. The collected wheat imperfect polarized light image information is redundant, the feature extraction capability of the ordinary classification model is weak, the calculation cost is high, and the time consumption is extremely long. Based on the above wheat transmission polarized light features, the YOLOv5 network is based on the convolutional neural network, the network perspective and the multi-scale prediction method, each layer of which can classify the imperfect area and the category of each wheat. Therefore, the technical scheme of the present application selects the YOLOv5 network as the main body to construct a wheat imperfect grain detection model.

[0025] In order to further match the features of the wheat transmission polarized light image, that is, for the problems of deep endosperm color, imperfect grain color blur, and small imperfect feature area, the YOLOv5 network is further optimized, an improved YOLOv5 network model architecture using the ECA-Net attention mechanism is adopted, and a new YOLOv5 network is proposed. The entire network structure is divided into four parts: input, backbone, neck and output.

[0026] Further optionally, the network layers of the corn detection model based on the LeNet-5 network are 12 layers, including: an input layer, 4 convolution layers, 4 pooling layers, 2 fully connected layers and an output layer;

[0027] Among them, a pooling layer is connected after each convolution layer, and 2 fully connected layers and an output layer are connected in turn after the last pooling layer. Batch normalization BN operation is added after each convolution layer and each fully connected layer.

[0028] The starch granules in corn are polygonal, and the surface has multiple planes and corners. The corn imperfect grains have clear characteristics under polarized light, and the damaged surface of the damaged grains is uneven, and the embryo or endosperm is damaged; the color of the corn kernel itself is light yellow, and the endosperm of the moldy grain is obviously green or red. Compared with other grains, the image features of the corn dataset have less redundant information, and a model with a deep network structure is not needed. Compared with other models, the LeNet-5 model has a simple network structure and strong classification ability, and is more suitable for corn dataset classification. Therefore, the technical scheme of the application selects the LeNet-5 network as the main body to construct the corn imperfect grain detection model. In order to adapt to the characteristics of corn, the LeNet-5 network is optimized.

[0029] Further optionally, the soybean detection model constructed based on the AlexNet network is to replace each convolutional layer in the original AlexNet network with a convolutional layer with three different convolutional kernels and a connection layer.

[0030] The starch crystal form in soybeans is C-type, which is composed of A-type and B-type crystals, and the spectrum image information reflected by the transmission polarized light through the soybeans is more. AlexNet is based on LeNet, and the network structure is deepened, which can learn more rich and high-dimensional image features. And the color of the soybean endosperm is the same as that of corn, and will not interfere with the color of the imperfect grain, therefore, a model with a relatively simple network structure is selected, which can ensure the classification efficiency without wasting resources.

[0031] Further optionally, before the transmission polarized light image of the grain sample is collected, the moisture content of the grain sample needs to be constant, wherein the rice, wheat, corn and soybeans correspond to the following constant moisture parameters: {rice, constant temperature: 30-40 DEG C, constant time: 10-20 min}, {wheat, constant temperature: 30-40 DEG C, constant time: 5-10 min}, {corn, constant temperature: 40-50 DEG C, constant time: 5-10 min}, {soybean, constant temperature: 45-55 DEG C, constant time: 10-20 min};

[0032] Rice, wheat and corn and soybean correspond to the following incident light source parameters: {rice, incident light wavelength: 800-1450nm, angle θ: 2kπ(k=0, 1, 2, 3…), incident light distance: 0.5-3cm}, {wheat, incident light wavelength: 1800-2450nm, angle θ: 2kπ-(2k+1)π(k=0, 1, 2, 3…), incident light distance: 3-10cm}, {corn, incident light wavelength: 1500-1750nm, angle θ: 2kπ-(2k+1)π(k=0, 1, 2, 3…), incident light distance: 1-5cm}, {soybean, incident light wavelength: 480-750nm, angle θ: 2kπ-(2k+1)π(k=0, 1, 2, 3…), incident light distance: 0.5-3cm}.

[0033] Polarized light is mainly due to the internal crystal structure of the grain itself, and the starch crystal forms of rice, wheat, corn and soybean are different. In addition, the different external colors will also affect the results of polarized light, so it is necessary to select appropriate incident light parameters to accurately indicate the characteristic influence of different grain structure information on polarized light.

[0034] In a second aspect, the present application provides a system based on the grain imperfect kernel identification method, comprising:

[0035] An image acquisition module is used to acquire grain samples and perform transmission polarized light image acquisition, and the category of the grain is two or more combinations of rice, wheat, corn and soybean;

[0036] A data set making module is used to preprocess the transmission polarized light image and then make a data set;

[0037] A grain classification model construction module is used to construct a deep learning network classification model suitable for grain classification, wherein the input data of the deep learning network classification model is the transmission polarized light image of the grain, and the output data is the grain category;

[0038] An imperfect kernel detection model construction module is used to construct a detection model suitable for imperfect kernel detection for each type of grain according to the transmission polarized light image features of each type of grain, and the input data of the detection model is the transmission polarized light image of the grain, and the output data is the imperfect kernel detection result;

[0039] Among them, the detection models of rice, wheat, corn and soybean are respectively constructed based on Inception V3 network, YOLOv5 network, LeNet-5 network and AlexNet network;

[0040] The detection module is used for classifying and identifying the to-be-identified grains by using the trained deep learning network classification model, and then selecting a matched detection model according to the identified grain category to perform the imperfect grain detection.

[0041] In a third aspect, the present application provides a computer terminal, comprising at least:

[0042] one or more processors;

[0043] a memory storing one or more computer programs;

[0044] the processor invokes the computer program to perform:

[0045] a step of a grain imperfect grain identification method based on a transmission polarized light image.

[0046] Among them, the selective polarization light image acquisition platform has many, the present application technical scheme does not carry out specific limitation, for example Paddy Check TM PC 6800 instrument.

[0047] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, the computer program is invoked by a processor

[0048] a step of a grain imperfect grain identification method based on a transmission polarized light image.

[0049] In a fifth aspect, the present application further provides a detection system based on a grain imperfect grain identification method, comprising: a computer terminal, a grain conveying device, a sampling platform and a transmission polarization imaging device; wherein the sampling platform and the transmission polarization imaging device are connected with the computer terminal;

[0050] The sampling platform comprises a power assembly, a rotating disc device and a light source; the grain conveying device comprises a motor, a conveying assembly and a collection box; the conveying channel on the feeding side of the conveying assembly directly reaches the rotating disc device, the conveying channel on the discharging side of the conveying assembly connects the rotating disc device and the collection box, and the collection box is arranged according to the grain imperfect grain classification;

[0051] The conveying assembly is used for conveying the grains to the rotating disc device of the sampling platform; the transmission polarization imaging device is arranged above the rotating disc device and is used for collecting the transmission polarized light image of the grains; the transmission polarization imaging device transmits the collected transmission polarized light image to the computer terminal; the computer terminal runs the computer program of the grain imperfect grain identification method to obtain the grain imperfect grain detection result, generates a classification conveying control instruction and sends the instruction to the sampling platform, so as to control the rotating disc device through the power assembly, and then convey the grains to the corresponding collection box through the conveying assembly.

[0052] Further, the sampling platform includes a rotating disc device, and a light source is arranged below the upper left corner sampling table and corresponds to the position of the lens, and the direction of the light source is from bottom to top. The transmission polarization imaging device includes a polarization camera, a polarizer, and a lens. The grain conveying device includes a stepping motor, a conveying device, and collection boxes (1-perfect grains; 2-damaged grains; 3-diseased grains; 4-moldy grains; 5-broken and immature grains; 6-germinated and heat-damaged grains).

[0053] Based on the above device, two prongs are arranged on the conveying channel of the grain conveying device, and a gap is arranged in the middle of the prongs. The size of the gap at the feeding end is between the radial sizes of one grain and two grains, so as to ensure that the grains entering the collection platform area are single grains. Then, the grains placed on the grain conveying device are sent to the polarization light grain detection platform and fall into the disc holes. The transmission polarization imaging device shoots the transmission polarization light image of the grains and transmits the image to the computer terminal.

[0054] Beneficial effects

[0055] 1. The technical scheme of the present application takes the transmission polarization light image as the processing object, and selects the best network model for each type of grain, including rice, wheat, corn and soybeans, to construct a detection model for detecting imperfect grains. Among them, the network structure is selected / set according to the correlation between the transmission polarization light image characteristics of each type of grain and the characteristics of the network model, thereby improving the detection accuracy of imperfect grains of each type of grain. Among them, Inception V3 network is selected for rice, YOLOv5 network is selected for the transmission polarization light image information of wheat, which is redundant, the feature extraction ability of the ordinary classification model is weak, the calculation cost is high, and the time consumption is extremely long; LeNet-5 network is selected for corn because the transmission polarization light image of corn has less redundant information than other grains and does not need a network structure deep model; and AlexNet network is selected for soybeans. That is, the best network model is selected for each type of grain, thereby improving the detection accuracy of imperfect grains of each type of grain.

[0056] 2. The technical scheme of the present application also adds grain category detection, that is, the grain category is detected first, and then the corresponding imperfect grain detection model is selected for detection according to the detected grain category. On the one hand, the matching degree of the grain category and the imperfect grain detection model is ensured; on the other hand, using the method described in the technical scheme, a complete detection process can be formed, that is, from the grain entering the warehouse to obtaining the imperfect grain detection result, manual classification and other intervention measures can be eliminated, thereby laying a foundation for subsequent grain warehousing, warehouse management and other unmanned and fully automatic operations.

[0057] 3.The transmission polarized light image collected by the application is mainly generated by birefringence of starch crystals in the endosperm of the grain and color reflection of the surface of the grain. The starch crystal structure of perfect and imperfect grains is different, so the polarized light image data presented is also different. For example, the perfect grain is full and hard; the starch in the immature grain is irregularly arranged in a powdery state, and the polarized light image of the endosperm will appear dark or black; the transmission polarized imaging of the worm-eaten grain can clearly present the part of the endosperm that has holes or is missing due to worm-eating. In addition, the polarized light maximizes the color change of the grain surface, and the moldy grain has color abnormalities, such as green or red, which can present the corresponding color under polarized light. Compared with common spectral detection technologies such as near-infrared spectroscopy and Raman spectroscopy, the polarized light technology used in the application can obtain more stable spectral information. The spectral analysis in the past has relatively high requirements for grain sample preparation and detection environment. If it is a powder sample, the spectral analysis is easily affected by various factors such as sample temperature and humidity, sample detection site, and sample loading conditions. The method described in the application can eliminate the influence of complex sample preparation operations on data. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of a grain imperfect kernel intelligent recognition method based on transmission polarized light image technology;

[0059] Figure 2 A grain imperfect kernel detection system based on transmission polarized light

[0060] Figure 3 is a dropout layer working diagram, wherein a diagram does not add a dropout layer, and b diagram adds a dropout layer;

[0061] Figure 4 is a CoT module network structure diagram;

[0062] Figure 5 is an ECA-Net network structure diagram;

[0063] Figure 6 is a VGG16 network structure diagram;

[0064] Figure 7 is a DG-InceptionV3 network structure diagram;

[0065] Figure 8 is an XM-YoLov5 network structure diagram;

[0066] Figure 9 is a YM-LeNet-5 network structure diagram;

[0067] Figure 10 is a DD-AlexNet network structure diagram. DETAILED DESCRIPTION

[0068] The method for identifying imperfect grains of cereals based on transmission polarized light images provided by the technical scheme can detect and identify imperfect grains of multiple types of cereals by constructing a deep learning network classification model for cereal classification and constructing an imperfect grain detection model for each type of cereal. The present application will be further described below in combination with embodiments.

[0069] The detection system for imperfect grains of cereals based on transmission polarized light provided by the embodiment of the present application. The detection system comprises a computer terminal, a cereal conveying device, a sampling platform and a transmission polarized imaging device. The sampling platform comprises a rotating disc device, a light source below the upper left corner sampling table, and the position corresponding to the lens. The direction of the light source is from bottom to top. The transmission polarized imaging device comprises a polarized camera, a polarizer and a lens. The cereal conveying device comprises a stepping motor, a conveying device, a collection box (1-perfect grain; 2-borers grain; 3-disease spot grain; 4-mold grain; 5-broken grain, immature grain; 6-germination grain, heat damage grain). Two prongs are arranged on the conveying channel of the cereal conveying device, and there is a gap between the two prongs. The size of the gap at the feeding end is between the radial size of one grain and the radial size of two grains, so as to ensure that the cereal entering the collection platform area is a single grain. Based on the above device, the cereal placed on the cereal conveying device is sent to the polarized light cereal detection platform for transmission polarized light image acquisition. The transmission polarized imaging device and the sampling platform are connected to the computer terminal. After image acquisition, the cereal is rotated to the imperfect grain classification hole, and the computer terminal gives an instruction (cereal imperfect grain category) according to the polarized image. If the cereal is a perfect grain, the classification hole 1 is opened, and the cereal falls to the corresponding conveying belt 1 below. After the cereal falls, the classification hole is closed, and the classification method of other imperfect grain categories is the same as above. The computer recognizes the category, and falls to the corresponding conveying device at the corresponding hole position, and finally enters the collection box to complete the classification.

[0070] The method for identifying imperfect grains of cereals based on transmission polarized light images provided by the embodiment of the present application comprises the following steps:

[0071] S1: acquiring a cereal sample and performing transmission polarized light image acquisition. The types of the cereal are two or more combinations of paddy, wheat, corn and soybeans. In this embodiment, the types of the cereal are paddy, wheat, corn and soybeans.

[0072] Before acquiring the transmission polarized light data of the cereal, the moisture content needs to be constant, and the incident light source parameters of the transmission polarized light need to be set according to the type of the cereal. In this embodiment, the Paddy Check TM PC 6800 instrument is selected as the transmission polarized light acquisition platform.

[0073] S2: preprocessing the transmitted polarized light image of step S1, and then making a data set.

[0074] In this embodiment, three data sets are preferably made. First, a data set for a deep learning network classification model for grain species identification, that is, transmitted polarized light images of rice, wheat, corn and soybeans are randomly selected, and the training set and the test set are divided in a ratio of 8:2. Second, a detection model for training and determining various grain perfect and imperfect grains, that is, the classified grain perfect and imperfect grain images are selected, and the samples of the original data set are divided into a training set and a test set in a ratio of 8:2. Third, the overall test of the built deep learning network classification model and the imperfect grain detection model of each grain is called the overall test set: the data set is independent and does not participate in the training of the above two models.

[0075] S3: constructing a deep learning network classification model suitable for grain classification, wherein the input data of the deep learning network classification model is a transmitted polarized light image of the grain, and the output data is a grain category.

[0076] Common deep learning network models include convolutional neural networks, recurrent neural networks, and generative adversarial networks, which can all be applied to the present technical solution. In image recognition technology, convolutional neural networks are the most widely used. In this embodiment, considering that the starch crystal forms of rice, wheat, corn and soybeans are different, and the shape and color differences are significant, the polarized light images are easy to distinguish, so a model with a complex network structure is not needed. Therefore, a lightweight VGG16 model is selected, which has a simple network structure and low computational complexity. Since the data feature samples differ significantly, 4 convolutional layers and 2 fully connected layers are reduced on the original 16 layers to reduce the computational complexity. Preferably, a Dropout layer is added before the fully connected layer to prevent model overfitting.

[0077] S4: constructing a detection model suitable for imperfect grain detection for each type of grain according to the transmitted polarized light image features of each type of grain.

[0078] The present technical solution takes into account the significant differences in appearance, color, texture and other aspects of rice, wheat, corn and soybeans. Therefore, different grains have different effects under different classification models. The present application builds different models according to the polarized image data for different grain categories, maximizes the use of grain polarized light image features, and improves the classification effect of perfect and imperfect grains.

[0079] Model one: selecting an Inception V3 network to build a detection model for rice.

[0080] The present embodiment optimizes the Inception V3 network to build a DG-Inception V3 network, as follows:Figure 7 As shown in FIG. 1, the DG-Inception V3 network is sequentially provided with an input layer, 3 convolutional layers, a CoT module, 2 convolutional layers, InceptionA-InceptionD modules, a pooling layer, a dropout layer, a full connection layer and an output layer. The input data of the input layer is the transmission polarized light image of the rice, and the output data of the output layer is the perfect grain and imperfect grain classification detection result.

[0081] In this embodiment, the DG-Inception V3 network is improved in the Inception V3 original network as follows:

[0082] a. The network parameter optimization is performed using the Adam optimizer algorithm, which can calculate the adaptive learning rate of each parameter and reduce the influence of interference factors such as the color of the rice hull. Since the Adam optimizer is an existing optimizer and the present application does not optimize it, it is only integrated into the training of the DG-Inception V3 network, and therefore it is not specifically described.

[0083] b. The ReLU function in the InceptionV3 network is replaced by PReLU, and the calculation method is as follows:

[0084]

[0085] wherein y i is the input of the nonlinear activation function at the i-th channel; a i is the negative slope of the activation function. For each channel, there is a learnable parameter to control the slope, which reduces the interference of the color of the hull.

[0086] c. Some neurons are randomly stopped without changing the input and output, that is, a dropout layer is added before the full connection layer (as shown in FIG. 2), which reduces the calculation amount and prevents overfitting phenomenon; Figure 3

[0087] d. The CoT module is introduced between the convolutional layer and the pooling layer to improve the recognition of the nearest neighbor space features (as shown in FIG. 3). Figure 4 ​As shown in the diagram, the CoT module addresses the limitation of convolution in effectively representing the interaction of features at different spatial locations when rice grains are similar in shape and imperfect grains lack distinct features. The CoT module improves the recognition of nearest-neighbor spatial features. It not only extracts local information but also extracts global information through the calculation of the attention matrix and fuses it with local information, resulting in a richer feature representation. Specifically, the CoT module first obtains the local features of the rice image through convolution operations. Then, it processes the local features using two 1x1 convolutional kernels to obtain the local feature vector and attention weight vector for each pixel. These weights represent the importance of each pixel in the image and are used to generate the attention matrix. Simultaneously, the input data is processed through 1x1 convolutional layers and the attention matrix to obtain the global feature vector of the image. Finally, the local and global feature vectors are added together and fused to obtain the final output.

[0088] e. Add Convolutional Attention (CBAM) modules before the three main Inception_A, B, and C modules respectively. After completing the convolution operation on each feature branch, apply channel attention and spatial attention respectively, and then concatenate the resulting feature maps to obtain the final feature representation. This strengthens the feature representation ability of each branch, thereby improving the classification accuracy of the entire network.

[0089] The convolutional attention module processes feature regions on the feature map, telling the model which regions should receive more attention. It performs global MaxPooling and AvgPooling on pixel values ​​at the same location on different feature maps along the axis, obtaining two spatial attention maps, and then concatenates them. The feature map is then passed through a 7×7 convolution and a PReLU activation function to obtain a spatial attention matrix of the same dimensions as the original feature map. That is: M S (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])). The areas of interest in the polarization image of rice grains are color anomalies, local color changes, and areas of insect damage.

[0090] It should be understood that performing the above optimizations simultaneously is the best approach in the embodiments of the present invention. However, in other feasible embodiments, partial optimization can be selected, such as selecting the above-mentioned optimization methods c, d, and e.

[0091] Model 2: For wheat, a detection model based on the YOLOv5 network is selected.

[0092] This invention optimizes the YOLOv5 network to construct the XM-YOLOv5 network, as shown in the embodiments of the present invention. Figure 8 As shown.

[0093] The XM-YOLOv5 network in this embodiment is improved in the YOLOv5 original network as follows:

[0094] a. The input image is processed by a homomorphic filtering image enhancement algorithm in the input layer to enhance the details of imperfect grain regions such as insect damage, disease spots and shriveling. The operation process is to perform logarithmic transformation on the input image, perform Fourier transformation on the transformed image, suppress low-frequency components and enhance high-frequency components, and use a high-pass filter: H(u, v) = (r H -r L )H hp (u, v) + r L ; wherein r H , r L represent the upper and lower bounds of the filter, H hp (u, v) represents the use of a Gaussian high-pass filter, H hp (u, v) = 1-exp[-c(D 2 (u, v) / D0 2) ]; c represents the steepness of the Gaussian function; D0 represents the standard deviation of the Gaussian function.

[0095] b. A Ghost module is designed in the network structure to replace the general convolution (the input of the general convolution module is a group of feature maps, and the output is also a group of feature maps). The Ghost module first performs convolution processing on the input features to obtain a part of the original feature map A; then performs convolution on the feature map A to obtain another part of the feature map B; finally, the feature map A and the feature map B are spliced to output the result of the Ghost Module. This process realizes feature reuse, wherein the convolution is equivalent to feature extraction in different subspaces, and the splicing operation preserves important information in different spaces, reducing the computational load of the network.

[0096] c. The efficient channel attention network (ECA-Net) is introduced as an improved network based on the channel attention mechanism, using one-dimensional convolution to replace the bottleneck structure composed of two fully connected layers in the channel attention mechanism. As shown in Figure 5 , given the aggregated features obtained by global average pooling (GAP), ECA generates channel weights by performing a fast 1D convolution of size k (k is adaptively determined by the mapping of channel dimension C), ECA-Net includes multiple convolution layers, BN layers, ECA modules and global pooling layers. For the aggregated features obtained by global average pooling, a local cross-channel interaction strategy without dimension reduction and a method for adaptively selecting convolution kernel size are proposed, which significantly reduces the complexity of the model while maintaining the performance of the model.

[0097] Model three: select a detection model based on LeNet-5 network for corn.

[0098] The embodiment of the present application optimizes LeNet-5 network to construct YM-LeNet-5 network, as shown in the figure. Figure 9 The traditional LeNet-5 network is originally designed for handwritten digit recognition. The traditional LeNet-5 is directly applied to the imperfect kernel classification, and the accuracy can be further effectively improved. The YM-LeNet-5 network is 12 layers, including: an input layer, four convolutional layers, four pooling layers, two fully connected layers and an output layer; wherein, a pooling layer is connected after each convolutional layer, and two fully connected layers and an output layer are sequentially connected after the last pooling layer. In the embodiment, the YM-LeNet-5 network is improved in the original LeNet-5 network as follows:

[0099] a. Increase the input image pixels, and expand the traditional 32*32 to 126*126.

[0100] b. Modify the activation function, add LeakyReLU function after each convolutional layer, which is simple in logic, fast in convergence and high in calculation efficiency. The equation is x is the input, and f(x) is the output.

[0101] c. Add batch normalization (BN) after each convolutional layer and fully connected layer. BN is to normalize the input of each layer in the neural network to speed up the training of the neural network, avoid gradient disappearance / explosion phenomenon in training, speed up the training speed of the model and ensure the stable distribution of the input data of each layer. The specific steps of BN are as follows: (1) for the output of the convolutional layer or the fully connected layer, the mean and the variance are calculated, that is, the mean Wherein, m is the number of corn samples in the current batch, x i is the i-th sample. (2) The output is normalized, that is: Wherein, represents the normalized sample, σ is the variance, and ε is a coefficient added to increase the stability of the value, which is generally a small value; (3) linear scaling and translation operation is performed by using the learnable parameters, that is: Wherein, γ and β are learnable parameters, and y i is the output sample of the BN layer. The number of network layers is increased, and the convolutional layer is increased from the traditional 2 layers to 4 layers. Compared with the handwritten digit image, the corn image is more complex, and the depth of the model needs to be increased to extract more features.

[0102] Model four: a detection model based on AlexNet network is selected for soybeans.

[0103] The embodiment of the present application optimizes AlexNet network to construct DD-AlexNet network, with reference to Figure 9The DD-AlexNet network is sequentially provided with a convolutional layer 1, a pooling layer 1, a convolutional layer 2, a pooling layer 2, a convolutional layer 3 to a convolutional layer 5, a pooling layer 3, a Dense layer, a Dropout layer and a full connection layer. The specific optimization contents are as follows:

[0104] a. Change the convolution kernel size to 5*5. A large convolution kernel can obtain the shape, position and other features of the image, while a small convolution kernel can obtain the color, texture and other features. Soybeans only occupy a small part of the image. When the convolution kernel is large, some feature information and original image information may be lost.

[0105] b. By increasing the number of layers and combining convolution kernels of different sizes, a single convolution layer is divided into three convolution layers with different convolution kernels, which can obtain features of different scales. In this way, more complex features can be learned, and the network accuracy can be further improved.

[0106] c. The present application selects to increase the batch normalization between the first and second convolution layers and the activation function, adjusts the output of the network by comparing the mean and variance of the batch data, reduces the data deviation, and avoids the gradient disappearance.

[0107] It should be understood that, regardless of which model in the model one to the model four, the above optimization method is considered as the best optimization method of the present embodiment, but it is not the only optimization method of the present technical solution. In other feasible embodiments, for each model, some of the above optimization methods can be selected for optimization, and the present application does not make specific limitations thereto.

[0108] The present embodiment further comprises: optimizing the network model by using a test set, wherein the model accuracy is a ratio of correct results predicted by the model to all images identified, and the higher the correct results, the higher the identification accuracy. The calculation formula is:

[0109]

[0110] TP represents a positive sample predicted as positive by the model, TN represents a negative sample predicted as negative by the model, FP represents a positive sample predicted as negative by the model, and FN represents a negative sample predicted as positive by the model.

[0111] S5: classifying and identifying the to-be-identified grains by using the trained deep learning network classification model, and selecting a matched detection model according to the identified grain category to detect imperfect grains.

[0112] In summary, the method of the present application introduces the transmission polarized light technology, the principle is: according to the principle of calculating the polarization degree of Stokes vector, the parallel light source is irradiated on the surface of the measured object to form the reflection of light, after the reflected light transmits through the polarizer, the image sensor of the camera can collect image data from multiple directions, and finally the polarization degree image combined can reflect the unique information of the polarization image. Compared with common visible light, remote sensing, infrared imaging and the like, polarization imaging can obtain multi-dimensional polarization information of the object. The transmission polarized light image collected by the present application is mainly generated by the birefringence of starch crystals in the grain endosperm and the color reflection of the grain surface. The starch crystal structure of perfect and imperfect grains of the grain is different, so the presented polarized light data is also different. Based on the obtained transmission polarized light image, the machine vision technology is introduced, and a method for detecting imperfect grains of multiple types of grains is realized.

[0113] Examples:

[0114] First step: prepare the grain sample and collect the image;

[0115] (1) Screen the purchased grains (rice, wheat, corn, soybeans) to remove inorganic impurities. According to the standards of "GB1350-2009 Rice", "GB 1351-2008 Wheat", "GB 1353-2018 Corn", "GB 1352-2009 Soybean", the grains are classified into perfect and imperfect grains. The imperfect grains of rice include immature grains, disease spot grains, moldy grains, germinated grains and insect damaged grains; the imperfect grains of wheat include broken grains, moldy grains, insect damaged grains, disease spot grains and germinated grains; the imperfect grains of corn include disease spot grains, insect damaged grains, broken grains, moldy grains and heat damaged grains; the imperfect grains of soybeans include immature grains, disease spot grains, insect damaged grains, broken grains, germinated grains and moldy grains.

[0116] (2) Constant sample moisture content: the classified rice samples are placed on the MB23MK moisture tester produced by Ohaus Instrument (Changzhou) Co., Ltd. The constant temperature of rice is 30℃, the constant time is 10min, and the equilibrium sample moisture content is 12%; the constant temperature of wheat is 30℃, the constant time is 10min, and the equilibrium sample moisture content is 12.5%; the constant temperature of corn is 40℃, the constant time is 10min, and the equilibrium sample moisture content is 14%; the constant temperature of soybeans is 40℃, the constant time is 10min, and the equilibrium sample moisture content is 12%. The experimental environment temperature is 25±5℃, and the environmental humidity is 65±15%.

[0117] (3) Determine the incident light source parameters: the selected wavelength of the incident light for rice is 1200 nm, the angle θ between the main section of the polarizer and the vibration direction of the incident light is set to 0°, and the incident light distance is set to 0.5 cm; the selected wavelength of the incident light for wheat is 1800 nm, the angle θ between the main section of the polarizer and the vibration direction of the incident light is set to 45°, and the incident light distance is set to 2 cm; the selected wavelength of the incident light for corn is 1500 nm, the angle θ between the main section of the polarizer and the vibration direction of the incident light is set to 0°, and the incident light distance is set to 3 cm; the selected wavelength of the incident light for soybean is 600 nm, the angle θ between the main section of the polarizer and the vibration direction of the incident light is set to 45°, and the incident light distance is set to 1 cm.

[0118] (4) Collect the polarization light data of the grain samples: the collection equipment is Paddy Check TM PC 6800 grain quality analyzer, the grain samples with constant moisture content are uniformly and appropriately placed in the Paddy Check grain quality analyzer for scanning, the number of each scanning is set to 300, and parallel scanning is performed. The image data for model establishment is collected as follows: 2000 images of perfect and imperfect grains (immature grains, disease spot grains, moldy grains, germinated grains and insect damaged grains) of rice are collected respectively, a total of 12000 images; 2000 images of perfect and imperfect grains (broken grains, moldy grains, insect damaged grains, disease spot grains and germinated grains) of wheat are collected respectively, a total of 12000 images; 2000 images of perfect and imperfect grains (disease spot grains, insect damaged grains, broken grains, moldy grains and heat damaged grains) of corn are collected respectively, a total of 12000 images; 2000 images of perfect and imperfect grains (immature grains, disease spot grains, insect damaged grains, broken grains, germinated grains and moldy grains) of soybean are collected respectively, a total of 12000 images. The image data for the total test set is collected as follows: 120 images of rice, wheat, corn and soybean are collected respectively, a total of 480 images.

[0119] Second step: data set making;

[0120] Image preprocessing is performed on the collected images: marking and adjusting the image size. For grain species identification, 1000 images of each of rice (DG), wheat (XM), corn (YM) and soybean (DD) are randomly selected, a total of 4000 images. The original data set is divided into a training set and a test set in a ratio of 8:2 by K-fold cross-validation.

[0121] Table 1 Grain species training set and test set division

[0122]

[0123]

[0124] Rice {perfect grain (DG0), insect damaged grain (DG1), disease spot grain (DG2), mildew grain (DG3), immature grain (DG4) and sprouted grain (DG5)}; wheat {perfect grain (XM0), insect damaged grain (XM1), disease spot grain (XM2), mildew grain (XM3), broken grain (XM4) and sprouted grain (XM5)}; corn {perfect grain (YM0), insect damaged grain (YM1), disease spot grain (YM2), mildew grain (YM3), broken grain (YM4), heat damaged grain (YM5)}; soybean {perfect grain (DD0), insect damaged grain (DD1), disease spot grain (DD2), mildew grain (DD3), broken grain (DD4), sprouted grain (DD5)}. The original data set is divided into training set and test set in the ratio of 8:2 by K-fold cross-validation. The data division is as follows:

[0125] Table 2 Training set and test set division of rice perfect and imperfect grains

[0126]

[0127] Table 3 Training set and test set division of wheat perfect and imperfect grains

[0128]

[0129] Table 4 Training set and test set division of corn perfect and imperfect grains

[0130]

[0131]

[0132] Table 5 Training set and test set division of soybean perfect and imperfect grains

[0133]

[0134] Step 3: Construct a deep learning network classification model suitable for grain (rice, wheat, corn and soybean) classification and construct a detection model suitable for each type of imperfect grain.

[0135] (1) Build a lightweight image recognition network VGG16 to identify the category (rice, wheat, corn and soybean) of grain image.

[0136] Referring to Figure 6The lightweight VGG16 model has a total of 11 layers, including 9 convolutional layers, 1 fully connected layer and 1 dropout layer. The mini-batch is set to 100, and the data is counted (4000 pictures) every 40 mini-batches. Epoch is 10, and learning rate is set to 1e-4. After training and testing, the grain classification result is good, and the accuracy is more than 95%.

[0137] (2) Construct DG-Inception V3, XM-YoLov5, YM-AlexNet, and DD-LeNet-5 models and train the models.

[0138] Referring to Figure 7 The DG-Inception V3 model uses the PReLU function as the activation function, adds Softmax as the activation function of the last layer set, and determines the number of classes to be 6. The loss function is the cross-entropy loss function, and the smaller the cross-entropy, the smaller the proximity: Where p is the expected output and q is the actual output. Due to the large amount of data used, Dropout is selected at 50% for training the model. The optimizer is determined to be Adam, the learning rate is 0.01, epochs are specified to be 15 for training and 25 for fine-tuning. The specific parameters are as follows:

[0139]

[0140] XM-YoLOv5 uses GhostNet network instead of general convolution to extract the features of the input image of the wheat imperfect grain. The Focus module slices the input wheat image, reduces the size of the feature map to 1 / 2 without losing information, and improves the inference speed of the network. ECA is an attention mechanism module that can be used by inserting, making the network pay more attention to the target itself and improving the detection accuracy. The input feature map is subjected to global average pooling, and the feature map is changed from a [h, w, c] matrix to a [1, 1, c] vector. The adaptive one-dimensional convolution kernel size kernel_size is calculated. kernel_size is used in one-dimensional convolution to obtain the weight for each channel of the feature map. The normalized weight and the original input feature map are multiplied channel by channel to generate the weighted feature map.

[0141] Referring to Figure 8, the input end mainly uses homomorphism filtering to enhance image features to uniformly process the input model picture. The role of Backbone is mainly to extract features, mainly including Focus, Ghost Module, and SPP module. The Focus module periodically extracts pixel points from the input picture and reconstructs them into a low-resolution image, that is, stacks the adjacent 4 positions of the image to improve the receptive field of each point and reduce the loss of original information and reduce the calculation amount to speed up. The SPP module uses max pooling and then performs Concat fusion to improve the receptive field. The Neck strengthens information transmission and has the ability to accurately retain spatial information, which helps to properly locate the pixels to form a mask. Prediction is mainly the detection part, which applies anchor frames to the feature map and generates classification probability, confidence and target anchor frame final vector.

[0142] Among them, referring to the application of YOLOv5 target detection algorithm in other fields, the network input size is 640x580, the batch size is set to 64 according to the performance of the GPU, the initial learning rate is set to 0.01, β1 is 0.937, β2 is 0.999, and the Adam optimization algorithm is used to optimize the network parameters.

[0143] Referring to Figure 9 , the specific parameters of the improved YM-LeNet-5 model are: {input layer pixels: 126x126x3; activation function: Leaky ReLU function; loss function: Cross Entropy Loss cross-entropy loss function; convolution layer number: 4 layers, pooling layer: 4 layers, convolution kernel size: 5x5, 2x2, convolution kernel number: 32, 64, 64, 64. The last layer of the full connection layer selects the softmax function for classification, and the output class is set to 6, which corresponds to the 6 categories of the corn data set.

[0144] Referring to Figure 10 , the DD-AlexNet model is composed of 11 different layers, including 5 convolution layers, 3 max pooling layers, 1 Dense layer, 1 Dropout layer and one fully connected output layer, and the image input size is 227x227.

[0145] Among them, 80% of the soybean images are randomly selected as the training set, and the remaining 20% are used as the test set. The data set is imported into the network, and the parameters used for training are set. The main model parameters are: Mini batch size, 64; iteration number: 50; initial learning rate: 0.0005; optimizer: Adam; image input size: 227x227. After 50 epochs, the training is stopped, and the accuracy and loss rate of the network are obtained.

[0146] Finally, the trained recognition model and classification model are deployed in the same environment to make them in series. The total test set is input into the trained adaptive deep learning network classification model to identify the grain type and classify the perfect and imperfect grains. The specific steps are as follows:

[0147] The images in the total test set are input into the trained VGG16 model, which automatically identifies the grain type. If it is rice, it will be automatically input into the InceptionV3 model with the highest accuracy to classify the perfect and imperfect grains. The final result presents two types, grain type identification and perfect and imperfect grain classification.

[0148] The model evaluation standard is established, and the model accuracy is the ratio of the correct results to the total number of images identified. The higher the correct result is, the higher the identification accuracy is. In this embodiment, the PaddyCheck device is used to collect the polarized light images of grains (rice, wheat, corn and soybeans), and the adaptive deep learning classification model is used to classify the images and distinguish the perfect and imperfect grains. The results are as follows:

[0149]

[0150] In this embodiment, the PaddyCheck device is used to collect the polarized light images of grains, and the adaptive deep learning classification model is used to identify the images and classify the perfect and imperfect grains. VGG16 can well identify the grain type with an accuracy of 0.99, and the four deep models can also well classify the perfect and imperfect grains with a classification accuracy of more than 0.9. The adaptive deep learning classification model built has an accuracy of more than 0.95, which is suitable for grain image classification and realizes the purpose of fast, accurate and intelligent detection.

[0151] Step 4: The trained deep learning network classification model is used to classify and identify the to-be-identified grains, and then the matched detection model is selected according to the identified grain type to detect the imperfect grains.

[0152] In a second aspect, the present application also provides a system based on the grain imperfect grain identification method, which comprises an image acquisition module, a data set making module, an imperfect grain detection model construction module, a grain classification model construction module and a detection module.

[0153] The image acquisition module is used to acquire grain samples and collect transmission polarized light images. The types of the grains are two or more combinations of rice, wheat, corn and soybeans.

[0154] The data set making module is used to preprocess the transmission polarized light images and make data sets.

[0155] a grain classification model construction module, configured to construct a deep learning network classification model suitable for grain classification, wherein input data of the deep learning network classification model is a transmission polarized light image of the grain, and output data is a grain category;

[0156] an imperfect grain detection model construction module, configured to construct a detection model suitable for imperfect grain detection for each grain category according to transmission polarized light image features of each grain category, wherein input data of the detection model is a transmission polarized light image of the grain, and output data is an imperfect grain detection result;

[0157] wherein the detection models for paddy, wheat, corn and soybean are respectively constructed based on an Inception V3 network, a YOLOv5 network, a LeNet-5 network and an AlexNet network;

[0158] a detection module, configured to perform classification and identification on a grain to be identified by using the trained deep learning network classification model, and then perform imperfect grain detection by using a matched detection model according to the identified grain category.

[0159] It should be understood that the implementation process of each module can be described with reference to the foregoing method, and the division of the foregoing functional modules is only a logical function division, and another division mode can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. Meanwhile, the integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0160] In a third aspect, the present application further provides an electronic terminal, comprising:

[0161] one or more processors;

[0162] a memory storing one or more computer programs;

[0163] the processor invokes the computer program to perform the steps of a grain imperfect grain identification method based on a transmission polarized light image. Specifically, the steps are performed as follows:

[0164] S1: acquiring a grain sample and performing transmission polarized light image acquisition, wherein the category of the grain is two or more combinations of paddy, wheat, corn and soybean;

[0165] S2: preprocessing the transmission polarized light image of step S1, and then making a data set;

[0166] S3: constructing a deep learning network classification model suitable for grain classification, wherein input data of the deep learning network classification model is a transmission polarized light image of the grain, and output data is a grain category;

[0167] S4: Constructing a detection model suitable for imperfect grain detection for each type of grain according to the transmission polarized light image features of each type of grain, wherein the input data of the detection model is the transmission polarized light image of the grain, and the output data is the imperfect grain detection result;

[0168] The detection models of rice, wheat, corn and soybean correspond to those constructed based on an Inception V3 network, a YOLOv5 network, a LeNet-5 network and an AlexNet network respectively.

[0169] S5: Classifying and identifying the to-be-identified grain by using the trained deep learning network classification model, and then selecting a matched detection model for imperfect grain detection according to the identified grain type.

[0170] The specific implementation process of each step can be referred to the description of the foregoing method.

[0171] The memory can include a high-speed RAM memory, and can also include a non-volatile defibrillator, for example, at least one disk memory.

[0172] If the memory, the processor and the communication interface are independently implemented, the memory, the processor and the communication interface can be connected to each other through a bus and complete communication between each other. The bus can be an industry standard architecture bus, an external device interconnection bus or an extended industry standard architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0173] Optionally, if the memory and the processor are integrated on a chip, the memory and the processor can complete communication between each other through an internal interface.

[0174] It should be understood that, in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The memory can include read-only memory and random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information.

[0175] In a fourth aspect, the present application also provides a readable storage medium storing a computer program, which is called by a processor to execute the steps of a grain imperfect kernel recognition method based on transmission polarized light images. Specifically, the steps are executed as follows:

[0176] S1: Obtain a grain sample and collect transmission polarized light images, and the grain category is two or more combinations of rice, wheat, corn and soybeans;

[0177] S2: Preprocess the transmission polarized light images of step S1, and then make a data set;

[0178] S3: Construct a deep learning network classification model suitable for grain classification, wherein the input data of the deep learning network classification model is the transmission polarized light images of the grain, and the output data is the grain category;

[0179] S4: Construct a detection model suitable for imperfect kernel detection for each category of grain according to the transmission polarized light image features of each category of grain, wherein the input data of the detection model is the transmission polarized light images of the grain, and the output data is the imperfect kernel detection result;

[0180] The detection models of rice, wheat, corn and soybeans correspond to those constructed based on Inception V3 network, YOLOv5 network, LeNet-5 network and AlexNet network, respectively.

[0181] S5: Classify the grain to be recognized by using the trained deep learning network classification model, and then select a matched detection model for imperfect kernel detection according to the recognized grain category.

[0182] The specific implementation process of each step is described in the foregoing method.

[0183] The readable storage medium is a computer readable storage medium, which can be an internal storage unit of the controller, such as a hard disk or a memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the readable storage medium can include both the internal storage unit and the external storage device of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0184] Based on such understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0185] It should be emphasized that the examples described in the present application are illustrative rather than restrictive, and therefore the present application is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art from the technical solutions of the present application, without departing from the purpose and scope of the present application, whether modified or replaced, also belong to the protection scope of the present application.

Claims

1. A method for identifying imperfect grains in grains based on transmitted polarized light images, characterized in that: Includes the following steps: S1: Obtain a grain sample and acquire a transmitted polarized light image. The grain is a combination of two or more of the following: rice, wheat, corn, and soybean. S2: Preprocess the transmitted polarized light image from step S1 to create a dataset; S3: Construct a deep learning network classification model suitable for grain classification, wherein the input data of the deep learning network classification model is the transmitted polarized light image of the grain, and the output data is the grain category; S4: Based on the transmitted polarized light image characteristics of various grains, a detection model suitable for detecting imperfect grains is constructed for each type of grain. The input data of the detection model is the transmitted polarized light image of the grain, and the output data is the detection result of imperfect grains. Among them, the detection models for rice, wheat, corn, and soybeans are respectively built based on the Inception V3 network, YOLOv5 network, LeNet-5 network, and AlexNet network; The rice detection model built on the Inception V3 network consists of an input layer, three convolutional layers, a CoT module, two convolutional layers, InceptionA-InceptionD modules, a pooling layer, a dropout layer, a fully connected layer, and an output layer. A convolutional attention module (CBAM) is added to the InceptionA, InceptionB, and InceptionC modules. The wheat detection model built on the YOLOv5 network first processes the input image using a homomorphic filtering image enhancement algorithm at the input layer; then it uses the Ghost module to replace the general convolution in the original YOLOv5 network, and uses the efficient channel attention network ECA-Net to replace the channel attention network in the original YOLOv5 network. The corn detection model based on LeNet-5 has 12 layers, including: an input layer, 4 convolutional layers, 4 pooling layers, 2 fully connected layers, and an output layer. Each convolutional layer is followed by a pooling layer, and the last pooling layer is followed by 2 fully connected layers and the output layer. Batch normalization (BN) is added after each convolutional layer and each fully connected layer. The soybean detection model built on the AlexNet network consists of the following layers in sequence: convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, convolutional layer 3-convolutional layer 5, pooling layer 3, Dense layer, Dropout layer, and fully connected layer. S5: Use the trained deep learning network classification model to classify and identify the grains to be identified, and then select a matching detection model to detect imperfect grains based on the identified grain categories.

2. The method according to claim 1, characterized in that: The convolutional attention module performs global MaxPooling and AvgPooling on the pixel values ​​at the same position on different feature maps along the axis, thereby obtaining two spatial attention maps, which are then connected. The connected feature maps are then convolved and activated to obtain a spatial attention matrix. M S (F)=σ(f([AvgPool(F);MaxPool(F)] In the formula, M S (F) represents the output of the convolutional attention module, i.e., the spatial attention matrix, F is the feature map of the input convolutional attention module, σ represents the activation function, and f represents convolution; The Cot module obtains local features of the image by convolving the input data with a k×k convolution kernel. Then, it concatenates and fuses the local information with the input data of the Cot module. The concatenated and fused data is then passed through two convolution kernels to obtain an attention matrix. Additionally, the input data of the Cot module is subjected to a 1×1 convolution kernel operation, and the result is multiplied with the attention matrix to obtain global features. Finally, the local features and the global features are added together to obtain the output features.

3. The method according to claim 1, characterized in that: In the wheat detection model built on the YOLOv5 network, the Ghost module first performs convolution processing on the input features to obtain a portion of the original feature map A; then, it performs convolution on feature map A one by one to obtain another portion of the feature map B; finally, it concatenates feature map A and feature map B to output the result. The efficient channel attention network ECA-Net replaces the bottleneck structure of the original channel attention network, which consists of two fully connected layers, with one-dimensional convolution.

4. The method according to claim 1, characterized in that: The specific steps of Batch Normalization (BN) in the corn detection model based on LeNet-5 network are as follows: (1) For the output of the convolutional layer or fully connected layer, calculate its mean and variance, i.e., mean Where m is the number of corn samples in the current batch, x i (1) For the i-th sample; (2) Normalize the output, i.e.: in, The normalized sample is represented by σ, which is the variance, and ε is a coefficient added to increase the stability of the values; (3) Linear scaling and translation operations are performed using learnable parameters, i.e.: Where γ and β are learnable parameters, y i These are samples output by the BN layer.

5. The method according to claim 1, characterized in that: The soybean detection model built on the AlexNet network adds a batch normalization method between the first and second convolutional layers and the activation function, adjusting the network output by comparing the mean and variance of the batch data.

6. The method according to claim 1, characterized in that: Before acquiring transmitted polarized light images of the grain samples, it is necessary to maintain a constant moisture content in the grain samples. The constant moisture parameters for rice, wheat, corn, and soybeans are as follows: {rice, constant temperature: 30-40℃, constant time: 10-20 min}, {wheat, constant temperature: 30-40℃, constant time: 5-10 min}, {corn, constant temperature: 40-50℃, constant time: 10-20 min}, {soybeans, constant temperature: 30-40℃, constant time: 5-10 min}. Rice, wheat, corn, and soybeans correspond to the following incident light source parameters: {Rice, incident light wavelength: 800~1450nm, included angle θ: 2kπ, k=0, 1, 2, 3…, incident light distance: 0.5~3cm}, {Wheat, incident light wavelength: 1500~1750nm, included angle θ: 2kπ~(2k+1)π, k=0, 1, 2, 3…, incident light distance: 3~10cm}, {Corn, incident light wavelength: 1800~2450nm, included angle θ: 2kπ~(2k+1)π, k=0, 1, 2, 3…, incident light distance: 1~5cm}, {Soybean, incident light wavelength: 480~750nm, included angle θ: 2kπ~(2k+1)π (k=0, 1, 2, 3…), incident light distance: 0.5~3cm}.

7. A system based on the method of any one of claims 1-6, characterized in that: include: An image acquisition module is used to acquire grain samples and perform transmitted polarized light image acquisition, wherein the grains are of two or more combinations of rice, wheat, corn and soybeans; The dataset creation module is used to preprocess transmitted polarized light images and then create datasets. A grain classification model building module is used to build a deep learning network classification model suitable for grain classification. The input data of the deep learning network classification model is the transmitted polarized light image of the grain, and the output data is the grain category. The imperfect grain detection model construction module is used to construct detection models suitable for imperfect grain detection for various types of grains based on the transmission polarization light image characteristics of various types of grains. The input data of the detection model is the transmission polarization light image of the grain, and the output data is the imperfect grain detection result. Among them, the detection models for rice, wheat, corn, and soybeans are respectively built based on the Inception V3 network, YOLOv5 network, LeNet-5 network, and AlexNet network; The detection module is used to classify and identify the grains to be identified using the trained deep learning network classification model, and then select a matching detection model to detect imperfect grains based on the identified grain categories.

8. A detection system based on the method of any one of claims 1-6, characterized in that: include: The system includes a computer terminal, a grain conveying device, a sampling platform, and a transmission polarization imaging device; wherein the sampling platform and the transmission polarization imaging device are both connected to the computer terminal. The sampling platform includes a power unit, a rotating disc device, and a light source; the grain conveying device includes a motor, a conveying component, and a collection box; the conveying channel on the feed side of the conveying component leads directly to the rotating disc device, and the conveying channel on the discharge side of the conveying component connects the rotating disc device and the collection box; the collection box is set according to the classification of imperfect grains. The grain is conveyed to the rotating disk device of the sampling platform using the conveying component; the transmission polarization imaging device is positioned above the rotating disk device to acquire transmitted polarized light images of the grain; the transmission polarization imaging device transmits the acquired transmitted polarized light images to the computer terminal; the computer terminal runs the computer program of the grain imperfect grain identification method to obtain the grain imperfect grain detection results, and generates classification and conveying control commands to send to the sampling platform, which controls the rotating disk device through the power component, thereby conveying the grain to the corresponding collection box through the conveying component.

9. A computer-readable storage medium, characterized in that: The computer program is stored and is invoked by the processor for execution. The steps of the method according to any one of claims 1-6.

Images