Method and system for automatic identification of agricultural product grades based on graph convolutional neural network

Abstract

The application discloses a kind of agricultural product grade automatic identification method and system based on graph convolutional neural network, first, the hyperspectral image of agricultural product is acquired, and data preprocessing is carried out;Then the hyperspectral image after preprocessing is input into graph convolutional neural network to carry out automatic identification evaluation of agricultural product grade;Output the probability value that agricultural product belongs to different quality grades, select the maximum quality grade of probability as the grade of the current agricultural product to be predicted.The hyperspectral image of tea leaf blade obtained by the hyperspectral imaging technology used in the application has the characteristics of nearly continuous spectral curve and atlas integration, can realize nondestructive testing of tea grade, and improve the accuracy of identification.In addition, using graph convolutional neural network model, the spectral features and spatial features of tea hyperspectral image can be extracted more flexibly, so as to realize accurate automatic identification of tea grade.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agricultural product safety detection, and relates to an agricultural product grade automatic identification method and system, in particular to an agricultural product grade automatic identification method and system based on hyperspectral technology and graph convolutional neural network, which can be applied to automatic identification of tea grades. BACKGROUND

[0002] At present, the tea tree grade identification methods mainly include artificial observation method, sensor detection method and spectral method. However, the artificial observation method has strong subjectivity, long time consumption and needs professional personnel to observe and judge. The sensor detection method realizes the process analysis and quality monitoring of tea through sensors such as electronic nose and electronic tongue, but needs professional technology and equipment support, and has high cost ([Document 1]). The traditional spectral technology has been widely used in the identification, identification and quality evaluation of tea products, and the near-infrared spectral technology has the characteristics of rapidity, non-damage and simultaneous detection of multiple components, but it cannot utilize spatial information, and the recognition result is not accurate ([Document 2]). In contrast, the hyperspectral image contains spectral information and spatial information at the same time, and can comprehensively detect the internal and external quality of the research object ([Document 3]). Therefore, it is necessary to develop a more scientific and efficient tea tree grade identification method based on hyperspectral technology to realize the rapid, non-destructive and accurate identification of tea tree grades.

[0003] [1] Zhou Yibin, Wang Jun. Status and Trend of Application of New Technology in Tea Quality Evaluation [J]. Tea Science, 2019, 24(2): 82-85.

[0004] [2] Yu Yingjie, Sun Weijiang. Application of Near-Infrared Spectroscopy and Hyperspectral Technology in Tea [J]. Subtropical Agricultural Research, 2014, 10(4): 269-273.

[0005] [3] Fan Tingting, Lu Jiangming, Kang Zhilong, et al. Nondestructive Detection of Qimen Black Tea Grades Based on Hyperspectral Imaging Technology [J]. Food Industry Science and Technology, 2021. SUMMARY

[0006] In order to solve the problems of large error, high cost and need for expert experience in the prior art, the application provides an agricultural product grade automatic identification method and system based on hyperspectral technology and graph convolutional neural network.

[0007] The technical scheme adopted by the method of the application is: an agricultural product grade automatic identification method based on graph convolutional neural network, comprising the following steps:

[0008] Step 1: Collecting hyperspectral images of agricultural products and performing data preprocessing;

[0009] Step 2: input the preprocessed hyperspectral image into a graph convolutional neural network to automatically identify the agricultural product grade.

[0010] The graph convolutional neural network comprises sequentially connected superpixel segmentation modules, a superpixel graph construction module, a first graph convolutional layer, a first channel attention layer, a second graph convolutional layer, a second channel attention layer, and a classifier.

[0011] The superpixel segmentation module is configured to project pixels to a superpixel space.

[0012] The superpixel graph construction module is configured to take superpixels as graph nodes, take superpixel features as node features, and obtain an adjacency matrix of the graph by using the similarity between superpixels, thereby obtaining superpixel graph structure data.

[0013] The first channel attention layer and the second channel attention layer each introduce two learnable weight matrices Ws and We, and for each channel of the output feature, a compression scalar zi is multiplied by the two weight matrices to obtain a scaling factor si and an offset ei; the features of each channel are multiplied by the scaling factor si and then added with the offset ei to obtain weighted features.

[0014] The classifier comprises a fully connected layer and a Softmax activation function, and is configured to map the features to different categories and output a prediction probability distribution of each category.

[0015] Preferably, in step 1, the preprocessing step comprises denoising and normalization; the minimum noise separation transform is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization, or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range and eliminate the influence of spectral differences caused by different response values between different bands.

[0016] Preferably, the superpixel segmentation module first initializes the superpixel center feature with the average pixel feature of M regular grids; then calculates the similarity between the pixel point feature and the superpixel center feature; then classifies the pixel points into the superpixel with the smallest distance; and finally recalculates the superpixel center feature; wherein M = H x W / λ, H and W are the length and width of the image respectively, and λ is the superpixel scale.

[0017] Preferably, the input channel of the first graph convolutional layer is the number of hyperspectral image channels, and the output channel is 128; the input channel of the second graph convolutional layer is 128, and the output channel is 128.

[0018] Preferably, in step 2, the agricultural product grade automatic identification graph convolutional neural network is a trained agricultural product grade automatic identification graph convolutional neural network; the training process comprises the following sub-steps:

[0019] Step S1: Collecting a number of agricultural product samples of different quality grades, and using an imaging spectrometer to obtain hyperspectral data;

[0020] Step S2: Hyperspectral image data preprocessing;

[0021] Step S3: Making an agricultural product grade recognition data set;

[0022] Step S4: Continuously optimizing network parameters using a back propagation algorithm to obtain a trained network.

[0023] Preferably, in step S2, the preprocessing step includes denoising and normalization; the minimum noise separation transform is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization or z-score normalization method is used for normalization processing of the hyperspectral data, so that all bands have the same data range, and the influence of spectral differences between different bands caused by different response values is eliminated.

[0024] Preferably, in step S3, the agricultural product grade recognition data set made mainly includes hyperspectral image samples of first, second, third, fourth, fifth and sixth grade agricultural products and corresponding labels.

[0025] Preferably, in step S4, the training process includes the following steps:

[0026] (1) Forward propagation: input the hyperspectral image sample into the network, extract the spectral-spatial features of the hyperspectral image from the low level to the high level through the graph convolutional neural network, then use the full connection layer to obtain the classifier output, and use the Softmax function to convert the output result into the probability of each class;

[0027] (2) Loss calculation: use the cross-entropy function to calculate the error of the model, and adjust the network parameters according to the loss size;

[0028] (3) Back propagation: use the back propagation algorithm to calculate the gradient, and update the network weight according to the gradient; the process of back propagation is from the loss, inversely passing through the parameters of each layer, to calculate the influence degree of each parameter on the loss;

[0029] (4) Parameter update: update the network parameters according to the gradient descent algorithm to minimize the loss of the network;

[0030] (5) Repeat the above steps until the network reaches the preset training number of rounds or reaches the preset accuracy.

[0031] The technical scheme adopted by the system of the application is: an agricultural product grade automatic recognition system based on a graph convolutional neural network, comprising:

[0032] One or more processors;

[0033] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the automatic identification method for agricultural product grades based on graph convolutional neural networks.

[0034] The advantages of this invention are that the hyperspectral images of agricultural products acquired using hyperspectral imaging technology have more bands and higher spectral resolution, which can more accurately reflect the grade of agricultural products and improve the accuracy of grade identification. Furthermore, by using a graph convolutional neural network model, the hyperspectral images of agricultural products are treated as graph-structured data, allowing for more flexible extraction of spectral-spatial features in non-Euclidean space, thereby achieving automatic and accurate identification of agricultural product grades. Attached Figure Description

[0035] The technical solutions described herein are further illustrated below using examples and specific implementation methods. Additionally, accompanying drawings are used in the description of the technical solutions. Those skilled in the art can, without any creative effort, obtain other drawings and the intent of the present invention based on these drawings.

[0036] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0037] Figure 2 This is a diagram of the convolutional neural network structure for automatic identification of agricultural product grades according to an embodiment of the present invention.

[0038] Figure 3 This is a flowchart of the training process of a convolutional neural network for automatic identification of agricultural product grades provided in an embodiment of the present invention. Detailed Implementation

[0039] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0040] This embodiment uses the detection of pesticide residues in tea as an example to further illustrate the present invention. Please see... Figure 1 This embodiment provides an automatic tea grade identification method based on graph convolutional neural networks, which includes the following steps:

[0041] Step 1: Acquire hyperspectral images of tea samples and perform data preprocessing;

[0042] In an embodiment, the preprocessing step includes denoising and normalization; the minimum noise separation transform technology is used to remove the noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization, z-score normalization and the like are used to normalize the hyperspectral data, so that all bands have the same data range, and the spectral difference between different bands caused by different response values is eliminated.

[0043] Step 2: inputting the preprocessed hyperspectral image into the agricultural product grade automatic identification graph convolutional neural network to automatically identify the tea grade;

[0044] See Figure 2 The agricultural product grade automatic identification graph convolutional neural network of the embodiment includes a superpixel segmentation module, a superpixel graph construction module, two graph convolutional layers, two channel attention layers and a classifier; the first channel attention layer is arranged after the first graph convolutional layer, and the second channel attention layer is arranged after the second graph convolutional layer. Two learnable weight matrices W s and W e are introduced into each channel attention, a scaling factor s i and an offset e i are obtained by multiplying the compression scalar z i of each channel of the output feature with the two weight matrices respectively. The features of each channel are multiplied by the scaling factor s i and then added with the offset e i to obtain the weighted features.

[0045] The superpixel segmentation module of the embodiment is used to project the pixels to the superpixel space, including four steps: first, initializing the superpixel center feature with the average pixel features of M regular grids. Then, the similarity between the pixel point feature and the superpixel center feature is calculated. Next, the pixel points are classified into the superpixel with the smallest distance. Finally, the superpixel center feature is recalculated. The last three steps are iterated until convergence to a stable state. Wherein, M = H x W / λ, wherein H and W are the length and width of the image respectively, and λ is the superpixel scale, and the superpixel scale λ is set to 100.

[0046] The superpixel graph construction module of the embodiment is used to take the superpixels as the graph nodes, take the superpixel features as the node features, and obtain the adjacency matrix of the graph by using the similarity between the superpixels, so as to obtain the superpixel graph structure data.

[0047] In an embodiment, the input channel of the first graph convolutional layer is the number of hyperspectral image channels, and the output channel is 128.

[0048] In an embodiment, the input channel of the second graph convolutional layer is 128, and the output channel is 128.

[0049] In an embodiment, the classifier comprises a fully connected layer with a Softmax activation function, which maps the features to different classes after feature extraction and outputs the prediction probability distribution of each class.

[0050] The output result is the probability value of each class, and the class label with the maximum output probability is selected as the classification to which the current image to be predicted belongs.

[0051] See Figure 3 The agricultural product grade automatic identification graph convolutional neural network of the embodiment is a trained agricultural product grade automatic identification graph convolutional neural network; the training process comprises the following sub-steps:

[0052] Step S1: Collect tea leaf samples of different quality grades, and use an imaging spectrometer to obtain hyperspectral data.

[0053] In an embodiment, a plurality of tea leaf samples of different qualities are collected in an experimental tea garden, and are transported to a laboratory. A MateSpec LAFH series push-broom hyperspectral imager is used to collect hyperspectral data, and the spectral range is 400-2500 nm.

[0054] Step S2: Hyperspectral image data preprocessing.

[0055] In an embodiment, the preprocessing step comprises denoising and normalization. The minimum noise separation transform is used to remove noise in the hyperspectral image, and the maximum-minimum value normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the influence of spectral differences between different bands due to different response values is eliminated.

[0056] Step S3: Making a tea leaf grade identification data set.

[0057] In an embodiment, the collected tea leaf samples are labeled with grade categories according to first, second, third, fourth, fifth and sixth grades. The first grade tea leaf sample has the best quality, and the sixth grade tea leaf sample has the worst quality.

[0058] Step S4: Continuously optimize network parameters using a back propagation algorithm to obtain a trained network.

[0059] The graph convolutional neural network in the embodiment comprises a superpixel segmentation module, a superpixel graph construction module, two graph convolutional layers, two channel attention layers and a classifier. The first channel attention layer is placed after the first graph convolutional layer, and the second channel attention layer is placed after the second graph convolutional layer. Two learnable weight matrices W s and W e are introduced into each channel attention. For the compression scalar z i, respectively multiplied by two weight matrices to obtain a scaling factor s i and an offset e i . The features of each channel are multiplied by the scaling factor s i and then added to the offset e i to obtain the weighted features.

[0060] The superpixel segmentation module of the embodiment projects the pixels into the superpixel space, including four steps: first, the superpixel center features are initialized with the average pixel features of M regular grids. Then, the similarity between the pixel point features and the superpixel center features is calculated. Next, the pixel points are grouped into the superpixel with the smallest distance. Finally, the superpixel center features are recalculated. The last three steps are iterated until convergence to a stable state.

[0061] The superpixel graph construction module of the embodiment takes the superpixels as graph nodes and the superpixel features as node features, and obtains the adjacency matrix of the graph using the similarity between the superpixels, thereby obtaining the superpixel graph structure data.

[0062] The first graph convolutional layer of the embodiment has an input channel of the number of hyperspectral image channels and an output channel of 128.

[0063] The second graph convolutional layer of the embodiment has an input channel of 128 and an output channel of 128.

[0064] The classifier of the embodiment includes a fully connected layer and a Softmax activation function, which maps the features to different categories after feature extraction and outputs the prediction probability distribution of each category.

[0065] In one embodiment, the cross-entropy function is used to calculate the loss of the network, and the backpropagation algorithm is used to calculate the gradient, and the network parameters are updated according to the descent method to minimize the loss of the network. The gradient descent algorithm updates the network parameter value by a certain step size in the direction of the current gradient. The training process can be divided into the following steps:

[0066] Forward propagation: input the image in the training set into the network, extract features through the convolutional layer and the pooling layer, and then pass through the fully connected layer to obtain the classifier output. Finally, the output result is converted into the probability of each category using the Softmax function.

[0067] Calculate the loss: use the cross-entropy function to calculate the error (loss) of the network, and adjust the network parameters according to the loss size. A larger loss means that there is a larger difference between the network prediction result and the actual label. The cross-entropy loss function is used to calculate the error of the network, and the backpropagation algorithm is used to update the network weights.

[0068] Backpropagation: The backpropagation algorithm is used to calculate the gradient, and the network weights are updated according to the gradient. The process of backpropagation starts from the loss, and the parameters of each layer are passed back to calculate the influence of each parameter on the loss. The influence of each parameter on the loss is calculated and passed back to update the network parameters.

[0069] Parameter update: The network parameters are updated according to the gradient descent algorithm to minimize the loss of the network. The gradient descent algorithm updates the network parameter values in a certain step size according to the direction of the current gradient.

[0070] Repeat the above steps until the network reaches the preset number of training rounds or reaches a certain accuracy.

[0071] In this embodiment, the trained network parameters are loaded, and the tea grade image to be predicted is input into the graph convolutional neural network for prediction. The output result is the probability value of each class, and the class label with the maximum output probability is selected as the classification of the current image to be predicted.

[0072] The tea leaf hyperspectral image obtained by the hyperspectral imaging technology has more wavebands and higher spectral resolution, which can more accurately reflect the grade of the tea leaf and improve the accuracy of grade recognition. In addition, the graph convolutional neural network can automatically extract the features of the tea grade, thereby realizing automatic recognition of the tea grade.

[0073] The present application is not only suitable for tea, but also can be used for automatic recognition of other leaf crops, and has a wide range of applications.

[0074] It should be understood that the above description of the preferred embodiments is more detailed and should not be considered as limiting the scope of patent protection of the present application. Those skilled in the art can make substitutions or modifications without departing from the scope of the present application, which falls within the scope of the present application. The scope of protection of the present application should be subject to the appended claims.

Claims

1. An agricultural product grade automatic recognition method based on a graph convolutional neural network, characterized in that, The method comprises the following steps: Step 1: collecting a hyperspectral image of agricultural products and performing data preprocessing; Step 2: inputting the preprocessed hyperspectral image into a graph convolutional neural network to automatically identify the grade of the agricultural products; The graph convolutional neural network comprises sequentially connected superpixel segmentation modules, a superpixel graph construction module, a first graph convolutional layer, a first channel attention layer, a second graph convolutional layer, a second channel attention layer and a classifier; The superpixel segmentation module is used for projecting pixels to a superpixel space; first, the superpixel center features are initialized with the average pixel features of M regular grids; Then, the similarity between the pixel point features and the superpixel center features is calculated; then, the pixel points are grouped into the superpixel with the smallest distance; finally, the superpixel center features are recalculated; wherein M = H * W / lambda, H and W are the length and width of the image respectively, and lambda is the superpixel scale; The superpixel graph construction module is used for taking superpixels as graph nodes and superpixel features as node features, and obtaining the adjacency matrix of the graph by using the similarity between the superpixels, so as to obtain superpixel graph structure data; The first channel attention layer and the second channel attention layer both introduce two learnable weight matrices Ws and We; for the compression scalar zi of each channel of the output feature, the two weight matrices are multiplied to obtain a scaling factor si and an offset ei; the features of each channel are multiplied by the scaling factor si and then added with the offset ei to obtain weighted features; The classifier comprises a fully connected layer and a Softmax activation function, and is used for mapping the features to different categories and outputting the prediction probability distribution of each category; The graph convolutional neural network is a trained agricultural product grade automatic identification graph convolutional neural network; the training process comprises the following sub-steps: Step S1: collecting a plurality of agricultural product samples of different quality grades, and obtaining hyperspectral data by using an imaging spectrometer; Step S2: pre-processing the hyperspectral image data; Step S3: preparing an agricultural product grade identification data set; Step S4: continuously optimizing network parameters by using a back propagation algorithm to obtain a trained network.

2. The method of claim 1, wherein the method comprises: In step 1, the preprocessing step comprises denoising and normalization; The minimum noise fraction is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the spectral difference between different bands caused by different response values is eliminated.

3. The method of claim 1, wherein the method comprises: The input channel of the first graph convolutional layer is the number of hyperspectral image channels, and the output channel is 128; the input channel of the second graph convolutional layer is 128, and the output channel is 128.

4. The method of claim 1, wherein the method comprises: In step S2, the preprocessing step comprises denoising and normalization; The minimum noise fraction is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the spectral difference between different bands caused by different response values is eliminated.

5. The method of claim 1, wherein the method comprises: In step S3, the prepared agricultural product grade identification dataset mainly includes hyperspectral image samples of first, second, third, fourth, fifth and sixth grade agricultural products and corresponding labels.

6. The method of claim 1, wherein the method is implemented by a computer system. In step S4, the training process includes the following steps: (1) Forward propagation: input the hyperspectral image sample into the network, extract the spectral-spatial features of the hyperspectral image from low level to high level through the graph convolutional neural network, then use the full connection layer to obtain the classifier output, and use the Softmax function to convert the output result into the probability of each category; (2) Loss calculation: use the cross-entropy function to calculate the error of the model, and adjust the network parameters according to the loss size; (3) Back propagation: use the back propagation algorithm to calculate the gradient, and update the network weight according to the gradient; The process of back propagation is to start from the loss and pass it to the parameters of each layer in reverse to calculate the influence degree of each parameter on the loss; (4) Parameter update: update the network parameters according to the gradient descent algorithm to minimize the loss of the network; (5) Repeat the above steps until the network reaches the preset training number or reaches the preset accuracy.

7. An agricultural product grade automatic recognition system based on a graph convolutional neural network, characterized in that, Comprise: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic identification method of agricultural product grade based on graph convolutional neural network as claimed in any one of claims 1 to 6.

Images