Soybean grain single-band image classification method and system based on deep learning

Abstract

The invention discloses a soybean grain single-band image classification method and system based on deep learning, and belongs to the technical field of hyperspectral application. The problem that in the hyperspectral data screening process, due to the fact that a hyperspectral image contains a large number of wave bands, once image noise is large or missing values exist, the classification accuracy class is low is solved. The method comprises the following steps: acquiring a soybean grain image and spectral data; carrying out black and white correction on the collected hyperspectral image to obtain a processed soybean grain image; selecting an optimal wave band according to SPA and KNN algorithms; carrying out image enhancement and threshold segmentation on the original data set and the new data set, and separating a soybean seed sample from a background by adopting a morphological method; the method comprises the following steps: training through an SENet-ResNet34-DCN model, and extracting a plurality of features of a test sample; and outputting a classification result. The method is suitable for soybean seed authenticity identification and quality analysis scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral application, and in particular relates to a soybean seed single-band image classification technology based on deep learning. Background Art

[0002] Soybeans play an important role in the world. They are not only one of the main agricultural crops, but also a nutritious food raw material. Soybeans are widely traded in the international market, especially used in feed, vegetable protein, edible oil and food processing industries. The United States, Brazil and other countries are the world's major soybean producers and exporters.

[0003] It is indeed difficult to distinguish fake seeds on the market, especially with the advancement of technology, the appearance and quality of fake seeds are getting closer and closer to real seeds. Fake seeds may not only cause economic losses, but also affect the quality and yield of agricultural production.

[0004] In recent years, hyperspectral imaging technology is an advanced technology that has developed rapidly. It combines spectral information and image information, providing a new way for non-destructive testing. Through this technology, not only can the spectral information of the sample be obtained, but also the image of the sample can be collected. This fusion enables more comprehensive and rich data and information to be obtained when classifying seeds. Through hyperspectral imaging technology, spectral information at different wavelengths and the morphological characteristics of the sample can be obtained. This helps to improve the accuracy and efficiency of quality analysis.

[0005] However, hyperspectral images usually contain a large number of bands, and if the image is noisy or has missing values, the classification accuracy may be affected. Summary of the invention

[0006] The present invention proposes a soybean seed single-band image classification method and system based on deep learning, which aims to solve the problem of low classification accuracy when screening hyperspectral data, because hyperspectral images contain a large number of bands. Once the image has more noise or missing values, it will lead to low classification accuracy.

[0007] The present invention proposes a soybean seed single-band image classification method based on deep learning, which includes: S1: Collect hyperspectral images and spectral data of soybean grains; S2: performing black and white correction on the hyperspectral image to obtain a processed soybean grain image, and selecting the optimal band; S3: dividing the hyperspectral image into a training set, a test set and a validation set; S4: Select the ResNet34 deep learning model; S5: Process the ResNet34 deep learning model to obtain a SENet-ResNet34-DCN model; S6: Input the training set, test set and validation set into the SENet-ResNet34-DCN model and output the classification results.

[0008] Furthermore, a preferred solution is provided: in S1, a hyperspectral image acquisition device is used to collect hyperspectral images and spectral data of soybean seeds, and the hyperspectral image acquisition device includes: a Pika hyperspectral imager, a linear moving platform, a mounting tower, a lighting device and a computer.

[0009] Furthermore, a preferred solution is provided: in S2, the SPA algorithm and the KNN algorithm are used to select the optimal band.

[0010] Furthermore, a preferred solution is provided: obtaining the spectral average of 462 spectra, and extracting the corresponding characteristic bands according to the mean square error of each iteration.

[0011] Furthermore, a preferred solution is provided: S2 includes: using the bands extracted by the SPA algorithm as a data set, each band as a feature, training each band using the KNN algorithm, and evaluating each band using 5-fold cross validation, and selecting the optimal band based on the score.

[0012] Furthermore, a preferred solution is provided: S5 is specifically: adding the SENet attention mechanism after the maximum pooling layer and Layer1 of the ResNet34 deep learning model to obtain the SENet-ResNet34-DCN model, where Layer1 in ResNet34 represents the first residual block part of the network.

[0013] Furthermore, a preferred solution is provided: the S5 also includes: in the SENet-ResNet34-DCN model, deformable convolution is used instead of ordinary convolution.

[0014] The present invention also proposes a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a single-band image classification method for soybean grains based on deep learning according to any one or more combinations of the above-mentioned schemes.

[0015] The present invention also proposes a computer-readable storage medium, which is used to store a computer program, and the computer program executes the steps of a soybean seed single-band image classification method based on deep learning described in any one or more combinations of the above-mentioned schemes.

[0016] The present invention also proposes a soybean seed single-band image classification system based on deep learning, the system is implemented based on a soybean seed single-band image method based on deep learning described in any one or more of the above schemes, and the system includes: Data acquisition module: used to collect hyperspectral images and spectral data of soybean seeds; Image correction module: used to perform black and white correction on the hyperspectral image, obtain the processed soybean grain image, and select the optimal band; Image division module: used for dividing the hyperspectral image into a training set, a test set and a validation set; Model selection module: used to select the ResNet34 deep learning model; Model improvement module: used to process the ResNet34 deep learning model to obtain the SENet-ResNet34-DCN model; Data classification module: used to input the training set, test set and validation set into the SENet-ResNet34-DCN model and output the classification results.

[0017] Compared with the prior art, the advantages of the present invention are: The soybean seed single-band image classification method based on deep learning proposed in the present invention uses SPA and KNN algorithms to select the optimal band to form a new data set. The SPA algorithm can extract key characteristic wavelengths from the entire band and remove redundant information in the original spectral matrix; the KNN algorithm performs classification by measuring distance. The combination of the two can improve the classification accuracy of soybean seeds. For example, by using KNN to train different bands and using 5-fold cross-validation to evaluate the selection of bands with higher importance, more effective features are provided for classification, thereby improving the classification accuracy.

[0018] The soybean seed single-band image classification method based on deep learning proposed in this invention adopts SENet -ResNet34 - DCN model to further optimize the model performance, generalization ability and efficiency. Adding SENet attention mechanism on the basis of ResNet34 model can make the convolutional network pay more attention to the features that contribute to the task and improve the recognition ability of important information; using deformable convolution to replace part of ordinary convolution can adapt to the non-uniform feature distribution in the input data, flexibly adjust the shape of convolution kernel, better capture local features, improve the classification accuracy of soybean seeds, and make the model better adapt to the specific soybean seed classification task requirements.

[0019] The present invention is applicable to the scenarios of soybean seed authenticity identification and quality analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a schematic flow chart of a soybean seed single-band image classification method based on deep learning according to a specific embodiment of the present invention; Figure 2 This is a flowchart of data set processing according to a first specific implementation mode of the present invention; Figure 3 This is a schematic diagram of the SENet-ResNet34-DCN model structure described in the first specific implementation mode of the present invention. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0023] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0024] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0025] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific implementation methods disclosed below.

[0027] Implementation method 1: Reference Figure 1 , Figure 2 , Figure 3 This embodiment will be described.

[0028] This embodiment proposes a soybean seed single-band image classification method based on deep learning, including using a hyperspectral image acquisition device to obtain soybean seed images and spectral data; performing black and white correction on the acquired hyperspectral image to obtain a processed soybean seed image; selecting the optimal band according to the SPA and KNN algorithms; performing image enhancement and threshold segmentation on the original data set and the new data set, and using morphological methods to separate the soybean seed sample from the background well; extracting multiple features of the test sample through SENet-ResNet34-DCN model training; and outputting the classification results. Specifically, it includes the following steps: S1: Collect hyperspectral images and spectral data of soybean grains; S2: performing black and white correction on the hyperspectral image to obtain a processed soybean grain image, and selecting the optimal band; S3: dividing the hyperspectral image into a training set, a test set and a validation set; S4: Select the ResNet34 deep learning model; S5: Processing the ResNet34 deep learning model to obtain a SENet-ResNet34-DCN model, specifically: adding a SENet attention mechanism after the maximum pooling layer and Layer1 of the ResNet34 deep learning model to obtain a SENet-ResNet34-DCN model, where layer1 in ResNet34 represents the first residual block part of the network, including 3 residual blocks, each residual block consists of two 3x3 convolutional layers, and data is processed by batch normalization and ReLU activation function; in the SENet-ResNet34-DCN model, deformable convolution is used instead of ordinary convolution; S6: Input the training set, test set and validation set into the SENet-ResNet34-DCN model, obtain the classification accuracy of the test set, and output the classification result.

[0029] Specifically: In S1, a hyperspectral image acquisition device is used to collect hyperspectral images and spectral data of soybean grains. The hyperspectral image acquisition device includes a Pika hyperspectral imager, a linear moving platform, a mounting tower, a lighting device, and a computer.

[0030] Among them, the imaging spectrometer is Pika XC2 (10.1cmx27.5cmx7.4cm), with a wavelength range of 400nm-1000nm, a spectral resolution of 1.3nm, a sampling interval of 0.67nm, and an exposure time of 56.37ms.

[0031] The spectrum acquisition and analysis software used was Spectronon.

[0032] The hyperspectral data collected has a total of 462 bands. There are seven types of soybean seeds, about 200 seeds in each type, and the collected images are saved in bil format in the device. When collecting spectral information, each variety is placed on the electric control displacement table as a research object, and try to keep the placement uniform and the intervals the same. As the platform moves, the camera scans the entire platform. To avoid interference from other indoor light, the hyperspectral image acquisition process is completed in a dark box.

[0033] In S2, the SPA algorithm and the KNN algorithm are used to select the optimal band. The SPA algorithm is extracted in the following manner: the spectral average of 462 spectra is obtained, and the corresponding feature band is extracted according to the mean square error of each iteration. The band extracted by the SPA algorithm is used as a data set, each band is used as a feature, and the KNN algorithm is used to train each band, and each band is evaluated using 5-fold cross validation, and the optimal band is selected according to the score.

[0034] SPA is a forward iterative search algorithm, which works by starting from one wavelength and gradually introducing new variables in each iteration until the preset number of variables N is reached. The algorithm uses projection analysis technology to compare and select the wavelength with the largest projection vector on other wavelengths as a candidate wavelength. SPA selects a combination of variables with minimal redundant information and collinearity. Its advantage is that it can extract several key characteristic wavelengths from the full band to remove redundant information in the original spectral matrix. In order to ensure the accuracy of data input into the model classification after feature extraction, and to reduce costs for the subsequent production of low-band and efficient spectral detection equipment related to soybean variety classification, this embodiment uses the SPA algorithm to extract the corresponding characteristic bands according to the mean square error (MSE) of each iteration after obtaining the spectral average of 462 spectra.

[0035] The K-nearest neighbor (KNN) algorithm is an instance-based learning algorithm and one of the classic methods in the field of traditional machine learning. It is based on statistical principles and classifies data points by measuring the distance between them. In the KNN algorithm, the label of a sample depends on its nearest neighbor in the feature space. k The algorithm will find the majority label of the new sample in the training data set. k samples, and based on this k The labels of samples are used to classify new samples.

[0036] The basic process of the KNN algorithm is as follows: (1) Collect a dataset with known category labels.

[0037] (2) For a new sample to be classified, calculate its distance from each sample in the training data set. This distance is usually calculated using the Euclidean distance method. Assume that all samples are defined in N In the dimensional space, usually each sample x can be obtained through the feature vector To indicate that here Represents the i-th eigenvalue of sample x. The Euclidean distance calculation formula is shown as follows: , The Euclidean distance is used to calculate the distance between two samples. The smaller the distance, the higher the similarity and the more likely they are neighbors. The larger the distance, the lower the similarity.

[0038] (3) Select the closest sample to the new sample from the training data set k samples.

[0039] (4) According to this k The labels of the nearest neighbor samples are used to determine the labels of new samples through a majority voting mechanism.

[0040] The KNN algorithm is simple and intuitive, easy to implement, and has shown good performance in many practical applications. However, it also has limitations. Its classification effect will decrease when the sample size is large or the feature attributes are more. In addition, the choice of K value also has an important impact on the performance of the KNN algorithm. If the K value is set too small, the algorithm may be affected by noisy data; if the K value is set too large, the algorithm cannot accurately reflect the data structure. Band selection can also be combined with KNN. The bands selected by SPA are used as data sets, and each band can be used as a feature. KNN is used to train each band, and each band is evaluated using 5-fold cross validation. The bands with higher importance are selected as the best features based on the scores.

[0041] In S3, the collected hyperspectral images are processed by an image processing algorithm and divided into a training set, a test set and a validation set. The image processing steps specifically include: identifying regions of interest, removing noise, threshold segmentation, detecting seed outlines, obtaining a seed shell matrix, and cropping the original image with a seed outer matrix to obtain a single seed image.

[0042] The identified region of interest is used to distinguish the original soybean seeds from the background.

[0043] The Gaussian filtering method used in the noise removal can smooth the image and reduce noise. The convolution operation based on the Gaussian function can effectively blur the details in the image. The formula is as follows: The formula for the one-dimensional Gaussian function is: , in, is the mean of x, is the variance of x, because when calculating the mean, the center point is the origin, so the mean is equal to 0.

[0044] , According to the one-dimensional Gaussian function, the two-dimensional Gaussian function can be derived: , Furthermore, the threshold processing formula is: , in, Represents the grayscale value of the point in the input image. Represents the grayscale value of the output image at that point. is the selected threshold.

[0045] In S5, the SENet attention mechanism is added on the basis of the ResNet34 model. It is a channel-type attention mechanism. The main contents are squeeze and excitation. On the basis of the original learning mechanism, a new network path is opened up. After operation, the attention degree of each channel in the feature map is obtained, and an attention weight is configured for each feature channel according to this degree, so that the convolution network pays more attention to these feature channels, thereby realizing the channels of the feature map that are useful for the current task and suppressing the feature channels that are not very useful for the current task. Before the SE attention mechanism is input, the attention degree of each channel of the feature map is the same. After passing through SENet, different colors represent different weights, making the importance of each feature channel different, so that the network consciously focuses on certain channels with large weight values. The network with the SENet attention mechanism added after the maximum pooling layer and Layer1 can pay more attention to the features that contribute to the task and improve the ability to recognize important information. SENet-ResNet34-DCN performs end-to-end training. The weights of the network are updated by backpropagation and gradient descent.

[0046] On the basis of SENet-ResNet34-DCN, deformable convolution is used to replace some ordinary convolutions. The deformable convolution layer can adapt to the non-uniform feature distribution in the input data and flexibly adjust the shape of the convolution kernel to better capture local features. Secondly, compared with the convolution kernel of fixed shape, the deformable convolution layer has stronger expression ability, which helps to improve the model's ability to learn complex patterns and abstract features. In addition, by introducing the deformable convolution layer, the network model can more effectively handle the spatial changes in the input data, thereby improving the classification accuracy of soybean seeds. In general, the use of deformable convolution layers is expected to optimize the performance of convolutional neural networks in tasks such as image processing. Taking advantage of these benefits of deformable convolution layers, soybean seed classification is applied.

[0047] Implementation method 2: This embodiment proposes a soybean seed single-band image classification system based on deep learning. The system is implemented based on the soybean seed single-band image classification method based on deep learning described in Embodiment 1. The system includes: Data acquisition module: used to collect hyperspectral images and spectral data of soybean grains; Image correction module: used to perform black and white correction on the hyperspectral image, obtain the processed soybean grain image, and select the optimal band; Image division module: used for dividing the hyperspectral image into a training set, a test set and a validation set; Model selection module: used to select the ResNet34 deep learning model; Model improvement module: used to process the ResNet34 deep learning model to obtain the SENet-ResNet34-DCN model; Data classification module: used to input the training set, test set and validation set into the SENet-ResNet34-DCN model and output the classification results.

[0048] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0049] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0050] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0051] It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may be implemented in the form of a complete hardware implementation, a complete software implementation or an implementation combining software and hardware. Moreover, the present disclosure may be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes. Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present disclosure rather than to limit its protection scope. Although the present disclosure has been described in detail with reference to the above implementation modes, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation modes of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the disclosed claims to be approved.

Claims

1. A soybean seed single-band image classification method based on deep learning, characterized in that: The method comprises: S1: Collect hyperspectral images and spectral data of soybean grains; S2: performing black and white correction on the hyperspectral image to obtain a processed soybean grain image, and selecting the optimal band; S3: dividing the hyperspectral image into a training set, a test set and a validation set; S4: Select the ResNet34 deep learning model; S5: Process the ResNet34 deep learning model to obtain a SENet-ResNet34-DCN model; S6: Input the training set, test set and validation set into the SENet-ResNet34-DCN model and output the classification result.

2. The soybean seed single-band image classification method based on deep learning according to claim 1, characterized in that: In S1, a hyperspectral image acquisition device is used to collect hyperspectral images and spectral data of soybean seeds. The hyperspectral image acquisition device includes: a Pika hyperspectral imager, a linear moving platform, a mounting tower, a lighting device, and a computer.

3. The soybean seed single-band image classification method based on deep learning according to claim 1, characterized in that: In S2, the SPA algorithm and the KNN algorithm are used to select the optimal band.

4. The soybean seed single-band image classification method based on deep learning according to claim 3, characterized in that: The extraction method of the SPA algorithm is: obtaining the spectral average value of 462 spectra, and extracting the corresponding characteristic band according to the mean square error of each iteration.

5. The soybean seed single-band image classification method based on deep learning according to claim 4, characterized in that: The S2 includes: using the bands extracted by the SPA algorithm as a data set, each band as a feature, training each band using the KNN algorithm, and evaluating each band using 5-fold cross validation, and selecting the optimal band according to the score.

6. The soybean seed single-band image classification method based on deep learning according to claim 1, characterized in that: Specifically, S5 is as follows: adding the SENet attention mechanism after the maximum pooling layer and Layer1 of the ResNet34 deep learning model to obtain the SENet-ResNet34-DCN model, wherein Layer1 in ResNet34 represents the first residual block part of the network.

7. The soybean seed single-band image classification method based on deep learning according to claim 1, characterized in that: The S5 also includes: in the SENet-ResNet34-DCN model, deformable convolution is used instead of ordinary convolution.

8. Computer device, characterized in that It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a soybean grain single-band image classification method based on deep learning according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program executes the steps of a soybean seed single-band image classification method based on deep learning as described in any one of claims 1-7.

10. A soybean seed single-band image classification system based on deep learning, characterized in that: The system is implemented based on a soybean seed single-band image classification method based on deep learning as described in any one of claims 1 to 7, and the system comprises: Data acquisition module: used to collect hyperspectral images and spectral data of soybean seeds; Image correction module: used to perform black and white correction on the hyperspectral image, obtain the processed soybean grain image, and select the optimal band; Image division module: used for dividing the hyperspectral image into a training set, a test set and a validation set; Model selection module: used to select the ResNet34 deep learning model; Model improvement module: used to process the ResNet34 deep learning model to obtain the SENet-ResNet34-DCN model; Data classification module: used to input the training set, test set and validation set into the SENet-ResNet34-DCN model and output the classification results.

Images