Detection method and detection system for identifying sorghum variety and storage medium

Abstract

The invention provides a detection method and a detection system for identifying sorghum varieties and a storage medium, and belongs to the technical field of variety identification. According to the method, a two-dimensional feature adaptive convolution model (DD-FACM) combining a one-dimensional convolutional neural network (1D CNN) and a two-dimensional convolutional neural network (2D CNN) is established; the classification accuracy of the method is as high as 100%, and is respectively improved by 8% and 4.2% compared with an SVM (Support Vector Machine) model and an OfficientNet-B3 model. And finally, the features extracted by the DD-FACM are visualized by using t-distribution-random neighbor embedding (t-SNE). The detection method provided by the invention can realize rapid, accurate and lossless identification of different sorghum varieties. The invention not only provides an efficient sorghum variety identification method for brewing enterprises, but also provides technical support for variety identification research in related fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of sorghum variety identification, and specifically relates to a detection method, a detection system and a storage medium for identifying sorghum varieties. Background Art

[0002] The main components in sorghum include starch, protein, fat, etc., and the component contents of different sorghum varieties vary. As one of the main raw materials for brewing liquor, the component contents of sorghum will directly affect the yield and quality of liquor brewing. Therefore, during the liquor brewing process, it is particularly important to identify whether there is a mixture of different types of sorghum. However, due to the fact that some sorghum varieties are extremely similar in morphology and color, it is very difficult to accurately distinguish them with the naked eye. Traditional sorghum identification methods include morphological identification, molecular biology identification, chemical identification, etc. These traditional methods mainly rely on chemical detection and manual experience judgment. The former is time-consuming and laborious and has a certain destructive effect on sorghum, while the latter is subject to subjective factors and it is difficult to ensure a high accuracy rate.

[0003] Patent CN117074599A discloses a method for identifying sorghum varieties for brewing Maotai-flavor liquor. This invention obtains the physical and chemical indexes and volatile substance information of sorghum, and inputs this information into the constructed PLS-DA discrimination model for discrimination calculation to obtain the discrimination result of sorghum varieties. The PLS-DA discrimination model establishes the corresponding relationship between the physical and chemical indexes and volatile substance information of sorghum and sorghum varieties (for liquor brewing or not for liquor brewing). However, as a destructive method, this method has relatively high requirements for sample preparation, requires continuous use of chemical reagents, and requires professional personnel for operation.

[0004] Hyperspectral imaging technology (HSI) is a combination of spectral technology and image technology, which can obtain spectral information and image information of samples. In recent years, hyperspectral imaging technology combined with machine learning algorithms has been widely applied in food analysis and detection. Relevant academic research reports that a partial least squares discriminant analysis (PLS-DA) was used to establish a sorghum adulteration model to detect whether sorghum was adulterated, and the comprehensive accuracy of the model reached about 90%; first, a competitive adaptive reweighted sampling (CARS) combined with a random forest (RF) model was used to classify and identify 27 sorghum varieties, and the accuracies of the training set and prediction set of this model reached 95.00% and 84.07% respectively; support vector machine (SVM), k-nearest neighbor algorithm (k-NN) and radial basis function neural network (RBFNN) models were used to identify different types of raisins; partial least squares discriminant analysis (PLS-DA) was used to distinguish organic eggs from traditional eggs, and the classification accuracy was 96.3%; local preserving projection (LPP) combined with support vector machine (SVM) model was used to extract the characteristics of rice leaves to identify 10 early rice varieties and 10 late rice varieties respectively, and the identification accuracies were 91.67% and 97.33% respectively; a competitive adaptive reweighted sampling (CARS) algorithm was combined with a support vector machine (SVM) model, and then image processing technology was used to optimize it for identifying different types of okra seeds, and the correct recognition rate of the prediction set was 94.83%; the above research results show that hyperspectral technology combined with traditional machine learning has achieved good classification results. However, these traditional machine learning algorithms need to preprocess spectral data and extract characteristic wavelengths before modeling, which consumes a lot of time.

[0005] Therefore, it is very necessary to explore a fast, accurate and non-destructive method for identifying sorghum varieties for quality control in brewing production. Summary of the Invention

[0006] The purpose of the present invention is to provide a detection method, a detection system and a storage medium for identifying sorghum varieties. By using hyperspectral imaging technology and 3D ultra-depth-of-field microscope to extract hyperspectral data and ultra-depth-of-field image data of sorghum grains respectively and combining them to construct a two-dimensional feature adaptive convolution model (DD-FACM), accurate identification of different sorghum varieties is realized.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions: The first object of the present invention is to provide a detection method for identifying sorghum varieties, including the following specific steps: S1. Obtain sorghum grain samples, collect hyperspectral images of sorghum grains using a hyperspectral imaging system, perform black-and-white correction on the hyperspectral images, and use the Otsu algorithm to segment the background and samples to extract the spectral data of the samples; collect the super-depth-of-field image data of the stigma remnants of the sorghum grain samples using a 3D super-depth-of-field microscope; S2. Construct a dataset with the obtained spectral data and super-depth-of-field image data, and input it into a two-dimensional feature adaptive convolution model for training to obtain a detection model; The two-dimensional feature adaptive convolution model consists of a spectral feature extraction module, an image feature extraction module, and a feature fusion module. The spectral feature extraction module is used to extract spectral features from the spectral data, the image feature extraction module is used to extract image features from the image data, and the feature fusion module is used to fuse and add the spectral features and image features. The fused features are sent to the output layer for classification and recognition; S3. Input the spectral data and super-depth-of-field image data of the sorghum samples to be tested into the detection model to obtain high-dimensional feature representations, and then use the t-SNE algorithm to perform dimensionality reduction processing on these high-dimensional data to achieve two-dimensional visualization classification and identification of sorghum varieties.

[0008] Further, the hyperspectral imaging system collects hyperspectral images of sorghum grains through push-broom linear scanning, and performs black-and-white correction on the collected hyperspectral images. The formula for black-and-white correction is: , where, is the corrected hyperspectral data; is the original hyperspectral image; is the dark reference image collected with the lens covered; is the standard whiteboard image collected.

[0009] Further, the process of obtaining the super-depth-of-field image data is as follows: Select a clear cross-section position of the stigma remnant, scan the sorghum grains along the Z-axis to realize the complete 3D construction of the true morphology of the stigma remnant position of the sorghum grains, and obtain the high-resolution image data of the sorghum grains.

[0010] Further, the spectral feature extraction module uses a one-dimensional convolutional neural network to extract the features of the spectral data. The one-dimensional convolutional neural network consists of 5 convolutional layers and 4 fully connected layers. In the feature extraction process of each convolutional layer, a first channel attention mechanism is used to adaptively weight and enhance the extracted spectral features.

[0011] Further, the calculation formula for the channel attention weight matrix generated by the first channel attention mechanism is: , where, is the first-channel attention weight matrix, is a multi-layer perceptron mechanism, is the input spectral feature size C×1×1, and are the average pooling operation and the global average pooling operation respectively, is the Sigmoid activation function.

[0012] Furthermore, the image feature extraction module uses a two-dimensional convolutional neural network to extract the features of the super-depth-of-field image data. The two-dimensional convolutional neural network uses the VGG16 network as the image feature extractor, which is composed of 13 convolutional layers, 5 max-pooling layers and 3 fully-connected layers. In the feature extraction process of each convolutional layer, a global attention mechanism is used to adaptively weight the extracted feature maps.

[0013] Furthermore, the global attention mechanism includes a second-channel attention mechanism and a spatial attention mechanism. The input feature map first passes through the channel of the second-channel attention mechanism to obtain the channel attention weight, and multiplies it with the original feature map to obtain a new feature map , and the feature map is input into the channel of the spatial attention mechanism to obtain the spatial attention weight, and multiplies it with the feature map F2 to output the final image feature , and the calculation formula is as follows: , , where, and M s are the channel attention module and the spatial attention module respectively; is the input feature map, is the new feature map obtained by multiplying the input feature map after being weighted by the channel attention, is the finally output feature.

[0014] Furthermore, the process of fusion and addition is as follows: the spectral features coefficients and the image features coefficients of the two shared connection layers are respectively added with 64 important coefficients, and the two coefficients are adjusted to between 0 and 1 through the linear regression coefficients, and then the Softmax function is used to calculate the weights of the coefficients. The processed and coefficients act on the weighted corresponding spectral features and image features respectively.

[0015] The second object of the present invention is to provide a detection system for identifying sorghum varieties. The system is based on the above method and at least includes the following components built in the system: A data acquisition module, configured to acquire hyperspectral images and super-depth-of-field image data of sorghum grain samples; A data processing module, configured to perform black-and-white correction on the acquired hyperspectral images, and use the Otsu algorithm to segment the background and samples to obtain spectral data; A spectral feature extraction module, configured to extract features of the spectral data using a one-dimensional convolutional neural network; An image feature extraction module, configured to extract features of the super-depth-of-field image data using a two-dimensional convolutional neural network; A feature fusion module, configured to fuse and add the spectral features and image features; A detection model construction module, configured to construct a data set with the acquired spectral data and super-depth-of-field image data, and input it into a two-dimensional feature adaptive convolutional model for training to obtain a detection model; the two-dimensional feature adaptive convolutional model is composed of the spectral feature extraction module, the image feature extraction module, and the feature fusion module; A detection module, configured to input the spectral data and image data of the sorghum grain sample to be tested into the detection model to obtain high-dimensional data, and use the t-distribution - stochastic neighborhood embedding algorithm to reduce the dimension and visualize the input high-dimensional data, so as to realize the visual detection of identifying sorghum varieties.

[0016] The third object of the present invention is to provide a computer-readable storage medium, which stores a program that can be executed by one or more processors to implement the above-mentioned detection method for identifying sorghum varieties.

[0017] The explanations of the English abbreviation characters in the present invention are as follows: FC-RGB: It is a special digital image representation method. Although it structurally contains pixel values of the red, green, and blue channels, its color does not reflect the true color of the photographed object. This kind of image is usually converted from single-channel data (such as grayscale images, thermal images, or specific band data) through a color mapping algorithm, artificially mapping the numerical range into the RGB color space to enhance the visual effect or highlight specific features.

[0018] ROI: Region of Interest; PLS-DA: Partial Least Squares Discriminant Analysis; 1DCNN: One-dimensional Convolutional Neural Network; 2DCNN: Two-dimensional Convolutional Neural Network; DD-FACM: Two-dimensional Feature Adaptive Convolutional Model.

[0019] Compared with the prior art, the beneficial effects brought by the technical solution provided by the present invention are: (1) The present invention provides a dual-dimensional feature adaptive convolution model (DD-FACM) established by combining spectral data extracted from a hyperspectral imaging (HSI) system and image data extracted from a 3D super-depth microscope, using a one-dimensional convolutional neural network (1D CNN) and a two-dimensional convolutional neural network (2D CNN); its classification accuracy is as high as 100%, and compared with the SVM and EfficientNet-B3 models, it is improved by 8% and 4.2% respectively. Finally, t-distributed stochastic neighbor embedding (t-SNE) is used to visualize the features extracted by DD-FACM.

[0020] (2) The detection method provided by the present invention can achieve rapid, accurate, and non-destructive identification of different sorghum varieties. It not only provides an efficient sorghum variety identification method for brewing enterprises but also provides technical support for variety identification research in related fields.

[0021] (3) Through multi-modal data fusion technology, the present invention makes full use of the spectral and morphological characteristics of sorghum grains. Compared with traditional identification methods based on a single data source, it can capture the subtle differences between varieties more comprehensively. The complementarity of dual-dimensional features significantly improves the robustness and generalization ability of the model, and it can still maintain high-precision identification even when there are certain variations in the samples.

[0022] (4) The adaptive convolution architecture adopted by the present invention can automatically learn and adjust the weight allocation of features in different dimensions without manual setting of fusion parameters, greatly simplifying the model usage process. Compared with traditional methods that require expert experience for feature selection and parameter tuning, the present invention reduces the technical application threshold and enables non-professionals to quickly get started.

[0023] (5) The non-destructive detection technology adopted by the present invention ensures the integrity of sorghum samples. The tested samples can be continued to be used for subsequent brewing processes or other analytical tests, avoiding sample waste caused by traditional destructive detection methods. This is of particularly important significance for the detection of precious varieties or small batches of samples.

[0024] (6) The detection system established by the present invention has good scalability and migration ability. By simple model fine-tuning, it can be applied to the identification tasks of other cereal varieties (such as wheat, barley, corn, etc.). At the same time, this technical framework can also be popularized and applied to agricultural quality control fields such as seed quality detection, pest and disease identification, and maturity determination, with broad application prospects and economic value. Brief Description of the Drawings

[0025] Figure 1 is a flowchart of the detection method for identifying sorghum varieties provided by the present invention; Figure 2 is a spectral curve graph of different sorghum varieties with abnormal effects removed; Figure 3 Average spectral curve diagrams of different varieties of sorghum; Figure 4 Images of the stigma relic positions of different varieties of sorghum under a super-depth-of-field microscope; Figure 5 Schematic diagram of the dual-dimensional feature adaptive convolution model (DD-FACM) combining a one-dimensional convolutional neural network (1D CNN) and a two-dimensional convolutional neural network (2D CNN); Figure 6 Schematic diagram of the channel attention mechanism structure of the one-dimensional convolutional neural network module; Figure 7 Schematic diagram of the channel attention mechanism structure of the two-dimensional convolutional neural network module; Figure 8 Schematic diagram of the spatial attention mechanism structure of the two-dimensional convolutional neural network module; Figure 9 Confusion matrix result diagram classified by the dual-dimensional feature adaptive convolution model (DD-FACM); Figure 10 t-SNE feature visualization result diagram of the test set before classification training of the dual-dimensional feature adaptive convolution model (DD-FACM); Figure 11 After visualizing the spectral and super-depth-of-field image data of the test set, the characteristic points corresponding to different sorghum varieties are fully clustered and separated in the 3D space for display; Figure 12 Schematic diagram of the detection system structure for identifying sorghum varieties provided by the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the following further describes the detailed implementation manners of the present invention in combination with specific embodiments and drawings. For those not specified in the embodiments in terms of specific technologies or conditions, they shall be carried out according to the technologies or conditions described in the literature in the field or according to the product instructions.

[0027] The raw materials, equipment and methods used in the present invention, unless otherwise specified, are all common raw materials, equipment or methods in the field.

[0028] As Figure 1 shown, it is the flowchart of the detection method for identifying sorghum varieties of the present invention, which includes the following steps: (1) Prepare sorghum samples of five different varieties; collect the hyperspectral images of the samples using a hyperspectral imaging system; (2) Perform black and white correction on the collected hyperspectral images; (3) Use the Otsu algorithm to segment the samples and background in the corrected hyperspectral image, and extract the spectral data of the samples after background segmentation; (4) Use a 3D ultra-depth-of-field microscope system to collect image data of sorghum; (5) Use the spectral feature extraction module in the two-dimensional feature adaptive convolutional model (DD-FACM) to extract the spectral features in the spectral data; (6) Use the image feature extraction module in the two-dimensional feature adaptive convolutional model (DD-FACM) to extract the image features in the sorghum data; (7) Input the extracted spectral features and image features into the feature fusion module, fuse and add the spectral features and image features, and the fused features are sent to the output layer for classification and recognition.

[0029] Example 1 The present invention provides a detection method for identifying sorghum varieties, specifically including the following steps: 1. Data collection 1.1 Sample preparation In this example, 5 sorghum varieties were selected: Aigan Kangxing No. 8 (A), Hongguan No. 1 (HG), and Hongta No. 2 (HT) from Jinan City, Shandong Province; Hongyingzi (HY) and Cuicuitian (C) from Taigu, Shanxi. For each of these five varieties, 100 complete sorghum seed samples were selected, for a total of 500 sorghum seed samples. Selecting samples from different geographical locations and including multiple varieties ensured the diversity and representativeness of the dataset of the present invention.

[0030] 1.2 Hyperspectral imaging system The hyperspectral imaging system used for collecting spectral information selected in the present invention is produced by Jiangsu Shuanglihepu Technology Co., Ltd. Its main components include a hyperspectral camera, a system auxiliary bracket, a calibration black and white board, computer data acquisition software, four 55W halogen lamps, etc. The spectral acquisition range of the hyperspectral camera is 900 - 1700nm, the spectral resolution is 5nm, the number of bands is 512, and the number of pixels is 660×640. In order to ensure the accuracy of the spectral information of the samples, the instrument parameters were carefully adjusted. The detailed parameters of the hyperspectral system are shown in Table 1. Each time of collection, 1 sorghum grain was placed in a petri dish, and the petri dish was placed at the center of the collection platform, 40 cm away from the camera lens. Finally, the hyperspectral image was collected through SpectraVIEW software.

[0031] Table 1. Hyperspectral imaging system parameters.

[0032]

[0033] 1.3 Black and white calibration To reduce the impact of noise introduced by the camera dark current and light source non-uniformity on the quality of hyperspectral images, it is necessary to perform black-and-white correction on the acquired hyperspectral images using white and black board images. This preprocessing step can effectively correct camera noise and the non-uniformity of the light source intensity distribution, thereby obtaining more accurate and stable spectral reflectance data and better reflecting the true spectral characteristics of the sample. The original image is black-and-white corrected according to the following formula:

[0034] In the formula, is the corrected hyperspectral image, is the hyperspectral image before correction, is the white reference image, is the black reference image.

[0035] 1.4 Extracting sample spectral data The hyperspectral image not only contains information about sorghum grains but also contains background information unrelated to them. To accurately extract the spectral information of sorghum grains, it is necessary to remove this background interference. In this study, the Otsu algorithm was used to achieve background removal. To obtain the spectral data of sorghum grains, each sorghum grain was taken as an independent region of interest (ROI). By collecting the spectral reflectance data of all pixels within each ROI and calculating their average value, the average spectral reflectance of each ROI was obtained. The obtained average spectral curve is the one-dimensional spectral data of sorghum grains in this study. While extracting the spectral data, the false color image (FC-RGB) in the hyperspectral image of sorghum grains was also retained as the two-dimensional image data used in subsequent comparative analysis.

[0036] Due to the existence of abnormal bands affected by dark current and noise during the acquisition process, it is necessary to remove the abnormal bands in the spectrum. The first 11 and the last 27 abnormal bands were removed respectively, and the number of bands was reduced from the original 512 bands to 474 bands. The original spectral range of 886 - 1735.34 nm was reduced to 904.28 - 1690.46 nm. After removing the abnormal influence, the spectral curves of all sorghum are as Figure 2 shown, and the average spectral reflectance is as Figure 3 shown.

[0037] 1.5 Acquisition of super-depth-of-field image data When collecting data of sorghum samples with a super-depth-of-field microscope, due to the obvious differences in the morphological remains of the styles of different sorghum varieties, the super-depth-of-field microscope image data at this position was collected, as Figure 4As shown below. The specific operation is as follows: Place the sorghum grains on the stage of the microscope. Use tweezers to move the sorghum grains so that the position of the stigma remains is facing upwards directly towards the center of the light passing hole. First, select a lower magnification, rotate the coarse focusing screw to find the position of the stigma remains of the sorghum sample, and then rotate the fine focusing screw for focal plane focusing to make the cross-section clear. Select a clear cross-section position, scan the sorghum grains along the Z-axis, and finally complete the full 3D construction of the true morphology of the position of the sorghum stigma remains to obtain high-resolution image information of the sorghum grains.

[0038] 2. Model construction and evaluation In this invention research, a two-dimensional feature adaptive convolution model was mainly constructed using 1DCNN and 2DCNN. In view of the characteristics of the spectral data and image data of sorghum grains, this research adopted the effective fusion of the image features extracted by 2DCNN and the spectral features extracted by 1DCNN to improve the classification and recognition accuracy of different types of sorghum grains. The constructed DD-FACM is as Figure 5 shown, mainly consisting of three parts: one is the 1DCNN module responsible for extracting the one-dimensional spectral features of sorghum grains; the second is the 2DCNN module for extracting the image features of sorghum grains; the third is the feature fusion module for fusing the features extracted by the first two.

[0039] 2.1 Spectral feature extraction module.

[0040] The 1DCNN module for extracting spectral features mainly consists of 5 convolutional layers and 4 fully connected layers. The ReLu activation function is used after the convolutional layer. The reciprocal of the ReLu activation function is always equal to 1 in the positive part, which performs a non-linear transformation on the input feature data, enhances the non-linear characteristics of the network, thus better learning features, and has a fast calculation speed, accelerating the training speed of the model. Its calculation is shown in the formula as follows:

[0041] where x is the feature value received by the neuron; In each convolutional layer, a batch normalization (BN) layer is introduced to normalize the input feature data, which not only helps to alleviate the problems of gradient disappearance and gradient explosion, but also can accelerate the convergence process of the model and improve the overall performance of the model. During the spectral feature extraction process, a channel attention mechanism is introduced to adaptively weight and enhance the extracted spectral features. For spectral features with an input size of C×1×W, the channel attention mechanism first performs global max pooling and global average pooling operations to generate two feature maps with a size of C×1×1. Next, these two feature maps are fed into a shared multi-layer perceptron ( ) and The outputs are added together and finally mapped through the Sigmoid activation function to generate the channel attention weight matrix ( ). The channel attention mechanism is as shown in Figure 6 . The formula for this weight matrix is as follows: , where is the first channel attention weight matrix, is the multi-layer perceptron mechanism, is the input spectral feature size C×1×1, and are the average pooling operation and the global average pooling operation respectively, is the Sigmoid activation function.

[0042] The feature map extracted by the convolutional layer is multiplied by the channel attention vector to enhance the output features of the convolutional layer. This multiplication operation aims to strengthen the key information in the output feature map while suppressing the less important parts. The enhanced feature map passes through a shared fully connected layer (k), which is used to compress the high-dimensional features into a one-dimensional spectral feature vector of length 64. This one-dimensional spectral feature vector contains the key information extracted from the original feature map. To further analyze and utilize these spectral features, the L1 norm of the weights in the shared fully connected layer Ⅰ is calculated, and then a feature norm vector of the same length 64 is generated. This step crucially reflects the relative importance of each spectral feature. Based on these feature norm vectors, an importance coefficient (λκ) can be calculated for each of the 64 spectral features, where the range of k is 0 to 63. These importance coefficients pay more attention to the most informative features in subsequent processing. The constructed 1DCNN structure parameters are shown in Table 2: Table 2. One-dimensional convolutional structure parameters.

[0043]

[0044] Note: "Conv" represents the convolutional layer; "FC" represents the fully connected layer; "Share_FC" represents the shared fully connected layer; 2.2 Image feature extraction module.

[0045] For the ultra-depth-of-field images of sorghum seeds, the classic VGG16 network was selected as the 2DCNN image feature extractor. This network consists of 13 convolutional layers, 5 max-pooling layers, and 3 fully connected layers, and the ReLU activation function was selected. The image (224×224) corresponding to the sorghum seed spectral data was input into the network, and batch normalization (BN) was performed on the features extracted from each convolutional layer. After multiple convolutional and pooling layers, the last shared fully connected layer II expands the output of the last pooling layer into a 1D image feature vector of length 64. By calculating the L2 norm of the weights in the shared fully connected layer II, the corresponding importance coefficients were calculated for the 64 image features, denoted as µκ, where κ ranges from 0 to 63. The global attention mechanism (GAM) was used to adaptively weight the feature maps extracted from each convolutional layer, which aims to retain global information and amplify cross-dimensional feature interactions. The global attention mechanism includes a channel attention mechanism and a spatial attention mechanism. The input size of the feature map (F1) is C×1×W, which passes through the channel attention and spatial attention modules in sequence. The global attention mechanism is as shown in Figure 7 shown. In the channel attention module, a 3D alignment method is used to process the information in three dimensions (width, height, and channel). Then, a two-layer multi-layer perceptron (MLP) is adopted to enhance the dependence between different channels. Finally, the Sigmoid activation function calculates the channel attention weights, which are multiplied by the original feature map to obtain a new feature map ( ); then is input into the spatial attention module. In the spatial attention module, two convolutional layers are used to fuse the spatial information to obtain the spatial attention weights, which are then multiplied by the feature map ( ) to output the final feature ( ). The channel attention module and the spatial attention module in the global attention mechanism are as shown in Figure 8 shown, and The formulas are as follows: , , where Mc2 and Ms are the channel attention module and the spatial attention module respectively; is the input feature map (C×H×W), is the new feature map obtained by multiplying the input feature map after being weighted by the channel attention, is the finally output feature.

[0046] 2.3 Feature fusion module.

[0047] The purpose of the feature fusion module is to fuse the obtained spectral features and image features. However, the inconsistency in the importance of different features may lead to a decline in model performance, poor generalization ability, and the inability to process different datasets. To address this issue, we used a linear regression model to calculate 64 important coefficients for the spectral features (λκ) and image features (µκ) of two shared connection layers, respectively. Among them, λκ and µκ represent the importance of spectral and image features, respectively. These coefficients are normalized by the Softmax function to make each (λκ + µκ) value constant at 1, where k ranges from 0 to 63. By ensuring that (λκ + µκ) equals 1, the relative ratio between the importance of spectral and image features can be maintained, and the fusion weights of spectral and image characteristics can be adaptively assigned to each feature channel, thereby making full use of complementary information.

[0048] 2.4 Model Evaluation The performance evaluation of the classification model mainly relies on the calculation and analysis of the confusion matrix. The confusion matrix comprehensively summarizes the distribution of the model's prediction results and the true labels, intuitively showing the performance of the model in each category. Through the confusion matrix, important indicators such as the accuracy, precision, and recall rate of the model can be calculated, and then the performance of the model can be comprehensively evaluated. To verify the performance of the proposed model, the accuracy (ACC), recall rate (Recall), and F1 coefficient of the sorghum grain test set were used as the evaluation indicators of the model. The range of each evaluation indicator is between 0 and 1, and the closer the indicator is to 1, the better the performance of the model. That is, the calculation formula is as follows:

[0049]

[0050]

[0051]

[0052] Among them, TP is the true positive, referring to the number of positive samples actually predicted; TN is the true negative, referring to the number of negative samples actually predicted; FP is the false positive, referring to the negative samples predicted as positive samples; FN is the false negative, referring to the positive samples predicted as negative samples.

[0053] Figure 9 It shows the classification confusion matrix obtained on the test set after inputting spectral data, super-depth image data, and different types of sample labels into the dual-dimensional feature adaptive convolutional model (DD-FACM). It can be intuitively seen from the figure that the model achieved 100% accuracy on the test set, fully demonstrating that the DD-FACM proposed in the present invention performs excellently in the sorghum variety identification task and has excellent discriminant ability.

[0054] t-SNE Visualization of Classification by the Dual-Dimensional Feature Adaptive Convolution Model (DD-FACM) To more intuitively demonstrate the effectiveness of the DD-FACM model based on fused spectral and image data in sorghum seed classification, the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm was used to visually downscale the input high-dimensional data, and the feature vectors extracted from the high-dimensional network were projected into a three-dimensional space, where the different color scatterings in this space correspond to different sorghum varieties. The distribution of sorghum data of different categories in the feature space was visualized through a 3D scatter plot. The sorghum dataset (spectral data and image data) was randomly divided into a training set and a test set in a ratio of 4:1. Figure 10 The t-SNE feature visualization results of the test set before model training are shown. At the same time Figure 11 it is shown that after visualizing the spectral and super-depth-of-field image data of the test set, the feature points corresponding to different sorghum varieties are completely clustered and separated in the 3D space, and the classification accuracy of sorghum seeds reaches 100%. The t-SNE method illustrates the effectiveness of DD-FACM based on spectral and image data in feature extraction for sorghum variety classification tasks.

[0055] 4 Ablation Experiments of the Dual-Dimensional Feature Adaptive Convolution Model (DD-FACM) The ablation experiment is an experimental method used to evaluate the impact of each component in a model on the overall performance. In this study, an ablation experiment was conducted on DD-FACM based on spectral data and super-depth-of-field image data to evaluate the effectiveness of 1DCNN, 2DCNN, and the attention mechanism. A total of six cases were designed for the experiment as shown in Table 6: For No.1 - 4, 1DCNN was used alone to process spectral data or 2DCNN was used alone to process super-depth-of-field image data, and the attention mechanism was introduced or not introduced in the corresponding module; For No.5 - 6, 1DCNN and 2DCNN were used simultaneously to process data, and the attention mechanism was introduced or not introduced respectively. By comparing the performance of the model in these six cases, the necessity of each component in DD-FACM was verified. From the results of the ablation experiment in Table 3, it can be seen that when compared with using 1DCNN or 2DCNN alone, introducing the attention mechanism can increase the accuracy by 2%, reaching 94% and 93%. This indicates that the attention mechanism can help the model adaptively focus on important features, suppress interference from irrelevant information, and improve the discrimination ability. When using 1DCNN and 2DCNN simultaneously and introducing the attention mechanism, the accuracy is further increased to 100%, which is 1% higher than when the attention mechanism is not introduced. This shows that the attention mechanism can not only enhance the feature representation of a single data type but also play an important role when fusing multi-source data. By adaptively adjusting the weights of different data sources and features, the model performance is improved. When using 1DCNN and 2DCNN models simultaneously, the classification accuracy of sorghum varieties has been significantly improved compared with using either of the models alone. Among all the experimental schemes, the best classification effect was achieved in No.6, with an accuracy as high as 100%. Compared with No.2 and No.4, the accuracy of the feature fusion model was increased by 6% and 7% respectively. This result indicates that combining 1DCNN for feature extraction of spectral data with 2DCNN for feature extraction of image data further demonstrates the importance of fusing these two features, which can effectively achieve feature complementarity and enhancement, thereby improving the classification accuracy.

[0056] In summary, the results of the ablation experiment fully prove the effectiveness of the attention mechanism, 1DCNN, and 2DCNN in DD-FACM. The combination of 1DCNN and 2DCNN can further extract key information from spectral data and super-depth-of-field image data, realizing the optimization of feature representation and the improvement of model performance.

[0057] Table 3. Results of the ablation experiment for DD-FACM.

[0058]

[0059] Comparative Example 1 The identification of sorghum varieties was carried out based on the partial least squares discriminant analysis model (PLS-DA).

[0060] In the PLS-DA model, the input one-dimensional spectral data is preprocessed by standardization to ensure that the mean of each feature is 0 and the standard deviation is 1. Through standardization preprocessing, the possible scale differences between features can be eliminated, enabling the model to more fairly evaluate the importance of each feature and improving the accuracy and reliability of the analysis results.

[0061] Comparative Example 2 Sorghum variety identification is carried out based on the support vector machine model (SVM).

[0062] In the support vector machine (SVM) model, the radial basis function (RBF) is selected as the kernel function, and the value of the regularization parameter C is 1. To make the model perform more stably on data of different scales, the gamma parameter is set to'scale' to automatically adjust according to the variance of the input data. This can avoid the overfitting or underfitting problems caused by manually adjusting the gamma parameter, thereby improving the generalization ability of the model.

[0063] Comparative Example 3 Sorghum variety identification is carried out based on the transfer learning model Efficientnet_b3.

[0064] The pseudo-color image (FC-RGB) obtained by the hyperspectral imaging system and the ultra-depth-of-field image (EDoF) obtained by the 3D ultra-depth-of-field microscope system are used as two-dimensional image data. To achieve accurate classification and recognition of the image data, the transfer learning model Efficientnet_b3 is selected as the feature extractor. To ensure the generalization ability of the model, 500 sorghum image samples are randomly divided into a training set, a test set, and a validation set according to a ratio of 3:1:1. The training set is used for model training, the test set evaluates the model performance, and the validation set is used for model hyperparameter tuning and preventing overfitting.

[0065] Comparative Example 4 Sorghum variety identification is carried out based on the transfer learning model Inception_V3.

[0066] The pseudo-color image (FC-RGB) obtained by the hyperspectral imaging system and the ultra-depth-of-field image (EDoF) obtained by the 3D ultra-depth-of-field microscope system are used as two-dimensional image data. To achieve accurate classification and recognition of the image data, the transfer learning model Inception_V3 is selected as the feature extractor. To ensure the generalization ability of the model, 500 sorghum image samples are randomly divided into a training set, a test set, and a validation set according to a ratio of 3:1:1. The training set is used for model training, the test set evaluates the model performance, and the validation set is used for model hyperparameter tuning and preventing overfitting.

[0067] The classification results of each model are compared as shown in Table 4.

[0068] Based on the single spectral data of 474 bands, the indicators of the training set and test set of the two models, PLS-DA and SVM, do not differ much, indicating that there is no overfitting phenomenon in the training through these two models. Among them, the accuracy rates of the training set and test set of SVM are 94% and 92% respectively, with the best classification effect, which are 3.3% and 3% higher than those of PLS-DA respectively. This is because the SVM model, as a non-linear classification algorithm, is superior to the linear classification algorithm. PLS-DA is a linear classification algorithm based on the PLS regression algorithm combined with the classification method. When implementing multi-classification, its classification accuracy will decrease significantly. The recall rate and F1 score of the training set and test set were also calculated. By comparing the performance of the spectral data used in this study on the SVM model and the PLS-DA model, it was found that the recall rate and F1 score of the SVM model were both about 3% higher than those of the PLS-DA model. This result indicates that on the spectral data set of this study, the SVM model can better capture the features in the spectral data and achieve better performance in the sample classification task.

[0069] Based on the FC-RGB images collected by HSI and the EDoF images collected by a 3D ultra-depth-of-field microscope, transfer learning models with the efficientnet_b3 and Inception_V3 network structures were used respectively. The model results are shown in Table 3. It can be seen from Table 3 that the classification accuracy rates of the training set and test set of the Inception_V3 model based on the ultra-depth-of-field images are 100% and 94.7% respectively, which are 19.3% and 15.6% higher than those of the pseudo-color images on the Inception_V3 model. The classification accuracy rates of the training set and test set of the Efficientnet_b3 model based on the ultra-depth-of-field images are 97.7% and 95.8% respectively, which are 27.1% and 29.1% higher than those of the hyperspectral images. It can also be seen from the training set and test set in the table that the recall rate and F1 value of the Inception_V3 and Efficientnet_b3 models on the EDoF images are significantly higher than those of the FC-RGB images. Thus, it can be further seen that the performance of the two models on the EDoF images is higher, the generalization ability is stronger, and both show higher classification performance. It can be seen that the EDoF images can be better recognized compared to the FC-RGB images. This is because the ultra-depth-of-field image (EDoF) has a larger depth-of-field range and higher spatial resolution. The larger depth-of-field range can provide more comprehensive spatial information of the sorghum grains. The high spatial resolution makes the edges, shapes and structures of the sorghum grains clearer and more accurate, capturing more spatial details and surface information of the sorghum grains. On the contrary, the spatial resolution of the pseudo-color image (FC-RGB) is low and the image features are not obvious. Therefore, in single image classification, the ultra-depth-of-field image can be better recognized.

[0070] Dual-dimensional Feature Adaptive Convolution Model (DD-FACM) based on 1DCNN and 2DCNN, which fuses the spectral features extracted by 1DCNN and the image features extracted by 2DCNN. In DD-FACM, the dataset is randomly divided into a training set and a test set at a ratio of 4:1. The parameters in the model are set as follows: the number of iterations is 100, the learning rate is 0.0001, the batch size is 4, and the Adam optimizer is selected. From the results in Table 4, the accuracies of the training set and the test set based on spectral data + hyper-depth-of-field image data reached 97.7% and 100% respectively, while the accuracies of the training set and the test set based on spectral data + pseudo-color image data were 97.5% and 95.9% respectively. Although the accuracies of the models with these two different data types on the training set are very close, on the test set, the model using EDoF images has an accuracy improvement of 4.1% compared to the model using FC-RGB images. This result indicates that on the basis of the same spectral data, combining EDoF images can improve the generalization ability of the model. From the recall rate and F1 score in the table, it shows that using spectral data + hyper-depth-of-field image data, DD-FACM exhibits better positive class recognition ability and more stable performance on the test data, while using spectral data + pseudo-color image data has mild overfitting, resulting in a slight decline in generalization ability. After comparison, compared with PLS-DA, SVM, Inception_V3, and Efficientnet_b3 based on a single dataset, the classification accuracy of DD-FACM has increased significantly. Among them, the model based on spectral features + hyper-depth-of-field image features has the best classification effect, and the accuracy can reach 100% on the test set. The results show that compared with the traditional single data source method, DD-FACM based on spectral data + hyper-depth-of-field image data has achieved significant improvements in classification accuracy and generalization ability.

[0071] Table 4. Comparison table of the results of each model.

[0072]

[0073] Example 2 Based on the design of Example 1, this example discloses a detection system for identifying sorghum varieties, as Figure 12 shown. This system is based on the above detection method and at least includes the following components built in the system: A data acquisition module, configured to obtain hyperspectral images and hyper-depth-of-field image data of sorghum grain samples; A data processing module, configured to perform black and white correction on the obtained hyperspectral images, and use the Otsu algorithm to segment the background and samples to obtain spectral data; A spectral feature extraction module, configured to extract the features of spectral data using a one-dimensional convolutional neural network; An image feature extraction module, configured to extract features of hyper-depth-of-field image data by using a two-dimensional convolutional neural network; A feature fusion module, configured to fuse and add spectral features and image features; A detection model construction module, configured to construct a data set with the obtained spectral data and hyper-depth-of-field image data, and input it into a two-dimensional feature adaptive convolutional model for training to obtain a detection model; the two-dimensional feature adaptive convolutional model is composed of the spectral feature extraction module, the image feature extraction module, and the feature fusion module; A detection module, configured to input the spectral data and image data of a to-be-detected sorghum grain sample into the detection model to obtain high-dimensional data, and use the t-distribution-stochastic neighborhood embedding algorithm to perform dimensionality reduction visualization on the input high-dimensional data to achieve visual detection of sorghum varieties.

[0074] Embodiment 3 Based on the design of Embodiments 1 and 2, this embodiment discloses a computer-readable storage medium storing a program that can be executed by one or more processors to implement the above method for detecting the moisture content of Daqu.

[0075] Without conflict, the above embodiments and the features in the embodiments in this article may be combined with each other.

[0076] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A detection method for identifying sorghum varieties, characterized in that, It includes the following specific steps: S1. Obtain sorghum grain samples, collect hyperspectral images of sorghum grains using a hyperspectral imaging system, perform black-and-white correction on the hyperspectral images, and use the Otsu algorithm to segment the background and samples to extract the spectral data of the samples; collect the ultra-depth-of-field image data of the style remnants of the sorghum grain samples using a 3D ultra-depth-of-field microscope; S2. Construct a dataset with the obtained spectral data and ultra-depth-of-field image data, and input it into a two-dimensional feature adaptive convolution model for training to obtain a detection model; The two-dimensional feature adaptive convolution model consists of a spectral feature extraction module, an image feature extraction module, and a feature fusion module. The spectral feature extraction module is used to extract spectral features from spectral data, the image feature extraction module is used to extract image features from image data, and the feature fusion module is used to fuse and add the spectral features and image features. The fused features are sent to the output layer for classification and recognition; S3. Input the spectral data and ultra-depth-of-field image data of the sorghum samples to be tested into the detection model to obtain a high-dimensional feature representation, and then use the t-SNE algorithm to perform dimensionality reduction processing on the high-dimensional data to achieve two-dimensional visual classification and identification of sorghum varieties.

2. The detection method according to claim 1, characterized in that The hyperspectral imaging system collects hyperspectral images of sorghum grains through push-broom linear scanning, and performs black-and-white correction on the collected hyperspectral images. The formula for black-and-white correction is: ， In the formula, is the corrected hyperspectral data; is the original hyperspectral image; is the dark reference image collected with the lens covered; is the standard whiteboard image collected.

3. The detection method according to claim 2, wherein The process of obtaining the ultra-depth-of-field image data is as follows: Select a clear cross-section position of the style remnants, scan the sorghum grains along the Z-axis to achieve a complete 3D construction of the true morphology of the sorghum style remnants position, and obtain high-resolution image data of the sorghum grains.

4. The detection method according to claim 3, wherein The spectral feature extraction module uses a one-dimensional convolutional neural network to extract the features of spectral data. The one-dimensional convolutional neural network consists of 5 convolutional layers and 4 fully connected layers. During the feature extraction process of each convolutional layer, a first-channel attention mechanism is used to adaptively weight and enhance the extracted spectral features.

5. The detection method according to claim 4, characterized in that, The calculation formula for the channel attention weight matrix generated by the first-channel attention mechanism is: ， Among them, is the first-channel attention weight matrix, is a multi-layer perceptron mechanism, is the input spectral feature size C×1×1, and are the average pooling operation and the global average pooling operation respectively, is the Sigmoid activation function.

6. The detection method according to claim 4 or 5, characterized in that The image feature extraction module uses a two-dimensional convolutional neural network to extract the features of ultra-depth-of-field image data. The two-dimensional convolutional neural network uses the VGG16 network as an image feature extractor, which consists of 13 convolutional layers, 5 max-pooling layers, and 3 fully connected layers. During the feature extraction process of each convolutional layer, a global attention mechanism is used to adaptively weight the extracted feature maps.

7. The detection method according to claim 6, characterized in that, The global attention mechanism includes a second-channel attention mechanism and a spatial attention mechanism. The input feature map first passes through the channels of the second-channel attention mechanism to obtain channel attention weights, which are multiplied by the original feature map to obtain a new feature map . The feature map is input into the channels of the spatial attention mechanism to obtain spatial attention weights, which are multiplied by the feature map F2 to output the final image features . The calculation formula is as follows: , , Among them, and M s are the channel attention module and the spatial attention module respectively; is the input feature map, is the new feature map obtained by multiplying the input feature map after being weighted by the channel attention, is the finally output feature.

8. The detection method according to claim 6, wherein, The process of fusion and addition is as follows: the spectral features of two shared connection layers coefficients and the image features coefficients are added with 64 important coefficients, and the two coefficient sums are adjusted to between 0 and 1 through the linear regression coefficients. Then, the Softmax function is used to calculate the weights of the coefficients. The processed and coefficients act on the weighted corresponding spectral features and image features respectively.

9. A detection system for identifying sorghum varieties, characterized in that, The system is based on the detection method described in any one of claims 6-8, and at least includes the following components built in the system: A data acquisition module configured to obtain hyperspectral images and ultra-depth-of-field image data of sorghum grain samples; A data processing module configured to perform black-and-white correction on the obtained hyperspectral images and use the Otsu algorithm to segment the background and samples to obtain spectral data; A spectral feature extraction module configured to use a one-dimensional convolutional neural network to extract the features of spectral data; An image feature extraction module configured to use a two-dimensional convolutional neural network to extract the features of ultra-depth-of-field image data; A feature fusion module configured to fuse and add spectral features and image features; The detection model construction module is configured to construct a data set with the acquired spectral data and super-depth-of-field image data, and input it into a two-dimensional feature adaptive convolution model for training to obtain a detection model; the two-dimensional feature adaptive convolution model is composed of the spectral feature extraction module, the image feature extraction module, and the feature fusion module; The detection module is configured to input the spectral data and image data of the to-be-detected sorghum grain sample into the detection model to obtain high-dimensional data, and use the t-distribution-stochastic neighborhood embedding algorithm to perform dimensionality reduction visualization on the input high-dimensional data, so as to realize the visual detection of sorghum varieties.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program, and the program can be executed by one or more processors to implement the detection method for identifying sorghum varieties according to any one of claims 1-8.

Images