Hyperspectral spatial-spectral fusion classification method, hyperspectral fast detection device and storage medium

Abstract

The invention provides a hyperspectral spatial-spectral fusion classification method, a hyperspectral fast detection device and a storage medium, and the method comprises the steps: employing a twin network layer, and enabling two convolutional neural network branches of the twin network layer to extract the feature value of the spatial dimension and the feature value of the spectral dimension of hyperspectral data respectively, the two convolutional neural network branches share a weight; inputting the feature value of the spatial dimension and the feature value of the spectral dimension into a feature fusion layer for fusion to form a fusion feature vector; inputting the fusion feature vector into a shared network layer, and carrying out global modeling by using a Transform model; and inputting the output features of the Transform model into a classifier formed by a full connection layer, and carrying out final classification prediction. According to the method, the spatial-spectral characteristics of the hyperspectral data are fully utilized, and the classification precision and robustness are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral nondestructive testing, and relates to a hyperspectral space-spectrum fusion classification method, device, and storage medium. Background Art

[0002] Hyperspectral imaging (HSI) technology combines imaging technology with spectral technology. It can not only obtain image information of samples in the spatial dimension, but also obtain continuous spectral information at the nanometer level, forming a space-spectral cube. Due to the imaging characteristics of hyperspectral and the rich spatial and spectral information it contains, it can obtain nearly continuous ground object information and has outstanding applications in the field of remote sensing. In addition, hyperspectral can obtain high-resolution continuous, narrow-band image data, thereby providing more effective information for the image processing process and providing data support for quantitative research such as biophysical and biochemical information extraction.

[0003] For hyperspectral data processing, most of them use single spectral dimension reduction or single image dimension reduction, which will lose some information. Hyperspectral imaging technology integrates machine vision and infrared spectroscopy to simultaneously obtain spectral and spatial image information of samples. It is fast, efficient, accurate and non-destructive, and is widely used in pesticide residue detection, food intrinsic component content analysis and food type identification.

[0004] In hyperspectral imaging data, the spectral information of each pixel represents the physical and chemical properties of the point, while the spatial information reflects the morphology and structure of the object. Using spectral information or spatial information alone for classification may lead to reduced classification accuracy, because these two types of information complement each other. Only by effectively integrating spatial and spectral information can high-precision classification be achieved.

[0005] Existing hyperspectral image classification methods usually only focus on spectral information or spatial information, which makes it difficult to fully utilize the spatial-spectral characteristics of hyperspectral data, thus affecting classification accuracy. Traditional spatial-spectral fusion methods often rely on simple feature connections or manually designed feature fusion strategies, which lack flexibility and adaptability and cannot fully capture the complex feature relationships of hyperspectral data. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a hyperspectral spatial-spectral fusion classification method, a hyperspectral rapid detection device, and a storage medium. The method extracts spatial and spectral information separately and performs feature fusion, and uses Transformer for global modeling. It makes full use of the spatial-spectral characteristics of hyperspectral data and improves classification accuracy and robustness.

[0007] In order to achieve the above object, the present invention provides a hyperspectral spatial-spectral fusion classification method, comprising:

[0008] Using a twin network layer, wherein two convolutional neural network branches of the twin network layer respectively extract the eigenvalues ​​of the spatial dimension and the eigenvalues ​​of the spectral dimension of the hyperspectral data, and the two convolutional neural network branches share weights;

[0009] Inputting the eigenvalue of the spatial dimension and the eigenvalue of the spectral dimension into a feature fusion layer for fusion to form a fused feature vector;

[0010] The fused feature vector is input into a shared network layer and the Transformer model is used for global modeling;

[0011] The output features of the Transformer model are input into the classifier composed of the fully connected layer for the final classification prediction.

[0012] In some embodiments, before extracting the spatial dimension features and spectral dimension features of the hyperspectral data using the twin network layer, the method further includes:

[0013] The original hyperspectral data is input into an input layer for preprocessing, including black-white correction and spectral normalization, to obtain first preprocessed hyperspectral data.

[0014] In some embodiments, the black-white correction step includes: calibrating the original hyperspectral data using a white reference image and a dark reference image to obtain a first calibrated reflectance image:

[0015]

[0016] Where R c is the first calibration reflectance image, R raw is the original reflectance image, R white is the white reference image, R dark is the dark reference image;

[0017] The step of spectral normalization includes: normalizing the original hyperspectral data to the interval [0,1], and obtaining the first normalized spectral data as follows:

[0018]

[0019] Among them, x min and x max are the minimum and maximum values ​​of the original hyperspectral data x in the band respectively;

[0020] The first pre-processed hyperspectral data includes the first calibrated reflectance image and first normalized spectral data.

[0021] In some embodiments, before extracting the spatial dimension features and spectral dimension features of the hyperspectral data using the twin network layer, the method further includes:

[0022] The first pre-processed hyperspectral data is compressed through a data network layer, including:

[0023] Inputting the first preprocessed hyperspectral data as a whole into a forward encoding network to obtain encoded feature representation;

[0024] The encoded feature representation is input into a quantizer, intermediate feature data of the encoded feature representation is extracted, the size of the intermediate feature data is reduced by a downsampling operation, and the quantized intermediate feature data is losslessly entropy encoded to obtain a compressed bit stream;

[0025] Reconstruct the first pre-processed hyperspectral data by using the compressed code stream through an inverse decoding network;

[0026] Through rate-distortion optimization, the first preprocessed hyperspectral data is flipped, and the data before and after flipping are respectively input into the two convolutional neural network branches of the twin network layer.

[0027] In some embodiments, the fusion feature layer uses pixel-level fusion to fuse the spatial dimension features and the spectral dimension features through a splicing operation to form the fusion feature vector.

[0028] In some embodiments, the step of performing global modeling by the Transformer model comprises:

[0029] Reshape the fused feature vector into a format suitable for Transformer model input;

[0030] adding a position code to the fused feature vector;

[0031] Through the self-attention mechanism, the dependency between the feature vectors in the fused feature vector sequence with position encoding is captured.

[0032] In some embodiments, the fully connected layer maps the output features of the Transformer model to the probability distribution of the classification label, uses the Softmax activation function to obtain the probability of each category, and selects the category with the highest probability as the final classification prediction result.

[0033] In some embodiments, the classification prediction results are input into an output layer, and the model is trained using a cross entropy loss function.

[0034] In another aspect, the present invention provides a hyperspectral rapid detection device for implementing the above-mentioned hyperspectral spatial-spectral fusion classification method, the device comprising:

[0035] Dark box, the inner wall is coated with high reflectivity polytetrafluoroethylene spray material;

[0036] A multi-faceted light source is installed on the inner wall of the dark box;

[0037] A light source controller connected to the multi-faceted light source;

[0038] A camera, mounted on the inner wall of the dark box, for collecting hyperspectral image data;

[0039] A hyperspectrometer, connected to the camera, for analyzing spectral characteristics based on the hyperspectral image data;

[0040] A liftable stage is installed on the inner wall of the dark box, and a sample is placed on the stage;

[0041] A motion control system connected to the liftable loading platform;

[0042] The main control computer is connected to the light source controller, the motion control system, and the hyperspectrometer, respectively, to drive the light source controller to control the on state of the multi-faceted light source; drive the motion control system to control the movement of the liftable stage to adjust the distance between the sample and each light source and the camera lens; and receive the spectral characteristics transmitted by the hyperspectrometer for analysis to obtain the final classification prediction result.

[0043] In another aspect, the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned hyperspectral spatial-spectral fusion classification method.

[0044] It can be seen from the above scheme that the advantages of the present invention are:

[0045] The hyperspectral spatial-spectral fusion classification method disclosed in the present invention introduces a twin neural network and a transformer model to simultaneously acquire rich spatial-spectral features, considers the sequence and position information of the hyperspectral image spectrum, extracts global features of the spectrum, and combines the self-attention mechanism and the residual structure to acquire global spatial features, thereby improving the change detection accuracy and ensuring a good and stable change detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The overall process diagram of the hyperspectral spatial-spectral fusion classification method provided by one embodiment of the present invention is shown;

[0047] Figure 2 The architecture diagram of the hyperspectral spatial-spectral fusion classification model is shown;

[0048] Figure 3 The schematic diagram of the structure of the hyperspectral rapid detection device is shown;

[0049] in:

[0050] 200: Hyperspectral rapid detection device;

[0051] 201: Camera obscura;

[0052] 202: light source;

[0053] 203: light source controller;

[0054] 204: hyperspectral instrument;

[0055] 205: camera lens;

[0056] 206: stage;

[0057] 207: motion control system;

[0058] 208: Main control computer. DETAILED DESCRIPTION

[0059] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0060] See also Figure 1 , Figure 2 , Figure 1 The overall process diagram of the hyperspectral spatial-spectral fusion classification method provided by one embodiment of the present invention is shown; Figure 2 The architecture diagram of the hyperspectral spatial-spectral fusion classification model is shown, and the hyperspectral spatial-spectral fusion classification model includes an input layer, a data compression layer, a twin network layer, a fusion feature layer, a shared feature layer, a fully connected layer, and an output layer. Among them: the data compression layer is composed of a forward encoding network, a quantizer, a reverse decoding network, and a rate-distortion optimization; the twin network layer includes a dual-branch convolutional neural network, and each branch includes multiple convolutional layers, activation layers, and pooling layers; one or more fully connected layers form a classifier, and each fully connected layer performs a linear transformation, and is followed by a nonlinear activation function. The hyperspectral spatial-spectral fusion classification method uses the hyperspectral spatial-spectral fusion classification model to achieve classification prediction. In some embodiments, the hyperspectral spatial-spectral fusion classification method can be applied to the lossless identification of Chinese medicinal materials such as Pueraria lobata and Pueraria lobata, but the present invention is not limited to this.

[0061] The hyperspectral spatial-spectral fusion classification method specifically includes the following steps:

[0062] Step S1, using a twin network layer, wherein two convolutional neural network branches of the twin network layer respectively extract the eigenvalues ​​of the spatial dimension and the eigenvalues ​​of the spectral dimension of the hyperspectral data, and the two convolutional neural network branches share weights.

[0063] In some embodiments, before using the twin network layer to extract the spatial dimension features and spectral dimension features of the hyperspectral data, the original hyperspectral data is first preprocessed by inputting the input layer of the hyperspectral spatial-spectral fusion classification model, and the preprocessing operation includes black and white correction and spectral normalization to obtain first preprocessed hyperspectral data.

[0064] Wherein: In one embodiment, the step of black and white correction includes: using a white reference image and a dark reference image to calibrate the original hyperspectral data, and obtaining a first calibrated reflectance image as follows:

[0065]

[0066] Where R c is the first calibration reflectance image, R raw is the original reflectance image, R white is the white reference image, and R dark is a dark reference image. The white reference image is acquired using a Teflon white board with a reflectivity close to 100%, and the dark reference image is acquired by covering the lens with a cap.

[0067] Spectral normalization is to scale the value of each spectral vector to a certain range to reduce the impact of illumination changes and other external factors. The commonly used spectral normalization method is maximum and minimum normalization, which normalizes the original hyperspectral data to the [0,1] interval, that is, the value of each spectral data point is compressed to the [0,1] range, so that the spectral features between different samples have better comparability. After normalizing the original hyperspectral data to the [0,1] interval, the first normalized spectral data is obtained:

[0068]

[0069] Among them, x min and x max are the minimum and maximum values ​​of the original hyperspectral data x in the band, respectively, and x' is the first normalized spectral data.

[0070] Through the above black-white correction and spectral normalization steps, the original hyperspectral data is preprocessed to obtain first preprocessed hyperspectral data, which includes the above first calibration reflectance image and first normalized spectral data.

[0071] In addition, in some embodiments, after the preprocessing of the original hyperspectral data is implemented, the first preprocessed hyperspectral data is further input into the data network layer for data compression processing.

[0072] Specifically, in the data compression processing step, the first pre-processed hyperspectral data is firstly inputted as a whole into the forward encoding network to obtain the encoded feature representation. The first pre-processed hyperspectral data is set as X, and the encoded feature representation is E: E = f encoder (X), where f encoder Represents a forward coding network. Further, the encoded feature is input into a quantizer to extract the intermediate feature data represented by the encoded feature, and the intermediate feature data includes the main spectral feature data and spatial feature data. And the size of the intermediate feature data is reduced by downsampling operation, and the entropy of the intermediate feature data is controlled by rate-distortion optimization to make the feature data distribution more compact; at the same time, the quantized intermediate feature data is losslessly entropy encoded to obtain a compressed bitstream. The quantized intermediate feature data Q is expressed as: Q = quantize (E), the downsampling operation is expressed as: E' = downsample (E), and the entropy coding is expressed as: C = entropy_encode (Q). Then, the compressed bitstream is reconstructed into the first preprocessed hyperspectral data using the reverse decoding network, and the compressed bitstream is reconstructed into the first preprocessed hyperspectral data through the inverse transformation process of entropy decoding, inverse quantization, upsampling, and deconvolution. The decoding process of the compressed bitstream is as follows: Q' = entropy_decode (C), the upsampling operation is expressed as: E" = upsample (Q'), and the decoded reconstructed image is expressed as: Among them, f decoder Represents the decoding network. Finally, rate-distortion optimization is used. Rate-Distortion Optimization (RDO) is a key concept in video and image compression, which aims to balance the compression rate and distortion when compressing data. In short, the purpose of RDO is to minimize the amount of data (i.e., improve the compression rate) while maintaining the original quality of the data as much as possible (i.e., reduce distortion). Through rate-distortion optimization, the first preprocessed hyperspectral data is flipped through the data so that the data before and after the flipping are respectively input into the two convolutional neural network branches of the twin network layer. The objective function of rate-distortion optimization is:

[0073]

[0075] Among them, D(X, ) represents the distortion metric function, X, They represent the data before and after flipping, respectively, and the mean square error (MSE) is usually used; R(C) represents the entropy of the compressed bitstream; λ is a hyperparameter that weighs the rate distortion.

[0076] After the raw hyperspectral data is preprocessed and compressed through the above steps, a twin network layer is used, wherein the two convolutional neural network branches of the twin network layer respectively extract the eigenvalues ​​of the spatial dimension and the eigenvalues ​​of the spectral dimension of the hyperspectral data. And the two convolutional neural network branches share weights. In some embodiments, a part of the pre-trained ResNet18 model is used to construct a shared convolutional neural network branch for extracting the eigenvalues ​​of the spatial dimension and the eigenvalues ​​of the spectral dimension. Each branch includes multiple convolutional layers, activation layers, and pooling layers. Each convolutional neural network branch can be expressed as:

[0077] f(x)=Pooling(Activation(Conv(x)))

[0078] Among them, Conv() represents the convolution operation, Activation() represents the activation function, and Pooling() represents the pooling operation.

[0079] Then, the two branches extract the eigenvalues ​​F1 of the spatial dimension and F2 of the spectral dimension of the hyperspectral data through multiple convolutional layers and pooling layers.

[0080] Step S2: input the eigenvalue of the spatial dimension and the eigenvalue of the spectral dimension into a feature fusion layer for fusion to form a fused feature vector.

[0081] In this step, the feature fusion layer further fuses the feature value of the spatial dimension and the feature value of the spectral dimension through a concatenation operation. In one embodiment, the fusion feature layer fuses the spatial dimension feature and the spectral dimension feature through a concatenation operation using a pixel-level fusion method to form the fused feature vector, and the fused feature vector is: F=contat(F1,F2), where concat() represents a feature concatenation operation.

[0082] Step S3: input the fused feature vector into a shared network layer and use the Transformer model for global modeling.

[0083] In step S3, the fused feature vector is input into the shared network layer, and the Transformer model is used for global modeling. Specifically, the fused feature vector F is first reshaped into a format suitable for the input of the Transformer model. Through the view function, its shape is adjusted to (-1, channels) so that each time step contains a complete feature vector, where channels corresponds to the number of spectral bands of the hyperspectral image.

[0084] In addition, since the Transformer model does not have the ability to process position information, it is necessary to add position encoding to the fused feature vector to retain the information of each position in the sequence. Position encoding can use sine and cosine functions:

[0085]

[0086] Among them, p is the position, i is the dimension index, and d is the encoding dimension.

[0087] Then add the above position encoding PE to the fused feature vector F, that is, F f =F+PE.

[0088] Finally, the fused feature vector F with position encoding is f The input is sent to the Transformer model for global modeling. The Transformer captures the dependencies between feature vectors in the fused feature vector sequence with position encoding through the self-attention mechanism.

[0089] The calculation formula of self-attention is:

[0090]

[0091] Among them, Q (Query), K (Key), and V (Value) represent query, key, and value matrices respectively. k is the dimension of the key vector.

[0092] The output features F of the Transformer model t : F t = Transformer(F f ).

[0093] Step S4: Input the output features of the Transformer model into the classifier composed of the fully connected layer to make the final classification prediction.

[0094] In this step, the output feature F of the Transformer model is t The input is sent to the classifier composed of a fully connected layer for final classification prediction. The fully connected layer maps the output features of the Transformer model to the probability distribution of the classification label, uses the Softmax activation function to obtain the probability of each category, and selects the category with the highest probability as the final classification prediction result.

[0095] Specifically, the output feature F of the Transformer model tThe output features are mapped through a fully connected layer and used as the input of the classifier. The classifier consists of one or more fully connected layers, each of which performs a linear transformation, possibly followed by a nonlinear activation function.

[0096] For a single fully connected layer, the operation formula is as follows: z = WF t +b, where W is the weight matrix of the fully connected layer, with a dimension of C×d, where C is the number of classification categories and d is the output feature F of the Transformer model t The dimension of F is C; b is the bias vector of the fully connected layer, and the dimension of F is C; t The feature vector used as input to the classifier has a dimension of d; z is the output logits vector with a dimension of C, where each element corresponds to a category (logit value).

[0097] The output z of the classifier is a vector of length C, representing the score (logit) for each class. This vector is usually passed to the Softmax activation function to convert the logits into a probability distribution.

[0098] After the output of the classifier, a Softmax activation function is applied to transform the logits vector z into a class probability distribution:

[0099]

[0100] Among them, z i is the logit value of the ith category, and C is the total number of categories.

[0101] Through the probability distribution output by the Softmax activation function, the model will select the category with the highest probability as the final classification prediction result:

[0102] In addition, in some embodiments, the classification prediction result is further input into the output layer, and the model is trained using a cross entropy loss function, where the loss function is expressed as:

[0103] Among them, y i is the actual label, is the classification prediction result, and the final classification prediction result is output as the category label of the hyperspectral data.

[0104] In addition, in some embodiments, during the training and optimization process of the entire hyperspectral spatial-spectral fusion classification model, the entire network is trained using labeled hyperspectral data. The Adam optimization algorithm is used to update the model parameters:

[0105]

[0106] Among them, θt is a parameter, η is the learning rate, m t and v t are the estimates of momentum and squared gradient, respectively, and ε is a constant to prevent division by zero.

[0107] In addition, in some embodiments, the entire hyperspectral spatial-spectral fusion classification model is further evaluated through cross-validation and standard evaluation indicators (such as accuracy, recall, F1 score) to optimize the model parameters and structure.

[0108] In summary, the hyperspectral spatial-spectral fusion classification method disclosed in the present invention introduces a twin neural network and a transformer model to simultaneously acquire rich spatial-spectral features, considers the sequence and position information of the hyperspectral image spectrum, extracts global features of the spectrum, and combines the self-attention mechanism and the residual structure to acquire global spatial features, thereby improving the change detection accuracy and ensuring a good and stable change detection effect.

[0109] In addition, it should be noted that the hyperspectral spatial-spectral fusion classification method disclosed in the present invention is not limited to being executed in the order of the above steps S1-S4. It can reorder, add or delete steps. As long as the expected results of the technical solution of the present invention can be achieved, the present invention is not limited here.

[0110] In addition, corresponding to the above-mentioned embodiment of the hyperspectral spatial-spectral fusion classification method, the present invention also provides an embodiment of a hyperspectral rapid detection device.

[0111] See also Figure 3 , a hyperspectral rapid detection device 200 provided in an embodiment of the present invention at least comprises: a dark box 201, the inner wall of which is coated with a polytetrafluoroethylene spray material with a high reflectance. A multi-faceted light source 202 is installed on the inner wall of the dark box 201. In one embodiment, the light source can be a halogen lamp array, each light source is connected to a light source controller 203, and the light source controller 203 is connected to a host computer 208. According to the instructions of the host computer, the light source controller controls the on state of the light source. A hyperspectrometer 204 and a camera lens 205 are installed on the dark box 201. The camera lens 205 collects hyperspectral image data in real time and transmits it to the hyperspectrometer 204 to analyze the spectral characteristics.

[0112] In addition, the inner wall of the dark box 201 is equipped with a liftable stage 206, and the sample is placed on the stage 206. The stage 206 is connected to a motion control system 207, and the motion control system 207 is connected to a host computer 208. According to the instructions of the host computer, the liftable stage is driven to move by the motion control system 207 to adjust the distance between the sample and the light source 202 and the camera lens 205.

[0113] In a specific implementation, the hyperspectral rapid inspection device 200 can implement the hyperspectral spatial-spectral fusion classification method disclosed in the above embodiment. The main control computer 208 controls and adjusts the distance between the sample and the light source 202 and the camera lens 205, as well as the on-state of the light source, so that the camera lens 205 cooperates with the hyperspectrometer 204 to collect a number of hyperspectral data, and the several hyperspectral data are transmitted to the main control computer 208. The hyperspectral spatial-spectral fusion classification model configured by the main control computer analyzes the collected hyperspectral data according to the hyperspectral spatial-spectral fusion classification method disclosed above to obtain the final classification prediction result.

[0114] Those skilled in the art can clearly understand that, for the convenience and simplicity of describing the hyperspectral spatial-spectral fusion classification method, the specific working process of each module of the described device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] According to an embodiment of the present invention, the present invention also provides a machine-readable medium, which may be a tangible medium, which may contain or store a program for use by an instruction execution system, apparatus or device or for use in conjunction with an instruction execution system, apparatus or device, and when the computer program is executed by a processor, the steps of the above-mentioned hyperspectral spatial-spectral fusion classification method are implemented.

[0116] In addition, the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0117] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A hyperspectral spatial-spectral fusion classification method, characterized in that: Include: Using a twin network layer, wherein two convolutional neural network branches of the twin network layer respectively extract the eigenvalues ​​of the spatial dimension and the eigenvalues ​​of the spectral dimension of the hyperspectral data, and the two convolutional neural network branches share weights; Inputting the eigenvalue of the spatial dimension and the eigenvalue of the spectral dimension into a feature fusion layer for fusion to form a fused feature vector; The fused feature vector is input into a shared network layer and the Transformer model is used for global modeling; The output features of the Transformer model are input into the classifier composed of the fully connected layer for the final classification prediction.

2. The method according to claim 1, characterized in that: Before using the twin network layer to extract the spatial dimension features and spectral dimension features of hyperspectral data, it also includes: The original hyperspectral data is input into an input layer for preprocessing, including black-white correction and spectral normalization, to obtain first preprocessed hyperspectral data.

3. The method according to claim 2, characterized in that in: The step of black-and-white correction includes: using a white reference image and a dark reference image to calibrate the original hyperspectral data, and obtaining a first calibrated reflectance image as follows: Where R c is the first calibration reflectance image, R raw is the original reflectance image, R white is the white reference image, R dark is the dark reference image; The step of spectral normalization includes: normalizing the original hyperspectral data to the interval [0,1], and obtaining the first normalized spectral data as follows: Among them, x min and x max are the minimum and maximum values ​​of the original hyperspectral data x in the band respectively; The first pre-processed hyperspectral data includes the first calibrated reflectance image and first normalized spectral data.

4. The method according to claim 2 or 3, characterized in that: Before using the twin network layer to extract the spatial dimension features and spectral dimension features of hyperspectral data, it also includes: The first pre-processed hyperspectral data is compressed through a data network layer, including: Inputting the first preprocessed hyperspectral data as a whole into a forward encoding network to obtain encoded feature representation; The encoded feature representation is input into a quantizer, intermediate feature data of the encoded feature representation is extracted, the size of the intermediate feature data is reduced by a downsampling operation, and the quantized intermediate feature data is losslessly entropy encoded to obtain a compressed bit stream; Reconstruct the first pre-processed hyperspectral data by using the compressed code stream through an inverse decoding network; Through rate-distortion optimization, the first preprocessed hyperspectral data is flipped, and the data before and after flipping are respectively input into the two convolutional neural network branches of the twin network layer.

5. The method according to claim 1, characterized in that The fusion feature layer adopts a pixel-level fusion method to fuse the spatial dimension feature and the spectral dimension feature through a splicing operation to form the fusion feature vector.

6. The method according to claim 1, characterized in that The steps of the Transformer model for global modeling include: Reshape the fused feature vector into a format suitable for Transformer model input; adding a position code to the fused feature vector; Through the self-attention mechanism, the dependency between the feature vectors in the fused feature vector sequence with position encoding is captured.

7. The method according to claim 1, characterized in that The fully connected layer maps the output features of the Transformer model to the probability distribution of the classification label, uses the Softmax activation function to obtain the probability of each category, and selects the category with the highest probability as the final classification prediction result.

8. The method according to claim 1, characterized in that The classification prediction results are input into an output layer, and the model is trained using the cross entropy loss function.

9. A hyperspectral rapid detection device, used to implement the hyperspectral spatial-spectral fusion classification method according to any one of claims 1 to 8, the device comprising: Dark box, the inner wall is coated with high reflectivity polytetrafluoroethylene spray material; A multi-faceted light source is installed on the inner wall of the dark box; A light source controller connected to the multi-faceted light source; A camera, mounted on the inner wall of the dark box, for collecting hyperspectral image data; A hyperspectrometer, connected to the camera, for analyzing spectral characteristics based on the hyperspectral image data; A liftable stage is installed on the inner wall of the dark box, and a sample is placed on the stage; A motion control system connected to the liftable loading platform; The main control computer is connected to the light source controller, the motion control system, and the hyperspectrometer, respectively, to drive the light source controller to control the on state of the multi-faceted light source; drive the motion control system to control the movement of the liftable stage to adjust the distance between the sample and each light source and the camera lens; and receive the spectral characteristics transmitted by the hyperspectrometer for analysis to obtain the final classification prediction result.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the steps of the hyperspectral spatial-spectral fusion classification method according to any one of claims 1 to 8 are implemented.

Images