A traditional Chinese medicinal material classification method based on hyperspectral processing model of space spectrum fusion

Abstract

The application discloses a traditional Chinese medicinal material classification method based on a hyperspectral processing model of spectral fusion, and comprises the following steps: first, acquiring a hyperspectral image of the traditional Chinese medicinal material to be classified; then, performing fusion on the wave bands of the hyperspectral image to reduce the number of wave bands, and performing normalization processing on the pixels in each wave band after the fusion; next, taking the pixel points in the image as the center one by one, extracting the neighborhood region of the center pixel point as a sub-image, and taking the center pixel point of each sub-image as a point to be recognized; inputting the sub-image into a classification model, extracting the image spectral features and image spatial features of the sub-image respectively, performing splicing, enhancing the channel features through an attention mechanism, and finally outputting the classification category of the traditional Chinese medicinal material. The application can overcome the problem caused by the color similarity of the traditional Chinese medicinal material, accurately distinguish the subtle differences between the traditional Chinese medicinal materials, and ensure the correctness and efficacy of a prescription.

Description

Technical Field

[0001] This invention belongs to the field of image data processing technology, and specifically relates to a method for classifying Chinese medicinal materials based on a spatial-spectral fusion hyperspectral processing model. Background Technology

[0002] Traditional classification and identification of Chinese medicinal herbs often rely on experienced pharmacists who distinguish the types of herbs by their shape, color, and odor. While effective, this method is highly dependent on the pharmacist's professional experience and is easily influenced by human factors, leading to identification errors or omissions, which can affect the accuracy and efficacy of prescriptions.

[0003] With the rapid development of deep learning technology, scholars have recently attempted to use deep learning methods to achieve automatic identification of Chinese medicinal herbs. Compared with traditional machine learning methods (such as support vector machines and K-nearest neighbors), deep learning can automatically learn features from data, avoiding the limitations of manual feature extraction, and exhibits better performance when processing complex, high-dimensional data. Especially in the field of image recognition, deep learning, through multi-layer network structures, can capture more complex image features, significantly improving the accuracy and efficiency of classification.

[0004] However, many existing deep learning methods still rely on RGB images for recognition. Although RGB images can visually reflect the appearance of medicinal materials, many materials are difficult to distinguish effectively in RGB space due to similarities in color and shape, resulting in low classification accuracy. In contrast, hyperspectral images provide richer spectral information than RGB images. Each pixel contains not only color information in the visible light range but also spectral data from the ultraviolet to near-infrared bands. The subtle differences in these bands allow hyperspectral images to more accurately distinguish different Chinese medicinal materials. Therefore, hyperspectral images can overcome the limitations of RGB images and significantly improve the accuracy and efficiency of Chinese medicinal material classification. Summary of the Invention

[0005] To address the issues that manual identification of medicinal materials requires high levels of pharmacist experience, has low efficiency, and is prone to errors and omissions, thus affecting the efficacy of traditional Chinese medicine prescriptions, current methods for classifying and identifying medicinal materials using RGB images or traditional machine learning suffer from low classification accuracy. This invention proposes a method for classifying traditional Chinese medicine materials based on a spatial-spectral fusion hyperspectral processing model.

[0006] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows:

[0007] A method for classifying Chinese medicinal materials based on a spatial-spectral fusion hyperspectral processing model includes the following steps:

[0008] S1, Obtain hyperspectral images of the Chinese medicinal materials to be classified, wherein the hyperspectral images include multiple bands representing different spectral information;

[0009] S2, perform band fusion on the hyperspectral image to reduce the number of bands, and then perform normalization on the pixels in each fused band;

[0010] S3: Using the pixels in the image processed by S2 as center pixels, extract the neighborhood regions of the center pixels as sub-images, and use the center pixel of each sub-image as the point to be identified, so as to classify each point to be identified based on the sub-images using a classification model.

[0011] S4. The sub-images are input into the classification model, and the image spatial features and image spectral features of the sub-images are extracted and then stitched together. Then, the channel features are enhanced through an attention mechanism, and finally the classification category of Chinese medicinal materials is obtained based on the channel features.

[0012] Furthermore, step S1 includes:

[0013] The hyperspectral data of the Chinese medicinal materials to be classified is obtained, including image metadata and actual hyperspectral image data, and a complete hyperspectral image including multiple bands is generated through association operations. The image metadata includes parameter information including image size, band information and coordinates. Each band in the complete hyperspectral image represents different spectral information, and the data of the complete hyperspectral image is a 3D array including height, width and number of bands, where the number of bands represents the number of spectral data channels of each pixel in the image.

[0014] Furthermore, in step S2, performing band fusion on the hyperspectral image to reduce the number of bands includes the following process:

[0015] The original bands are grouped and fused, which means that a predetermined number of adjacent bands are grouped together, and the bands in each group are added together and then averaged to form a fused band. After performing group fusion a preset number of times on the original bands, if there are any bands that have not been fused, they are discarded.

[0016] Furthermore, in step S2, the normalization process for pixels in each fused band includes the following steps:

[0017] The maximum pixel value of each pixel in the image obtained after fusion is counted. Then, all data in the image of each band is divided by the maximum pixel value of the corresponding band and the result is normalized, thereby limiting the image data value to [0, 1]. If the image data in a certain band does not change, that is, the maximum value is equal to the minimum value, the normalized value of this band is directly set to 0.

[0018] Furthermore, step S3 includes:

[0019] The preset number of pixels and the shape of the pixel arrangement are used as the neighborhood region for forming the sub-image. Each pixel in the image processed in step S2 is used as the center pixel of a sub-image. All pixels are traversed to extract sub-images one by one based on the neighborhood region. Each sub-image uses the center pixel as the point to be identified. When there are missing pixels in the sub-image range, the mirror filling method is used to fill them. When extracting sub-images, the extracted sub-images are processed in batches.

[0020] Furthermore, the extracted sub-images are processed in batches. This involves loading the extracted sub-images into memory, and whenever the number of sub-images loaded into memory reaches a preset threshold, this batch of sub-images is transferred to the hard disk to free up memory space.

[0021] Furthermore, in step S4, the classification model is a trained convolutional neural network model, which includes parallel spatial feature channels and spectral feature channels. The outputs of the two channels are concatenated and then processed by an attention-based SE module to enhance the channel features.

[0022] The spatial feature channel includes multiple cascaded spatial feature extraction blocks, and convolutional layers, batch normalization layers, and sigmoid activation functions connected after the spatial feature extraction blocks. Each spatial feature extraction block includes cascaded deep convolutional layers, pointwise convolutional layers, batch normalization layers, an SE module based on channel attention mechanism, and a ReLU activation function.

[0023] The spectral feature channel includes a series of fully connected layers and a ReLU activation function. After the ReLU activation function, multiple spectral feature extraction blocks are connected in series. The input of each spectral feature extraction block is concatenated with the output through residual connection and then used as the final output. The spectral feature extraction block includes a series of batch normalization layers, fully connected layers, GELU activation functions, Dropout layers, fully connected layers, and Sigmoid activation functions.

[0024] The SE module compresses the number of channels in the input feature map through a fully connected layer and a batch normalization layer. After processing by the ReLU activation function, it is restored to the original number of channels through a fully connected layer. Then, it is processed by the ReLU activation function again. Finally, the fully connected layer maps the multi-channel data to the channels representing the categories, thereby outputting the predicted probability of each category.

[0025] Furthermore, during training, the convolutional neural network model uses pre-collected hyperspectral images of various medicinal materials as training data, and adjusts its parameters using a loss function. The expression for the loss function is:

[0026] ;

[0027] Where C represents the total number of categories, and the categories include medicinal material types and background. It is the true probability of category c in image label b. It is the model's predicted output. It is a smoothed label. ,in It is the smoothing coefficient. The predicted value is an array of length equal to the number of medicinal herb types plus one, representing the probability of each type and background in the hyperspectral image of medicinal herbs. This represents the standard cross-entropy loss, used to calculate the difference between the prediction and the true label; This represents the label smoothing loss, used to smooth out the true labels.

[0028] Furthermore, in step S4, when extracting spatial features from a sub-image, the sub-image is first input into a spatial feature extraction block. In the spatial feature extraction block, it is first processed by a depthwise convolutional layer with a 3×3 convolutional kernel, where each channel of the sub-image is convolved independently, and the number of channels in the output feature map is the same as that in the input. Then, a pointwise convolutional layer using a 1×1 convolutional kernel is used to perform pointwise convolution and expand the number of channels. Finally, a batch normalization layer is used to make the mean of the output of each channel 0 and the variance 1.

[0029] The features are then enhanced by an attention-based SE module and finally processed by a ReLU activation function as the output.

[0030] After being processed by three concatenated spatial feature extraction blocks with the same structure, the output is processed by an nn.Conv2d convolutional layer with a kernel size of 3×3, and the output has a feature map with 1 channel. After being processed by a batch normalization layer, the final output is mapped to the range of 0 to 1 by the Sigmoid activation function.

[0031] When extracting spectral features from a sub-image, the sub-image is first input into a fully connected layer to map it into a high-channel representation space. After being corrected by the ReLU activation function, it is input into the spectral feature extraction block. In the spectral feature extraction block, it is first processed by a batch normalization layer to make the mean of the features 0 and the variance 1. Then, a nonlinear transformation is performed by a fully connected layer, followed by processing by the GELU activation function. Subsequently, 50% of the output is randomly discarded by a Dropout layer. Then, it is processed by a fully connected layer and a GELU activation function in sequence to enhance the nonlinear representation capability. Each spectral feature extraction block is connected by a residual connection to add the input and output of each spectral feature extraction block. The output of the last fully connected layer in the last spectral feature extraction block is then processed by a sigmoid layer to serve as the image spectral feature output.

[0032] Furthermore, in step S4, image spatial features and image spectral features are spliced ​​together, and then channel features are enhanced through an attention mechanism. Finally, the classification categories of Chinese medicinal materials are output, including:

[0033] The outputs of the two channels are concatenated. Then, the number of channels in the input tensor is compressed to reduce the number of features. After processing with the ReLU activation function, the compressed features are restored to the original number of channels. Then, the ReLU activation function is applied again. The feature data after the above channel compression and expansion is passed through the last fully connected layer to map the number of channels to the number of categories, and is used as the output to represent the predicted probability distribution of each category.

[0034] The technical advantage of this invention lies in its ability to effectively improve the accuracy and efficiency of Chinese medicinal herb classification by combining multi-band information from hyperspectral images with the powerful feature extraction capabilities of deep learning models. Compared to traditional RGB image-based classification methods, this invention overcomes the problems caused by the color similarity of Chinese medicinal herbs, accurately distinguishing subtle differences and ensuring the correctness and efficacy of prescriptions. Furthermore, by utilizing techniques such as convolutional neural networks, residual connections, and spatial-spectral fusion, the model's expressiveness and stability are enhanced.

[0035] Furthermore, this invention utilizes hyperspectral image data, rather than traditional RGB images, in the classification of medicinal materials, thus providing more accurate and abundant spectral information for the identification of Chinese medicinal materials. This method is highly practical and can be widely applied to the automatic identification, quality control, and intelligent management of Chinese medicinal materials, effectively reducing human identification errors and ensuring the efficacy of Chinese medicinal prescriptions.

[0036] The invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0037] Figure 1This is a flowchart illustrating an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the structure of a convolutional neural network model according to an embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram of the spatial feature channel structure of the convolutional neural network model according to an embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of the spectral feature channel structure of the convolutional neural network model according to an embodiment of the present invention.

[0041] Figure 5 This is a schematic diagram of the input and output of an embodiment of the present invention. The left side is an image synthesized from the input hyperspectral images of Chinese medicinal materials. From top to bottom, the images show four Chinese medicinal materials: licorice, astragalus, polygonatum, and maca. The right side is the output.

[0042] To keep the illustrations concise, each figure only schematically depicts the relevant parts of the invention and does not represent the structure of the actual product. Furthermore, for the sake of brevity, components with the same structure or function are described only once in the figures; this does not mean there is only one such component, and more than one may be depicted. Detailed Implementation

[0043] The method for classifying Chinese medicinal materials based on a spatial-spectral fusion hyperspectral processing model provided in this embodiment includes the following steps:

[0044] Step 1: Acquire hyperspectral images.

[0045] This step first requires reading the hyperspectral data to be identified. Hyperspectral data typically consists of two files: an .hdr file, which contains metadata about the image (such as image size, band information, coordinates, reflectance information, etc.), and an .img file, which contains the actual hyperspectral image data. Then, by parsing the metadata information in the .hdr file and associating it with the corresponding image data in the .img file, the complete hyperspectral image data is obtained.

[0046] This embodiment uses this step to effectively read hyperspectral data and generate an image matrix containing multiple bands, each representing different spectral information. This information will be used for subsequent tasks such as feature extraction and classification. Image data is typically a three-dimensional array, as shown below. Figure 5 Taking the image on the left as an example, it includes a height of 2466, a width of 567, and a band count of 597, where the band count represents the number of spectral data channels for each pixel in the image. Figure 5 The image on the left is a pseudo-color image synthesized from the input hyperspectral images of Chinese medicinal herbs. From top to bottom, the images show four Chinese medicinal herbs: licorice, astragalus, polygonatum, and maca.

[0047] The key function of this step is to load the raw data of the hyperspectral image, thereby providing a data foundation for subsequent processing and analysis.

[0048] Step 2: Perform channel fusion on the hyperspectral image.

[0049] Step 2.1: Channel fusion operation. In this embodiment, considering that the number of bands in the input image is 597, while the number of input channels in the classification model in this embodiment is 64, it is set that every 9 bands are fused into a new band. Specifically, every 9 bands are grouped together, added, and then averaged. After performing the above operation 64 times, the final generated image will contain 64 new fused bands. This operation simplifies the image by reducing the number of bands, while preserving the spectral information of each group of bands as much as possible. In specific execution, the shape of the original image is first obtained, and the maximum number of effective channels to be retained is calculated, that is, the maximum fused channel value is 9×64, which means that 576 bands are fused. At this time, there are still 21 bands in the image that have not been fused, and these 21 bands are discarded. Because hyperspectral data has many bands, high dimensionality, and large redundancy, direct modeling will lead to high computational load, high GPU / RAM usage, slow training, and difficult deployment; and some band noise will weaken the model's discrimination ability. This step significantly reduces the number of input channels, thereby lowering training / inference computation and memory usage, and improving project deployability. This embodiment uses "adjacent band" fusion instead of random dimensionality reduction, preserving the continuous trend of the spectral curve, which is more suitable for the "spectral shape differences" characteristic of traditional Chinese medicine materials.

[0050] Step 2.2: Normalize the pixels of each band in the image. For each band after image fusion, which corresponds to a channel in the classification model, normalization is performed. For each band, the maximum value in the image is calculated, and then all image data is divided by the maximum value. This ensures that the image data range is between [0,1]. For bands where pixel values ​​do not change (i.e., the maximum value equals the minimum value), the value of that band is set to 0 to avoid invalid bands affecting subsequent operations. This step, through averaging, weakens occasional noise in individual bands, improves input stability, and indirectly enhances classification robustness.

[0051] Step 3: Perform pixel-by-pixel sub-image extraction on the hyperspectral image obtained in Step 2 after channel fusion.

[0052] Step 3.1: In this step, a sliding window technique is used to traverse each pixel in the image, and each pixel is used as the center of a sub-image. The size of each sub-image is generated according to a preset `patch_size`, and the pixel at the center of the sub-image serves as the identification point for the classification model. In this embodiment, `patch_size` is set to 9, meaning the sub-image size is 9×9 pixels. This method extracts a large number of small regions from the entire image. These small regions contain local spectral information, which serves as input to the subsequent classification model. Classifying the sub-image is essentially classifying the identification point, while the pixels surrounding the identification point (i.e., other pixels in the sub-image) play a role similar to auxiliary judgment. When extracting sub-images, the neighborhood region centered on each pixel is extracted based on its coordinates. For the edges of the sub-image, if a pixel cannot completely capture its neighborhood region (i.e., it exceeds the original image boundary), a mirroring method is used to expand the image boundary. This method avoids incomplete sub-images due to missing edge pixels during extraction. Images of Chinese medicinal herbs often suffer from problems such as background clutter, fragmentation, and subtle differences in local textures. If only the entire image is classified or only a single point of the spectrum is taken, it is easily overwhelmed by background / local noise, making it difficult to distinguish fine-grained categories. However, this embodiment uses local neighborhoods, i.e., sub-images, as modeling units. Compared with the average or global features of the entire image, it is less susceptible to local occlusion and background dominance.

[0053] Step 3.2: This step aims to address the issue that the number of sub-images in an image may be very large, causing memory overflow if all sub-images are directly loaded into memory. To efficiently process and store these sub-images, this embodiment uses batch processing. Whenever a certain number of sub-images are extracted, they are stored on the hard disk; the specific number can be controlled by the `batch_size` parameter. In practice, this can be achieved by creating a memory-mapped file to gradually write the sub-image data to the hard disk.

[0054] Step 4: Use a convolutional neural network as a classification model to identify and classify each pixel in the input hyperspectral image.

[0055] This step classifies the output sub-images described in step 3 using a trained convolutional neural network classification model. The number of input sub-images can also be set using `batch_size` to avoid memory overflow issues caused by an excessive number of sub-images.

[0056] like Figure 2As shown, after importing the sub-image, the classification model has two paths: extracting spectral features and extracting spatial features. After extracting features from both levels, the model uses the `torch.cat` function to concatenate them on specified channels, then inputs them into the SE module to enhance feature representation. Finally, a custom classifier function is used for classification. When classifying Chinese medicinal materials, relying solely on spatial features can lead to misclassification in the "similar color / texture" category; relying solely on spectral features may ignore morphology and texture, and be affected by local noise. Therefore, this embodiment extracts texture / morphology through the spatial channel and spectral curve discrimination information through the spectral channel. The two feature paths are then fused to ensure that for medicinal materials with similar appearances but different chemical compositions, the spectral channel provides key discrimination; for medicinal materials with similar compositions but different morphology and texture, the spatial channel provides supplementary discrimination. This approach also effectively improves generalization stability. When spatial or spectral feature information is interfered with (e.g., illumination affects texture contrast, or noise in some bands affects the spectrum), the other feature can still provide effective classification evidence. The specific implementation of each layer in the two channels and the model training method are described below.

[0057] Step 4.1: As Figure 3 As shown, the sub-image data output in step 3 enters the spatial feature extraction channel, and is first processed by a depthwise convolutional layer. The specific convolution operation can be represented as follows:

[0058] ;

[0059] in This represents the result of processing by a deep convolutional layer. This indicates that the input channels are accumulated, where c is the number of input channels. The number of channels in the input sub-image; This indicates that the spatial channels of the convolution kernel are accumulated. Indicates the height of the input sub-image. This represents the width of the input sub-image, where m and n are the height and width of the convolution kernel, respectively. This indicates the position of the c-th channel of the input feature map in the convolutional region. Pixel values; This indicates the position of the convolution kernel from the c-th input channel to the o-th output channel. The weight parameters, This represents the learnable bias parameter of the o-th output channel.

[0060] This deep convolutional layer uses a 3×3 kernel with a stride of 1 and padding of 1, employing depthwise separable convolution. Each channel is convolved independently, and the output feature map has the same number of channels as the input. Subsequently, the image undergoes pointwise convolution with a 1×1 kernel and a stride of 1, mapping the input channels from 3 to 8. The convolution results are then batch normalized to ensure that the mean of each channel's output is 0 and the variance is 1, thereby improving model stability. After standardization, the data undergoes feature enhancement using a custom SE module to enhance important spatial features. Finally, the data is activated by ReLU, generating a feature map as shown below. Figure 3 The output shown is out1, which is the output of the first spatial feature extraction block.

[0061] After entering the second spatial feature extraction block, the data first undergoes depthwise convolution to further extract spatial features. Next, the data is expanded from 8 to 16 channels through pointwise convolution. Subsequently, the convolution results are batch normalized, as shown in the diagram. To ensure data stability. This represents the input, where the subscript i indicates the i-th channel. This represents the batch mean of the current batch B samples in this channel. This represents a learnable scaling parameter (scale), used to scale up or down the standardized results (mean 0, variance 1) by a certain ratio. It is a learnable translation parameter used to shift the result of the previous step up or down to change its mean. It is a small positive constant used to prevent the denominator from becoming 0, which could lead to numerical instability or overflow.

[0062] The data was then further enhanced with features using the SE module, and subsequently generated using the ReLU activation function, as shown below. Figure 3 The output shown is out2, which is the output of the second feature extraction block.

[0063] Upon entering the third spatial feature extraction block, the data first undergoes depthwise convolution to further extract spatial features. Next, the data is expanded from 16 to 24 channels through pointwise convolution to obtain more feature information. After batch normalization, the data undergoes feature enhancement through the SE module, and then generates features using the ReLU activation function. Figure 3 The output shown is out3, which is the output of the third depthwise separable feature extraction block.

[0064] Finally, the processed data is convolved using nn.Conv2d, with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel of 1, to generate the final feature map. The data undergoes batch normalization to ensure the stability of the convolutional output. Finally, a sigmoid activation function is applied. The final output is mapped to a range of 0 to 1 to obtain the spatial feature result, where e represents the natural base and x represents the input.

[0065] Step 4.2: As Figure 4 As shown, the hyperspectral image output after fusion in step 2 has 63 channels. When the sub-image data output in step 3 enters the spectral feature extraction channel, it first passes through a fully connected layer. ,in For input, Represents the weight matrix. This represents the bias vector. This represents the activation function. This is the output of the fully connected layer. The fully connected layer has 63 input channels and 128 output channels. This fully connected operation maps the input data of the hyperspectral image from 63 channels to a representation space of 128 hidden channels, which is then corrected using a ReLU activation function.

[0066] The processed data enters a normalization layer, which standardizes the features of each input sample to ensure that the mean of the 128 features for each sample is 0 and the variance is 1. This improves the stability of the model, accelerates convergence, and prevents gradient explosion or vanishing problems. The normalized data is then input into a fully connected layer for linear transformation, which maps the data back to 128 channels through a linear transformation.

[0067] The data is then passed through a GELU activation function. To enhance the model's expressive power and increase its nonlinear characteristics, among which... The input is tanh, which represents the hyperbolic tangent function. A Dropout layer is connected after the GELU activation function to prevent overfitting. The Dropout layer randomly discards 50% of the neurons, which means randomly discarding 50% of the output of the GELU activation function, thereby enhancing the model's robustness and effectively improving its generalization ability.

[0068] Next, the data is processed again through a fully connected layer for further nonlinear mapping, followed by another pass through the GELU activation function. This layer enhances the nonlinear representation of the features and prepares the output for subsequent network layers. In the spectral feature extraction channel, each spectral feature extraction block is connected via residual connections. The input and output of each spectral feature extraction block are summed, where Indicates the output. This represents the input. This residual connection helps mitigate the vanishing gradient problem, ensuring the network remains effectively trained as depth increases. After multiple layers of processing, the data is processed through a final fully connected output layer to generate the final spectral feature results.

[0069] Step 4.3: After obtaining the spectral and spatial feature data, the program concatenates the outputs of the two channels using the `torch.cat` function, and then enhances the feature representation by applying the SE module. The specific process of the SE module is as follows: First, the number of channels in the input tensor, `in_channels` (256), is compressed to `in_channels / / reduction` (256 / 8) to reduce the number of features; after passing through the ReLU activation function, the compressed features are restored back to the original number of channels, `in_channels` (256); finally, the ReLU activation function is applied again. This SE block learns the importance of each channel by compressing and restoring the feature channels, and adjusts the feature representation of each channel in the network by channel weighting. The feature data after passing through the above SE block is mapped from the 256 channels to the output of 5 channels through the last fully connected layer, representing the predicted probability distribution of the five classes, i.e., the output. In this embodiment, the five classes are licorice, astragalus, polygonatum, maca, and the background. The final output image of the Chinese medicinal materials classification is as follows. Figure 5 As shown on the right, each pixel provides a separate output result.

[0070] In this embodiment, when training the classification model, a loss function for the convolutional neural network model is constructed to optimize the model during training. The expression for the loss function is as follows:

[0071] ;

[0072] Where C represents the total number of categories, and the categories include medicinal material types and background. It is the true probability of category c in image label b. It is the model's predicted output. It is a smoothed label. ,in It is the smoothing coefficient. The predicted value is an array of length equal to the number of medicinal herb types plus one, representing the probability of each type and background in the hyperspectral image of medicinal herbs. This represents the standard cross-entropy loss, used to calculate the difference between the prediction and the true label; This represents label smoothing loss, used to smooth out the true labels to prevent the model from overconfident in certain categories.

[0073] This embodiment uses hyperspectral images of Chinese medicinal herbs, including licorice, astragalus, maca, and polygonatum, as the dataset. Classification tests were conducted based on existing mainstream hyperspectral image classification algorithms SSANet, Res3Dnet, HybridSN, SSRNh, and DBDA, as well as the scheme provided by this invention. The detection results are shown in Table 1 below. The parameter count is the number of parameters used by the model, in M ​​(millions); the computational cost is the number of operations performed to obtain the result, in M ​​(millions); the inference time is the time taken to obtain the classification result, in seconds; and F1 is an indicator used to statistically measure the model accuracy.

[0074] Table 1. Classification results of the hyperspectral image dataset of Chinese medicinal materials.

[0075] .

[0076] As shown in Table 1, the experimental results on the hyperspectral image dataset of Chinese medicinal materials show that the classification accuracy of the method proposed in this invention is higher than that of existing mature algorithms, while also achieving lower parameter quantity, computational quantity and faster inference time. Therefore, this invention can effectively balance recognition accuracy, robustness and engineering deployability, and can provide classification accuracy superior to existing mainstream hyperspectral image classification algorithms under limited computing resources, proving the feasibility and effectiveness of this invention in the classification of Chinese medicinal materials.

[0077] It should be emphasized that the above-described embodiments are illustrative and not limiting. Therefore, the present invention is not limited to the embodiments described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of the present invention, without departing from the spirit and scope of the present invention, whether modifications or substitutions, shall also fall within the protection scope of the present invention.

Claims

1. A method for classifying Chinese medicinal materials based on a spatial-spectral fusion hyperspectral processing model, characterized in that, Includes the following steps: S1, Obtain hyperspectral images of the Chinese medicinal materials to be classified, wherein the hyperspectral images include multiple bands representing different spectral information; S2, perform band fusion on the hyperspectral image to reduce the number of bands, and then perform normalization on the pixels in each fused band; S3: Using the pixels in the image processed by S2 as center pixels, extract the neighborhood regions of the center pixels as sub-images, and use the center pixel of each sub-image as the point to be identified, so as to classify each point to be identified based on the sub-images using a classification model. S4. The sub-images are input into the classification model, and the image spatial features and image spectral features of the sub-images are extracted and then stitched together. Then, the channel features are enhanced through an attention mechanism, and finally the classification category of Chinese medicinal materials is obtained based on the channel features. In step S4, the classification model is a trained convolutional neural network model, which includes parallel spatial feature channels and spectral feature channels. The outputs of the two channels are concatenated and then processed by an attention-based SE module to enhance the channel features. The spatial feature channel includes multiple cascaded spatial feature extraction blocks, and convolutional layers, batch normalization layers, and sigmoid activation functions connected after the spatial feature extraction blocks. Each spatial feature extraction block includes cascaded deep convolutional layers, pointwise convolutional layers, batch normalization layers, an SE module based on channel attention mechanism, and a ReLU activation function. The spectral feature channel includes a series of fully connected layers and a ReLU activation function. After the ReLU activation function, multiple spectral feature extraction blocks are connected in series. The input of each spectral feature extraction block is concatenated with the output through residual connection and then used as the final output. The spectral feature extraction block includes a series of batch normalization layers, fully connected layers, GELU activation functions, Dropout layers, fully connected layers, and Sigmoid activation functions. The SE module compresses the number of channels in the input feature map through a fully connected layer and a batch normalization layer. After processing by the ReLU activation function, it is restored to the original number of channels through a fully connected layer. Then, it is processed by the ReLU activation function again. Finally, the fully connected layer maps the multi-channel data to the channels representing the categories, thereby outputting the predicted probability of each category.

2. The method according to claim 1, characterized in that, Step S1 includes: The hyperspectral data of the Chinese medicinal materials to be classified is obtained, including image metadata and actual hyperspectral image data, and a complete hyperspectral image including multiple bands is generated through association operations. The image metadata includes parameter information including image size, band information and coordinates. Each band in the complete hyperspectral image represents different spectral information, and the data of the complete hyperspectral image is a 3D array including height, width and number of bands, where the number of bands represents the number of spectral data channels of each pixel in the image.

3. The method according to claim 1, characterized in that, In step S2, performing band fusion on the hyperspectral image to reduce the number of bands includes the following process: The original bands are grouped and fused, which means that a predetermined number of adjacent bands are grouped together, and the bands in each group are added together and then averaged to form a fused band. After performing group fusion a preset number of times on the original bands, if there are any bands that have not been fused, they are discarded.

4. The method according to claim 1, characterized in that, In step S2, the normalization process for pixels in each fused band includes the following steps: The maximum pixel value of each pixel in the image obtained after fusion is counted. Then, all data in the image of each band is divided by the maximum pixel value of the corresponding band and the result is normalized, thereby limiting the image data value to [0, 1]. If the image data in a certain band does not change, that is, the maximum value is equal to the minimum value, the normalized value of this band is directly set to 0.

5. The method according to claim 1, characterized in that, Step S3 includes: The preset number of pixels and the shape of the pixel arrangement are used as the neighborhood region for forming the sub-image. Each pixel in the image processed in step S2 is used as the center pixel of a sub-image. All pixels are traversed to extract sub-images one by one based on the neighborhood region. Each sub-image uses the center pixel as the point to be identified. When there are missing pixels in the sub-image range, the mirror filling method is used to fill them. When extracting sub-images, the extracted sub-images are processed in batches.

6. The method according to claim 5, characterized in that, The extracted sub-images are processed in batches. The extracted sub-images are loaded into memory. Whenever the number of sub-images loaded into memory reaches a preset threshold, this batch of sub-images is transferred to the hard disk to free up memory space.

7. The method according to claim 1, characterized in that, During training, the convolutional neural network model uses pre-collected hyperspectral images of various medicinal materials as training data, and adjusts its parameters using a loss function. The expression for the loss function is as follows: ； Where C represents the total number of categories, and the categories include medicinal material types and background. It is the true probability of category c in image label b. It is the model's predicted output. It is a smoothed label. ,in It is the smoothing coefficient. The predicted value is an array of length equal to the number of medicinal herb types plus one, representing the probability of each type and background in the hyperspectral image of medicinal herbs. This represents the standard cross-entropy loss, used to calculate the difference between the prediction and the true label; This represents the label smoothing loss, used to smooth out the true labels.

8. The method according to claim 1, characterized in that, In step S4, when extracting spatial features from a sub-image, the sub-image is first input into a spatial feature extraction block. In the spatial feature extraction block, it is first processed by a depthwise convolutional layer with a 3×3 convolutional kernel. Each channel of the sub-image is convolved independently, and the number of channels in the output feature map is the same as that in the input. Then, a pointwise convolutional layer using a 1×1 convolutional kernel is used to perform pointwise convolution and expand the number of channels. Finally, a batch normalization layer is used to make the mean of the output of each channel 0 and the variance 1. The features are then enhanced by an attention-based SE module and finally processed by a ReLU activation function as the output. After being processed by three concatenated spatial feature extraction blocks with the same structure, the output is processed by an nn.Conv2d convolutional layer with a kernel size of 3×3, and the output has a feature map with 1 channel. After being processed by a batch normalization layer, the final output is mapped to the range of 0 to 1 by the Sigmoid activation function. When extracting spectral features from a sub-image, the sub-image is first input into a fully connected layer to map it into a high-channel representation space. After being corrected by the ReLU activation function, it is input into the spectral feature extraction block. In the spectral feature extraction block, it is first processed by a batch normalization layer to make the mean of the features 0 and the variance 1. Then, a nonlinear transformation is performed by a fully connected layer, followed by processing by the GELU activation function. Subsequently, 50% of the output is randomly discarded by a Dropout layer. Then, it is processed by a fully connected layer and a GELU activation function in sequence to enhance the nonlinear representation capability. Each spectral feature extraction block is connected by a residual connection to add the input and output of each spectral feature extraction block. The output of the last fully connected layer in the last spectral feature extraction block is then processed by a sigmoid layer to serve as the image spectral feature output.

9. The method according to claim 1, characterized in that, In step S4, the spatial features and spectral features of the image are spliced ​​together, and then the channel features are enhanced through an attention mechanism. Finally, the classification categories of Chinese medicinal materials are output, including: The outputs of the two channels are concatenated. Then, the number of channels in the input tensor is compressed to reduce the number of features. After processing with the ReLU activation function, the compressed features are restored to the original number of channels. Then, the ReLU activation function is applied again. The feature data after the above channel compression and expansion is passed through the last fully connected layer to map the number of channels to the number of categories, and is used as the output to represent the predicted probability distribution of each category.

Images