Pigment hyperspectral image classification method, storage medium and computer equipment

Abstract

The invention relates to the technical field of pigment image processing, and particularly discloses a pigment hyperspectral image classification method, a storage medium and computer equipment. The pigment hyperspectral image classification method comprises the following steps: acquiring a hyperspectral image of a simulated painting through a hyperspectral camera, firstly extracting low-level detail information by using a spectrum-space attention module, and then extracting deep semantic features in combination with ViT; the multi-scale features are then effectively fused for classification. According to the method, detail and global information in the hyperspectral image can be more comprehensively captured, so that the analysis and classification capability of complex hyperspectral data is improved, the pigment detection accuracy is improved, lossless and reliable technical support is provided for scientific repair and value protection of ancient calligraphy and painting, and the method is suitable for popularization and application. And a technical foundation is laid for intelligent development of cultural relic protection.

Description

Technical Field

[0001] The present invention relates to the technical field of pigment image processing, and in particular to a pigment hyperspectral image classification method, a storage medium and a computer device. Background Art

[0002] Pigments are an important part of cultural relics, which not only give them color and beauty, but also carry rich historical, cultural and technological information. Traditional Chinese painting pigments can be divided into "stone colors" (mineral colors) and "water colors / grass colors" (plant colors), as well as pigments synthesized from chemical raw materials used in modern times, that is, chemical colors. In order to better protect and display the artistic value of ancient calligraphy and paintings, it is usually necessary to perform color restoration and reproduction work on them. At present, although technical means such as laser Raman spectroscopy and X-ray fluorescence have been widely used in the identification of the categories of pigments in ancient calligraphy and paintings, due to the limitations of factors such as sample size and surface flatness, and the need for sampling for some analysis methods, the analysis of pigments in ancient calligraphy and paintings faces many challenges.

[0003] Hyperspectral imaging (HSI) has the advantages of non-destructive, wide-area scanning, and acquisition of complete spectral information. It can accurately analyze the pigments of ancient paintings and calligraphy, and is not limited by the surface flatness and size of the sample. This provides an efficient and reliable method for pigment identification. Hyperspectral imaging technology can quickly obtain detailed spectral information and spatial distribution characteristics of samples in a non-invasive way, and has the ability to collect data efficiently and quickly; however, hyperspectral image data has high dimensionality and high complexity, and traditional analysis methods are difficult to fully utilize this rich information. Therefore, it is particularly important to use deep learning technology to help analyze hyperspectral data.

[0004] In recent years, significant progress has been made in this field: a hyperspectral image classification method based on 3D-Convolutional Neural Networks (3D-CNN) combined with Convolutional Block Attention Module (CBAM) has been proposed. By combining the visual attention mechanism, the convolutional neural network can better focus on important features [Zhang Yan, Shen Jinyue, Shao Yuyi, et al. Computer & Digital Engineering, 2024, 52(03): 898-902+949]. A method for hyperspectral image classification based on combined pixel embedding and dual attention has been proposed. By adding a channel-space dual attention mechanism, the interference of redundant information is suppressed and the feature weights of the hyperspectral image space and spectrum are enhanced [Tan Yunfei, Li Ming, Luo Yonghang, et al. Computer & Digital Engineering, 1+3-9[2024-09-11]]. Someone proposed a semi-supervised network based on graph convolution to learn the importance between spatially adjacent regions by focusing on important information, and the image classification accuracy is greatly improved compared with traditional algorithms [Y. Ding, X. Zhao, Z. Zhang, W. Cai and N. Yang. in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 5504205]. Others have constructed a discriminative learning network RADN for hyperspectral target detection that combines regional guidance and dual attention mechanisms, alleviating the problem of difficult distinction between background and target caused by high similarity and large differences between different categories under the conditions of labeled samples [Zhong Jiaping, Li Yunsong, Xie Weiying, et al (Zhong Jiaping, Li Yunsong, Xie Weiying, etc.). Acta Electronica Sinica (Journal of Electronics), 2024, 52 (05): 1716-1729].

[0005] However, due to the high dimensionality and complexity of hyperspectral image data, there are still some problems in its effective feature extraction and classification. For example, traditional convolutional neural networks (CNN) have certain applications in hyperspectral image classification, but their continuous convolution and pooling operations may lead to the loss of detail information of high-dimensional spectral features, affecting classification performance; methods that rely solely on low-level or high-level features have limitations: although low-level features can capture detail information such as texture and edges, they may lack understanding of the overall semantics; and although high-level features can provide high-level semantic information, they may ignore detail features, such as small objects and subtle changes in the image.

[0006] The above technical problems need to be solved. Summary of the invention

[0007] In view of this, in order to fuse low-level and high-level features to more comprehensively capture the details and global information in hyperspectral images, thereby improving the analysis and classification capabilities of complex hyperspectral data, the present invention provides a pigment hyperspectral image classification method, storage medium and computer device.

[0008] In a first aspect, the present invention provides a method for classifying pigment hyperspectral images, comprising the following steps:

[0009] S1, obtaining a hyperspectral image of the pigment to be classified;

[0010] S2, inputting the hyperspectral image into a multi-scale spatial-spectral feature extraction network to obtain the fusion features of the hyperspectral image;

[0011] S3, input the fused features into the linear layer for classification output;

[0012] The multi-scale spatial-spectral feature extraction network includes a spatial attention module, a spectral attention module and a Vit module; the specific working steps are:

[0013] S21, the input hyperspectral image is subjected to the spatial attention module to extract spatial features, the spectral attention module to extract spectral features, and the Vit module to extract global spatial-spectral features;

[0014] S22, fusing the spectral feature, the spatial feature and the global spatial-spectral feature to obtain a fused feature.

[0015] As a further improvement of the above technical solution, the spectral attention module is used to analyze the reflectance characteristics of each band of the hyperspectral image, so as to extract the spectral characteristics of different materials and substances; the specific working steps are:

[0016] First, the collected hyperspectral data is input as The spatial dimension is expressed as (H, W), where H represents the height of the hyperspectral image, W represents the width, and B represents the number of spectral bands;

[0017] Then, the band information of each pixel in the hyperspectral image is extracted through maximum pooling and global average pooling to form a 1×1×B spectral feature vector, which is expressed as: h，w =[X h，w，1 , X h，w，2 , X h，w，3 , ..., X h，w，B ]; where X h，w Represents the spectral feature vector at position (h, w);

[0018] Then, these feature vectors are input into a multi-layer perceptron MLP to extract the weight of each band in the hyperspectral image, and then the extracted features are passed through the Sigmoid activation function to obtain the spectral weight coefficient, which is expressed as:

[0019] M Se (X) = σ(MLP(AvgPool(X))) + σ(MLP(MaxPool(X))) where X is the output hyperspectral image, σ is the sigmoid activation function, and M Se is the weight coefficient;

[0020] Finally, the calculated attention weights are weighted summed for each band of the original hyperspectral image, expressed as:

[0021] Where, X h，w，i represents the characteristics of the i-th band at position (h, w), M Se (X) is the attention weight.

[0022] As a further improvement of the above technical solution, the spatial attention module is used to analyze the geometric structure and texture information of the hyperspectral image, capture the spatial dependency between pixels, and thus obtain spatial features;

[0023] The specific working steps are as follows: first, the hyperspectral image is pooled in the channel dimension, and then the results of global pooling and average pooling, with dimensions of (H, W, 1), are concatenated according to the channel, and then a 7×7 convolution kernel is used to obtain the spatial features, and then the attention weight of each position is obtained by using the Sigmoid activation function. Finally, the calculated attention weights are weighted summed for the original hyperspectral image to enhance the key spatial features; the spatial attention calculation formula is summarized as:

[0024] M S (X′)=σ(f 7×7([AvgPool(X′);Maxpool(X′)]));

[0025] Where X′ represents the weighted result in the hyperspectral image, f 7×7 represents a convolutional layer with a convolution kernel of 7×7, and σ is the sigmoid activation function.

[0026] As a further improvement of the above technical solution, in the Vit module, the input hyperspectral image is divided into small blocks through convolution operation, and these small blocks are processed using position encoding; the processed image blocks are then input into the Transformer Encoder module to extract the global spatial-spectral features of the hyperspectral image.

[0027] As a further improvement to the above technical solution, the fusion features of the hyperspectral image are obtained by using the attention weighted fusion method, and the specific working steps are as follows:

[0028] After extracting the multi-scale spatial features Fp, spectral features Fspe and global spatial-spectral features Fh, we first generate query Q, key K and value V vectors for each feature, expressed as:

[0029] Where W Q , W K , W V They represent the learnable weight matrices, and F represents the input features;

[0030] Then, by calculating the dot product of the query and key vectors and normalizing the results, we get the attention weights α corresponding to the spatial features Fp, spectral features Fspe, and global spatial-spectral features Fh. p , α spe , α h ;

[0031] Finally, these weights are used to perform weighted summation on the features and fuse different features into a comprehensive feature F fused , as shown below:

[0032] F fused =α spe F spe +α p F p +α h F h .

[0033] In a second aspect, the present invention provides a storage medium, wherein the storage medium stores instructions and data for implementing the aforementioned pigment hyperspectral image classification method.

[0034] In a third aspect, the present invention provides a computer device, including a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the aforementioned pigment hyperspectral image classification method.

[0035] Compared with the prior art, the present invention has the following beneficial technical effects:

[0036] The present invention provides a pigment hyperspectral image classification method, which combines different levels of information in the analysis process: using the spectral-spatial attention mechanism to capture detail features, and obtaining the overall high-level semantic information of the image through the Vision Transformer (ViT) model, thereby enhancing the representation ability and classification performance of complex pigment features. Experimental results show that the classification performance of this method on simulated painting samples is significantly better than traditional and other deep learning models: compared with the support vector machine (SVM), the classification accuracy is improved by 34.35%; compared with the HyBridSN and SSRN models, the accuracy is improved by 8.93% and 5.6%, respectively.

[0037] The present invention not only improves the accuracy of pigment detection, but also provides non-destructive and reliable technical support for the scientific restoration and value protection of ancient calligraphy and paintings, and lays a technical foundation for the intelligent development of cultural relics protection.

[0038] A storage medium and a computer device provided by the present invention, since they include the above-mentioned classification method, naturally also have the above-mentioned beneficial technical effects.

[0039] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0041] Figure 1 This is a flow chart of a pigment hyperspectral image classification method according to an embodiment of the present invention;

[0042] Figure 2 A color block mixing diagram simulating ancient calligraphy and painting pigments in an embodiment of the present invention;

[0043] Figure 3 This is a graph showing the spectrum of pigments in the embodiment of the present invention;

[0044] Figure 4A network structure diagram of a multi-scale spatial-spectral feature extraction network proposed in an embodiment of the present invention;

[0045] Figure 5 A comparison chart of ablation experiment evaluation indicators in an embodiment of the present invention;

[0046] Figure 6 A classification diagram of data sets using different models in an embodiment of the present invention;

[0047] Figure 7 It is a schematic diagram of the operation of the hardware device in the embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0049] Example

[0050] 1. Experimental part

[0051] 1. Pigment hyperspectral image acquisition system

[0052] This embodiment uses the iSpecHyper-VS1000 hyperspectral camera of Leson Optics for data acquisition. The camera adopts the push-broom imaging principle, collects the spectral information of the object line by line through the relative movement between the sample and the camera, and generates a three-dimensional data cube in the two dimensions of spectrum and space; the working spectrum range of the camera is 380-1020nm, covering the visible spectrum from ultraviolet to near-infrared region, suitable for spectral analysis of materials such as pigments; the instrument contains 300 spectral channels, and the spectral resolution is better than 2.5nm, ensuring the capture of subtle spectral differences; the lateral scanning angle is less than 60°, ensuring the uniformity of illumination within the field of view during imaging; the system uses a 300W halogen lamp as a stable continuous light source, and the emission spectrum covers a wide spectral range of 380nm-25000nm, which can provide smooth spectral output and avoid spectral discontinuity.

[0053] In order to ensure the diversity and accuracy of the experimental data, the collected samples are composed of color blocks mixed with different pigments; these color blocks are made by mixing specific ancient pigments in different proportions, simulating the distribution and mixing of pigments in actual ancient calligraphy and painting; this method not only ensures the authenticity of the samples, but also enables the collected hyperspectral data to better reflect the complex situations in actual applications. The collected samples and the spectra of each pigment are shown in Figure 2. Figure 2 , 3 shown.

[0054] (II) Multi-scale spatial-spectral feature extraction network

[0055] As a cube-like data, a hyperspectral image contains hundreds of band images at continuous wavelengths, each of which reflects the reflection characteristics of an object in a different spectral range. Through these band information, the spectral characteristics of different materials and substances can be finely distinguished. For such hyperspectral data, this embodiment proposes a multi-scale spatial-spectral feature extraction network, the network structure of which is as follows: Figure 4 shown.

[0056] In this multi-scale spatial-spectral feature extraction network, the input hyperspectral image first passes through the spectral attention module and the spatial attention module to extract spectral features and spatial features respectively. Specifically, the spectral attention module focuses on analyzing the reflectance characteristics of each band to extract the spectral features of different materials and substances; while the spatial attention module focuses on the geometric structure and texture information of the image to capture the spatial dependencies between pixels. In addition, the network divides the input hyperspectral image into small blocks (Tokens) through convolution operations and processes these small blocks using position encoding (Position Embedding); the processed image blocks are then input into the Transformer Encoder of the Vit (Vision Transformer) module to extract the global spatial-spectral features of the hyperspectral image.

[0057] The core of the attention mechanism is to assign weights according to the relevance of the input data, so that the model can focus on important parts and ignore irrelevant information, thereby extracting useful information more effectively; for hyperspectral image data, the spectral attention module highlights the characteristics of the spectral band, while the spatial attention module emphasizes the spatial features around the classified pixels; the method proposed in this embodiment uses the attention mechanism to capture the detailed information in the hyperspectral image, namely the spectral and spatial features, to complete the classification task of the pigment hyperspectral image.

[0058] Here, the collected hyperspectral data input is The spatial dimension is expressed as (H, W), where H represents the height of the hyperspectral image, W represents the width, and B represents the number of spectral bands.

[0059]

[0060] Then, the band information of each pixel in the hyperspectral image is extracted through maximum pooling and global average pooling to form a 1×1×B spectral feature vector:

[0061] X h，w =[X h，w，1 , X h，w，2 , X h，w，3 , …, X h，w，B] (2)

[0062] Among them, X h，w Represents the spectral feature vector at position (h, w).

[0063] Then, these feature vectors are input into a multi-layer perceptron (MLP) to extract the weight of each band in the hyperspectral image, and then the extracted features are passed through the Sigmoid activation function to obtain the spectral weight coefficient. Since the spectral features contain both global pattern information (overall spectral trend) and strong responses in local bands (key information contained in specific bands),

[0064] In order to capture these two types of information, this embodiment uses both average pooling and maximum pooling. Average pooling is used to extract the average trend of the overall spectrum, helping the model to focus on the global spectral distribution characteristics, smoothing abnormal band responses, and ensuring that the model does not rely too much on local drastic changes; while maximum pooling is used to capture local strong responses in the spectrum, which helps the model to be more sensitive when it needs to identify small but critical spectral features. This achieves the purpose of highlighting the bands containing important information and suppressing redundant or noisy bands.

[0065] The spectral attention calculation formula can be summarized as:

[0066] M Se (X)=σ(MLP(AvgPool(X)))+σ(MLP(MaxPool(X))) (3)

[0067] Where X is the output hyperspectral image, σ is the sigmoid activation function, and M Se is the weight coefficient.

[0068] Finally, the calculated attention weights are weighted summed for each band of the original hyperspectral image to enhance the key spectral features and improve the classification and recognition performance.

[0069]

[0070] Among them, X h，w，i represents the characteristics of the i-th band at position (h, w), M Se (X) is the attention weight.

[0071] In this embodiment, spatial information refers to the distribution pattern and local structural characteristics of pigments in the painting in two-dimensional space; the spectral information of each pixel not only reflects the composition of the local pigment, but also carries characteristics such as color changes, textures, and pigment overlaps in the spatial neighborhood. The introduction of spatial information helps to capture the spatial relationships between different pigments (such as boundaries, textures, and local differences). This information is of great significance for the classification and restoration of pigments in paintings.

[0072] Similar to spectral attention, the spatial attention mechanism calculates the weight of each position by analyzing the spatial features in the hyperspectral image, thereby highlighting the areas containing important information and suppressing irrelevant or noisy areas.

[0073] Specifically, the hyperspectral image is first pooled in the channel dimension to compress the channel size for subsequent feature learning. Then the results of global pooling and average pooling, with a dimension of (H, W, 1), are concatenated according to the channel. A 7×7 convolution kernel is then used to obtain the spatial features. The Sigmoid activation function is then used to obtain the attention weight of each position. Finally, the calculated attention weights are weighted summed for the original hyperspectral image to enhance the key spatial features.

[0074] The spatial attention calculation formula can be summarized as:

[0075] M S (X′)=σ(f 7×7 ([AvgPool(X′); Maxpool(X′)])) (5)

[0076] Where X′ represents the weighted result in the hyperspectral image, f 7×7 represents a convolutional layer with a convolution kernel of 7×7, and σ is the sigmoid activation function.

[0077] Vision Transformer (ViT) is a novel method for applying Transformer models to computer vision tasks such as image classification. The core idea of ​​ViT is to divide the image into small blocks (Tokens) and process these small blocks using a self-attention mechanism to capture long-range dependencies. This enables ViT to effectively extract global features of the image, thereby improving the understanding and classification performance of complex spectral and spatial information in hyperspectral images. In hyperspectral image processing, applying ViT to feature extraction can give full play to its powerful capabilities and help capture detailed spectral and spatial information in hyperspectral data, thereby improving classification accuracy and enhancing the understanding of complex spatial and spectral relationships in hyperspectral images.

[0078] In this embodiment, in order to meet the input requirements of the ViT model, the hyperspectral image is divided into small blocks (Tokens). Specifically, the image is divided into tokens through convolution operations, and the resolution of the image blocks is controlled by setting the convolution kernel size and compensation parameters.

[0079] The serialized representation processed by the embedding layer is used as input and passed to the encoder to process the serialized representation. After being processed by a series of Transformer encoders, the global features are extracted for subsequent classification tasks.

[0080] After extracting the multi-scale spatial features Fp, spectral features Fspe, and global spatial-spectral features Fh extracted by ViT, how to effectively fuse these features becomes a key step to improve the classification performance of the model. The core goal of feature fusion is to combine these features of different dimensions so that the model can more comprehensively understand the complex structure of the hyperspectral image, thereby more accurately identifying pigments. To achieve this goal, this embodiment adopts the method of attention weighted fusion. The attention mechanism dynamically assigns weights to different features to ensure that the most important features are fully expressed in the fusion process, thereby enhancing the expressiveness of the model.

[0081] Specifically, first generate query Q (Query), key K (Key) and value V (Value) vectors for each feature. Among them:

[0082] Q=W Q F (6)

[0083] N=W K F (7)

[0084] V=W V F (8)

[0085] W Q , W K , W V They represent the learnable weight matrices, and F represents the input features.

[0086] Then, the attention weight α of each feature is obtained by calculating the dot product of the query and key vectors and normalizing the result spe , α p , α h Next, these weights are used to perform weighted summation on the features, and different features are fused into a comprehensive feature as shown in formula (9).

[0087] F fused =α spe F spe +α p F p +αh F h (9)

[0088] This weighted fusion method not only ensures the effective integration of various features, but also strengthens the focus on key information through dynamic weight allocation.

[0089] See also Figure 7 , Figure 7 9 is a schematic diagram of the hardware device operation of the classification method corresponding to this embodiment, wherein the hardware device specifically includes: a computer device 901, a processor 902 and a storage medium 903; the storage medium 903 stores instructions and data for implementing the hyperspectral image classification method; the processor 902 loads and executes the instructions and data in the storage medium 903 for implementing the hyperspectral image classification method.

[0090] The storage medium 903 is a computer-readable storage medium used for computer-readable storage. The storage medium 903 stores one or more programs, and the one or more programs can be executed by one or more processors 902 to implement the above-mentioned image classification method.

[0091] 2. Experimental Results and Discussion

[0092] This section will discuss the experimental results in detail. First, the hyperspectral image data used in the experiment will be introduced, and then the impact of spectral spatial attention and ViT modules on model performance will be analyzed through ablation experiments. Finally, the proposed model will be compared with other mainstream models, and the experimental results will be analyzed in depth.

[0093] (I) Dataset Introduction

[0094] The hyperspectral image dataset used in this embodiment covers a variety of color block mixing diagrams that simulate ancient calligraphy and painting pigments. The image spectral range is 380-1020nm with 300 channels and a resolution of 2.5nm. In the data preprocessing stage, the image was processed by reflectivity correction, noise removal, etc. to ensure the accuracy and consistency of the data. The obtained raw data is divided into two parts, a training set and a test set, with the ratio set to 80% training set and 20% test set. The total number of training set samples is 22560, and the number of test set samples is 5690. The class names and the number of samples of each data set are shown in Table 1 below.

[0095] Table 1 Class names and number of samples in the dataset

[0096]

[0097]

[0098] (II) Experimental environment configuration and hyperparameter setting

[0099] In order to verify the pigment hyperspectral image classification algorithm based on multi-scale spatial-spectral feature extraction proposed in this embodiment, all algorithm models in this chapter are programmed in Python3.9 programming language using Pycharm2022.3.3, and the experiment is performed on a personal computer with an Intel(R) Xeon(R) Platinum 8157@2.30GHz CPU, an NVIDIA GeForceRTX3090 GPU, and 24GB RAM. In this embodiment, all deep models uniformly use the cross entropy loss function (CrossEntropyLoss), Adam optimizer, stochastic gradient descent (StochasticGradientDescent, SGD), the learning rate is 0.0001, and the training round Epoch is 200 rounds.

[0100] The quantitative evaluation method of the experiment uses overall accuracy OA (Overall Accuracy), average accuracy AA (Average Accuracy), Kappa coefficient K, precision (Precision), and F1 score as comprehensive evaluation indicators. The formulas for several evaluation indicators are as follows

[0101] OA represents the ratio of the number of correctly predicted samples of all categories to the total number of samples, and its calculation formula is as follows:

[0102]

[0103] AA is the ratio of the number of samples correctly predicted by the model to the total number of samples. It measures the overall prediction accuracy of the model for all categories, and its calculation formula is as follows:

[0104]

[0105] The Kappa coefficient is a multivariate discrete method for evaluating the classification accuracy and error matrix of remote sensing images. Its calculation formula is as follows:

[0106]

[0107] Among them, p0 represents the proportion of observation accuracy or consistency units, p e The proportion of cells that show accidental agreement or expected accidental agreement:

[0108]

[0109] Recall is the ratio of the number of positive samples to the number of true samples. The calculation formula is as follows:

[0110]

[0111] F1score is the harmonic mean of precision and recall, which is used to measure the model's balance between precision and recall. The higher the F1score, the better the model performs in these two indicators.

[0112]

[0113] Among them, TP represents the number of true positive examples; FP represents the ratio of negative positive examples; FN represents the number of false negative examples; TP, FP, and FN are all obtained from the confusion matrix of classification results.

[0114] (III) Ablation experiment

[0115] In order to evaluate the contribution of each module in the proposed model to the classification performance, two sets of ablation experiments were designed and implemented, removing the spectral-spatial attention module and the VisionTransformer (ViT) module respectively, and comparing them with the complete model. The following is the specific design and analysis method of the ablation experiment:

[0116] (A) The spectral-spatial attention module in the model is removed, while other network structures remain unchanged, aiming to evaluate the role of this module in low-level spectral and spatial feature extraction. By removing this module, it is expected to observe changes in the model in capturing detailed information of hyperspectral images and classification performance.

[0117] (B) A network without the ViT module is designed, using only the basic convolutional neural network and the spectral-spatial attention mechanism. This experiment aims to verify whether the self-attention mechanism of ViT significantly enhances the global feature extraction ability of the model and improves the classification accuracy.

[0118] Through ablation experiments, the performance of the network proposed in this embodiment and the network without the spectral-spatial attention module and the Vision Transformer module in the classification task were evaluated respectively; the experimental results were obtained through 10 repeated experiments and reported in the form of mean ± standard deviation.

[0119] Table 2 and Figure 5 The key evaluation indicators of the ablation experiments are summarized, including average accuracy (AA), overall accuracy (OA) and precision.

[0120] Table 2 Ablation experiment evaluation indicators

[0121]

[0122] As can be seen from Table 2, after removing the spectral-spatial attention module, the AA and OA of the model dropped to 86.47% and 82.68%, respectively, which shows that this module is essential for enhancing the model's ability to capture spectral and spatial features. After removing the ViT module, AA and OA dropped to 87.51% and 79.95%, respectively, which shows the importance of ViT in global feature extraction. The decrease in the Precision index further shows that the lack of these two modules will significantly affect the classification accuracy, resulting in an increase in the classification error rate.

[0123] These results show that after removing the spectral-spatial attention module, the model's performance in low-level feature extraction and classification accuracy drops significantly. This suggests that the spectral-spatial attention mechanism plays a vital role in enhancing effective band and pixel information. The lack of this module leads to the omission of key information during feature extraction, thereby reducing the model's sensitivity to the details of hyperspectral images. In addition, after removing ViT, the model's performance in global feature capture and classification tasks also drops significantly. The self-attention mechanism of ViT is essential for capturing long-range dependencies and global contextual information in images. After removing ViT, the model loses its ability to deeply analyze global features, resulting in incomplete feature representation, which affects the classification accuracy.

[0124] (IV) Comparison of classification effects of different models

[0125] The pigment hyperspectral image classification algorithm based on multi-scale spatial-spectral feature extraction proposed in this embodiment is compared with several other methods, including the machine learning method SVM (Support Vector Machine, support vector machine), and two deep learning methods SSRN (Spectral-Spatial Residual Network) and HyBridSN (Hybrid Spectral Network). In order to ensure the fairness of the experiment, all networks were compared under the same experimental conditions. The experimental results are shown in Table 3 and Figure 6 Each experiment was repeated 10 times, and the results are shown as mean ± standard deviation.

[0126] Table 3 Classification accuracy of different models on the dataset

[0127]

[0128]

[0129] The experimental results show that the hyperspectral image classification model based on multi-scale spatial-spectral feature extraction proposed in this embodiment is significantly superior to the traditional machine learning model SVM and two mainstream deep learning models HyBridSN and SSRN in multiple evaluation indicators. In terms of overall classification accuracy (OA), the model proposed in this embodiment reached 87.37±0.25%, which is close to 87.22±0.09% of HyBridSN, and significantly higher than 56.95±0.21% of SVM and 85.55±0.20% of SSRN, indicating that it has stronger performance in the overall classification task. In terms of average classification accuracy (AA), the proposed model is significantly higher than other methods with 95.89±0.12%, indicating that it is more balanced when processing various types of samples and can better deal with the imbalance problem between different categories in hyperspectral data. The Kappa coefficient (K) is 0.95, which is significantly higher than 0.53 of SVM and 0.83 of SSRN, showing the advantage of the model in classification consistency. In addition, the F1-score and Precision of the proposed model are 0.89 and 91.54±0.17% respectively, which are not only far higher than SVM’s 0.59 and 57.19±0.21%, but also better than SSRN’s 0.73 and 85.94±0.23%, reflecting an overall improvement in classification accuracy and recall rate.

[0130] These results fully demonstrate that the proposed multi-scale spatial-spectral feature extraction method has significant advantages in fusing the multi-scale spatial information and spectral features of hyperspectral images, and can effectively improve the classification performance in complex hyperspectral data scenarios. Compared with traditional machine learning and other deep learning methods, this model has a stronger ability to capture subtle spectral and spatial differences in hyperspectral images, and shows excellent robustness and accuracy in pigment category recognition tasks. These characteristics make it have important application prospects in fields such as hyperspectral image processing and cultural heritage protection.

[0131] Conclusion

[0132] This embodiment proposes a pigment hyperspectral image classification method that combines spatial-spectral attention and Vision Transformer (ViT); a hyperspectral image of a simulated painting is acquired through a hyperspectral camera, and the model first uses a spectral-spatial attention module to extract low-level detail information, and then combines ViT to extract deep semantic features; subsequently, these multi-scale features are effectively fused for classification.

[0133] Based on the above content, it can be seen that the model / method proposed in this embodiment is significantly better than the traditional support vector machine (SVM) method in feature extraction and classification performance, and is significantly improved compared to the existing HyBridSN and SSRN models. Specifically, the classification accuracy of the training set and the test set reached 98% and 91.54%, respectively. Compared with SVM, HyBridSN and SSRN, the classification accuracy was improved by 34.35%, 8.93% and 5.6%, respectively. Compared with the existing deep learning model that only extracts single-scale features, the multi-scale spatial-spectral feature extraction network of this embodiment performs better in terms of feature extraction sufficiency and recognition accuracy; the network is expected to be widely used in future cultural relics protection work, especially in pigment identification and classification tasks based on hyperspectral images, providing more accurate and efficient technical support for the refined restoration and scientific research of cultural relics such as ancient paintings and calligraphy.

[0134] Finally, it should be noted that this article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only to help understand the core idea of ​​the present invention. Without departing from the principles of the present invention, the present invention may also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A pigment hyperspectral image classification method, characterized in that: The following steps are involved: S1, obtaining a hyperspectral image of the pigment to be classified; S2, inputting the hyperspectral image into a multi-scale spatial-spectral feature extraction network to obtain the fusion features of the hyperspectral image; S3, input the fused features into the linear layer for classification output; The multi-scale spatial-spectral feature extraction network includes a spatial attention module, a spectral attention module and a Vit module; the specific working steps are: S21, the input hyperspectral image is subjected to the spatial attention module to extract spatial features, the spectral attention module to extract spectral features, and the Vit module to extract global spatial-spectral features; S22, fusing the spectral feature, the spatial feature and the global spatial-spectral feature to obtain a fused feature.

2. A pigment hyperspectral image classification method according to claim 1, characterized in that: The spectral attention module is used to analyze the reflectance characteristics of each band of the hyperspectral image, so as to extract the spectral characteristics of different materials and substances; the specific working steps are: First, the collected hyperspectral data is input as The spatial dimension is expressed as (H, W), where H represents the height of the hyperspectral image, W represents the width, and B represents the number of spectral bands; Then, the band information of each pixel in the hyperspectral image is extracted through maximum pooling and global average pooling to form a 1×1×B spectral feature vector, which is expressed as: X h，w =[X h，w，1 , X h，w，2 , X h，w，3 , …, X h，w，B ]; where X h，w Represents the spectral feature vector at position (h, w); Then, these feature vectors are input into a multi-layer perceptron MLP to extract the weight of each band in the hyperspectral image, and then the extracted features are passed through the Sigmoid activation function to obtain the spectral weight coefficient, which is expressed as: M Se (X) = σ(MLP(AvgPool(X))) + σ(MLP(MaxPool(X))) where X is the output hyperspectral image, σ is the sigmoid activation function, and M Se is the weight coefficient; Finally, the calculated attention weights are weighted summed for each band of the original hyperspectral image, expressed as: In the formula, X h，w，i represents the characteristics of the i-th band at position (h, w), M Se (X) is the attention weight.

3. A pigment hyperspectral image classification method according to claim 1, characterized in that: The spatial attention module is used to analyze the geometric structure and texture information of the hyperspectral image, capture the spatial dependency between pixels, and thus obtain spatial features; The specific working steps are as follows: first, the hyperspectral image is pooled in the channel dimension, and then the results of global pooling and average pooling, with dimensions of (H, W, 1), are concatenated according to the channel, and then a 7×7 convolution kernel is used to obtain the spatial features, and then the attention weight of each position is obtained by using the Sigmoid activation function. Finally, the calculated attention weights are weighted summed for the original hyperspectral image to enhance the key spatial features; the spatial attention calculation formula is summarized as: M S (X′)=σ(f 7×7 ([AvgPool(X′);Maxpool(X′)])); Where X′ represents the weighted result in the hyperspectral image, f 7×7 represents a convolutional layer with a convolution kernel of 7×7, and σ is the sigmoid activation function.

4. The pigment hyperspectral image classification method according to claim 1, characterized in that: In the Vit module, the input hyperspectral image is divided into small blocks through convolution operations, and these small blocks are processed using position encoding; the processed image blocks are then input into the Transformer Encoder module to extract the global spatial-spectral features of the hyperspectral image.

5. The pigment hyperspectral image classification method according to claim 1, characterized in that: The fusion features of the hyperspectral image are obtained by using the attention weighted fusion method, and the specific working steps are as follows: After extracting the multi-scale spatial features Fp, spectral features Fspe and global spatial-spectral features Fh, we first generate query Q, key K and value V vectors for each feature, expressed as: Where W Q , W K , W v They represent the learnable weight matrices, and F represents the input features; Then, by calculating the dot product of the query and key vectors and normalizing the results, we get the attention weights α corresponding to the spatial features Fp, spectral features Fspe, and global spatial-spectral features Fh. p , α spe , α h ; Finally, these weights are used to perform weighted summation on the features and fuse different features into a comprehensive feature F fused , as shown below: F fused =a spe F spe +a p F p +a h F h 。 6. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a pigment hyperspectral image classification method as described in any one of claims 1 to 5.

7. A computer device comprising a processor and a storage medium; characterized in that: The processor loads and executes instructions and data in the storage medium to implement a pigment hyperspectral image classification method as described in any one of claims 1 to 5.

Images