A ground feature classification method based on statistical and semantic features for on-site security

Abstract

The embodiment of the application relates to the technical field of temporary security, and discloses a ground feature classification method based on statistical and semantic features for temporary security, which comprises the following steps: inputting an acquired hyperspectral image to be classified into a classification model, acquiring features output by the first three convolution layers of the classification model, performing statistics on the features output by the first two convolution layers by using a statistical feature enhancement module of the classification model to obtain statistical texture features; acquiring multi-scale semantic features by using a multi-scale feature extraction module of the classification model based on the features output by the third convolution layer; fusing the statistical texture features and the multi-scale semantic features by using a statistical and semantic feature fusion module of the classification model to obtain space-spectrum joint features; and performing ground feature classification by using a classifier of the classification model based on the space-spectrum joint features to obtain ground feature categories corresponding to the hyperspectral image to be classified, so that the discrimination ability for shallow features is effectively enhanced, and the precision of ground feature classification of the hyperspectral image is improved.

Description

Technical Field

[0001] This application relates to the field of local security technology, and in particular to a method for classifying land features based on statistical and semantic features for local security. Background Technology

[0002] Temporary security refers to a diversified, cross-domain, three-dimensional, collaborative, and intelligent technical system addressing the needs of defense, protection, production, safety, and rescue within space. Specific application scenarios include low-altitude security, underwater security, and cross-domain security. The temporary security technology system encompasses six major research directions: related light detection, stability detection, multimodal cognitive computing, swarm intelligence decision-making, waterborne optics, and cross-domain remote sensing. Obtaining accurate information on ground cover categories is crucial for the effective implementation of typical temporary security tasks such as low-altitude UAV detection, extreme disaster early warning and rescue, and target reconnaissance. Hyperspectral images contain fine-grained spectral information and rich spatial information, serving as an important information carrier in the cross-domain remote sensing research direction within the temporary security technology system. Analyzing hyperspectral remote sensing images to classify ground cover is key to the successful implementation of temporary security applications; therefore, ground cover classification in hyperspectral images has received widespread attention.

[0003] Due to the complexity and variability of ground features, their irregular regional distribution, numerous types, and imbalanced sample sizes, coupled with the fact that the acquisition of hyperspectral remote sensing images is affected by meteorological conditions, sensor type, and shooting angle, and involves issues such as "different spectra for the same object" and "different objects for the same spectrum," accurately identifying the types of ground features contained in hyperspectral images is a challenging task. Constructing a discriminative feature extraction method for hyperspectral remote sensing images is a key step in overcoming these difficulties and achieving accurate identification of ground feature categories in hyperspectral remote sensing.

[0004] Hyperspectral image land cover classification methods can be categorized into handcrafted feature-based methods and depth feature-based methods, depending on the modeling and feature construction techniques used. Depth feature-based methods effectively alleviate the reliance on specialized knowledge inherent in handcrafted feature-based methods and address the limitation of adequately fitting data distributions when dealing with high-dimensional data, making them the mainstream technique for hyperspectral image land cover classification. Convolutional neural networks (CNNs), in particular, are widely used for joint spatial-spectral feature modeling of hyperspectral images. However, most CNN-based methods employ neighborhood image patch sampling, which presents challenges in capturing long-range dependencies and leads to computational redundancy due to overlapping sampling. To address these issues, Fully Convolutional Networks (FCNs) have been introduced into hyperspectral land cover classification, freeing the model from the constraints of neighborhood image patch size selection and offering new possibilities for enhancing classification performance.

[0005] However, land cover classification methods based on fully convolutional networks for hyperspectral images only focus on the deep, high-level semantic features captured by the convolutional neural network, ignoring the capture of shallow features, including texture information, contained in the image, resulting in a low accuracy rate for land cover classification of hyperspectral images. Summary of the Invention

[0006] The purpose of this application is to provide a land cover classification method based on statistical and semantic features for local security, which captures the texture information contained in the image in a statistical manner, enhances the discrimination ability of shallow features, and effectively improves the accuracy of land cover classification of hyperspectral images.

[0007] To address the aforementioned technical problems, embodiments of this application provide a method for land cover classification based on statistical and semantic features for near-site security, comprising the following steps: inputting the acquired hyperspectral image to be classified into a pre-trained classification model; obtaining the features output by the first three convolutional layers of the classification model; based on the features output by the first two convolutional layers, performing statistical analysis using the statistical feature enhancement module of the classification model to obtain statistical texture features; based on the features output by the third convolutional layer, using the multi-scale feature extraction module of the classification model to obtain multi-scale semantic features; fusing the statistical texture features and the multi-scale semantic features using the statistical and semantic feature fusion module of the classification model to obtain spatial-spectral joint features; and based on the spatial-spectral joint features, using the classifier of the classification model to classify land cover to obtain the land cover category corresponding to the hyperspectral image to be classified.

[0008] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for classifying ground features based on statistical and semantic features for local security.

[0009] This application provides a method for classifying ground features based on statistical and semantic features for near-site security. The classification model includes a statistical feature enhancement module and a multi-scale feature extraction module, which can simultaneously capture statistical texture features and high-level semantic features (multi-scale semantic features) from hyperspectral images. Statistical texture features can effectively characterize the shallow features of hyperspectral images, thus improving the completeness of feature modeling. The presence of a statistical and semantic feature fusion module enables adaptive fusion of statistical texture features and multi-scale semantic features, enhancing the discriminative ability of the extracted features. Based on this classification model, the accuracy of ground feature classification in hyperspectral images can be overcome by addressing the influence of climate conditions, sensor type and shooting angle, uneven distribution of ground features in hyperspectral images, and issues such as "different objects with the same spectrum" and "different spectra with the same object," significantly improving the accuracy of ground feature classification in hyperspectral images. This provides effective and accurate data support for typical near-site security tasks requiring precise ground feature category information, such as low-altitude UAV security, disaster early warning and rescue, and target reconnaissance.

[0010] In some optional embodiments, the step of using the statistical feature enhancement module of the classification model to perform statistical analysis on the features output by the first two convolutional layers to obtain statistical texture features includes: adding the features output by the first two convolutional layers element-wise to obtain shallow features; performing average pooling and max pooling operations on the shallow features to obtain global mean features and global maximum features, and calculating the cosine similarity between the shallow features and the global mean features and the global maximum features to obtain a first similarity matrix and a second similarity matrix; performing multi-level quantization on the first similarity matrix and the second similarity matrix to obtain a first quantization encoding matrix and a second quantization encoding matrix; performing counting and self-attention reconstruction on the first quantization encoding matrix and the second quantization encoding matrix to obtain global mean texture features and global maximum texture features; and adding the global mean texture features and the global maximum texture features element-wise to obtain statistical texture features. The features output by the first two convolutional layers are shallow features of the hyperspectral image to be classified, but these shallow features are still not obvious. In this application, the shallow features are input into the statistical feature enhancement module, and after pooling, similarity calculation, multi-level quantization, counting statistics and self-attention reconstruction, statistical texture features are obtained, which effectively enhances the feature expression of texture information, thereby enhancing the classification model's ability to discriminate statistical texture features.

[0011] In some optional embodiments, the first similarity matrix and the second similarity matrix are subjected to multi-level quantization using the following formulas to obtain the first quantization coding matrix and the second quantization coding matrix:

[0012]

[0013]

[0014]

[0015]

[0016] m∈[1,H×W]

[0017] Where N is the total number of orders of magnitude of the multi-level quantization, m represents the cosine similarity index, H×W represents the size of the first similarity matrix, the size of the second similarity matrix is ​​the same as the size of the first similarity matrix, and S avg Let S be the first similarity matrix. max This is the second similarity matrix. This represents the m-th item in the first similarity matrix. The m-th item of the second similarity matrix, This indicates that the first similarity matrix is ​​quantized to the order of n. This indicates that the second similarity matrix is ​​quantized to the order of n. This represents the m-th item in the first quantization encoding matrix obtained by quantization at the n-th order of magnitude. Let m represent the m-th item in the second quantization encoding matrix obtained by quantization at the n-th order of magnitude, where max(·) represents the maximum value function and min(·) represents the minimum value function.

[0018] In some optional embodiments, the first quantization coding matrix and the second quantization coding matrix are counted and reconstructed using the following formulas to obtain the global mean texture feature and the global maximum texture feature:

[0019]

[0020]

[0021]

[0022]

[0023] Among them, F avg Let F be the global mean feature. max For the global maximum feature, Cat(·) represents the concatenation operation, MLP(•) represents the multilayer perceptron operation, and V avg V is the first quantization encoding matrix. max The second quantization encoding matrix, where W1(·), W2(·), and W3(•) all represent 1×1 convolution operations, and S(·) represents the Softmax function. Represents global mean texture features. This represents the global maximum texture feature.

[0024] In some optional embodiments, the step of fusing the statistical texture features and the multi-scale semantic features using the statistical and semantic feature fusion module of the classification model to obtain spatial-spectral joint features includes: performing average pooling and max pooling operations on the statistical texture features respectively and concatenating them to obtain processed statistical texture features; performing average pooling and max pooling operations on the multi-scale semantic features respectively and concatenating them to obtain processed multi-scale semantic features; adding the processed statistical texture features and the processed multi-scale semantic features element-wise, and obtaining fusion weights by passing them through a convolutional layer and activating them based on the Sigmoid function; fusing the statistical texture features and the multi-scale semantic features based on the fusion weights to obtain spatial-spectral joint features. Statistical texture features effectively characterize the shallow visual cues of the hyperspectral image to be classified, while multi-scale semantic features well characterize high-dimensional semantic cues. Adaptively fusing the two further improves the discriminative ability of the classification model, thereby better improving the accuracy of land cover classification of hyperspectral images.

[0025] In some optional embodiments, the processed statistical texture features and the processed multi-scale semantic features are added element-wise, and the fusion weights are obtained by applying activation based on the Sigmoid function through a convolutional layer, as shown in the following formula:

[0026]

[0027] Among them, F′ sta For the processed statistical texture features, F′ sem The processed multi-scale semantic features, The expression represents element-wise addition, Conv(·) represents a 3×3 convolution operation, Sig(·) represents the Sigmod activation function, and λ is the fusion weight.

[0028] In some optional embodiments, the adaptive fusion of the statistical texture features and the multi-scale semantic features based on the fusion weights to obtain the spatial-spectral joint features is achieved through the following formula:

[0029]

[0030] Among them, F sta For the statistical texture features, F sem For the aforementioned multi-scale semantic features, Cat(·) represents element-wise multiplication, Cat(·) represents concatenation, and F represents element-wise multiplication. spe-spa The spatial-spectral joint feature is described above.

[0031] In some optional embodiments, the multi-scale feature extraction module includes a channel attention module, a spatial attention module, a 3×3 convolutional layer, and an atrous spatial convolutional pooling pyramid. The process of obtaining multi-scale semantic features based on the features output from the third convolutional layer using the multi-scale feature extraction module of the classification model includes: downsampling the features output from the third convolutional layer, passing them through the channel attention module, the spatial attention module, and the 3×3 convolutional layer, and then inputting them into the atrous spatial convolutional pooling pyramid for multi-scale feature extraction; using bilinear interpolation to restore the size of the features output from the atrous spatial convolutional pooling pyramid to obtain the multi-scale semantic features. The atrous spatial convolutional pooling pyramid (ASPP) is composed of atrous convolutions, which can balance a large receptive field and high feature map resolution, and can effectively extract multi-scale semantic features from the hyperspectral image to be classified.

[0032] In some optional embodiments, the loss function used when training the classification model includes a cross-entropy loss term and an online hard sample mining loss term, the loss function being expressed as:

[0033] L = L CE +α·L OHEM

[0034] Among them, L CE Let L be the cross-entropy loss term. OHEM Here, α is the online hard sample mining loss term, α is a preset balance factor, and L represents the loss function. The online hard sample mining loss term can automatically improve mining performance without requiring manual sample selection. Combining it with the cross-entropy loss term to generate a loss function for training the classification model enables the model to quickly achieve high-accuracy hyperspectral image land cover classification. Attached Figure Description

[0035] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0036] Figure 1 This is a flowchart of a method for classifying ground features based on statistical and semantic features for local security, provided in one embodiment of this application;

[0037] Figure 2 This is a schematic diagram of an SSFLNet network structure provided in one embodiment of this application;

[0038] Figure 3This is a flowchart provided by an embodiment of the present application, which describes how statistical texture features are obtained by using the statistical feature enhancement module of the classification model to perform statistical analysis based on the features output by the first two convolutional layers.

[0039] Figure 4 This is a schematic diagram illustrating the architecture and working principle of a statistical feature enhancement module provided in one embodiment of this application;

[0040] Figure 5 This is a flowchart provided by an embodiment of the present application, which describes how statistical texture features and multi-scale semantic features are fused using the statistical and semantic feature fusion module of a classification model to obtain spatial-spectral joint features;

[0041] Figure 6 This is a schematic diagram illustrating the architecture and working principle of a statistical and semantic feature fusion module provided in one embodiment of this application;

[0042] Figure 7 This is a schematic diagram illustrating the impact of different training sample rates on the classification results of SSFLNet on three datasets, provided by one embodiment of this application.

[0043] Figure 8 This is a schematic diagram illustrating the impact of different quantization levels on the classification results of SSFLNet on three datasets, provided by one embodiment of this application.

[0044] Figure 9 This is a schematic diagram illustrating the impact of different loss functions on the classification results of SSFLNet on three datasets, provided by one embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0046] One embodiment of this application relates to a land cover classification method based on statistical and semantic features for perimeter security. The implementation details of the land cover classification method based on statistical and semantic features for perimeter security in this embodiment are described in detail below. The following implementation details are provided for ease of understanding and are not necessary for implementing this solution.

[0047] The specific process of the land cover classification method based on statistical and semantic features for local security proposed in this embodiment can be described as follows: Figure 1 As shown, it includes:

[0048] Step 101: Input the acquired hyperspectral image to be classified into the pre-trained classification model, obtain the features output by the first three convolutional layers of the classification model, and use the statistical feature enhancement module of the classification model to perform statistics based on the features output by the first two convolutional layers to obtain statistical texture features.

[0049] In the specific implementation, let the hyperspectral image to be classified be... H and W represent the height and width of the hyperspectral image to be classified, and D represents the spectral dimension of the hyperspectral image. The classification model (also known as SSFLNet) consists of three 3×3 convolutional layers in its first three layers. Between any two adjacent 3×3 convolutional layers, there is a channel attention module and a spatial attention module. The features output by these three convolutional layers can be represented by F1, F2, and F3, respectively. The classification model can use a statistical feature enhancement module to perform statistical analysis based on the features output by the first two convolutional layers (i.e., F1 and F2) to obtain statistical texture features, which can be denoted as F... sta .

[0050] In one example, the network structure of SSFLNet can be as follows: Figure 2 As shown, the first three layers are three 3×3 convolutional layers. Between two adjacent 3×3 convolutional layers, there is a channel attention module and a spatial attention module. The features output by the first two convolutional layers are added pixel by pixel and then enter the statistical feature enhancement module. The features output by the third convolutional layer are then passed through a channel attention module and a spatial attention module before entering the fourth convolutional layer. The features output by the fourth convolutional layer enter the multi-scale feature extraction module. After the statistical feature enhancement module and the multi-scale feature extraction module, there is a statistical and semantic feature fusion module. The spatial-spectral joint features output by the statistical and semantic feature fusion module enter the last few convolutional layers and the classifier to obtain the prediction result (classification result).

[0051] In one example, when training the classification model SSFLNet, the loss function used includes a cross-entropy loss term and an online hard sample mining loss term, which can be expressed as:

[0052] L = L CE +α·L OHEM

[0053] In the formula, L CE L is the cross-entropy loss term. OHEMHere, α is the online hard sample mining loss term, α is a preset balance factor, and L represents the loss function. The online hard sample mining loss term can automatically improve mining performance without requiring manual sample selection. Combining it with the cross-entropy loss term to generate a loss function for training the classification model enables the model to quickly achieve high-accuracy hyperspectral image land cover classification.

[0054] In one example, based on the features output from the first two convolutional layers, statistical feature enhancement modules of the classification model are used to perform statistical analysis to obtain statistical texture features, which can be achieved through methods such as... Figure 3 The steps shown are implemented as follows:

[0055] Step 1011: Add the features output by the first two convolutional layers element by element to obtain shallow features.

[0056] In the specific implementation, the features F1 and F2 output by the first two convolutional layers are first added element-wise to obtain the shallow feature F. This indicates element-wise addition. H×W represents the spatial dimensions of the shallow features, C represents the number of shallow feature channels, and the shallow feature F is the input to the statistical feature enhancement module. The statistical feature enhancement module extracts texture information from the shallow features using statistical methods to obtain the statistical texture feature F. sta .

[0057] Step 1012: Perform average pooling and max pooling operations on the shallow features to obtain the global mean feature and the global maximum feature, and calculate the cosine similarity between the shallow features and the global mean feature and the global maximum feature to obtain the first similarity matrix and the second similarity matrix.

[0058] In the specific implementation, after the shallow feature F is input to the statistical feature enhancement module, the module first performs average pooling and max pooling operations on the shallow feature F to obtain the global mean feature. and global maximum value characteristics Then, the shallow feature F and the global mean feature F are calculated respectively. avg and global maximum feature F max The cosine similarity between them yields the first similarity matrix. Second similarity matrix

[0059] Step 1013: Perform multi-level quantization on the first similarity matrix and the second similarity matrix respectively to obtain the first quantization coding matrix and the second quantization coding matrix.

[0060] In the specific implementation, the first similarity matrix S is obtained.avg The second similarity matrix S max Then, the statistical feature enhancement module respectively applied the first similarity matrix S avg The second similarity matrix S max Multi-level quantization is performed to obtain the first quantization encoding matrix V. avg Second quantization encoding matrix V max .

[0061] In one example, the statistical feature enhancement module applies the first similarity matrix S. avg The multi-level quantization performed can be expressed as For the second similarity matrix S max The multi-level quantization performed can be expressed as N is the total number of orders of magnitude in the multi-level quantization, Q avg and Q max The calculation process is as follows:

[0062]

[0063]

[0064] In the formula, This indicates that for the first similarity matrix S avg Perform quantization on the nth order of magnitude. This indicates that the second similarity matrix S max For quantization of the nth order of magnitude, max(·) represents the maximum value function and min(·) represents the minimum value function.

[0065] In one example, the first quantization coding matrix V avg Second quantization encoding matrix V max It can be calculated using the following formula:

[0066]

[0067]

[0068] m∈[1,H×W]

[0069] In the formula, m represents the cosine similarity index, and H×W represents the first similarity matrix S. avg The size of the second similarity matrix S max Size and first similarity matrix S avg The same size This represents the m-th item in the first similarity matrix. The m-th item of the second similarity matrix This represents the m-th item in the first quantization encoding matrix obtained by quantization at the n-th order of magnitude. This represents the m-th item in the second quantization encoding matrix obtained by quantization at the n-th order of magnitude.

[0070] Step 1014: Count and self-attention reconstruction are performed on the first quantization coding matrix and the second quantization coding matrix respectively to obtain the global mean texture feature and the global maximum texture feature.

[0071] In the specific implementation, each quantization level represents a type of texture information. By counting the quantization encoding matrix, the statistical features of the texture information can be obtained. The statistical feature enhancement module then applies the first quantization encoding matrix V to... avg Second quantization encoding matrix V max By performing counting and self-attention reconstruction, the global mean texture feature is obtained. and global maximum texture features

[0072] In one example, the first quantization encoding matrix V avg Second quantization encoding matrix V max Counting and self-attention reconstruction are performed to obtain the global mean texture features. and global maximum texture features This can be achieved using the following formula:

[0073]

[0074]

[0075]

[0076]

[0077] In the formula, Cat(·) represents a concatenation operation, MLP(·) represents a multilayer perceptron operation, W1(·), W2(·), and W3(·) all represent 1×1 convolution operations, and S(·) represents the Softmax function. The self-attention mechanism obtains new-level texture information by perceiving statistical information of all original levels, thereby enhancing statistical features and improving the ability to represent texture details.

[0078] Step 1015: Element-wise summation of the global mean texture feature and the global maximum texture feature to obtain the statistical texture feature.

[0079] In the specific implementation, the global mean texture feature is obtained. and global maximum texture features After that, you can... and Element-wise summation yields the statistical texture feature F. sta ,Right now The features output by the first two convolutional layers are shallow features of the hyperspectral image to be classified, but these shallow features are still not obvious. The shallow features are input into the statistical feature enhancement module, and after pooling, similarity calculation, multi-level quantization, counting statistics and self-attention reconstruction, statistical texture features are obtained, which effectively enhances the feature expression of texture information, thereby enhancing the classification model's ability to discriminate statistical texture features.

[0080] In one example, the architecture and working principle of the statistical feature enhancement module can be as follows: Figure 4 As shown.

[0081] Step 102: Based on the features output by the third convolutional layer, multi-scale semantic features are obtained using the multi-scale feature extraction module of the classification model.

[0082] In the specific implementation, the channel attention module, spatial attention module, and fourth convolutional layer after the third convolutional layer can be regarded as part of the multi-scale feature extraction module. That is, the multi-scale feature extraction module includes a channel attention module, a spatial attention module, a 3×3 convolutional layer, and a dilated spatial convolutional pooling pyramid (ASPP). The classification model downsamples the features F3 output from the third convolutional layer, passes them through the channel attention module, spatial attention module, and 3×3 convolutional layer, and inputs them into the ASPP for multi-scale feature extraction. The size of the features output by the ASPP is restored using bilinear interpolation to obtain multi-scale semantic features. The ASPP is composed of dilated convolutions, which can balance a large receptive field and high resolution of feature maps, and can effectively extract multi-scale semantic features from the hyperspectral image to be classified.

[0083] Step 103: The statistical texture features and multi-scale semantic features are fused using the statistical and semantic feature fusion module of the classification model to obtain the spatial-spectral joint features.

[0084] In the specific implementation, the statistical texture feature F output by the statistical feature enhancement module sta The multi-scale semantic features F output by the multi-scale feature extraction module sem The inputs are fed into the statistical and semantic feature fusion module (also known as SSFFM), where SSFFM adaptively weights and fuses F... sta and F sem Finally, the joint spatial-spectral feature F is obtained. spe-spa .

[0085] In one example, the statistical and semantic feature fusion module of the classification model is used to fuse statistical texture features and multi-scale semantic features to obtain spatial-spectral joint features, which can be obtained through methods such as... Figure 5 The steps shown are implemented as follows:

[0086] Step 1031: Perform average pooling and max pooling operations on the statistical texture features respectively and concatenate them to obtain the processed statistical texture features.

[0087] In the specific implementation, the statistical and semantic feature fusion module performs statistical texture feature F sta Perform average pooling and max pooling operations separately, and then concatenate the results of the two pooling operations to obtain the processed statistical texture feature F′. sta The calculation process can be expressed as follows:

[0088] F' sta =Conv{Cat[AvgPool(F sta MaxPool(F) sta )]}

[0089] In the formula, AvgPool(·) represents the average pooling operation, MaxPool(·) represents the max pooling operation, and Conv(·) represents the 3×3 convolution operation.

[0090] Step 1032: Perform average pooling and max pooling operations on the multi-scale semantic features respectively and concatenate them to obtain the processed multi-scale semantic features.

[0091] In the specific implementation, the statistical and semantic feature fusion module performs multi-scale semantic feature F sem Perform average pooling and max pooling operations separately, and concatenate the results of the two pooling operations to obtain the processed statistical texture features F'. sem Its calculation process can be expressed as: F' sem =Conv{Cat[AvgPool(F sem MaxPool(F) sem )]}.

[0092] Step 1033: The processed statistical texture features and the processed multi-scale semantic features are added element-wise, and the fusion weights are obtained by activation based on the Sigmoid function through a convolutional layer.

[0093] In the specific implementation, the processed statistical texture features F' sta and the processed multi-scale semantic features F' sem Element-wise addition is performed, and the fusion weights are obtained by passing through a convolutional layer and activating the Sigma function. This can be achieved using the following formula:

[0094]

[0095] In the formula, Sig(·) represents the Sigmod activation function, and λ is the fusion weight.

[0096] Step 1034: Adaptively fuse statistical texture features and multi-scale semantic features based on fusion weights to obtain spatial-spectral joint features.

[0097] In practice, statistical texture features effectively characterize the shallow visual cues of the hyperspectral image to be classified, while multi-scale semantic features well characterize the high-dimensional semantic cues. Adaptive fusion of the two further enhances the discriminative ability of the classification model, thereby improving the accuracy of land cover classification of hyperspectral images.

[0098] In the specific implementation, the statistical texture features F are based on the fusion weight λ. sta and multi-scale semantic features F sem Adaptive fusion is performed to obtain the joint spatial-spectral feature F. spe-spa This can be achieved using the following formula:

[0099]

[0100] In the formula, This indicates element-wise multiplication.

[0101] In one example, the architecture and working principle of the statistical and semantic feature fusion module can be as follows: Figure 6 As shown.

[0102] Step 104: Based on the joint spatial-spectral features, use the classifier of the classification model to classify land cover and obtain the land cover category corresponding to the hyperspectral image to be classified.

[0103] In the specific implementation, the statistical and semantic feature fusion module outputs the joint spatial-spectral features F. spe-spa The features are first extracted through two 3×3 convolutional layers and then input into the SSFLNet Softmax classifier to obtain the land cover category corresponding to the hyperspectral image to be classified.

[0104] In this embodiment, the classification model includes a statistical feature enhancement module and a multi-scale feature extraction module. This allows for the simultaneous capture of statistical texture features and high-level semantic features (multi-scale semantic features) from hyperspectral images. Statistical texture features effectively characterize the shallow features of hyperspectral images, significantly improving the completeness of feature modeling. The statistical and semantic feature fusion module adaptively fuses statistical texture features and multi-scale semantic features, enhancing the discriminative ability of the extracted features. Based on this classification model, the accuracy of hyperspectral image land cover classification can be effectively overcome by addressing factors such as climate conditions, sensor type and shooting angle, uneven distribution of ground features in hyperspectral images, and issues like "different objects with the same spectrum" and "different spectra with the same object." This significantly improves the accuracy of hyperspectral image land cover classification, providing effective and accurate data support for typical on-site security tasks requiring precise land cover category information, such as low-altitude UAV security, disaster early warning and rescue, and target reconnaissance.

[0105] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0106] In another embodiment, to verify the effectiveness of SSFLNet, classification experiments were conducted on three publicly available hyperspectral image datasets: Indian Pines, Pavia University, and Salinas. To alleviate the problem of redundant computation caused by neighborhood image patch sampling, the hyperspectral images were directly fed into the model for training, and finally tested on the test set to obtain the final classification results. The specific experiments are as follows:

[0107] (1) Hyperspectral image dataset

[0108] The Indian Pines dataset was collected by the AVIRIS airborne visible / infrared imaging spectrometer in a farmland area in northern Indiana. The dataset has dimensions of 145×145×200, with a spatial dimension of 145×145 and 200 spectral bands. It contains 16 different land cover categories, including various materials found in real-world scenes such as coniferous forests and farmland, including corn, soybeans, and pine trees. 5% of the samples from each category were randomly selected as the training set, and the remaining 95% were used as the test set.

[0109] The Pavia University dataset consists of hyperspectral urban remote sensing images acquired by the ROSIS-3HS imaging spectrometer sensor near the University of Pavia, Italy. The dataset has dimensions of 610×340×103, with a spatial dimension of 610×340 and 103 spectral bands. The dataset contains 9 land cover categories, mainly including urban land cover categories such as asphalt, grass, and gravel. 30 samples are randomly selected from each category as the training set, and the remaining samples are used as the test set.

[0110] The Salinas dataset is a rural hyperspectral remote sensing dataset captured by the AVIRIS airborne visible / infrared imaging spectrometer sensor in the Salinas Valley, California. The dataset has dimensions of 512×217×204, with a spatial dimension of 512×217 and 204 spectral bands. The dataset contains 16 land cover categories, including crop land cover categories such as corn, soybeans, cucumbers, and tomatoes. 30 samples were randomly selected from each category as the training set, and the remaining samples were used as the test set.

[0111] (2) Experimental setup

[0112] In the experiment, the training optimizer was set to stochastic gradient descent (SGD) with momentum, with a momentum of 0.9. The initial learning rate was set to 0.01, and the learning rate was updated using a multinomial adjustment strategy with each iteration. The update rule was: rate × {1 - [iter / (max_iter)]} 0.9 `iter` is the current iteration number, `max_iter` is the maximum iteration number, and SSFLNet is trained for a total of 1000 epochs with a batch size of 1.

[0113] (3) Evaluation indicators

[0114] Overall Accuracy (OA) is the ratio of the number of correctly classified pixels to the total number of sample pixels.

[0115] Average Accuracy (AA) is the average classification accuracy, representing the average classification accuracy across different categories.

[0116] The Kappa coefficient is used to evaluate the consistency between the classification results and the true ground values.

[0117] (4) Comparative Experiment

[0118] Experiments were conducted to compare the land cover classification methods on 10 representative hyperspectral images from the three publicly available hyperspectral image datasets mentioned above with SSFLNet. The experimental results on the Indian Pines dataset are shown in Table 1, the results on the Pavia University dataset are shown in Table 2, and the results on the Salinas dataset are shown in Table 3.

[0119] Table 1: Results of comparative experiments using 10 representative methods and SSFLNet on the Indian Pines dataset.

[0120] OA 94.96 96.20 93.31 93.89 96.28 94.77 AA 89.74 78.24 85.69 83.02 90.51 93.57 Kappa 94.25 95.67 92.36 93.02 95.75 94.03 Method ENL-FCN SSFCN FContNet-PSP FContNet-ASPP SSFLNet OA 96.01 69.29 94.96 95.66 96.59 AA 89.37 66.04 89.77 91.28 93.21 Kappa 95.46 65.05 94.26 95.05 96.11

[0121] Table 2: Results of comparative experiments using 10 representative methods and SSFLNet on the Pavia University dataset.

[0122]

[0123]

[0124] Table 3: Results of comparative experiments using 10 representative methods with SSFLNet on the Salinas dataset.

[0125] OA 94.95 94.76 95.98 97.15 96.42 93.06 AA 97.76 97.62 97.68 98.80 98.47 95.79 Kappa 94.39 94.17 95.53 96.83 96.01 92.29 Method ENL-FCN SSFCN FContNet-PSP FContNet-ASPP SSFLNet OA 95.76 87.08 97.21 97.49 98.90 AA 98.25 69.55 98.65 98.80 98.96 Kappa 95.28 85.65 96.90 97.30 98.78

[0126] As can be seen from Tables 1 to 3, SSFLNet has higher classification accuracy.

[0127] In another embodiment, based on the Indian Pines, Pavia University, and Salinas datasets, the impact of different training sample rates on classification results was also investigated (e.g., Figure 7 As shown), the impact of different quantization levels on classification results (e.g.) Figure 8 As shown), and the impact of different loss functions on classification results (e.g. Figure 9 (As shown). When the training sample rate is 50% and the quantization level is 64, SSFLNet achieves good classification accuracy. A mixed loss function Li is used. C +L O The effect is obvious because only the cross-entropy loss function L is used. C .

[0128] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0129] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for classifying ground features based on statistical and semantic features for perimeter security, characterized in that, include: The acquired hyperspectral image to be classified is input into a pre-trained classification model. The features output by the first three convolutional layers of the classification model are obtained. Based on the features output by the first two convolutional layers, the statistical feature enhancement module of the classification model is used to perform statistics to obtain statistical texture features. Based on the features output by the third convolutional layer, multi-scale semantic features are obtained using the multi-scale feature extraction module of the classification model. The statistical texture features and the multi-scale semantic features are fused using the statistical and semantic feature fusion module of the classification model to obtain the spatial-spectral joint features; Based on the spatial-spectral joint features, the classifier of the classification model is used to classify land features and obtain the land feature category corresponding to the hyperspectral image to be classified. The features output from the first two convolutional layers are statistically analyzed using the statistical feature enhancement module of the classification model to obtain statistical texture features, including: The features output by the first two convolutional layers are added element by element to obtain shallow features; Average pooling and max pooling operations are performed on the shallow features to obtain global mean features and global maximum features. The cosine similarity between the shallow features and the global mean features and the global maximum features is calculated to obtain a first similarity matrix and a second similarity matrix. The first similarity matrix and the second similarity matrix are quantized at multiple levels to obtain the first quantization coding matrix and the second quantization coding matrix; The first quantization coding matrix and the second quantization coding matrix are counted and reconstructed by self-attention respectively to obtain the global mean texture feature and the global maximum texture feature; The statistical texture features are obtained by adding the global mean texture features and the global maximum texture features element by element.

2. The method for classifying ground features based on statistical and semantic features for perimeter security according to claim 1, characterized in that, The first similarity matrix and the second similarity matrix are quantized at multiple levels using the following formulas to obtain the first quantization encoding matrix and the second quantization encoding matrix: ； ； ； ； ； in, The total number of orders of magnitude of the multi-level quantization. Represents the cosine similarity index. This represents the size of the first similarity matrix, and the size of the second similarity matrix is ​​the same as the size of the first similarity matrix. For the first similarity matrix, This is the second similarity matrix. The first similarity matrix represents the first similarity matrix. item, The second similarity matrix item, This indicates that the first similarity matrix is ​​subjected to the [further processing / processing]. Quantification of magnitude This indicates that the second similarity matrix is ​​subjected to the first... Quantification of magnitude Indicates the first The first quantization encoding matrix obtained by quantization of the order of magnitude is the first... item, Indicates the first The second quantization encoding matrix obtained by quantization of the order of magnitude is the first... item, This represents the function that takes the maximum value. This represents the function that takes the minimum value.

3. The method for classifying ground features based on statistical and semantic features for perimeter security according to claim 2, characterized in that, The global mean texture feature and the global maximum texture feature are obtained by counting and self-attention reconstruction of the first quantization encoding matrix and the second quantization encoding matrix respectively using the following formulas. feature: ； ； ； ； in, The global mean feature is... The global maximum value feature is... Indicates a serial operation. This indicates the operation of a multilayer sensor. This is the first quantization encoding matrix. This is the second quantization encoding matrix. , and All indicate Convolution operation, This represents the Softmax function. Represents global mean texture features. This represents the global maximum texture feature.

4. The method for classifying ground features based on statistical and semantic features for perimeter security according to claim 1, characterized in that, The statistical and semantic feature fusion module of the classification model fuses the statistical texture features and the multi-scale semantic features to obtain spatial-spectral joint features, including: The statistical texture features are subjected to average pooling and max pooling operations respectively and then concatenated to obtain the processed statistical texture features. The multi-scale semantic features are subjected to average pooling and max pooling operations respectively and then concatenated to obtain the processed multi-scale semantic features. The processed statistical texture features and the processed multi-scale semantic features are added element-wise, and the fusion weights are obtained by passing through a convolutional layer and activating based on the Sigmoid function. Based on the fusion weights, the statistical texture features and the multi-scale semantic features are adaptively fused to obtain the spatial-spectral joint features.

5. A method for classifying ground features based on statistical and semantic features for perimeter security, as described in claim 4, is characterized in that... The processed statistical texture features and the processed multi-scale semantic features are added element-wise, and the fusion weights are obtained by passing them through a convolutional layer and activating them based on the Sigmoid function, as shown in the following formula: ； in, The processed statistical texture features, The processed multi-scale semantic features, This indicates the element-by-element addition. express Convolution operation, This represents the Sigmod activation function. The fusion weight is denoted as .

6. A method for classifying ground features based on statistical and semantic features for perimeter security, as described in claim 5, is characterized in that... The adaptive fusion of the statistical texture features and the multi-scale semantic features based on the fusion weights to obtain the spatial-spectral joint features is achieved through the following formula: ； in, For the statistical texture features, For the aforementioned multi-scale semantic features, This indicates element-wise multiplication. Indicates a serial operation. The spatial-spectral joint feature is described above.

7. A method for classifying ground features based on statistical and semantic features for perimeter security according to any one of claims 1 to 6, characterized in that, The multi-scale feature extraction module includes a channel attention module, a spatial attention module, a 3×3 convolutional layer, and a dilated spatial convolutional pooling pyramid. Based on the features output from the third convolutional layer, the multi-scale semantic features obtained using the classification model's multi-scale feature extraction module are: After downsampling the features output by the third convolutional layer, the features are passed through the channel attention module, the spatial attention module, and the 3×3 convolutional layer, and then input into the dilated spatial convolutional pooling pyramid for multi-scale feature extraction. The size of the features output by the dilated spatial convolutional pooling pyramid is restored by bilinear interpolation to obtain multi-scale semantic features.

8. A method for classifying ground features based on statistical and semantic features for perimeter security according to any one of claims 1 to 6, characterized in that, When training the classification model, the loss function used includes a cross-entropy loss term and an online hard sample mining loss term, which is expressed as: ； in, For the cross-entropy loss term, For the online hard sample mining loss term, The preset balance factor, This represents the loss function.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a method for classifying ground features based on statistical and semantic features for local security as described in any one of claims 1 to 8.

Images