Hyperspectral image superpixel spatial-spectral fusion identification method and system, and medium

Abstract

The invention discloses a hyperspectral image super-pixel spatial-spectral fusion recognition method and system and a medium, and the method comprises the following steps: S101, extracting a super-pixel feature map of a to-be-recognized hyperspectral image, and carrying out the classification recognition of the super-pixel feature map, extracting a spatial-spectral feature map from the hyperspectral image to be recognized, and carrying out classification recognition on the spatial-spectral feature map; and S102, fusing the two obtained classification identification results to obtain a final surface feature category identification result. The invention aims to realize super-pixel and spatial-spectral fusion recognition of the hyperspectral image, so as to solve the problem of poor recognition effect caused by noise and scarce sample quantity in the hyperspectral image, and improve the precision of surface feature category recognition of the hyperspectral image.

Description

Technical Field

[0001] The present invention relates to a hyperspectral image ground object recognition technology in the field of image processing, and in particular to a hyperspectral image superpixel spatial-spectral fusion recognition method, system and medium. Background Art

[0002] Hyperspectral remote sensing is a multi-dimensional information acquisition technology that combines imaging technology and spectral technology. It can simultaneously detect the two-dimensional spatial geometry and one-dimensional spectral information of the target and obtain continuous, narrow-band image data with high spectral resolution. Hyperspectral remote sensing data has the characteristics of "image-spectrum integration". It combines the spectrum that determines the properties of the material with the image containing spatial geometric information, which can provide richer information on the earth's surface and is widely used in land object recognition tasks.

[0003] The classifier structure used in traditional hyperspectral image object recognition methods is usually relatively simple, and the single model structure limits the classification accuracy to a certain extent. In addition, the recognition performance of these traditional methods is highly dependent on the quantity and quality of training data. However, in practical applications, hyperspectral images are interfered by various factors during the imaging process, such as atmospheric scattering, cloud occlusion, complex illumination changes, etc., which will reduce the signal-to-noise ratio of hyperspectral images, affect the accuracy and robustness of traditional recognition methods, and significantly reduce recognition performance. In addition, due to the high cost of manually annotated samples, hyperspectral object recognition often faces the problem of small samples, that is, the number of samples that can be used for classifier training is limited. Under the condition of small samples, the classification model cannot fully learn the characteristics of the data, which further leads to a decrease in recognition accuracy. Summary of the invention

[0004] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method, system and medium for superpixel spatial-spectral fusion recognition of hyperspectral images are provided. The present invention aims to realize superpixel and spatial-spectral fusion recognition of hyperspectral images to solve the problem of poor recognition effect caused by noise and scarce sample number in hyperspectral images, and to improve the accuracy of hyperspectral image ground object category recognition.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A hyperspectral image superpixel spatial-spectral fusion recognition method comprises the following steps: S101, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps, and extracting spatial spectrum feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps; S102, fusing the two classification recognition results to obtain a final ground object category recognition result.

[0006] Optionally, the function expression for fusing the two classification recognition results obtained in step S102 to obtain the final ground object category recognition result is: , In the above formula, is the final category label, is the probability of the i-th category obtained by classifying and identifying the superpixel feature map, is the probability of the ith category obtained by classifying and identifying the spatial spectrum feature map, is a free parameter.

[0007] Optionally, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying them in step S101 includes: S201, performing principal component analysis (PCA) on the hyperspectral image to be identified to extract multiple leading principal components; S202, performing super-pixel segmentation on the plurality of principal components to obtain a super-pixel image; S203, extracting a superpixel feature map according to the superpixel images of the plurality of principal components; S204, using a classifier to classify the superpixel feature map to obtain the ground object category probability of each superpixel; S205 , using an extended random walk algorithm to refine the ground object category probability of each superpixel to obtain a category probability map C1 as a result of classification and recognition of the superpixel feature map.

[0008] Optionally, step S205 includes: S301, for each type of feature , according to the ground object category in the superpixel feature map The ground object category probability matrix of the pixels is constructed to obtain the ground object category The probability matrix ; S302, using the extended random walk algorithm to classify the ground objects based on the following objective function The probability matrix To optimize: , In the above formula, for The optimization result of The pixel belongs to the ground feature category The probability matrix of for The spatial term, is the balance factor, for The non-spatial terms of , and there are: , , In the above formula, is the Laplacian matrix of the weight graph, The pixel belongs to the ground feature category The probability matrix of For feature category The diagonal matrix of the diagonal matrix consists of the preset initial probabilities. is the transpose operation; the Laplacian matrix of the weight graph The calculation function expression is: , In the above formula, is the first element in the Laplacian matrix of the weight graph. Line List, is a constant parameter, is the first principal component in the superpixel image superpixels, is the first principal component in the superpixel image superpixels, and The adjacent represents the first principal component in the superpixel image superpixels and The distance between superpixels is less than the set value.

[0009] Optionally, in step S203, the function expression for extracting a superpixel feature map according to the superpixel images of multiple principal components is: , In the above formula, is the first superpixel feature map The pixel value of a superpixel, For the The number of pixels in a superpixel, The first The pixel value of a pixel.

[0010] Optionally, the classifier used in step S204 is a support vector machine SVM, the input of the support vector machine SVM is the superpixel of the superpixel feature map, and the output is the category label in the category probability map C1. The function expression of the support vector machine SVM is: , , In the above formula, Indicates to the maximum value, is a penalty parameter greater than zero, and is the Lagrange multiplier, is the first superpixel feature map Superpixels The corresponding category label, , Represents the first Superpixels The superpixel feature map Superpixels The feature vector after mapping to a higher dimensional space, is the number of superpixels.

[0011] Optionally, extracting a spatial spectrum feature graph from the hyperspectral image to be identified and classifying and identifying it in step S101 includes: S401, using a preset iterative algorithm to iteratively solve the hyperspectral image using the total variation-based structural feature extraction model shown in the following formula to obtain a spatial spectrum feature map of the hyperspectral image: , In the above formula, For a number not greater than polynomial space, where Pixel The polynomial terms of The number of polynomial terms in , Represents pixel is a two-dimensional real number space An element in is a vector of non-negative integers, For vector The value range and is a positive integer, For vector The absolute value and of all elements in refers to the degree of the local polynomial, is the i-th pixel The polynomial of is the i-th pixel, For hyperspectral images I From the i-th pixel The pixel value at the determined position, For pixels and the i-th pixel The weight function, weight function Used to determine the construction polynomial When each i-th pixel The contribution of is a free parameter, For polynomial exist The gradient of S402, reducing the dimension of the spatial spectrum feature map of the hyperspectral image; S403, using a classifier to classify the space-spectrum feature map after dimensionality reduction to obtain a category probability map C2 as a result of classification and recognition of the space-spectrum feature map.

[0012] Optionally, the weight function in step S401 The calculation function expression is: , In the above formula, For comparing pixels and the i-th pixel The local area of ​​the image block, is the offset value, For hyperspectral images I At the i-th pixel and offset value The pixel value at the determined position, It refers to the hyperspectral image I composed of pixels and offset value The pixel value at the determined position, The standard deviation is Gaussian function, is the scale parameter set to 1.

[0013] Optionally, in step S401, when the hyperspectral image is iteratively solved by using the total variation-based structural feature extraction model shown in the following formula to obtain the spatial spectrum feature map of the hyperspectral image, each iterative solution process includes: Solve based on the following formula : , Solve based on the following formula : , Solve based on the following formula : , in, is the i-th pixel at the k+1th iteration The polynomial of , is the auxiliary variable for solving the kth and k+1th iterations of the L1-regularization problem. 、 It is an auxiliary variable for the kth and k+1th iterations when dealing with constrained optimization problems. For the k+1th iteration, the polynomial is The gradient of is a soft threshold operation, is a free parameter.

[0014] In addition, the present invention also provides a hyperspectral image superpixel spatial-spectral fusion recognition system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the hyperspectral image superpixel spatial-spectral fusion recognition method.

[0015] In addition, the present invention also provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used to be programmed or configured by a microprocessor to execute the hyperspectral image superpixel spatial-spectral fusion recognition method.

[0016] Compared with the prior art, the present invention mainly has the following advantages: superpixel segmentation can divide an image into multiple adjacent pixel sets with similar features such as color, brightness, and texture. Existing research results show that the spatial consistency of pixels within superpixels is high. When performing object recognition, processing superpixels can constrain the consistency of the recognition results within superpixels and effectively reduce the influence of noise. In addition, superpixels can provide spatial distribution constraints for similar pixels, providing a basis for high-confidence sample expansion. Therefore, superpixels can be used to solve the noise interference and small sample problems faced in the task of hyperspectral image object recognition. The present invention includes extracting superpixel feature maps from the hyperspectral image to be recognized for classification and recognition, and extracting spatial spectrum feature maps from the hyperspectral image to be recognized for classification and recognition, fusing the obtained two classification and recognition results to obtain the final object category recognition result, and using a dual-branch structure to estimate the probability of the object category of the hyperspectral image, and extracting different spatial spectrum features through the dual-branch structure, which can solve the problem of poor recognition effect caused by noise and scarcity of samples in the hyperspectral image, and improve the accuracy of hyperspectral image object category recognition. Compared with the single-branch structure, the results obtained in different situations are more robust and more versatile. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0018] Figure 2 Detailed flowchart of the method of the embodiment of the present invention.

[0019] Figure 3 The figure compares the experimental results of the method of the embodiment of the present invention and the existing method. DETAILED DESCRIPTION

[0020] In order to solve the problem of poor recognition effect caused by noise and scarce sample number in hyperspectral images, this paper proposes a novel superpixel spatial-spectral fusion recognition technology. Figure 1 As shown, the hyperspectral image superpixel spatial-spectral fusion recognition method of this embodiment includes the following steps: S101, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps, and extracting spatial spectrum feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps; S102, the two classification and recognition results are fused to obtain a final ground object category recognition result, and the function expression for fusion of the two classification and recognition results to obtain the final ground object category recognition result is: , In the above formula, is the final category label, is the probability of the i-th category obtained by classifying and identifying the superpixel feature map, is the probability of the ith category obtained by classifying and identifying the spatial spectrum feature map, The hyperspectral image superpixel spatial-spectral fusion recognition method of this embodiment uses a dual-branch structure of extracting superpixel feature maps for classification and recognition and extracting spatial-spectral feature maps for classification and recognition to respectively identify the ground objects in the input hyperspectral image and estimate the category probability. After both branches obtain the ground object category probability, the ground object category probability is integrated to obtain the final recognition result.

[0021] like Figure 2 As shown, in step S101 of this embodiment, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying them includes: S201, performing principal component analysis PCA (Principal Component Analysis) on the hyperspectral image to be identified to extract multiple leading principal components; S202, performing super-pixel segmentation on the plurality of principal components to obtain a super-pixel image; S203, extracting a superpixel feature map according to the superpixel images of the plurality of principal components; S204, using a classifier to classify the superpixel feature map to obtain the ground object category probability of each superpixel; S205 , using an extended random walk algorithm to refine the ground object category probability of each superpixel to obtain a category probability map C1 as a result of classification and recognition of the superpixel feature map.

[0022] In the branch one structure, this embodiment first uses principal component analysis (PCA) to reduce the dimension of the hyperspectral image containing simulated noise, selects the first three principal components and performs superpixel segmentation on them, and then averages the pixel values ​​in the superpixels to obtain the superpixel feature image of the hyperspectral image, and then uses support vector machine (SVM) to identify the superpixel feature image to obtain the ground object category probability of each superpixel. On this basis, the extended random walk (ERW) algorithm is used to refine the ground object category probability map to obtain the category probability result C1.

[0023] Step S201 uses principal component analysis (PCA) to reduce the dimension of the hyperspectral input image containing simulated noise. This step can effectively reduce the redundancy in the spectral information and improve the execution efficiency of the method of this embodiment. The hyperspectral image containing noise is represented as a series of The combination of dimensional vectors is analyzed by principal component analysis PCA. The principal component analysis (PCA) sorts the importance of the obtained "principal element" vectors, retains the most important information in the front and omits the dimensions behind, thus retaining the important information of the hyperspectral image to the greatest extent.

[0024] In the present embodiment, step S202 is specifically to process the first three principal components using a superpixel segmentation algorithm (Entropy Rate Superpixel, ERS) based on entropy rate. Superpixels group pixels using the similarity of features between pixels, which can reduce the complexity of subsequent image processing tasks, and the superpixel segmentation algorithm based on entropy rate can control the number of segmented blocks through an objective function consisting of an entropy rate function and a balance function, while being able to well retain the edge of the image, and the method has a high accuracy rate and low computational complexity. The method of the present embodiment introduces a superpixel structure as a guide when processing hyperspectral images, and performs sample expansion based on the superpixel segmentation result. In addition, in one of the branches, a superpixel spatial spectrum feature image is extracted for object recognition, which effectively reduces image complexity while retaining key image features, especially edge information. This method has achieved a good balance in reducing the computational burden of subsequent image processing tasks and maintaining image quality. In performing advanced image processing and analysis, especially in the case of limited computing resources, this method has shown its unique effectiveness and practicality. This embodiment uses an entropy rate-based superpixel segmentation algorithm (Entropy RateSuperpixel, ERS) that can be well adapted to the processing of high-resolution images, while ensuring high recognition accuracy and effectively reducing the resource consumption of ultra-high resolution images such as hyperspectral images.

[0025] In step S203, the superpixel feature map is extracted according to the superpixel images of the multiple principal components. A feasible implementation method can be adopted as needed. For example, as an optional implementation method, in step S203 of this embodiment, after obtaining the superpixel image, the first The average value of the pixels in the superpixel is calculated, and then the pixels in each superpixel are assigned the same average value, that is, , the re-assigned super-pixel image is used as the super-pixel feature image, and the function expression for extracting the super-pixel feature map according to the super-pixel images of multiple principal components in step S203 is: , In the above formula, is the first superpixel feature map The pixel value of a superpixel, For the The number of pixels in a superpixel, The first In order to extract the feature image of a superpixel, the corresponding pixel average value in each superpixel is first calculated, and then all pixels in each superpixel are assigned the corresponding pixel average value to obtain a superpixel feature image. The advantage of this superpixel feature image is that it can reduce the spectral difference between pixels in the same superpixel and reduce the computational complexity.

[0026] After obtaining the superpixel feature image, thanks to the great advantages of SVM in solving high-dimensional small samples and nonlinear problems, the classifier can use the support vector machine SVM to identify the ground objects and obtain the class probability results of the ground objects. The classifier used in step S204 of this embodiment is a support vector machine SVM, the input of which is the superpixel of the superpixel feature map, and the output is the class label in the class probability map C1, specifically, the superpixel of the superpixel feature map is used as the sample point enter, , For the corresponding category label, the function expression of the support vector machine SVM is: , , In the above formula, Indicates to the maximum value, is a penalty parameter greater than zero, and is the Lagrange multiplier, is the first superpixel feature map Superpixels The corresponding category label, , Represents the first Superpixels The superpixel feature map Superpixels The feature vector after mapping to a higher dimensional space, is the number of superpixels. It should be noted that, in addition to the support vector machine, other classifiers can also be used as needed.

[0027] After obtaining the object category probability map after SVM classification, in order to solve the problem that some small object information will be lost in the feature extraction stage, this embodiment uses a post-processing spatial probability optimization method based on extended random walk to optimize the posterior probability of the obtained category label, and obtains the category probability optimization result C1 by modeling the spatial correlation of adjacent superpixels. By modeling the spatial correlation of adjacent superpixels, small-scale targets are highlighted and the over-smoothing phenomenon caused by feature extraction is avoided. Specifically, step S205 of this embodiment includes: S301, for each type of feature , according to the ground object category in the superpixel feature map The ground object category probability matrix of the pixels is constructed to obtain the ground object category The probability matrix ; S302, using the extended random walk algorithm to classify the ground objects based on the following objective function The probability matrix To optimize: , In the above formula, for The optimization result of The pixel belongs to the ground feature category The probability matrix of for The spatial term, is the balance factor, for The above formula is composed of spatial terms and non-spatial terms. Used to model the spatial correlation between adjacent superpixels; and: , , In the above formula, is the Laplacian matrix of the weight graph, The pixel belongs to the ground feature category The probability matrix of For feature category The diagonal matrix of the diagonal matrix consists of the preset initial probabilities. is a transpose operation; and the calculation function expression of the Laplacian matrix of the weight graph is: , In the above formula, is the first element in the Laplacian matrix of the weight graph. Line List, is a constant parameter, is the first principal component in the superpixel image superpixels, is the first principal component in the superpixel image superpixels, and The adjacent represents the first principal component in the superpixel image superpixels and The distance between superpixels is less than the set value. After obtaining the superpixel feature image, take the first principal component to construct the weight map. represents the first principal component superpixels. Non-spatial terms Make full use of the initial probability (which can be set as needed). Finally, take The maximum value of gets the final category probability optimization result C1.

[0028] The second branch of the method of this embodiment uses a total variation-based structural feature extraction model to extract structural features from noisy hyperspectral input images. It has a strong texture removal capability, and the extracted structural features are less affected by noise. It is suitable for hyperspectral images of various scenes, and aims to reduce the influence of unimportant information such as noise in the image, and improve the robustness of the algorithm to noise. Figure 2 As shown, in step S101 of this embodiment, extracting the spatial spectrum feature graph of the hyperspectral image to be identified and classifying and identifying it includes: S401, using a preset iterative algorithm to iteratively solve the hyperspectral image using the total variation-based structural feature extraction model shown in the following formula to obtain a spatial spectrum feature map of the hyperspectral image: , In the above formula, For a number not greater than polynomial space, where Pixel The polynomial terms of The number of polynomial terms in , Represents pixel is a two-dimensional real number space An element in is a vector of non-negative integers, For vector The value range and is a positive integer, For vector The absolute value and of all elements in refers to the degree of the local polynomial, is the i-th pixel The polynomial of is the i-th pixel, For hyperspectral images I From the i-th pixel The pixel value at the determined position, For pixels and the i-th pixel The weight function, weight function Used to determine the construction polynomial When each i-th pixel The contribution of is a free parameter, For polynomial exist The gradient of S402, reducing the dimension of the spatial spectrum feature map of the hyperspectral image. In order to further increase the separability of different objects in the hyperspectral image, in this embodiment, kernel principal component analysis KPCA is used to reduce the dimension of the obtained structural features to obtain structural components with stronger separability. In addition, other dimensionality reduction methods can also be used as needed; S403, using a classifier to classify the reduced spatial spectrum feature map to obtain a category probability map C2 as a result of classification and identification of the spatial spectrum feature map. For example, the classifier in step S403 in this embodiment specifically uses a support vector machine SVM, takes the feature image processed by KPCA as input, uses the support vector machine SVM as a classifier to identify the ground object, and obtains a category probability map C2 of ground object identification.

[0029] The structural feature extraction model based on total variation can be expressed as: , In the above formula, represents a hyperspectral image, Represents structural components, weights Used to control the similarity of adjacent pixels. is a free parameter. In order to smooth the texture components, a Local polynomial of , denoted as .in yes The number of polynomial terms in . For each pixel In terms of, assuming yes exist A finite set of pixels in which yes The total number of pixels. Structural components Can be described as , .Then The high-order approximation of can be transformed into the following optimization problem: , in, For a number not greater than The polynomial space of Determines the polynomial used to construct The weight function in step S401 of this embodiment The calculation function expression is: , In the above formula, For comparing pixels and the i-th pixel The local area of ​​the image block, is the offset value, For hyperspectral images I At the i-th pixel and offset value The pixel value at the determined position, It refers to the hyperspectral image I composed of pixels and offset value The pixel value at the determined position, The standard deviation is Gaussian function, is a scale parameter set to 1. On this basis, a model for iteratively solving the spatial spectrum feature map of the hyperspectral image based on the total variation structural feature extraction model can be obtained: , Then, the desired iterative algorithm can be used to solve the spatial spectrum feature map of the hyperspectral image. As an optional implementation, the present embodiment uses the Split Bregman iterative algorithm (see: Goldstein T, Osher S. The Split Bregman Method for L1-regularized Problems[J]. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343.) to iteratively solve and obtain the spatial spectrum feature map of the hyperspectral image.

[0030] In step S401 of this embodiment, when the hyperspectral image is iteratively solved using the total variation-based structural feature extraction model shown in the following formula to obtain the spatial spectrum feature map of the hyperspectral image, each iterative solution process includes: Solve based on the following formula : , Solve based on the following formula : , Solve based on the following formula : , in, For the k+1th iteration, The polynomial of the points, is the number of pixels in the hyperspectral image, It refers to the hyperspectral image I in The pixel value at the point, , is the auxiliary variable for solving the kth and k+1th iterations of the L1-regularization problem. 、 It is an auxiliary variable for the kth and k+1th iterations when dealing with constrained optimization problems. For the k+1th iteration, the polynomial is The gradient of is a soft threshold operation, is a free parameter, and: , In the above formula, is the symbolic function, To get the maximum value.

[0031] Solution When , the weight norm can be defined for: , In the above formula, For hyperspectral images exist The pixel value at the point.

[0032] , , In the above formula, the hyperspectral image Defined as The pixel value set in , the polynomial Defined as The polynomial at the same pixel position in A collection of values ​​of ; To surround The local pixel set, similarly, the variable b and d It can also be expressed as above. The dimension is constructed as of , p Expressed as Linear combination of : , In the above formula, is the unknown coefficient, is a linear combination The basis polynomial of for The dimension of . It can be seen that the above formula is a least squares problem, which can be solved by solving the polynomial The coefficient of Solved. The variables of the objective function can be converted into : , In the above formula, For polynomial The coefficient of The objective function is is the regularization parameter, , , Auxiliary variables introduced for Split Bregman iteration; for polynomials The coefficient of This can be solved by setting the derivative of the above equation to zero: .

[0033] make: , , In the above formula, Representation Matrix middle The position elements are basis polynomials exist The value of The basis polynomial exist The value of, matrix Representation Matrix middle Position elements are weight functions The value of , then the polynomial The coefficient of The solution results are as follows: , In the above formula, for The set of pixel values ​​in , For the matrix The matrix formed, for The matrix formed, is the regularization parameter, Auxiliary variables introduced for Split Bregman iteration; thus, we can solve: , In the above formula, is the polynomial of the k+1th iteration , for The dimension of is a linear combination The basis polynomial of .

[0034] In the structure of branch two, the method of this embodiment first constructs a hyperspectral image structural feature extraction model for hyperspectral images containing simulated noise based on the total variation structural feature extraction method, and then selects a certain number of principal components from the structural feature model for object recognition through kernel principal component analysis (KPCA). Finally, based on the structural features extracted by KPCA, the SVM classifier is used to estimate the probability of the object category, and the category probability result of the object is obtained, namely: category probability graph C2. In terms of model training, the method of this embodiment randomly selects each type of object in branch one. (In this embodiment ) samples, the superpixel segmentation results are used to expand the samples, and then the expanded samples are used to train the classification model of branch two. Specifically, the sample expansion finds the superpixel where each initial training sample is located, and then all pixels in the superpixel are used as training samples, which can solve the lack of accuracy in the case of small samples. After obtaining the probability results of the ground object category estimation of the two branches, the final classification result, that is, the final category label, can be obtained by integrating the probabilities of all categories through weighted fusion. ,After integration, the class probability belonging to the same object will be higher than the recognition probability of the object by a single classifier, and different objects can be identified more accurately. In order to verify the hyperspectral image superpixel spatial-spectral fusion recognition method of this embodiment, the method of this embodiment and multiple existing methods are experimented as follows. The multiple existing methods include: MSTV, see the literature: P. Duan, X. Kang, S. Li, et al. Noise-Robust Hyperspectral Image Classification via Multi-Scale Total Variation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(6): 1948-1962; FSDI, see the literature: P. Duan, P. Ghamisi, X. Kang, et al. Fusion of Dual Spatial Information for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9): 7726-7738; KNNRS, see the literature: B. Tu, J. Wang, X. Kang, et al. KNN-Based Representation of Superpixels for Hyperspectral ImageClassification[J]. IEEE Journal of Selected Topics in Applied EarthObservations and Remote Sensing, 2018, 1(11): 4032-4047; RCSPP, see the literature for details: C.Zheng, N. Wang and J. Cui. Hyperspectral Image Classification With SmallTraining Sample Size Using Superpixel-Guided Training Sample Enlargement[J].IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10): 7307-7316; SSSAN, see the literature for details: X. Zhang et al.Spectral–Spatial Self-Attention Networks for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-15; SSFTT, see the literature for details: L. Sun, G. Zhao, Y. Zheng, etal. Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60:1-14; SSAtt, for details, see the literature: R. Hang, Z. Li, Q. Liu, et al. Hyperspectral ImageClassification With Attention-Aided CNNs[J]. IEEE Transactions on Geoscienceand Remote Sensing, 2021, 59(3): 2281-2293; A2S2K, for details, see the literature: SK Roy, S.Manna, T. Song, et al. Attention-Based Adaptive Spectral–Spatial KernelResNet for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9): 7831-7843. When comparing the classification accuracy of the method in this embodiment with that of various existing methods on Pavia University data, the accuracy evaluation indicators used are Overall Accuracy (OA), Average Accuracy (AA) and Kappa coefficient. The results are shown in Table 1 and. Figure 3 shown.

[0035] Table 1: Comparison of classification accuracy between the method in this embodiment and various existing methods

[0036] Figure 3 (a) is the actual feature classification map, (b) is the feature classification result map of SSAtt, (c) is the feature classification result map of SSSAN, (d) is the feature classification result map of A2S2K, (e) is the feature classification result map of SSFTT, (f) is the feature classification result map of FDSI, (g) is the feature classification result map of MSTV, (h) is the feature classification result map of RCSPP, (i) is the feature classification result map of KNNRS, and (j) is the feature classification result map of the method of this embodiment.

[0037] As shown in Table 1 and Figure 3 It can be seen that on the Pavia University dataset, the method of this embodiment shows significant advantages over other comparison algorithms. The highest scores were achieved in the three important performance indicators of OA (Overall Accuracy), AA (Average Accuracy), and Kappa coefficient. This shows the excellent performance of the method of this embodiment in the task of hyperspectral image object recognition.

[0038] As mentioned above, the method of this embodiment uses a dual-branch structure to estimate the category probability of the objects in the noisy hyperspectral image respectively. This method solves the small sample problem of hyperspectral images. The dual-branch spatial-spectral fusion model structure effectively improves the accuracy of object recognition and increases the robustness of the model to image noise, providing strong support for actual object recognition. Experimental results show that the method of this embodiment has high accuracy and robustness in the hyperspectral image object recognition task. In addition, the present embodiment further provides a hyperspectral image superpixel spatial-spectral fusion recognition system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the hyperspectral image superpixel spatial-spectral fusion recognition method. In addition, the present embodiment further provides a hyperspectral image superpixel spatial-spectral fusion recognition system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the hyperspectral image superpixel spatial-spectral fusion recognition method.

[0039] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0040] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A hyperspectral image superpixel spatial-spectral fusion recognition method, characterized in that: The steps include: S101, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps, and extracting spatial spectrum feature maps from the hyperspectral image to be identified and classifying and identifying the superpixel feature maps; S102, the two classification and recognition results are fused to obtain a final ground object category recognition result, and the function expression for fusion of the two classification and recognition results to obtain the final ground object category recognition result is: ， In the above formula, is the final category label, is the probability of the i-th category obtained by classifying and identifying the superpixel feature map, is the probability of the ith category obtained by classifying and identifying the spatial spectrum feature map, is a free parameter.

2. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 1 is characterized in that: In step S101, extracting superpixel feature maps from the hyperspectral image to be identified and classifying and identifying the same includes: S201, performing principal component analysis (PCA) on the hyperspectral image to be identified to extract multiple leading principal components; S202, performing super-pixel segmentation on the plurality of principal components to obtain a super-pixel image; S203, extracting a superpixel feature map according to the superpixel images of the plurality of principal components; S204, using a classifier to classify the superpixel feature map to obtain the ground object category probability of each superpixel; S205 , using an extended random walk algorithm to refine the ground object category probability of each superpixel to obtain a category probability map C1 as a result of classification and recognition of the superpixel feature map.

3. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 2 is characterized in that: Step S205 includes: S301, for each type of feature , according to the ground object category in the superpixel feature map The ground object category probability matrix of the pixels is constructed to obtain the ground object category The probability matrix ; S302, using the extended random walk algorithm to classify the ground objects based on the following objective function The probability matrix To optimize: ， In the above formula, for The optimization result of The pixel belongs to the ground feature category The probability matrix of for The spatial term, is the balance factor, for The non-spatial terms of , and there are: ， ， In the above formula, is the Laplacian matrix of the weight graph, The pixel belongs to the ground feature category The probability matrix of For feature category The diagonal matrix of the diagonal matrix consists of the preset initial probabilities. is the transpose operation; the Laplacian matrix of the weight graph The calculation function expression is: ， In the above formula, is the first element in the Laplacian matrix of the weight graph. Line List, is a constant parameter, is the first principal component in the superpixel image superpixels, is the first principal component in the superpixel image superpixels, and The adjacent represents the first principal component in the superpixel image superpixels and The distance between superpixels is less than the set value.

4. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 2 is characterized in that: The function expression for extracting the superpixel feature map from the superpixel images of multiple principal components in step S203 is: ， In the above formula, is the first superpixel feature map The pixel value of a superpixel, For the The number of pixels in a superpixel, The first The pixel value of a pixel.

5. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 2 is characterized in that: The classifier used in step S204 is a support vector machine SVM, the input of the support vector machine SVM is the superpixel of the superpixel feature map, and the output is the category label in the category probability map C1. The function expression of the support vector machine SVM is: ， ， In the above formula, Indicates to the maximum value, is a penalty parameter greater than zero, and is the Lagrange multiplier, is the first superpixel feature map Superpixels The corresponding category label, , Represents the first Superpixels The superpixel feature map Superpixels The feature vector after mapping to a higher dimensional space, is the number of superpixels.

6. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 1 is characterized in that: In step S101, extracting the spatial spectrum feature graph of the hyperspectral image to be identified and classifying and identifying the hyperspectral image to be identified includes: S401, using a preset iterative algorithm to iteratively solve the hyperspectral image using the total variation-based structural feature extraction model shown in the following formula to obtain a spatial spectrum feature map of the hyperspectral image: ， In the above formula, For a number not greater than polynomial space, where Pixel The polynomial terms of The number of polynomial terms in , Represents pixel is a two-dimensional real number space An element in is a vector of non-negative integers, For vector The value range and is a positive integer, For vector The absolute value and of all elements in refers to the degree of the local polynomial, is the i-th pixel The polynomial of is the i-th pixel, For hyperspectral images I From the i-th pixel The pixel value at the determined position, For pixels and the i-th pixel The weight function, weight function Used to determine the construction polynomial When each i-th pixel The contribution of is a free parameter, For polynomial exist The gradient of S402, reducing the dimension of the spatial spectrum feature map of the hyperspectral image; S403, using a classifier to classify the space-spectrum feature map after dimensionality reduction to obtain a category probability map C2 as a result of classification and recognition of the space-spectrum feature map.

7. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 6 is characterized in that: The weight function in step S401 The calculation function expression is: ， In the above formula, For comparing pixels and the i-th pixel The local area of ​​the image block, is the offset value, For hyperspectral images I At the i-th pixel and offset value The pixel value at the determined position, It means that the hyperspectral image I consists of pixels and offset value The pixel value at the determined position, The standard deviation is Gaussian function, is the scale parameter set to 1.

8. The hyperspectral image superpixel spatial-spectral fusion recognition method according to claim 6 is characterized in that: In step S401, when the hyperspectral image is iteratively solved using the total variation-based structural feature extraction model shown in the following formula to obtain the spatial spectrum feature map of the hyperspectral image, each iterative solution process includes: Solve based on the following formula : ， Solve based on the following formula : ， Solve based on the following formula : ， in, is the i-th pixel at the k+1th iteration The polynomial of , is the auxiliary variable for solving the kth and k+1th iterations of the L1-regularization problem. 、 It is an auxiliary variable for the kth and k+1th iterations when dealing with constrained optimization problems. For the k+1th iteration, the polynomial is The gradient of is a soft threshold operation, is a free parameter.

9. A hyperspectral image superpixel spatial-spectral fusion recognition system, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the hyperspectral image superpixel spatial-spectral fusion recognition method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored therein, characterized in that: The computer program is used to be programmed or configured by a microprocessor to execute the hyperspectral image superpixel spatial-spectral fusion recognition method according to any one of claims 1 to 8.

Images