Spatial-spectral feature selection method and device based on multimode data and storage medium

Abstract

The embodiment of the invention provides a spatial-spectral feature selection method and device based on multimode data and a storage medium, and the method comprises the steps: carrying out the adaptive multimode spectrum analysis of a hyperspectral image multimode data set, and obtaining a multimode low-dimensional representation; introducing the multi-mode low-dimensional representation as a regression target into an RFS algorithm, and determining an objective function of the AMFS; optimizing and solving the objective function of the AMFS to obtain a regression coefficient matrix; and sorting elements of the regression coefficient matrix from large to small, and screening out the first p spatial spectrum features. The method and the device are used for solving the problems of data redundancy and large calculation amount caused by the fact that hidden complementary information of multimode data cannot be fully utilized in the prior art.

Description

Technical Field

[0001] The present application relates to the technical field of interpretation and processing of hyperspectral remote sensing image data, and in particular to a method and device for selecting spatial spectrum features based on multi-mode data and a storage medium. Background Art

[0002] In the field of interpretation and processing of hyperspectral remote sensing image data, feature mining and classification technology based on multimodal data has made great progress in recent years to address the problem that a single spectral feature cannot effectively express the spatial shape, texture and layout of objects.

[0003] The spatial features of hyperspectral images, such as texture features and mathematical morphological features (each feature is called a pattern), can effectively represent the spatial shape attributes and spatial distribution characteristics of different objects, making up for the shortcomings of single spectral features in their ability to express deep spatial context information of images.

[0004] In the early days, each single feature space algorithm directly processed the serial vector formed by multi-mode features, simply concatenating multiple features into a long vector. At this time, multiple modes are equal, and weights are not allocated according to the importance of different data. It is impossible to fully utilize the complementary information hidden in multi-mode data, and the dimension of the multi-mode feature set is higher, the redundancy is more obvious, and the amount of calculation is greater. Therefore, a spectral feature extraction method that fully utilizes the complementary information hidden in multi-mode data needs to be proposed. Summary of the invention

[0005] The embodiments of the present application provide a method, device and storage medium for selecting spatial spectrum features based on multi-mode data, so as to solve the problem in the prior art that the complementary information hidden in multi-mode data cannot be fully utilized, resulting in data redundancy and large amount of calculation.

[0006] On the one hand, an embodiment of the present application provides a method for selecting spatial spectrum features based on multimodal data, comprising:

[0007] Adaptive multimodal spectrum analysis is performed on the multimodal dataset of hyperspectral images to obtain a multimodal low-dimensional representation;

[0008] The multimodal low-dimensional representation is introduced as the regression target into the RFS (Robust Feature Selection) algorithm to determine the objective function of AMFS;

[0009] Optimize and solve the objective function of AMFS (Adaptive Multiview Feature Selection) to obtain the regression coefficient matrix;

[0010] Sort the elements of the regression coefficient matrix from large to small, and filter out the first p empty spectrum features.

[0011] Optionally, adaptive multimodal spectrum analysis is performed on the hyperspectral image multimodal dataset according to the following formula to obtain a multimodal low-dimensional representation:

[0012]

[0013] in, Represents the low-dimensional representation of n sample points, R n×d represents a real number set of n×d dimensions, I represents the identity matrix, T represents the transpose, d represents the embedding dimension, L i Represents the Laplace matrix of the i-th mode data, each L i Hyperspectral image multimodal dataset The data in each mode is calculated, X=[X1,X2,…,X s ], where the i-th pattern is m i Represents the dimension, R n×m represents a real number set of n×m dimensions, s represents the number of types of multimodal data, and tr() represents the trace of the matrix.

[0014] Optionally, the objective function of AMFS is:

[0015]

[0016] in, is the regression coefficient matrix, w i and w j are the i-th row and j-th column of the matrix W, respectively.

[0017] Represents the low-dimensional representation of n sample points, R n×d represents a real number set of n×d dimensions, column vector The elements of are all 1, is the bias term.

[0018] Optionally, the objective function of AMFS is optimized and solved to obtain a regression coefficient matrix including:

[0019] According to the hyperspectral image training sample set and the dimension of low-dimensional representation, the Laplacian matrix nearest neighbor number is constructed and the low-dimensional representation is solved;

[0020] The regression coefficient matrix is ​​solved based on the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter.

[0021] Optionally, the specific steps of constructing the Laplacian matrix neighbor number and solving the low-dimensional representation according to the sample set of the hyperspectral image training sample set and the dimension of the low-dimensional representation include:

[0022] Input: A sample set containing n hyperspectral image training samples, the dimension of the low-dimensional representation, represented by d, the number of neighbors of the Laplace matrix, and the weight value of each mode initialized to α i =1 / s;

[0023] Step 101: Find the Laplacian matrix of multimodal data

[0024] Step 102: Solve the generalized eigenvalue problem Ly=λy, arrange the eigenvalues ​​from small to large, select the 2nd to d+1th empty spectrum eigenvectors to form Y, where λ is a regularization parameter;

[0025] Step 103: Determine whether the objective function converges; if converged, exit; if not, update α i , repeat steps 101 to 103 until convergence;

[0026] Output: low-dimensional representation, denoted by Y.

[0027] Optionally, solving the regression coefficient matrix according to the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter includes:

[0028] Input: sample set of hyperspectral image training samples, low-dimensional representation, regularization parameters.

[0029] Initialization: Initialize C and D to the identity matrix, where C is the eigenvalue matrix and D is the eigenvector matrix.

[0030] Step 201: Solve W and b in the objective function of AMFS based on the loss function of robust least squares regression and the regularization term of group sparsity constraint T ;

[0031] Step 202: Re-solve matrices C and D;

[0032] Step 203: Determine whether the objective function converges. If converged, exit; if not, repeat steps 201 to 203 until converged;

[0033] Output: Regression coefficient matrix.

[0034] The present application also proposes a spatial spectrum feature selection device based on multi-mode data, comprising:

[0035] Adaptive multi-mode spectrum analysis module, used to perform adaptive multi-mode spectrum analysis on hyperspectral image multi-mode data sets to obtain multi-mode low-dimensional representation;

[0036] The objective function determination module is used to introduce the multi-modal low-dimensional representation as the regression target into the RFS algorithm and determine the objective function of AMFS;

[0037] The objective function solving module is used to optimize and solve the objective function of AMFS to obtain the regression coefficient matrix;

[0038] The output module is used to sort the elements of the regression coefficient matrix from large to small and filter out the first p empty spectrum features.

[0039] The present application also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, a method for selecting spatial spectrum features based on multi-mode data is implemented as described in any one of the above embodiments.

[0040] The technical solution in this application has the following advantages:

[0041] The present application proposes a method for selecting spatial spectral features based on multimodal data, which is an unsupervised adaptive method for selecting spatial spectral features based on multimodal data. This method adaptively adjusts the weight coefficient of each mode data according to the physical meaning and statistical characteristics of each mode data to obtain the optimal low-dimensional representation; and takes this as the regression target, and utilizes group sparsity constraints to regularize the weight matrix (regression coefficient matrix) of the robust least squares regression model, so as to fully discover the complementary information and potential consistency of each mode data, and in the process of overall measurement of all features, selects the spatial spectral feature subset with the strongest discrimination and resolution ability for multimodal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A step diagram of a method for selecting spatial spectrum features based on multimodal data provided in an embodiment of the present application.

[0044] Figure 2 This is a classification accuracy curve of different Indian Pines algorithms provided in an embodiment of the present application.

[0045] Figure 3 A convergence state diagram of each algorithm when selecting the first 50 features provided in an embodiment of the present application.

[0046] Figure 4 A convergence state diagram of each algorithm when selecting the first 100 features provided in an embodiment of the present application.

[0047] Figure 5A schematic diagram of a spatial spectrum feature selection device based on multi-mode data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] It should be noted that the spatial spectrum feature selection method based on multimodal data provided in this application is based on the RFS algorithm, which uses the multi-cluster low-dimensional embedding obtained by adaptive multimodal spectrum analysis to replace the category labeling matrix of RFS to achieve unsupervised adaptive multimodal feature selection.

[0050] Figure 1 A step diagram of a method for selecting spatial spectrum features based on multimodal data provided in an embodiment of the present application. The embodiment of the present application provides a method for selecting spatial spectrum features based on multimodal data, specifically, including S1-S4:

[0051] S1: Adaptive multimodal spectrum analysis is performed on the hyperspectral image multimodal dataset to obtain a multimodal low-dimensional representation.

[0052] In one embodiment, a hyperspectral image multimodal dataset comprising n samples is constructed. Perform adaptive multimode spectral analysis.

[0053]

[0054] in, Represents the low-dimensional representation of n sample points, R n×d represents a real number set of n×d dimensions, I represents the identity matrix, T represents the transpose, d represents the embedding dimension, L i Represents the Laplace matrix of the i-th mode data, each L i Hyperspectral image multimodal dataset The data in each mode is calculated, X=[X1,X2,…,X s ], where the i-th pattern is m i Represents the dimension, R n×m represents a real number set of n×m dimensions, s represents the number of types of multimodal data, and tr() represents the trace of the matrix.

[0055] S2: Introduce the multimodal low-dimensional representation into the RFS algorithm as the regression target and determine the objective function of AMFS.

[0056] In one embodiment, the objective function of RFS is first constructed as:

[0057]

[0058] in, Represents n data, is the regression coefficient matrix, also the weight matrix, w i and w j are the i-th row and j-th column of the matrix W respectively. is a category label matrix consisting of 0 and 1, column vector The elements of are all 1, is the bias term, and ||·||2 represents L 2,1 norm, used to regularize the matrix, m means that the matrix W has a total of m rows, c means that the matrix W has a total of c columns, w ij Represents the element in the i-th row and j-th column of the matrix W. The category labeling matrix Y0 represents the global similarity relationship between every two samples, that is, the global similarity structure of the sample set. The matrix W obtained by the RFS algorithm has a row sparse effect, which reflects the contribution of different features in preserving the global similarity structure of the data set, and achieves the effect of feature selection.

[0059] In this embodiment, this chapter uses the multi-mode low-dimensional representation obtained by adaptive multi-mode spectrum analysis, The RFS algorithm is introduced to replace the category label matrix Y0, that is, the low-dimensional representation of the multimodal dataset is used as the regression target of the RFS algorithm. Determine the objective function of AMFS:

[0060]

[0061] Among them, X represents the hyperspectral image multi-modal dataset, is the regression coefficient matrix, represents the regression coefficient matrix, and the i-th row and j-th column of the matrix W are represented by w i and w j , Represents the low-dimensional representation of n sample points, R n×d represents a real number set of n×d dimensions, column vector The elements of are all 1, is the bias term, λ represents the coefficient, ||·|| 2,1 Indicates L 2,1 Norm, used to regularize the matrix.

[0062] It is easy to solve formula (4). This is a convex optimization problem. The local optimal solution is the global optimal solution, and the optimal solution is unique.

[0063] S3: Optimize and solve the objective function of AMFS to obtain the regression coefficient matrix.

[0064] In one embodiment, when solving the objective function of AMFS, it is necessary to first solve formula (1) to obtain a low-dimensional representation of the multimodal data, and then select a feature subset by formula (4). Therefore, the solution process of AMFS is divided into two steps.

[0065] The first step is to construct the Laplace matrix neighbor number and solve the low-dimensional representation based on the hyperspectral image training sample set and the dimension of the low-dimensional representation; the second step is to solve the regression coefficient matrix based on the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter.

[0066] Specifically, according to the sample set of the hyperspectral image training sample set and the dimension of the low-dimensional representation, the specific steps of constructing the Laplacian matrix neighbor number and solving the low-dimensional representation include:

[0067] Input: A sample set containing n hyperspectral image training samples, the dimension of the low-dimensional representation, represented by d, the number of neighbors of the Laplace matrix, and the weight value of each mode initialized to α i =1 / s.

[0068] Step 101: Find the Laplacian matrix of multimodal data

[0069] Step 102: Solve the generalized eigenvalue problem Ly=λy, arrange the eigenvalues ​​from small to large, select the 2nd to d+1th empty spectrum eigenvectors to form Y, where λ is a regularization parameter.

[0070] Step 103: Determine whether the objective function converges; if converged, exit; if not, update α i , repeat steps 101 to 103 until convergence.

[0071] Output: low-dimensional representation, denoted by Y.

[0072] It should be noted that in the solution process, the similarity between samples is calculated based on the sample set and the number of neighbors, and is represented by the Laplace matrix, which is calculated based on the adjacency matrix and degree matrix of the graph. Specifically, if there is a graph G with n nodes and its adjacency matrix A and degree matrix D, the Laplace matrix L can be calculated by L=DA. In addition, in order to eliminate the influence of the degree difference of different nodes on the calculation results, the Laplace matrix can also be normalized. The methods for solving the problem of generalized eigenvalues ​​include iteration, deformation, decomposition, reduction, etc.

[0073] The second step is to solve the regression coefficient matrix based on the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter, including:

[0074] Input: sample set of hyperspectral image training samples, low-dimensional representation, regularization parameter, denoted by λ.

[0075] Initialization: Initialize C and D to the identity matrix, where C is the eigenvalue matrix and D is the eigenvector matrix.

[0076] Step 201: Solve W and b in the objective function of AMFS based on the loss function of robust least squares regression and the regularization term of group sparsity constraint T .

[0077] Step 202: Resolve matrices C and D.

[0078] Step 203: Determine whether the objective function converges. If converged, exit; if not, repeat steps 201 to 203 until converged;

[0079] Output: Regression coefficient matrix

[0080] It should be noted that since the regression coefficient matrix W solved by the objective function based on AMFS has row sparsity, the regression coefficient matrix of the robust least squares regression model is regularized using group sparsity constraints. The regularized decomposition here mainly uses principal component analysis (PCA), a data dimensionality reduction technology that reduces the dimension of the data by finding the main components of the data (i.e., the direction with the largest variance). Decompose the eigenvalue matrix C and the eigenvector matrix D. It should be noted that both matrices C and D have unit matrices, which are finally obtained by iteration. The size of the elements in the eigenvalue matrix C determines the importance of each pattern data, that is, it is also the distribution weight of each pattern data, so as to fully discover the complementary information and potential consistency of each pattern data. In addition, in this step, by optimizing the objective function, the optimal solution of the regression coefficient matrix can be obtained. This optimization process usually involves iterative algorithms, such as gradient descent method, coordinate descent method, etc. In each iteration, the value of the regression matrix is ​​updated according to the current objective function value and gradient information, and the optimal output regression coefficient matrix is ​​finally output.

[0081] S4: Sort the elements of the regression coefficient matrix from large to small, and filter out the first p empty spectrum features.

[0082] In one embodiment, by sorting the elements of the regularized regression coefficient matrix from large to small, the first p empty spectrum features can be quickly screened out, that is, the empty spectrum feature subset with the strongest discrimination and resolution ability for multimodal data is selected.

[0083] It should be noted that in the embodiment of this method, the spatial features such as texture features and mathematical morphology features of the hyperspectral remote sensing image data set are fully considered, which can effectively represent the spatial shape attributes and spatial distribution characteristics of different objects. The unsupervised adaptive multi-mode feature selection proposed in this chapter can adaptively adjust the contribution weight of each mode to the spatial spectrum feature mining according to the statistical characteristics and physical meaning of each mode data, better discover the complementary information and potential consistency of each mode data, and then select the multi-mode feature subset with the strongest discrimination and resolution ability to effectively identify and distinguish different objects. Experimental results show that the feature subset selected by the algorithm in this chapter has stronger discrimination and resolution ability, stronger classification and recognition ability for different objects, and higher classification accuracy.

[0084] In addition, the present application also provides a set of simulation results based on the above-mentioned method for selecting spatial spectrum features based on multi-mode data.

[0085] In order to evaluate the classification performance of hyperspectral images using the AMFS algorithm in the technical solution of this application, experiments were conducted using representative Indian Pines and hyperspectral remote sensing datasets. Among them, the Indian Pines dataset removed four types of objects with a small number of reference samples, such as "Alfalfa", "Orats", "Grass / Pasture-mowed" and "Stone-steel towers", and only the remaining 12 types of objects with a large number of reference samples were selected for experiments.

[0086] This application uses spectral features, texture features and morphological profile features for experiments. X=[X1,X2,X3], m=m1+m2+m3. Here X1 represents the spectral feature, Represents texture features and the dimension m2 is 60, Represents the morphological profile characteristics and the dimension m3 is 80.

[0087] like Figure 2As shown in the figure, MFCL21 (The feature selection algorithm based on multi-feature weighted combination and group sparse regression can be translated into, feature selection algorithm based on multi-feature weighted combination and group sparse regression), MCFS (multi-cluster feature selection), MaxVar (Minimum Redundancy Maximum Relevance and Feature Selection, minimum redundancy maximum relevance feature selection algorithm), LS (Least Squares, least squares method), MaxVar and other algorithms are selected to compare with AMFS, and the upper limit of the number of selected features (dimensions) is set to 100. MFCL21 is a feature selection algorithm based on multi-feature weighted combination and group sparse regression; MCFS uses graph analysis to obtain the multi-cluster manifold structure of the data set, and selects the feature subset with the strongest ability to distinguish the multi-cluster structure through sparse regression; LS calculates the ability of each spatial spectrum feature to preserve the local geometric characteristics of the data set, and selects the top-ranked features; MaxVar is a classic algorithm that calculates the variance of each spatial spectrum feature and selects features with larger variance, which maximizes the representation ability of the data set. Each algorithm adjusts parameters to achieve optimal performance.

[0088] First, the training sample set X is used to learn each feature selection algorithm, and then the feature subset is obtained, and then the feature subset is used for classification experiments. When classifying, all samples except X are used as test samples. The nearest neighbor classifier is used for classification, and the overall accuracy (OA), average accuracy (AA) and Kappa coefficient are used as evaluation indicators to evaluate the performance of the algorithm based on the classification accuracy. In order to improve the accuracy and reliability of the experiment, it is repeated 10 times, and the training samples are randomly selected each time, and the classification accuracy of the 10 experiments is averaged.

[0089] Figure 2 The classification accuracy curves of each algorithm when the number of training samples for each type of land object is 50 are given. It is easy to see that the classification accuracy of each algorithm continues to increase with the increase in the number of selected features, and remains stable or gradually decreases after reaching the maximum value. This is because the more features are selected, the more effective information is contained, which improves the classification accuracy. When the number of selected features is between 20 and 70, the classification accuracy of AMFS is relatively high; when the number of features is more than 60, the classification accuracy of AMFS gradually decreases. This is because AMFS is based on The robust feature selection algorithm of norm has a strong ability to remove redundant information and noise, so selecting fewer features can achieve relatively good classification results; when more features are selected, redundancy and noise gradually increase, the "Hughes" effect appears, and the classification accuracy gradually decreases. This also proves that AMFS can measure all features as a whole, and consider the correlation and combination effect between features, so as to give priority to selecting feature subsets with richer information and stronger complementarity, and obtain better data representation effect and classification accuracy.

[0090] according to Figure 3 and Figure 4 As shown in the figure, when the first 50 and first 100 empty spectrum features are selected, the AMFS algorithm is compared with MFCL21, MCFS, LS, MaxVar and other algorithms. The low-dimensional representation and robust least squares regression of AMFS are basically stable when the objective function value increases with the number of iterations greater than 5. This proves that the algorithm has high computational efficiency and only requires fewer iterations to ensure convergence. In actual calculations, the number of iterations can be 10 to 20.

[0091] This application also proposes a spatial spectrum feature selection device based on multi-mode data, such as Figure 5 As shown, including:

[0092] Adaptive multi-mode spectrum analysis module, used to perform adaptive multi-mode spectrum analysis on hyperspectral image multi-mode data sets to obtain multi-mode low-dimensional representation;

[0093] The objective function determination module is used to introduce the multi-modal low-dimensional representation as the regression target into the RFS algorithm and determine the objective function of AMFS;

[0094] The objective function solving module is used to optimize and solve the objective function of AMFS to obtain the regression coefficient matrix;

[0095] The output module is used to sort the elements of the regression coefficient matrix from large to small and filter out the first p empty spectrum features.

[0096] It should be noted that the spatial spectrum feature selection device based on multi-mode data provided in the present application can implement method steps that are completely consistent with the above-mentioned embodiment of the spatial spectrum feature selection method based on multi-mode data provided, and will not be repeated here.

[0097] In one embodiment, the present application also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating device of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the above-mentioned embodiment of a method for selecting spatial spectrum features based on multi-mode data.

[0098] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, storage media, or computer program products. Therefore, the application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A method for selecting spatial spectrum features based on multimodal data, characterized in that: include: Adaptive multimodal spectrum analysis is performed on the multimodal dataset of hyperspectral images to obtain a multimodal low-dimensional representation; The multi-modal low-dimensional representation is introduced into the RFS algorithm as a regression target to determine the objective function of AMFS; Optimizing and solving the objective function of the AMFS to obtain a regression coefficient matrix; The elements of the regression coefficient matrix are sorted in descending order, and the first p empty spectrum features are screened out.

2. A method for selecting spatial spectrum features based on multi-mode data as claimed in claim 1, characterized in that: According to the following formula, adaptive multi-mode spectrum analysis is performed on the multi-mode data set of hyperspectral images to obtain a multi-mode low-dimensional representation: in, Represents the low-dimensional representation of n sample points, R n×d represents a real number set of n×d dimensions, I represents the identity matrix, T represents the transpose, d represents the embedding dimension, L i Represents the Laplace matrix of the i-th mode data, each L i Hyperspectral image multimodal dataset The data in each mode is calculated, X=[X1,X2,…,X s ], where the i-th pattern is m i Represents the dimension, R n×m represents a real number set of n×m dimensions, s represents the number of types of multimodal data, and tr() represents the trace of the matrix.

3. A method for selecting spatial spectrum features based on multi-mode data as claimed in claim 1, characterized in that: The objective function of the AMFS is: Where X represents the hyperspectral image multi-modal dataset, represents the regression coefficient matrix, and the i-th row and j-th column of the matrix W are represented by w i and w j , Represents the low-dimensional representation of n sample points, that is, the low-dimensional representation of the hyperspectral image multimodal dataset, R n×d represents a real number set of n×d dimensions, column vector The elements of are all 1, is the bias term, λ represents the coefficient, ‖·|| 2,1 Indicates L 2,1 Norm, used to regularize the matrix.

4. The method for selecting spatial spectrum features based on multi-mode data according to claim 1, characterized in that: The objective function of the AMFS is optimized and solved to obtain a regression coefficient matrix including: According to the hyperspectral image training sample set and the dimension of low-dimensional representation, the Laplacian matrix nearest neighbor number is constructed and the low-dimensional representation is solved; A regression coefficient matrix is ​​solved according to the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter.

5. A method for selecting spatial spectrum features based on multi-mode data as claimed in claim 4, characterized in that: According to the sample set of the hyperspectral image training sample set and the dimension of the low-dimensional representation, the specific steps of constructing the Laplacian matrix neighbor number and solving the low-dimensional representation include: Input: A sample set containing n hyperspectral image training samples, the dimension of the low-dimensional representation, represented by d, the number of neighbors of the Laplace matrix, and the weight value of each mode initialized to α i =1 / s; Step 101: Find the Laplacian matrix of multimodal data Step 102: Solve the generalized eigenvalue problem Ly=λy, arrange the eigenvalues ​​from small to large, select the 2nd to d+1th eigenvectors to form Y, where λ is a regularization parameter; Step 103: Determine whether the objective function converges; if converged, exit; if not, update α i , repeat steps 101 to 103 until convergence; Output: low-dimensional representation, denoted by Y.

6. A method for selecting spatial spectrum features based on multi-mode data as claimed in claim 4, characterized in that: Solving the regression coefficient matrix according to the hyperspectral image training sample set, the solved low-dimensional representation and the regularization parameter includes: Input: sample set of hyperspectral image training samples, low-dimensional representation, regularization parameters; Initialization: Initialize C and D to the identity matrix, where C is the eigenvalue matrix and D is the eigenvector matrix; Step 201: Solve W and b in the objective function of AMFS based on the loss function of robust least squares regression and the regularization term of group sparsity constraint T ; Step 202: Re-solve matrices C and D; Step 203: determine whether the objective function converges; if converged, exit; if not, repeat steps 201 to 203 until converged; Output: Regression coefficient matrix.

7. A spatial spectrum feature selection device based on multi-mode data, characterized in that: include: Adaptive multi-mode spectrum analysis module, used to perform adaptive multi-mode spectrum analysis on hyperspectral image multi-mode data sets to obtain multi-mode low-dimensional representation; An objective function determination module, used to introduce the multi-modal low-dimensional representation as a regression target into the RFS algorithm to determine the objective function of AMFS; An objective function solving module is used to optimize and solve the objective function of the AMFS to obtain a regression coefficient matrix; The output module is used to sort the elements of the regression coefficient matrix from large to small, and filter out the first p empty spectrum features.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a spatial spectrum feature selection method based on multi-mode data is implemented as described in any one of claims 1 to 6.

Images