Hyperspectral remote sensing image recognition method and device based on spectral space feature coupling

Abstract

The application discloses a hyperspectral remote sensing image recognition method and device based on spectral space feature coupling, and utilizes different linear spectral filters to perform feature extraction on the whole hyperspectral remote sensing image, so that multi-class spectral features are obtained. Compared with the traditional technology, the application does not need to perform spectral dimension reduction, thereby avoiding the problem that the traditional technology weakens the subtle spectral information and high-order spectral correlation contained in the hyperspectral data. Meanwhile, the application takes the minimization of image background energy as an objective function to optimize the optimal real spectral feature vector of each linear spectral filter to the target class. Based on this, when the constructed linear spectral filter is used for feature extraction, the background energy can be minimized while the target spectral response is maintained and enhanced, so that the spectral clutter and noise in the complex environment are suppressed, and thus the detection precision of the model in the complex environment background can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image target detection, and particularly relates to a hyperspectral remote sensing image recognition method and device based on spectral space feature coupling. BACKGROUND

[0002] Under the background of rapid development of remote sensing technology and deep learning, due to the spectral characteristics of "narrow band and high dimension", hyperspectral images have become an important data source in the field of ground object recognition and segmentation. However, in the current hyperspectral remote sensing semantic segmentation, in order to balance the calculation efficiency, most methods usually simplify the input data by spectral dimension reduction methods (such as principal component analysis, independent component analysis, etc.) or only select part of the waveband. The above methods weaken the subtle spectral information and high-order spectral correlation contained in the hyperspectral data to some extent, which leads to insufficient model resolution for targets with extremely high spectral similarity or dramatic spectral changes, and also limits the detection accuracy and robustness of the model in complex backgrounds. Therefore, based on the foregoing deficiencies, how to provide a hyperspectral remote sensing image recognition method based on spectral space feature coupling with strong target resolution and high detection accuracy has become a problem to be solved. SUMMARY

[0003] The purpose of the present application is to provide a hyperspectral remote sensing image recognition method and device based on spectral space feature coupling, which solves the problems of insufficient resolution for targets with extremely high spectral similarity or dramatic spectral changes and low detection accuracy in complex backgrounds existing in the prior art.

[0004] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0005] In a first aspect, a hyperspectral remote sensing image recognition method based on spectral space feature coupling is provided, comprising:

[0006] A plurality of linear spectral filters are constructed, wherein any linear spectral filter is an optimal real spectral feature vector of the target class for the any linear spectral filter, which is obtained by taking the minimization of the image background energy as the objective function, and is constructed based on the optimal real spectral feature vector, and the target class is the class corresponding to the ground object to be identified in the hyperspectral remote sensing image;

[0007] The plurality of linear spectral filters are used to perform spectral feature extraction processing on the hyperspectral remote sensing image respectively to obtain a plurality of spectral features, and the plurality of spectral features are feature spliced to generate the spectral space coupling feature of the hyperspectral remote sensing image;

[0008] The texture feature of the hyperspectral remote sensing image is extracted to obtain the spatial texture feature of the hyperspectral remote sensing image;

[0009] Based on the spectral spatial coupling feature and the spatial texture feature, a fusion feature of the hyperspectral remote sensing image is generated, and the fusion feature is input into an image recognition model to obtain a ground object recognition result of the hyperspectral remote sensing image.

[0010] Based on the above disclosure, the present application extracts spectral features of a hyperspectral remote sensing image by constructing a plurality of linear spectral filters, thereby obtaining a plurality of spectral features of the hyperspectral remote sensing image. Then, the present application generates a spectral spatial coupling feature of the hyperspectral remote sensing image by feature splicing of the plurality of spectral features. Then, the present application extracts a spatial texture feature of the hyperspectral remote sensing image, thereby obtaining the spatial texture feature of the hyperspectral remote sensing image. Then, based on the spectral spatial coupling feature and the spatial texture feature, a fusion feature of the hyperspectral remote sensing image is generated. Finally, the fusion feature is input into an image recognition model, thereby obtaining a ground object recognition result of the hyperspectral remote sensing image.

[0011] Through the above design, the present application extracts features of the entire hyperspectral remote sensing image by using different linear spectral filters, thereby obtaining a plurality of categories of spectral features. Compared with the traditional technology, the present application does not need to perform spectral dimension reduction, and therefore can extract comprehensive feature information, thereby avoiding the problem that the traditional technology weakens the subtle spectral information and high-order spectral correlation contained in the hyperspectral data, and thereby improving the resolution capability for targets with extremely high spectral similarity or dramatic spectral changes. Meanwhile, the present application takes minimizing the image background energy as an objective function to optimize the optimal real spectral feature vector of each linear spectral filter for the target category. Based on this, when the linear spectral filter constructed is used for feature extraction, the background energy can be minimized while maintaining and enhancing the target spectral response, thereby suppressing spectral clutter and noise in a complex environment. Therefore, the detection accuracy of the model in a complex environment background can be improved. In this way, the present application can improve the target resolution capability while ensuring the detection accuracy in a complex environment background, and is therefore very suitable for large-scale application and promotion.

[0012] In one possible design, the independent variable of the objective function is the real spectral feature vector of the linear spectral filter for the target category;

[0013] The objective function takes minimizing the image background energy as an objective function to optimize the optimal real spectral feature vector of the linear spectral filter for the target category, and includes:

[0014] The sample hyperspectral remote sensing image and the real spectral feature vector of the linear spectral filter at the tth iteration are obtained, where the initial value of t is 1, and when t is 1, the real spectral feature vector is an initial feature vector;

[0015] putting the real spectral feature vector at the tth iteration into the objective function, and judging whether the iteration stopping condition is met based on a function value of the objective function;

[0016] If not, performing spectral feature extraction processing on the sample hyperspectral remote sensing image by using the any linear spectral filter to obtain a sample spectral feature corresponding to each pixel element in the sample hyperspectral remote sensing image;

[0017] updating the real spectral feature vector at the tth iteration based on the sample spectral feature corresponding to each pixel element to obtain a real spectral feature vector at a (t+1) th iteration;

[0018] increasing t by 1 and reacquiring the real spectral feature vector of the any linear spectral filter at the tth iteration until the iteration stopping condition is met, and obtaining the optimal real spectral feature vector.

[0019] In one possible design, updating the real spectral feature vector at the tth iteration based on the sample spectral feature corresponding to each pixel element to obtain a real spectral feature vector at a (t+1) th iteration includes:

[0020] calculating gradients of the respective sample spectral features relative to the real spectral feature vector at the tth iteration to obtain a plurality of filter output response gradients, and taking an average of the plurality of filter output response gradients as an update gradient;

[0021] acquiring a learning rate and taking a product between the update gradient and the learning rate as an update parameter;

[0022] calculating a difference between the real spectral feature vector at the tth iteration and the update parameter to obtain the real spectral feature vector at the (t+1) th iteration.

[0023] In one possible design, the objective function is:

[0024] (1)

[0025] In formula (1), denotes a parameter vector of the any linear spectral filter, denotes a correlation matrix of the sample hyperspectral remote sensing image, denotes a real spectral feature vector of the any linear spectral filter for a target category, denotes a transposition operation, denotes a linear equality constraint condition of the objective function;

[0026] wherein, , , wherein, represents the total number of pixel elements in the sample hyperspectral remote sensing image. represents the total number of pixel elements in the sample hyperspectral remote sensing image. represents the total number of pixel elements in the sample hyperspectral remote sensing image.

[0027] In one possible design, a plurality of linear spectral filters are utilized to perform spectral feature extraction processing on the hyperspectral remote sensing image respectively, to obtain a plurality of spectral features, including:

[0028] The hyperspectral remote sensing image is input into each linear spectral filter to obtain an output response of each linear spectral filter;

[0029] The output response of each linear spectral filter is subjected to square error loss processing to obtain a plurality of spectral features.

[0030] In one possible design, for any pixel element in the hyperspectral remote sensing image, the output response of the any linear spectral filter to the any pixel element is:

[0031] (2)

[0032] In formula (2), is the output response of the any linear spectral filter to the any pixel element, represents an optimal parameter vector of the any linear spectral filter, represents the total channel spectral value of the any pixel element, wherein, , and represents a correlation matrix of the hyperspectral remote sensing image, represents an optimal real spectral feature vector of the any linear spectral filter to a target category;

[0033] Correspondingly, the output response of each linear spectral filter is subjected to square error loss processing to obtain a plurality of spectral features, which includes:

[0034] For the output response of the any linear spectral filter to the any pixel element in the hyperspectral remote sensing image, the output response of the any pixel element is subjected to square error loss processing by using formula (3) as follows:

[0035] (3)

[0036] In formula (3), represents the spectral feature corresponding to the any pixel element.

[0037] In one possible design, texture feature extraction processing is performed on the hyperspectral remote sensing image to obtain a spatial texture feature of the hyperspectral remote sensing image, including:

[0038] performing two-dimensional convolution processing on the hyperspectral remote sensing image to obtain an initial spatial texture feature;

[0039] performing nonlinear processing on the initial spatial texture feature by using a nonlinear activation function to obtain the spatial texture feature after the nonlinear processing.

[0040] In one possible design, based on the spectral spatial coupling feature and the spatial texture feature, a fusion feature of the hyperspectral remote sensing image is generated, including:

[0041] performing feature splicing on the spectral spatial coupling feature and the spatial texture feature in the spectral channel dimension to obtain a spliced feature;

[0042] performing 1x1 convolution processing on the spliced feature to obtain an initial fusion feature;

[0043] performing nonlinear processing on the initial fusion feature to obtain the fusion feature of the hyperspectral remote sensing image.

[0044] In a second aspect, a hyperspectral remote sensing image recognition apparatus based on spectral spatial feature coupling is provided, including:

[0045] a filter construction unit configured to construct a plurality of linear spectral filters, wherein any linear spectral filter is constructed based on an optimal real spectral feature vector of a target class of the any linear spectral filter, which is obtained by optimizing the any linear spectral filter with the optimal real spectral feature vector as an objective function, and the target class is a class corresponding to a to-be-recognized ground feature in the hyperspectral remote sensing image;

[0046] a spectral feature extraction unit configured to perform spectral feature extraction processing on the hyperspectral remote sensing image by using the plurality of linear spectral filters to obtain a plurality of spectral features, and perform feature splicing on the plurality of spectral features to generate a spectral spatial coupling feature of the hyperspectral remote sensing image;

[0047] a texture feature extraction unit configured to perform texture feature extraction processing on the hyperspectral remote sensing image to obtain a spatial texture feature of the hyperspectral remote sensing image;

[0048] a ground feature recognition unit configured to generate a fusion feature of the hyperspectral remote sensing image based on the spectral spatial coupling feature and the spatial texture feature, and input the fusion feature into an image recognition model to obtain a ground feature recognition result of the hyperspectral remote sensing image.

[0049] In a third aspect, another hyperspectral remote sensing image recognition device based on spectral spatial feature coupling is provided, taking an electronic device as an example, comprising a memory, a processor and a transceiver connected in sequence in communication, wherein the memory is configured to store a computer program, the transceiver is configured to transceive messages, and the processor is configured to read the computer program and execute the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling as in the first aspect or any possible design in the first aspect.

[0050] In a fourth aspect, a storage medium is provided, and instructions are stored on the storage medium, and when the instructions are run on a computer, the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling as in the first aspect or any possible design in the first aspect is executed.

[0051] In a fifth aspect, a computer program product containing instructions is provided, and when the instructions are run on a computer, the computer is caused to execute the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling as in the first aspect or any possible design in the first aspect.

[0052] Advantages:

[0053] (1) The present application extracts features of the entire hyperspectral remote sensing image by using different linear spectral filters, thereby obtaining multi-class spectral features. Compared with the prior art, the present application does not need to perform spectral dimension reduction, so comprehensive feature information can be extracted, thereby avoiding the problem that the traditional technology weakens the subtle spectral information and high-order spectral correlation contained in the hyperspectral data, and further improving the resolution capability of the target with extremely high spectral similarity or drastic spectral change.

[0054] (2) The present application constructs a linear spectral filter with the optimization target of minimizing background energy, and extracts spectral features based on the linear spectral filter. In this way, the background energy can be minimized while maintaining and enhancing the target spectral response, thereby fully utilizing the high-resolution spectral information of the hyperspectral data, and further realizing adaptive suppression of spectral clutter and noise in a complex environment, and learning and strengthening the subtle spectral curve features of the target class.

[0055] (3) The present application constructs an explicit derivable target response function, so that the target spectral learning mechanism has a clear mathematical explanation, avoiding the uncertainty of black box learning, and further improving the controllability of the training process and the clarity of the gradient optimization.

[0056] (4) The present application fuses the spectral spatial coupling features and the spatial texture features extracted by convolution in the channel dimension, and based on this, the depth complementarity of spectral sensitivity and spatial discriminability is realized, thereby significantly improving the distinguishing ability and detail recovery precision of the model for ground object classes. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 A step flow diagram of the hyperspectral remote sensing image recognition method based on spectral space feature coupling provided by the embodiment of the present application is shown in the figure.

[0058] Figure 2 A flow chart of the hyperspectral remote sensing image recognition method provided by the embodiment of the present application is shown in the figure.

[0059] Figure 3 A schematic diagram of the spectral space coupling features corresponding to different linear spectral filters provided by the embodiment of the present application is shown in the figure.

[0060] Figure 4 A comparison schematic diagram of image recognition provided by the embodiment of the present application is shown in the figure.

[0061] Figure 5 A structural diagram of the hyperspectral remote sensing image recognition device based on spectral space feature coupling provided by the embodiment of the present application is shown in the figure.

[0062] Figure 6 A structural schematic diagram of the electronic device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the present application will be briefly introduced below in combination with the drawings and the descriptions of the embodiments or the prior art. Obviously, the following description of the drawings structure is only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings. It should be noted that the description of these embodiment modes is used to help understand the present application, but does not constitute a limitation on the present application.

[0064] It should be understood that although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element can be called a second element, and similarly a second element can be called a first element, without departing from the scope of the example embodiments of the present application.

[0065] It should be understood that, for the term "and / or" that can appear in the present text, it only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent three cases of A alone, B alone, and A and B together; for the term " / and" that can appear in the present text, it describes another association object relationship, which means that there can be two relationships, for example, A / and B, which can represent two cases of A alone and A and B together; in addition, for the character " / " that can appear in the present text, it generally represents an "or" relationship between the associated objects before and after it.

[0066] Embodiment:

[0067] Referring to Figure 1 As shown in the figure, the hyperspectral remote sensing image recognition method based on spectral space feature coupling provided by the embodiment creates a plurality of trainable linear spectral filters, and updates the parameters thereof (i.e., the real spectral feature vector of the target category) by using the back propagation of the neural network, so as to output the spectral space coupling features of different categories by using the plurality of linear spectral filters after optimizing the optimal parameters of the foregoing filters, and thereby suppress the background interference in a complex environment; then, the spectral space coupling features and the spatial texture features extracted by convolution are deeply fused, and the feature analysis and target recognition are performed in combination with the Vision-Transformer architecture, and thereby the ground object recognition result in the image is obtained; based on this, compared with the traditional technology, the present method does not need to perform spectral dimension reduction, thereby avoiding the problem that the traditional technology weakens the subtle spectral information and high-order spectral correlation contained in the hyperspectral data, and realizing the suppression of the background interference in a complex environment, based on which the accuracy and robustness of target recognition in a complex scene are improved, thereby the present method is very suitable for large-scale application and promotion; wherein, for example, the present method can but is not limited to run at the image recognition end side, and optionally, for example, the image recognition end can but is not limited to a personal computer (PC) or a server; it can be understood that the foregoing execution subject does not constitute a limitation on the embodiments of the present application, and correspondingly, the running steps of the present method can but are not limited to the steps S1-S4 shown below.

[0068] S1. A plurality of linear spectral filters are constructed, wherein any linear spectral filter is constructed based on the optimal real spectral feature vector of the target category, which is optimized by taking the minimization of the image background energy as the objective function, and the target category is the category corresponding to the ground object to be identified in the hyperspectral remote sensing image.

[0069] In a specific application, hyperspectral remote sensing data usually contains hundreds or even thousands of continuous spectral channels, which contains a large amount of spectral information that can be used for semantic segmentation tasks (i.e. the semantic segmentation task here is feature recognition). In order to fully capture the slight differences of different features in the spectral dimension, and use their distribution patterns in the spatial position to construct a spectral-spatial coupling feature, K improved constraint energy minimization (CEM) filters (i.e. the aforementioned linear spectral filter) are designed to extract the spectral-spatial coupling feature based on the K filters designed above. In this embodiment, the value of K can be flexibly set according to the specific task requirements (this embodiment is set to 32), which is not limited here.

[0070] Further, the CEM filter does not need the background information of the image, but only needs to know the prior spectral information to be detected (i.e. the real spectral feature vector of the target category, and the target category is the category of the feature to be identified). The traditional method is to determine the real spectral feature vector by the prior known target to be detected. Therefore, using prior knowledge to determine the real spectral feature vector of the target category has the problem of poor background suppression effect. Based on this, this embodiment provides an improved linear spectral filter, that is, by constructing a target function with the optimization goal of minimizing the image background energy, the optimal real spectral feature vector of the target category for each linear spectral filter is optimized to suppress the response intensity of non-target pixels and maintain an approximate unit amplitude output for target pixels, thereby retaining and enhancing the spectral response characteristics of the target category.

[0071] Optionally, the independent variable of the aforementioned target function is the real spectral feature vector of the target category for the any linear spectral filter, wherein it is assumed that a sample hyperspectral remote sensing image , and respectively represent the height and width of the image, represent the number of spectral channels), the parameter optimization of the any linear spectral filter is performed, then the target function can be represented as:

[0072] (1)

[0073] In formula (1), represents the parameter vector of the any linear spectral filter, represents the correlation matrix of the sample hyperspectral remote sensing image input to the any linear spectral filter during training, represents the real spectral feature vector of the target category for the any linear spectral filter, represents the transposition operation.

[0074] wherein,​ , , wherein, represents the full-channel spectral value of the i-th pixel element in the sample hyperspectral remote sensing image (i.e. the spectral value of the i-th pixel element in different spectral channels, which is essentially a vector), and represents the total number of pixel elements in the sample hyperspectral remote sensing image. Meanwhile, in formula (1), the linear equality constraint condition of the objective function is represented by

[0075] , which means that the response output of the filter to must be 1. This condition ensures that the output response of the filter to the target pixel remains at a unit amplitude, thereby enhancing the identification ability of the target category.

[0076] Thus, after the aforementioned objective function is constructed, the parameter optimization of the any linear spectral filter can be performed based on the objective function, and the process is shown in the following steps S11-S15.

[0077] S11. Obtain a sample hyperspectral remote sensing image and a real spectral feature vector of the any linear spectral filter at the t-th iteration, wherein the initial value of t is 1, and when t is 1, the real spectral feature vector is an initial feature vector. In this embodiment, the elements of the initial feature vector can be, but are not limited to, random numbers with a mean of 0 and a variance of 1. Thus, after the real spectral feature vector at the t-th iteration is obtained, it can be determined whether the vector is the optimal parameter by means of the objective function, and the process is shown in the following step S12.

[0078] S12. Substitute the real spectral feature vector at the t-th iteration into the objective function, and determine whether the iteration stopping condition is met based on the function value of the objective function. In specific applications, the correlation matrix corresponding to the sample hyperspectral remote sensing image is calculated, and the filter parameter of the any linear spectral filter at the t-th iteration is calculated using the aforementioned correlation matrix and the real spectral feature vector at the t-th iteration. Finally, the filter parameter is substituted into formula (1) to obtain the background energy at the t-th iteration. Based on this, it can be determined whether the background energy is less than the energy threshold to determine whether parameter optimization is needed. If it is greater than the energy threshold, it means that the iteration stopping condition is not met, and parameter optimization needs to be continued, and the process is shown in the following steps S13-S15.

[0079] ​​S13. If no, a spectral feature extraction processing is performed on the sample hyperspectral remote sensing image by using the any linear spectral filter to obtain a sample spectral feature corresponding to each pixel element in the sample hyperspectral remote sensing image; in the embodiment, the sample hyperspectral remote sensing image is input into the any linear spectral filter to obtain an output response of each pixel element in the sample hyperspectral remote sensing image; then, a square error loss is introduced to process the response value, so as to obtain a final sample spectral feature of each pixel element.

[0080] In a specific application, the output response of the any linear spectral filter to the jth pixel element in the sample hyperspectral remote sensing image is: , wherein, in an ideal case, is a scalar, which is closer to 1, indicating that is closer to the target; meanwhile, represents a filter parameter of the any linear spectral filter at the tth iteration, that is, , and represents a real spectral feature vector at the tth iteration.

[0081] In the embodiment, the reason for introducing the square error loss to process the output response of each pixel element is to strengthen the accurate response of the model and improve the gradient expression ability of the model training; wherein, the processing formula of the square error loss is: , wherein, is the sample spectral feature of the jth pixel element.

[0082] Therefore, after obtaining the sample spectral feature of each pixel element in the sample hyperspectral remote sensing image based on the foregoing step S13, the real spectral feature vector can be updated, and the process is shown in the following step S14.

[0083] S14. Based on the sample spectral feature corresponding to each pixel element, the real spectral feature vector at the tth iteration is updated to obtain a real spectral feature vector at the t+1th iteration; in a specific application, the iteration update of the real spectral feature vector can be completed by using, for example but not limited to, the following steps S14a-S14c.

[0084] S14a. The gradient of each sample spectral feature with respect to the real spectral feature vector at the tth iteration is calculated to obtain a plurality of filter output response gradients, and the mean value of the plurality of filter output response gradients is taken as an update gradient.

[0085] In the embodiment, a learning mechanism is introduced to the real spectral feature vector, and the real spectral feature vector is updated through a neural network back propagation manner, wherein, as mentioned above, the pixel element in the sample hyperspectral remote sensing image , and the filter response is:

[0086] ;

[0087] Then, The formula for calculating the gradient of the real spectral feature vector is:

[0088] ;

[0089] In the formula, represents The gradient of the real spectral feature vector.

[0090] In this way, after introducing the square error loss, the gradient of the sample spectral feature of the jth pixel element in the sample hyperspectral remote sensing image to the real spectral feature vector at the tth iteration can be represented as:

[0091] ;

[0092] Based on this, in combination with the aforementioned The gradient of the real spectral feature vector, can be represented as:

[0093] .

[0094] Therefore, based on the aforementioned formula, after calculating the gradient of the real spectral feature vector at the tth iteration for each sample spectral feature, the mean of all the calculated gradients can be taken as the update gradient,

[0095] wherein the update gradient is:

[0096] .

[0097] After calculating the update gradient based on step S14a, the update parameter can be calculated, and the process is shown in the following step S14b.

[0098] S14b. Obtain a learning rate, and take the product between the update gradient and the learning rate as the update parameter.

[0099] After calculating the update parameter, the real spectral feature vector at the t+1th iteration can be obtained in combination with the real spectral feature vector at the tth iteration, and the process is shown in the following step S14c.

[0100] S14c. Calculate the difference between the real spectral feature vector at the tth iteration and the update parameter to obtain the real spectral feature vector at the t+1th iteration.

[0101] In this embodiment, the aforementioned update process of the true spectral feature vector can be summarized by the following formula:

[0102] In the formula, This represents the true spectral eigenvector at the (t+1)th iteration. This represents the learning rate.

[0103] After updating the true spectral feature vector through the aforementioned steps S14a to S14c, it can be substituted into the objective function to determine whether the iteration stopping condition is met. If not, the iteration update continues. The aforementioned iteration update process is shown in step S15 below.

[0104] S15. Increment t by 1, and re-obtain the true spectral feature vector of any linear spectral filter at the t-th iteration, until the iteration stopping condition is met, to obtain the optimal true spectral feature vector.

[0105] Therefore, through the aforementioned steps S11 to S15, the optimal true spectral feature vector for each linear spectral filter for the target category can be found by constructing the objective function, thereby constructing the final linear spectral filter.

[0106] After constructing multiple linear spectral filters, spectral feature extraction can be performed, as shown in step S2 below.

[0107] S2. Multiple linear spectral filters are used to extract spectral features from the hyperspectral remote sensing image to obtain multiple spectral features. These multiple spectral features are then stitched together to generate the spectral spatial coupling features of the hyperspectral remote sensing image. In this embodiment, it is equivalent to using each linear spectral filter to filter non-target pixels in the hyperspectral remote sensing image, thereby allowing the target of interest to be output through the filter.

[0108] For specific applications, see Figure 2 As shown, the hyperspectral remote sensing image is first input into each linear spectral filter to obtain the output response of each linear spectral filter; then, the output response of each linear spectral filter is processed by squared error loss, thereby obtaining multiple spectral features after squared error loss processing; among them, Figure 2 The result of target category spectral response enhancement (spectral feature extraction) corresponds to the output response of each linear spectral filter, while Figure 2 The images obtained after spectral feature extraction represent the extracted spectral features, which are obtained by applying squared error loss to the output response of each linear spectral filter. Finally, the spectral features are concatenated along the channel dimension to obtain the final image.Figure 2 spectral space coupling features in the hyperspectral remote sensing image.

[0109] Further, for any pixel element in the hyperspectral remote sensing image, the output response of the any linear spectral filter to the any pixel element is:

[0110] (2)

[0111] In formula (2), is the output response of the any linear spectral filter to the any pixel element, represents an optimal parameter vector of the any linear spectral filter, represents a full-channel spectral value of the any pixel element, wherein, , and represents a correlation matrix of the hyperspectral remote sensing image, represents an optimal real spectral feature vector of the any linear spectral filter to a target category; in the embodiment, the optimal real spectral feature vector is a real spectral feature vector of the target category in the hyperspectral remote sensing image. The calculation process of the optimal real spectral feature vector can be referred to the aforementioned formula (1), which is not described herein again.

[0112] Similarly, the process of the square error loss processing of any pixel element in the hyperspectral remote sensing image is the same as the process of the square error loss processing in the aforementioned parameter optimization process, that is:

[0113] For the output response of the any linear spectral filter to the any pixel element in the hyperspectral remote sensing image, the square error loss processing is performed on the output response of the any pixel element by using the following formula (3).

[0114] (3)

[0115] In formula (3), represents a spectral feature corresponding to the any pixel element.

[0116] In this way, each linear spectral filter sequentially processes each pixel element in the input hyperspectral remote sensing image to obtain a corresponding output response, and then the square error loss processing is performed, so that the spectral features of each pixel element in the hyperspectral remote sensing image can be obtained; then, the spectral features of each pixel element in the hyperspectral remote sensing image are used to form the spectral features of the entire image for each linear spectral filter; finally, the spectral features of the entire image for each linear spectral filter are spliced in the channel dimension, so that the spectral space coupling features of the hyperspectral remote sensing image can be obtained.

[0117] As shown in FIG. 1, Figure 3 Figure 3 ​The paper presents the spectral spatial coupling feature map obtained by extracting features from hyperspectral remote sensing images using four different sets of linear spectral filters; specifically, Figure 3 Figure (a) is a hyperspectral remote sensing image. Figure 3 Figures (b)-(e) in the figure represent the spectral spatial coupling feature maps obtained by extracting features from hyperspectral remote sensing images using four different sets of linear spectral filters. In this embodiment, the aforementioned four different sets of linear spectral filters refer to the fact that the optimal true spectral feature vectors of each filter in each set are different for the target category (a set of linear spectral filters contains K linear spectral filters, and the optimal true spectral feature vectors of the linear spectral filters within the same set are different, and the optimal true spectral feature vectors of each linear spectral filter in each set are also different from the optimal true spectral feature vectors of the linear spectral filters in the other sets). Thus, different true spectral feature vectors will result in different parameter vectors for each linear spectral filter (i.e., the aforementioned...). The differences in the linear spectral filters lead to variations in the output responses of each filter within each group. Therefore, by using four different groups of linear spectral filters, we can obtain... Figure 3 Figure (b) in the middle Figure 3 Figure (c) in the middle Figure 3 (d) diagram and Figure 3 The spectral spatial coupling feature map shown in Figure (e) is shown in the figure.

[0118] Thus, after the extraction of spectral spatial coupling features is completed, spatial texture features can be extracted, as shown in step S3 below.

[0119] S3. Perform texture feature extraction processing on the hyperspectral remote sensing image to obtain the spatial texture features of the hyperspectral remote sensing image; in this embodiment, see... Figure 2 As shown, in this embodiment, the hyperspectral remote sensing image is first subjected to two-dimensional convolution processing to obtain initial spatial texture features; then, a nonlinear activation function is used to perform nonlinear processing on the initial spatial texture features, thereby obtaining the spatial texture features after nonlinear processing.

[0120] In practical applications, for example, when performing two-dimensional convolution processing, the size of the convolution kernel used is... Furthermore, the stride and padding are both set to 1 to maintain the spatial dimensions. Simultaneously, after convolution, a nonlinear activation function is used to nonlinearly process the extracted features to enhance their expressive power, thereby obtaining spatial texture features.

[0121] Furthermore, spatial texture features can be represented as:

[0122] ;

[0123] wherein, represents a spatial texture feature, represents a two-dimensional convolution operation, represents a hyperspectral remote sensing image, represents a nonlinear activation function, which is BatchNorm + ReLU.

[0124] After obtaining the spatial texture feature, the spectral-spatial coupling feature can be combined to perform feature fusion, and based on the fused feature, the ground object recognition in the image is performed; wherein the feature fusion and ground object recognition process are shown in the following step S4.

[0125] S4. Based on the spectral-spatial coupling feature and the spatial texture feature, a fused feature of the hyperspectral remote sensing image is generated, and the fused feature is input into an image recognition model to obtain a ground object recognition result of the hyperspectral remote sensing image; in specific application, first, the spectral-spatial coupling feature and the spatial texture feature are feature spliced in the spectral channel dimension to obtain a spliced feature; then, the spliced feature is subjected to 1x1 convolution processing to obtain an initial fused feature; finally, the initial fused feature is subjected to nonlinear processing, so that after the nonlinear processing, the fused feature of the hyperspectral remote sensing image is obtained.

[0126] In specific implementation, the spliced feature (i.e. mixed feature map) is represented as:

[0127] ;

[0128] wherein, represents a spliced feature map, represents a feature splicing operation in the channel dimension, represents a spectral-spatial coupling feature, and B represents the number of convolution channels when the two-dimensional convolution operation is used to extract the spatial texture feature.

[0129] Similarly, the fused feature is represented as: , wherein, is a 1x1 convolution operation.

[0130] In this way, after obtaining the aforementioned fused feature, it is input into an image recognition model, so that the ground object recognition result of the hyperspectral remote sensing image can be obtained; wherein the image recognition model may, but is not limited to, use a Vision-Transformer model; of course, other neural network models can be selected according to actual use, which will not be described here.

[0131] In addition, the embodiment gives a comparison diagram of ground object recognition using the method and ground object recognition using traditional technology, as shown in Figure 4 Figure 4 ​The (a) figure in the figure is an original image (i.e. a hyperspectral remote sensing image), Figure 4 The (b) figure in the figure is a label image corresponding to the original image, Figure 4 The (c) figure in the figure represents a ground object recognition map obtained by a conventional technology (i.e. an extraction effect map without a spectral-spatial coupling mechanism), and Figure 4 The (d) figure in the figure represents a ground object recognition map obtained by using the method provided in the embodiment. Figure 4 By comparison, it can be seen that the ground object recognition effect of the method provided in the embodiment is obviously higher than that of the conventional technology, proving the improvement of the resolution capability and detection precision of the method for ground object recognition.

[0132] By the hyperspectral remote sensing image recognition method based on spectral-spatial feature coupling described in detail in the foregoing steps S1-S4, the present application creates a plurality of trainable linear spectral filters, and updates the parameters thereof by using the back propagation of the neural network, so as to output spectral-spatial coupling features of different categories by using the plurality of linear spectral filters after optimizing the optimal parameters of the foregoing filters, thereby suppressing the background interference in a complex environment. Then, the spectral-spatial coupling features are deeply fused with the spatial texture features extracted by convolution, and the features are analyzed and the target is recognized by combining the Vision-Transformer architecture, thereby obtaining the ground object recognition result in the image. Based on this, compared with the conventional technology, the present application does not need to perform spectral dimension reduction, thereby avoiding the problem that the conventional technology weakens the subtle spectral information and high-order spectral correlation contained in the hyperspectral data, and realizing the suppression of the background interference in a complex environment. Based on this, the accuracy and robustness of target recognition in a complex scene are improved, thereby being very suitable for large-scale application and promotion.

[0133] As shown in Figure 5 The second aspect of the present embodiment provides a hardware device for implementing the hyperspectral remote sensing image recognition method based on spectral-spatial feature coupling described in the first aspect of the embodiment, comprising:

[0134] A filter construction unit is configured to construct a plurality of linear spectral filters, wherein any linear spectral filter is constructed based on an optimal real spectral feature vector of a target category of the any linear spectral filter, which is obtained by optimizing the any linear spectral filter with the target category as the objective function of minimizing the background energy of the image, and the target category is the category corresponding to the ground object to be recognized in the hyperspectral remote sensing image.

[0135] A spectral feature extraction unit is configured to perform spectral feature extraction processing on the hyperspectral remote sensing image by using the plurality of linear spectral filters, to obtain a plurality of spectral features, and to perform feature splicing on the plurality of spectral features to generate spectral-spatial coupling features of the hyperspectral remote sensing image.

[0136] The texture feature extraction unit is configured to perform texture feature extraction processing on the hyperspectral remote sensing image to obtain spatial texture features of the hyperspectral remote sensing image.

[0137] The ground object recognition unit is configured to generate fusion features of the hyperspectral remote sensing image based on the spectral-spatial coupling features and the spatial texture features, and input the fusion features into the image recognition model to obtain a ground object recognition result of the hyperspectral remote sensing image.

[0138] The working process, working details and technical effects of the device provided in this embodiment can be referred to the first aspect of the embodiment, which will not be repeated here.

[0139] As shown in Figure 6 The third aspect of the embodiment provides another device for recognizing a hyperspectral remote sensing image based on spectral-spatial feature coupling. Taking the device as an electronic device, the device includes a memory, a processor and a transceiver connected in sequence. The memory is configured to store a computer program. The transceiver is configured to receive and send messages. The processor is configured to read the computer program and execute the method for recognizing a hyperspectral remote sensing image based on spectral-spatial feature coupling as described in the first aspect of the embodiment.

[0140] For example, the memory can include, but is not limited to, a random access memory (RAM), a read only memory (ROM), a flash memory, a first input first output (FIFO) memory and / or a first in last out (FILO) memory, etc. Specifically, the processor can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), a FPGA (Field-Programmable Gate Array) and a PLA (Programmable Logic Array). Meanwhile, the processor can include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state.

[0141] In some embodiments, the processor can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content required to be displayed on the display screen, for example, the processor can not be limited to a microprocessor of STM32F105 series, a RISC (reduced instruction set computer) microprocessor, an X86 architecture processor, or a processor integrated with an embedded NPU (neural-network processing unit); the transceiver can be but not limited to a WIFI transceiver, a Bluetooth transceiver, a GPRS (General Packet Radio Service) transceiver, a ZigBee transceiver, a 3G transceiver, a 4G transceiver, and / or a 5G transceiver, etc. In addition, the device can also include but not limited to a power module, a display screen, and other necessary components.

[0142] The working process, working details and technical effects of the electronic device provided in the embodiment can be referred to the first aspect of the embodiment, and will not be repeated here.

[0143] The fourth aspect of the embodiment provides a storage medium storing instructions of the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling described in the first aspect of the embodiment, that is, the storage medium stores instructions, and when the instructions run on a computer, the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling described in the first aspect of the embodiment is executed.

[0144] The storage medium refers to a carrier for storing data, which can include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk, and / or a Memory Stick, etc., and the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0145] The working process, working details and technical effects of the storage medium provided in the embodiment can be referred to the first aspect of the embodiment, and will not be repeated here.

[0146] The fifth aspect of the embodiment provides a computer program product containing instructions, which, when running on a computer, causes the computer to execute the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling described in the first aspect of the embodiment, wherein the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0147] It should be pointed out finally that the above only describes the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A hyperspectral remote sensing image recognition method based on spectral space feature coupling, characterized in that, The method comprises the following steps: Constructing a plurality of linear spectral filters, wherein any linear spectral filter is optimized to minimize the image background energy as the objective function, and the optimal real spectral feature vector of the target class of the any linear spectral filter is obtained based on the optimal real spectral feature vector constructed. Using a plurality of linear spectral filters to perform spectral feature extraction processing on the hyperspectral remote sensing image respectively to obtain a plurality of spectral features, and performing feature splicing on the plurality of spectral features to generate spectral space coupling features of the hyperspectral remote sensing image. Performing texture feature extraction processing on the hyperspectral remote sensing image to obtain spatial texture features of the hyperspectral remote sensing image. Based on the spectral space coupling features and the spatial texture features, generating fusion features of the hyperspectral remote sensing image, and inputting the fusion features into an image recognition model to obtain a ground object recognition result of the hyperspectral remote sensing image. Using a plurality of linear spectral filters to perform spectral feature extraction processing on the hyperspectral remote sensing image respectively to obtain a plurality of spectral features, comprising: Inputting the hyperspectral remote sensing image into each linear spectral filter to obtain an output response of each linear spectral filter. Performing square error loss processing on the output response of each linear spectral filter to obtain a plurality of spectral features. Based on the spectral space coupling features and the spatial texture features, generating fusion features of the hyperspectral remote sensing image, comprising: Performing feature splicing on the spectral space coupling features and the spatial texture features in the spectral channel dimension to obtain spliced features. Performing 1×1 convolution processing on the spliced features to obtain initial fusion features. Performing nonlinear processing on the initial fusion features to obtain the fusion features of the hyperspectral remote sensing image.

2. The method of claim 1, wherein, The independent variable of the objective function is the real spectral feature vector of the target class of the any linear spectral filter; Wherein, the optimal real spectral feature vector of the target class of the any linear spectral filter is optimized to minimize the image background energy as the objective function, comprising: Obtaining a sample hyperspectral remote sensing image and a real spectral feature vector of the any linear spectral filter at the tth iteration, wherein the initial value of t is 1, and when t is 1, the real spectral feature vector is an initial feature vector; Substituting the real spectral feature vector at the tth iteration into the objective function, and judging whether it meets the iteration stopping condition based on the function value of the objective function; If not, using the any linear spectral filter to perform spectral feature extraction processing on the sample hyperspectral remote sensing image to obtain a sample spectral feature corresponding to each pixel element in the sample hyperspectral remote sensing image; Based on the sample spectral feature corresponding to each pixel element, updating the real spectral feature vector at the tth iteration to obtain a real spectral feature vector at the (t+1) th iteration; Adding 1 to t, and reobtaining the real spectral feature vector of the any linear spectral filter at the tth iteration until the iteration stopping condition is met to obtain the optimal real spectral feature vector.

3. The method of claim 2, wherein, updating the real spectral feature vector at the tth iteration based on the sample spectral feature corresponding to each pixel element, to obtain a real spectral feature vector at a (t+1)th iteration, comprising: calculating the gradient of each sample spectral feature relative to the real spectral feature vector at the tth iteration to obtain a plurality of filter output response gradients, and taking the mean of the plurality of filter output response gradients as an update gradient; obtaining a learning rate, and taking the product of the update gradient and the learning rate as an update parameter; calculating the difference between the real spectral feature vector at the tth iteration and the update parameter to obtain the real spectral feature vector at the (t+1)th iteration.

4. The method of claim 2, wherein, The objective function is: （1） In formula (1), denotes a parameter vector of any linear spectral filter, denotes a correlation matrix of the sample hyperspectral remote sensing image, denotes a real spectral feature vector of the target class for any linear spectral filter, denotes a transposition operation, denotes a linear equality constraint condition of the target function; in, , In the formula, Indicating the first hyperspectral remote sensing image of the sample The full-channel spectral values ​​of each pixel, and This represents the total number of pixels in the sample hyperspectral remote sensing image.

5. The method of claim 1, wherein, For any pixel element in the hyperspectral remote sensing image, the output response of the any linear spectral filter to the any pixel element is: （2） In formula (2), an output response of the any linear spectral filter to the any pixel element, an optimal parameter vector of the any linear spectral filter, a full-channel spectral value of the any pixel element, wherein, , and a correlation matrix of the hyperspectral remote sensing image, an optimal real spectral feature vector of the any linear spectral filter to a target class; Correspondingly, the output response of each linear spectral filter is subjected to square error loss processing to obtain a plurality of spectral features, which comprises: For the output response of the any linear spectral filter to the any pixel element in the hyperspectral remote sensing image, the output response of the any pixel element is subjected to square error loss processing using the following formula (3); （3） In formula (3), represents the spectral feature corresponding to any pixel element.

6. The method of claim 1, wherein, performing texture feature extraction processing on the hyperspectral remote sensing image to obtain spatial texture features of the hyperspectral remote sensing image, comprising: performing two-dimensional convolution processing on the hyperspectral remote sensing image to obtain initial spatial texture features; using a nonlinear activation function to perform nonlinear processing on the initial spatial texture features to obtain the spatial texture features after nonlinear processing.

7. A hyperspectral remote sensing image recognition device based on spectral space feature coupling, characterized in that, The device for performing the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling according to any one of claims 1-6, wherein the device comprises: a filter construction unit configured to construct a plurality of linear spectral filters, wherein any linear spectral filter is constructed based on an optimal real spectral feature vector of a target category that is optimized by taking minimization of image background energy as an objective function, and the target category is a category corresponding to a ground object to be identified in the hyperspectral remote sensing image; a spectral feature extraction unit configured to perform spectral feature extraction processing on the hyperspectral remote sensing image using the plurality of linear spectral filters to obtain a plurality of spectral features, and to perform feature splicing on the plurality of spectral features to generate spectral spatial coupling features of the hyperspectral remote sensing image; a texture feature extraction unit configured to perform texture feature extraction processing on the hyperspectral remote sensing image to obtain spatial texture features of the hyperspectral remote sensing image; a ground object identification unit configured to generate fusion features of the hyperspectral remote sensing image based on the spectral spatial coupling features and the spatial texture features, and to input the fusion features into an image recognition model to obtain a ground object identification result of the hyperspectral remote sensing image.

8. A hyperspectral remote sensing image recognition device based on spectral space feature coupling, characterized in that, comprise: a memory, a processor and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to transmit and receive messages, and the processor is configured to read the computer program and execute the hyperspectral remote sensing image recognition method based on spectral spatial feature coupling according to any one of claims 1-6.

Images