Hyperspectral remote sensing image target detection method based on target-background reconstruction bias

Abstract

The application discloses a hyperspectral remote sensing image target detection method based on target-background reconstruction deviation, comprising the following steps: acquiring a to-be-detected hyperspectral remote sensing image, inputting the pre-trained asymmetric auto-encoding network, and obtaining the first output and the second output of each pixel in the image; the pre-trained asymmetric auto-encoding network comprises a feature extraction subnetwork, a feature fusion subnetwork and a feature reconstruction subnetwork; the first output is the output of the feature extraction subnetwork, and the second output is the output of the feature reconstruction subnetwork; the pre-trained asymmetric auto-encoding network is obtained by training a mixed target spectrum and a mixed background spectrum; the mixed target spectrum and the mixed background spectrum are generated based on a first hyperspectral remote sensing sample image, a prior spectrum of a preset target and a bilinear spectrum mixing model; according to the first output and the second output, the spectral angle distance of the pixel is determined; and the spectral angle distance is smoothed to obtain the detection result of the to-be-detected hyperspectral remote sensing image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning, and particularly relates to a hyperspectral remote sensing image target detection method based on target-background reconstruction bias. BACKGROUND

[0002] With the development of sensor technology, digital electronic technology and hardware computing resources, the development of hyperspectral remote sensing technology is greatly promoted. Hyperspectral remote sensing technology can simultaneously capture the spectral and spatial characteristics of ground objects, providing more abundant information for the accurate detection of targets of interest in complex backgrounds, and has attracted widespread attention in many fields such as ocean observation, search and rescue, and city planning, and has very important theoretical research significance and practical application value.

[0003] Under the condition that the spectral information of the target of interest is known in advance, the hyperspectral remote sensing image target detection aims to quantitatively interpret and analyze the to-be-detected hyperspectral remote sensing image, and then judge the probability of the existence of the target of interest in each to-be-detected pixel. Influenced by atmospheric conditions, ground object composition, sensor noise and other uncertain factors, spectral distortion phenomenon in the hyperspectral remote sensing image is inevitable, which specifically manifests as the difference between the spectral characteristics of the same ground object and the similarity between the spectral characteristics of different ground objects. This phenomenon seriously limits the accuracy of spectral data interpretation and analysis.

[0004] In view of this problem, existing work has carried out research from aspects of statistical analysis, kernel function, machine learning and deep learning. Among them, the deep learning technology shows great potential in the hyperspectral remote sensing image target detection task due to its strong discriminant feature extraction capability. At present, the research on the hyperspectral remote sensing image target detection based on deep learning mainly develops from three aspects of labeled spectrum simulation, network structure design and knowledge effective migration. In the aspect of spectrum simulation, the existing method is usually based on a linear spectral mixing model, which is difficult to accurately describe the ground object spectral characteristics in a real scene; in the aspect of network structure design, in order to more thoroughly mine the potential effective features of the hyperspectral remote sensing image, the model structure is complex, the data processing efficiency is low, and the requirement for computing resource configuration is high, such as the method "Htd-net: A deep convolutional neural network for target detection in hyperspectral imagery" (HTD-Net) proposed by Zhang et al.; in the aspect of knowledge effective migration, the data distribution offset problem under different sensors and different collection conditions has not been fully considered, so that the performance of the model is greatly reduced when detecting new hyperspectral remote sensing images, such as the method "Transferred deep learning for hyperspectral target detection" (CNND) proposed by Li et al. SUMMARY

[0005] In order to solve the above problems existing in the related art, the present application provides a hyperspectral remote sensing image target detection method based on target-background reconstruction deviation.

[0006] The present application provides a hyperspectral remote sensing image target detection method based on target-background reconstruction deviation, comprising:

[0007] Obtaining a hyperspectral remote sensing image to be detected;

[0008] Inputting the hyperspectral remote sensing image to be detected into a pre-trained asymmetric auto-encoding network to obtain a first output and a second output corresponding to each pixel in the hyperspectral remote sensing image to be detected; the pre-trained asymmetric auto-encoding network comprises a pre-trained feature extraction subnetwork, a pre-trained feature fusion subnetwork and a pre-trained feature reconstruction subnetwork; the input of the feature fusion subnetwork is the output of the feature extraction subnetwork, and the input of the feature reconstruction subnetwork is the output of the feature fusion subnetwork; the first output is the output of the feature extraction subnetwork, and the second output is the output of the feature reconstruction subnetwork; the pre-trained asymmetric auto-encoding network is trained by using a plurality of mixed target spectra and a plurality of mixed background spectra; the plurality of mixed target spectra and the plurality of mixed background spectra are generated based on a first hyperspectral remote sensing sample image, a prior spectrum of a preset target and a bilinear spectral mixing model;

[0009] According to the first output and the second output, the spectral angle distance of the pixel is determined;

[0010] Through smoothing processing of the spectral angle distance of the pixel, a detection result of whether the hyperspectral remote sensing image to be detected contains the preset target is obtained.

[0011] The present application has the following beneficial technical effects:

[0012] By adopting the bilinear spectral mixing model containing linear terms and bilinear terms to generate target spectra and background spectra, compared with the data simulation strategy based on the linear spectral mixing model of the prior art, the bilinear spectral mixing model can accurately describe the nonlinear interaction between features, is more in line with the characteristics of real data, and thus the generated target spectra and background spectra are more accurate; and by migrating the effective feature extraction capability learned in the hyperspectral remote sensing image classification task to the hyperspectral remote sensing image target detection task, the performance drop problem in the model migration process is relieved by training on the target detection domain, so that the target detection result is more accurate; in addition, the asymmetric auto-encoding network has a simple structure, which greatly reduces the network complexity. Therefore, by the method provided by the present application, accurate and efficient target detection of the hyperspectral remote sensing image can be realized.

[0013] The present application will be further described in detail below with reference to the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A flowchart of the hyperspectral remote sensing image target detection method based on target-background reconstruction deviation provided by the embodiment of the present application;

[0015] Figure 2 A structural schematic diagram of an exemplary asymmetric auto-encoding network provided by the embodiment of the present application;

[0016] Figure 3 A structural schematic diagram of each multi-scale residual convolution block provided by the embodiment of the present application;

[0017] Figure 4 A structural schematic diagram of each residual attention block provided by the embodiment of the present application;

[0018] Figure 5 A structural schematic diagram of each residual deconvolution block provided by the embodiment of the present application;

[0019] Figure 6A An exemplary pseudo-color image of the hyperspectral remote sensing image with pixel labels collected by the airborne imaging spectrometer AVIRIS provided by the embodiment of the present application;

[0020] Figure 6B An exemplary distribution of 16 types of features contained in the hyperspectral remote sensing image with pixel labels collected by the airborne imaging spectrometer AVIRIS provided by the embodiment of the present application;

[0021] Figure 7A An exemplary pseudo-color image of the hyperspectral remote sensing image to be detected collected by the airborne imaging spectrometer SIG.MA provided by the embodiment of the present application;

[0022] Figure 7B The distribution of vehicles contained in the hyperspectral remote sensing image to be detected, acquired by the exemplary airborne imaging spectrometer SIG.MA in this embodiment of the invention;

[0023] Figure 7C The figure shows exemplary experimental results using the HTD-Net method provided for embodiments of the present invention.

[0024] Figure 7D An exemplary experimental result diagram using the CNND method is provided for an embodiment of the present invention;

[0025] Figure 7E The figure shows an exemplary experimental result of using the method of the present invention, provided as an embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0027] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0029] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0030] Figure 1 This is a flowchart of a hyperspectral remote sensing image target detection method based on target-background reconstruction bias provided in an embodiment of the present invention, as follows: Figure 1 As shown, the method includes the following steps:

[0031] S101. Acquire the hyperspectral remote sensing image to be detected.

[0032] In this embodiment of the invention, the number of hyperspectral remote sensing images to be detected can be one or more, without limitation. The hyperspectral remote sensing image to be detected can be an image taken of any area, such as farmland, urban landscapes, etc. The size of the hyperspectral remote sensing image I to be detected can be n. h ×n w ×n λ , where n h n w n λ These represent the width, height, and number of bands of the hyperspectral remote sensing image I to be detected, respectively.

[0033] For example, the hyperspectral remote sensing image I to be detected can be acquired by an airborne imaging spectrometer SIG.MA, with a size of 375×450×110.

[0034] S102. Input the hyperspectral remote sensing image to be detected into a pre-trained asymmetric autoencoder network to obtain the first output and the second output corresponding to each pixel in the hyperspectral remote sensing image to be detected. The pre-trained asymmetric autoencoder network includes: a pre-trained feature extraction sub-network, a pre-trained feature fusion sub-network, and a pre-trained feature reconstruction sub-network. The input of the feature fusion sub-network is the output of the feature extraction sub-network, and the input of the feature reconstruction sub-network is the output of the feature fusion sub-network. The first output is the output of the feature extraction sub-network, and the second output is the output of the feature reconstruction sub-network. The pre-trained asymmetric autoencoder network is trained using multiple mixed target spectra and multiple mixed background spectra. The multiple mixed target spectra and multiple mixed background spectra are generated based on the first hyperspectral remote sensing sample image, the prior spectrum of the preset target, and the bilinear spectral mixing model.

[0035] Here, the feature extraction subnetwork includes: a first convolutional processing module and N multi-scale residual convolutional blocks. The feature fusion subnetwork includes: N residual attention blocks and a second convolutional processing module. The feature reconstruction subnetwork includes: N residual deconvolutional blocks; the output of the feature extraction subnetwork includes: the outputs of N residual attention blocks, where N is an integer greater than or equal to 3. For example, as... Figure 2 As shown, N is 3.

[0036] Here, the first convolutional processing module includes: convolutional layers, batch normalization layers, and non-linear activation layers. Specifically, as follows... Figure 2 As shown, the first layer of the first convolution processing module is a one-dimensional convolution with a kernel size of 1, followed by a batch normalization operation layer and a non-linear activation layer (ReLU function).

[0037] Here, each multi-scale residual convolutional block includes: a first sub-convolution processing module, a second sub-convolution processing module, a third sub-convolution processing module, a fourth sub-convolution processing module, and a first summation processing module; wherein, the first and second sub-convolution processing modules have the same input, the output of the first sub-convolution processing module serves as the input of the third sub-convolution processing module, the output of the third sub-convolution processing module serves as the input of the fourth sub-convolution processing module, and the outputs of the fourth and second sub-convolution processing modules, together, serve as the input of the first summation processing module. Specifically, as shown... Figure 3 As shown, the first sub-convolution processing module sequentially includes: a one-dimensional convolution with a kernel size of 1, a batch normalization operation layer, and a non-linear activation layer (ReLU function); the second sub-convolution processing module sequentially includes: a one-dimensional convolution with a kernel size of 1 and a batch normalization operation layer; the third sub-convolution processing module includes three sub-modules with different kernel sizes, and the kernel sizes are K1, K2, and K3 (K1, K2, and K3 are three different preset values), each sub-module is followed by a batch normalization operation layer and a non-linear activation layer (ReLU function) after the one-dimensional convolution; the fourth sub-convolution processing module sequentially includes: a summation operation layer, a one-dimensional convolution with a kernel size of 1, and a batch normalization operation layer; the first summation processing module sequentially includes: a summation operation layer and a non-linear activation layer (ReLU function).

[0038] Here, each residual attention block sequentially includes: a pooling layer, a mapping activation layer, a summation layer, a multiplication layer, and a convolution layer; wherein the input of the pooling layer and the output of the summation layer are used together as the input of the multiplication layer. Each residual deconvolution block includes: a first deconvolution processing module, a second deconvolution processing module, a third deconvolution processing module, a fourth deconvolution processing module, and a second summation processing module; wherein the first and second deconvolution processing modules have the same input, the output of the first deconvolution processing module is used as the input of the third deconvolution processing module, the output of the third deconvolution processing module is used as the input of the fourth deconvolution processing module, and the output of the fourth deconvolution processing module and the output of the second deconvolution processing module are used together as the input of the second summation processing module. Specifically, as shown... Figure 4 As shown, the pooling layer is an average pooling layer; the mapping activation layer consists of two sub-layers, each of which includes a fully connected layer (with n′ neurons). λ ′ / r, where r is a preset value, n′ λ ′ is to make n λ The value n obtained after processing by a one-dimensional convolutional layer with a kernel size of 1 in the first convolutional processing module. λ The second sub-layer has a number of bands in the hyperspectral remote sensing image to be detected and a non-linear activation layer (Sigmoid activation function). The output of the second sub-layer and the value 1 are used as inputs to the summation operation layer. The convolutional layer is a one-dimensional convolution with a kernel size of 1.

[0039] Here, the second convolution processing module includes a summation layer and a convolutional layer. Specifically, as described above. Figure 2 As shown, the second convolution processing module includes a summation operation layer and a one-dimensional convolution with a kernel size of 1.

[0040] Here, each residual deconvolution block includes: a first deconvolution processing module, a second deconvolution processing module, a third deconvolution processing module, a fourth deconvolution processing module, and a second summation processing module; wherein, the first and second deconvolution processing modules have the same input, the output of the first deconvolution processing module serves as the input of the third deconvolution processing module, the output of the third deconvolution processing module serves as the input of the fourth deconvolution processing module, and the outputs of the fourth and second deconvolution processing modules are jointly used as the input of the second summation processing module. Specifically, as shown... Figure 5As shown, the first deconvolution processing module sequentially includes: a one-dimensional deconvolution with a kernel size of 1, a batch normalization operation layer, and a non-linear activation layer (ReLU function); the second deconvolution processing module sequentially includes: a one-dimensional deconvolution with a kernel size of 1 and a batch normalization operation layer; the third deconvolution processing module includes: a one-dimensional deconvolution with a kernel size of K (K is a preset value, and its meaning is different from that of K in the aforementioned training times), a batch normalization operation layer, and a non-linear activation layer (ReLU function); the fourth deconvolution processing module sequentially includes: a one-dimensional deconvolution with a kernel size of 1 and a batch normalization operation layer; the second summation processing module sequentially includes: a summation operation layer and a non-linear activation layer (ReLU function).

[0041] In some embodiments, the first hyperspectral remote sensing sample image may be an image belonging to the same image category as the hyperspectral remote sensing image to be detected. For example, both the first hyperspectral remote sensing sample image and the hyperspectral remote sensing image to be detected may be images of farmland, or both may be images of grassland, and so on. The number of first hyperspectral remote sensing sample images may be one or more, without limitation.

[0042] Here, the preset target can be one or more objects, such as a car, an airplane, a rice paddy, etc.

[0043] Here, the size of the prior spectrum of the preset target can be n. λ ×n t , where n λ n represents the number of bands in the prior spectrum of each preset target. t This indicates the number of prior spectra of the preset target. The prior spectra of the preset target can be selected from a public spectral library or collected in the field using a ground-based spectrometer.

[0044] In some embodiments, the size of the first hyperspectral remote sensing sample image can also be n. h ×n w ×n λ .

[0045] For example, the first hyperspectral remote sensing sample image can be acquired by an airborne imaging spectrometer SIG.MA with a size of 375×450×110, and the prior spectrum of the preset target can be acquired by a handheld spectroradiometer FieldSpec with a size of 110×2.

[0046] In some embodiments, prior to S102 above, the method further includes: S001 to S004:

[0047] S001. Acquire the first hyperspectral remote sensing sample image and the prior spectrum of the preset target, respectively.

[0048] S002. Based on the prior spectrum and the automatic endmember extraction method, extract the background spectrum from the first hyperspectral remote sensing sample image; the background spectrum is different from the prior spectrum.

[0049] Here, after obtaining the prior spectrum, the background spectrum b, which is significantly different from the prior spectrum t, can be extracted from the first hyperspectral remote sensing sample image by combining the prior spectrum t of the target of interest with the automatic endmember extraction method (ATGP).

[0050] Specifically, the number of background spectra to be extracted can be initialized to n. b The prior spectrum is used as the initial endmember spectrum; n b The value is an integer greater than 1. In the Kth iteration, an orthogonal subspace projector is constructed based on the identity matrix and the endmember spectra obtained in the (K-1)th iteration. When K = 1, the endmember spectra obtained in the (K-1)th iteration are the initial endmember spectra. The identity matrix is ​​determined based on the prior spectrum. Using the Kth orthogonal subspace projector, each pixel in the first hyperspectral remote sensing sample image is projected onto the orthogonal subspace, obtaining the magnitude of each pixel after the Kth projection. The pixel corresponding to the maximum value among the magnitudes after the Kth projection is used as the background spectrum extracted in the Kth iteration and the endmember spectrum obtained in the Kth iteration, respectively. This process is repeated until the background spectrum extracted in the nth iteration and the endmember spectrum obtained in the nth iteration are obtained. b The endmember spectrum obtained in the next iteration.

[0051] For example, the background spectrum can be extracted using the following steps:

[0052] (1a1) Initialize the background spectrum b (0) [ ] (empty), endmember spectrum u (0) Let be the prior spectrum t, and let n be the number of background spectra to be extracted. b The initial iteration count K = 1; where the HSC method can be used to determine the number of background categories present in the first hyperspectral remote sensing sample image, and the determined number of background categories is used as n. b n can also be set according to actual needs. b ;

[0053] (1a2) In the Kth iteration, construct the orthogonal subspace projector:

[0054] Where E represents a size of n λ ×n λ The identity matrix, u (K-1) This represents the endmember spectrum in the (K-1)th iteration, where T is the transpose. This represents the orthogonal subspace projector constructed during the Kth iteration;

[0055] (1a3) Project each pixel in the first hyperspectral remote sensing sample image onto an orthogonal subspace, and select the pixel with the largest modulus value after projection as the background: Among them, I j This represents the spectral vector of the j-th pixel in the first hyperspectral remote sensing sample image (i.e., equivalent to a 1×1×n vector). λ (vector), b (K) This represents the background spectrum extracted in the Kth iteration;

[0056] (1a4) Update the endmember spectrum to [u (K-1) ,b (K) ], background spectrum is [b (K-1) ,b (K) If K <n b If the iteration count is K = K + 1, proceed to step (1a2); otherwise, output the background spectrum.

[0057] S003. Based on the prior spectrum and background spectrum, a bilinear spectral mixing model is used to generate multiple mixed target spectra and multiple mixed background spectra.

[0058] In this embodiment of the invention, the number of prior spectra is n. t The number of background spectra is n b ;n t and n b All are integers greater than 2; multiple mixed target spectra and multiple mixed background spectra can be generated using steps S1 to S2:

[0059] S1, each time from n t Select one prior spectrum from n prior spectra, and then, based on the selected prior spectrum and n... b Based on the background spectrum and the first preset abundance value, n are generated. b A mixed target spectrum, until n are traversed. t When there are n prior spectra, t The generated mixed target spectrum is used as multiple generated mixed target spectra.

[0060] S2, each time from n b One background spectrum is selected from the background spectra. Based on the selected background spectrum, the second preset abundance value, and n... b From the background spectra whose index follows the selected background spectrum, at least one mixed background spectrum is generated, until all n background spectra have been traversed. b When there are background spectra, n t -1 times the generated mixed background spectrum is used as the multiple generated mixed background spectra.

[0061] Here, S1 and S2 can be executed sequentially or simultaneously, without any restriction.

[0062] Here, the mixed target spectrum is expressed by the following formula:

[0063] T α,m,n = α1t m + (1 - α1)b n + α1(1 - α1)t m ⊙ b n ;

[0064] where, T α,m,n represents the mixed target spectrum, t m represents one prior spectrum selected from n t prior spectra each time, m = 1, …, n t , b n represents n b background spectra, n = 1, …, n b , α1 represents the first preset abundance value, and ⊙ represents element-wise multiplication operation.

[0065] Here, the mixed background spectrum is expressed by the following formula:

[0066]

[0067] where,[[]] represents the mixed background spectrum, represents one background spectrum selected from n b background spectra each time, n1 = 1, …, n b , represents the background spectra with serial numbers after b among n background spectra, n1 < n2, and α2 represents the second preset abundance value.

[0068] S004. Use multiple mixed target spectra and multiple mixed background spectra to train an asymmetric auto-encoding network including a pre-trained feature extraction sub-network, an initial feature fusion sub-network, and an initial feature reconstruction sub-network, and obtain a pre-trained asymmetric auto-encoding network.

[0069] Here, when training the asymmetric auto-encoding network each time, the network parameters of the pre-trained feature extraction sub-network can be not adjusted, and the network parameters of the initial feature fusion sub-network and the initial feature reconstruction sub-network are adjusted. Thus, an asymmetric auto-encoding network including a pre-trained feature extraction sub-network, a pre-trained feature fusion sub-network, and a pre-trained feature reconstruction sub-network is obtained, that is, a pre-trained asymmetric auto-encoding network is obtained.

[0070] Here, during the i-th training iteration, the mixed target spectrum and mixed background spectrum are selected from multiple mixed target spectra and multiple mixed background spectra for training. These are then input into the asymmetric autoencoder network after the (i-1)-th training iteration to obtain the output of the i-th feature extraction sub-network and the output of the i-th feature reconstruction sub-network. The asymmetric autoencoder network after the (i-1)-th training iteration includes: a pre-trained feature extraction sub-network, a feature fusion sub-network after the (i-1)-th training iteration, and a feature reconstruction sub-network after the (i-1)-th training iteration; i is an integer greater than or equal to 1. Based on the output of the i-th feature extraction sub-network and the output of the i-th feature reconstruction sub-network, the... Determine the spectral angular distance for the i-th iteration and obtain the loss value for the i-th iteration. Based on the loss value for the i-th iteration, update the feature fusion subnetwork and the feature reconstruction subnetwork after the (i-1)-th iteration of training to obtain the asymmetric autoencoder network after the i-th iteration of training. Iterate in this way until the asymmetric autoencoder network after the (i+p)-th iteration of training is obtained; p is an integer greater than 1. The asymmetric autoencoder network after the i-th iteration of training includes: a pre-trained feature extraction subnetwork, a feature fusion subnetwork after the i-th iteration of training, and a feature reconstruction subnetwork after the i-th iteration of training. Use the asymmetric autoencoder network after the (i+p)-th iteration of training as the pre-trained asymmetric autoencoder network.

[0071] Specifically, during the i-th training iteration, the mixed target spectrum and mixed background spectrum used for training can be input together into the first convolutional processing module of the asymmetric autoencoder network after the (i-1)-th training iteration for processing. The output of the first convolutional processing module is input into the j-th multi-scale residual convolutional block of the asymmetric autoencoder network after the (i-1)-th training iteration. After processing the input, the j-th multi-scale residual convolutional block inputs its output into the (j+1)-th multi-scale residual convolutional block and the N-th residual attention block, respectively; j is 1. After processing the input, the (j+1)-th multi-scale residual convolutional block inputs its output into the (j+2)-th multi-scale residual convolutional block and the (N-1)-th residual attention block, respectively; the (j+2)-th multi-scale residual convolutional block inputs its output into the (j+2)-th multi-scale residual convolutional block and the (N-1)-th residual attention block, respectively. After processing the input, the multi-scale residual convolutional block processes the input and then inputs the output into the (j+3)th multi-scale residual convolutional block and the (N-2)th residual attention block, respectively. This process continues until the Nth multi-scale residual convolutional block processes the input and then inputs the output into the first residual attention block. The outputs of the N residual attention blocks are then fed into the second convolutional processing module for processing. The second convolutional processing module then inputs the output into the kth residual deconvolutional block for processing. The kth residual deconvolutional block inputs its output into the (k+1)th residual deconvolutional block for processing, and so on, until the (N-1)th residual deconvolutional block inputs its output into the Nth residual deconvolutional block for processing, resulting in the output of the Nth residual deconvolutional block; where k is 1. For example, as described above. Figure 2As shown, when N is 3 and i = 1, the mixed target spectrum and mixed background spectrum used for training are jointly input into the first convolutional processing module in the asymmetric autoencoder network for processing. The first convolutional processing module inputs its output into the first multi-scale residual convolutional block. After processing the input, the first multi-scale residual convolutional block inputs its output into the second multi-scale residual convolutional block and the first residual attention block, respectively. Then, after processing the input, the second multi-scale residual convolutional block inputs its output into the third multi-scale residual convolutional block, respectively. The first residual attention block processes the input and outputs it to the second residual attention block. The second residual attention block processes the input and outputs it to the first residual attention block. Then, the outputs of the N residual attention blocks are fed into the second convolution processing module for processing. The output of the second convolution processing module is fed into the first residual deconvolution block for processing. The output of the first residual deconvolution block is fed into the second residual deconvolution block for processing. The output of the second residual deconvolution block is fed into the third residual deconvolution block for processing, resulting in the output of the third residual deconvolution block.

[0072] Here, for the i-th training iteration, the output of the feature reconstruction sub-network in the i-th iteration includes: N outputs corresponding to N multi-scale residual convolutional blocks; the output of the feature reconstruction sub-network in the i-th iteration includes: N outputs corresponding to N residual deconvolutional blocks. Based on this, the loss value for the i-th iteration can be calculated using the following formula:

[0073]

[0074] Where L is the loss value in the i-th iteration, s represents each training sample, s = 0 indicates that the training sample is a mixed background spectrum, s = 1 indicates that the training sample is a mixed target spectrum, |s| represents the total number of training samples used in the i-th training iteration, c represents the band dimension, d represents all elements in the channel dimension, and q is 1, 2, ..., N-1, N. Let z represent the q-th first output (i.e., the output of the q-th multi-scale residual convolutional block) corresponding to the training sample s, where z is N, N-1, ..., 2, 1. Let T represent the z-th second output (i.e., the output of the z-th residual deconvolution block) corresponding to the training sample s, where T is the transpose symbol and ||.|| represents the norm symbol.

[0075] For example, an asymmetric autoencoder network can be trained 100 times with an initial learning rate of lr = 0.005. After each training iteration, the relationship between the current loss value and the previous loss value can be determined. When the loss values ​​of two adjacent training iterations no longer decrease, the learning rate can be updated to half of the current learning rate. Then, training continues using the updated learning rate until training ends. Furthermore, when updating the network parameters of the feature fusion subnetwork and the feature reconstruction subnetwork, the Adam optimizer can be used for the update.

[0076] In some embodiments, S0001 to S0003 may be included before S004 above:

[0077] S0001. Obtain a second hyperspectral remote sensing sample image with pixel labels; the pixel label of each pixel in the second hyperspectral remote sensing sample image represents the target category to which the pixel belongs.

[0078] Here, the second hyperspectral remote sensing sample image with pixel tags can be one or more images, and can be selected from a public hyperspectral remote sensing image library, or can be a hyperspectral remote sensing image with pixel-level tags acquired in the field using an imaging spectrometer.

[0079] Here, the size of the second hyperspectral remote sensing sample image with pixel labels can be n. h ′×n′ w ×n′ λ n h ′,n′ w ,n′ λ These represent the image width, height, and number of bands, respectively. For example, the second hyperspectral remote sensing sample image with pixel labels was acquired by an airborne visible / near-infrared imaging spectrometer and has a size of 512×217×204.

[0080] S0002. The hyperspectral classification network, which includes the initial feature extraction sub-network and the initial classifier, is trained using the second hyperspectral remote sensing sample image. After each training, the loss value is determined based on the cross-entropy between the prediction result of each pixel in the obtained second hyperspectral remote sensing sample image and the pixel label corresponding to that pixel. The parameters of the hyperspectral classification network are updated based on the loss value. Through multiple iterations of training, the pre-trained hyperspectral classification network is obtained.

[0081] Here, the classifier can include two fully connected layers. For example, the first fully connected layer can have 100 neurons, and the second fully connected layer can have the number of neurons equal to the number of categories in the pixel-level labeled hyperspectral remote sensing image, for example, 16. Based on this, during each training iteration of the asymmetric autoencoder network, the prediction result for each pixel can be...

[0082] For example, a hyperspectral classification network can be trained 100 times with an initial learning rate of lr = 0.005. After each training iteration, the relationship between the current loss value and the previous loss value can be determined. When the loss values ​​of two adjacent training iterations no longer decrease, the learning rate can be updated to half of the current learning rate. Then, training can continue using the updated learning rate until training ends. Furthermore, when updating the network parameters of the feature extraction subnetwork and the classifier, the Adam optimizer can be used for the update.

[0083] S0003. Use the feature extraction subnetwork in the pre-trained hyperspectral classification network as the pre-trained feature extraction subnetwork.

[0084] Here, when the pre-trained hyperspectral classification network is obtained, the pre-trained classifier can be removed, and the trained feature extraction sub-network can be retained.

[0085] S103. Determine the spectral angular distance of the pixel based on the first output and the second output.

[0086] Here, the first output includes N first outputs; the second output includes N second outputs; the spectral angular distance of each pixel is calculated using the following formula:

[0087]

[0088] Where D' represents the spectral angular distance of each pixel, and q is 1, 2, ..., N-1, N. Let c represent the band dimension, d represent all elements in the channel dimension, and z be N, N-1, ..., 2, 1. This indicates the z-th second output.

[0089] S104. By smoothing the spectral angular distance of the pixels, the detection result of whether the hyperspectral remote sensing image to be detected contains the preset target is obtained.

[0090] Here, a Gaussian filter can be used to smooth the spectral angular distance of each pixel to obtain the confidence level of that pixel. The confidence level is used to characterize whether the preset target is contained. When there are pixels in the hyperspectral remote sensing image to be detected with a confidence level greater than or equal to the preset threshold, the detection result of the hyperspectral remote sensing image to be detected is that it contains the preset target. When the confidence levels of all pixels in the hyperspectral remote sensing image to be detected are less than the preset threshold, the detection result of the hyperspectral remote sensing image to be detected is that it does not contain the preset target.

[0091] Here, when there are pixels in the hyperspectral remote sensing image to be detected that have a confidence level greater than or equal to a preset threshold, the position of the preset target in the hyperspectral remote sensing image to be detected can be determined based on the position of these pixels in the hyperspectral remote sensing image to be detected.

[0092] This invention, through the use of a bilinear spectral mixture model containing linear and bilinear terms to generate target and background spectra, achieves a more accurate representation of nonlinear interactions between ground features compared to existing data simulation strategies based on linear spectral mixture models. This better aligns with real-world data characteristics, resulting in more accurate target and background spectra. Furthermore, by transferring effective feature extraction capabilities learned in hyperspectral remote sensing image classification tasks to target detection tasks, and training on the target detection domain, the performance degradation during model transfer is mitigated, leading to more accurate target detection results. Additionally, the use of an asymmetric autoencoder network simplifies the structure and significantly reduces network complexity. Therefore, the method provided by this invention enables accurate and efficient target detection in hyperspectral remote sensing images.

[0093] The technical effects of the method provided in the embodiments of the present invention will be further explained below through experimental data.

[0094] (1) Experimental equipment:

[0095] The experimental hardware environment for this invention is a workstation equipped with an NVIDIA GeForce GTX Titan Xp GPU, and the software environment is Python and the PyTorch framework.

[0096] (2) Experimental content and results analysis:

[0097] Experimental content: The method of this invention is used to optimize the feature extraction sub-network on the data acquired by the airborne imaging spectrometer AVIRIS. The method of this invention, along with existing deep learning-based hyperspectral remote sensing image target detection methods HTD-Net and CNND, are used to conduct target detection experiments on the data acquired by the airborne imaging spectrometer SIG.MA.

[0098] Reference Figures 6A-6B The hyperspectral classification network in the method of this invention is trained to obtain a trained feature extraction sub-network model. Figure 6A A pseudo-color image (of farmland) of pixel-tagged hyperspectral remote sensing imagery acquired by the AVIRIS airborne imaging spectrometer. Figure 6B This image shows the distribution of 16 land cover types in a pixel-tagged hyperspectral remote sensing image acquired by the AVIRIS airborne imaging spectrometer. The white area represents the non-study area.

[0099] Reference Figures 7A-7B The target detection experiments conducted using the method of this invention are described in detail. Figure 7A This is a pseudo-color image of the hyperspectral remote sensing image to be tested, acquired by the airborne imaging spectrometer SIG.MA. Figure 7B The distribution of vehicles in the hyperspectral remote sensing image to be tested, acquired by the airborne imaging spectrometer SIG.MA. Figures 7C-7E The figures show the experimental results using HTD-Net, CNND, and the method of this invention, respectively.

[0100] Compare Figures 7C-7E It was found that, compared with the two existing technologies, the present invention provides more accurate target localization, with a brighter target area and a darker background area. Therefore, the present invention detects targets more accurately than the two existing methods, has a lower probability of misclassifying targets as background, and achieves better detection results.

[0101] To compare the target detection performance of the three methods, this method uses the AUC value for evaluation, which is obtained through the following formula:

[0102]

[0103] Where τ is the threshold, TP (τ) FP is the number of pixels that are considered as targets. (τ) N is the number of background pixels that were identified as the target. tgt and N bkg These are the total number of target pixels and the total number of background pixels, respectively. and These represent the detection rate and false alarm rate at the threshold τ, respectively. AUC is the detection rate and false alarm rate at the threshold τ. and The area under the curve plotted for the x and y axes.

[0104] The AUC accuracy of the method of this invention and two existing methods is shown in Table 1. As can be seen from Table 1, the method of this invention can achieve higher detection accuracy.

[0105] Method HTD-Net CNND The method of the invention Precision 0.8496 0.4555 0.9996

[0106] Table 1

[0107] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A hyperspectral remote sensing image target detection method based on target-background reconstruction deviation, characterized in that, The method comprises: acquiring a hyperspectral remote sensing image to be detected; inputting the hyperspectral remote sensing image to be detected into a pre-trained asymmetric auto-encoding network to obtain a first output and a second output corresponding to each pixel in the hyperspectral remote sensing image to be detected; the pre-trained asymmetric auto-encoding network comprises a pre-trained feature extraction subnetwork, a pre-trained feature fusion subnetwork and a pre-trained feature reconstruction subnetwork; the input of the feature fusion subnetwork is the output of the feature extraction subnetwork, and the input of the feature reconstruction subnetwork is the output of the feature fusion subnetwork; the first output is the output of the feature extraction subnetwork, and the second output is the output of the feature reconstruction subnetwork; the pre-trained asymmetric auto-encoding network is trained by using a plurality of mixed target spectra and a plurality of mixed background spectra; the plurality of mixed target spectra and the plurality of mixed background spectra are generated based on a first hyperspectral remote sensing sample image, a prior spectrum of a preset target and a bilinear spectral mixing model; determining a spectral angle distance of the pixel according to the first output and the second output; obtaining a detection result of whether the hyperspectral remote sensing image to be detected contains the preset target through smoothing processing of the spectral angle distance of the pixel; wherein the generation method of the plurality of mixed target spectra and the plurality of mixed background spectra comprises: acquire the first hyperspectral remote sensing sample image and the prior spectrum of the preset target, the number of the prior spectrum being ; According to the prior spectrum and the automatic endmember extraction method, a background spectrum is extracted from the first hyperspectral remote sensing sample image; the background spectrum is different from the prior spectrum, and the number of the background spectrum is , and are integers greater than 2. each time from one prior spectrum, according to the selected prior spectrum, one background spectrum and the first preset abundance value, generate one mixed target spectrum, until the prior spectrum is traversed , the mixed target spectrum generated for times is taken as the generated plurality of mixed target spectra; the mixed target spectrum is represented as: , wherein, the mixed target spectrum is represented as one prior spectrum selected each time from one prior spectrum, is , , wherein, one background spectrum is represented as is , the first preset abundance value is represented as the element-by-element multiplication operation is represented as select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time select one background spectrum from the background spectra each time 2. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 1, characterized in that, Before the step of inputting the hyperspectral remote sensing image to be detected into a pre-trained asymmetric auto-encoding network to obtain a first output and a second output corresponding to each pixel in the hyperspectral remote sensing image to be detected, the method further comprises: training an asymmetric auto-encoding network comprising a pre-trained feature extraction subnetwork, an initial feature fusion subnetwork and an initial feature reconstruction subnetwork by using the plurality of mixed target spectra and the plurality of mixed background spectra to obtain the pre-trained asymmetric auto-encoding network.

3. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 2, characterized in that, Before the step of training an asymmetric auto-encoding network comprising a pre-trained feature extraction subnetwork, an initial feature fusion subnetwork and an initial feature reconstruction subnetwork by using the plurality of mixed target spectra and the plurality of mixed background spectra to obtain the pre-trained asymmetric auto-encoding network, the method further comprises: acquiring a second hyperspectral remote sensing sample image with pixel labels; the pixel label of each pixel in the second hyperspectral remote sensing sample image represents a target category to which the pixel belongs; training a hyperspectral classification network comprising an initial feature extraction subnetwork and an initial classifier by using the second hyperspectral remote sensing sample image, wherein after each training, a loss value is determined according to the cross entropy between the prediction result of each pixel in the second hyperspectral remote sensing sample image obtained and the pixel label corresponding to the pixel, the parameter of the hyperspectral classification network is updated according to the loss value, and through multiple iterations of training, a pre-trained hyperspectral classification network is obtained; using the feature extraction subnetwork in the pre-trained hyperspectral classification network as the pre-trained feature extraction subnetwork.

4. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 2, characterized in that, The pre-trained asymmetric auto-encoding network is obtained by training the asymmetric auto-encoding network comprising a pre-trained feature extraction sub-network, an initial feature fusion sub-network and an initial feature reconstruction sub-network using the plurality of mixed target spectra and the plurality of mixed background spectra, comprising: In the i-th training, mixed target spectra and mixed background spectra for training are selected from the plurality of mixed target spectra and the plurality of mixed background spectra, and are input into the asymmetric auto-encoding network after the (i-1)-th training to obtain the output of the i-th feature extraction sub-network and the output of the i-th feature reconstruction sub-network; the asymmetric auto-encoding network after the (i-1)-th training comprises the pre-trained feature extraction sub-network, the feature fusion sub-network after the (i-1)-th training and the feature reconstruction sub-network after the (i-1)-th training; i is an integer greater than or equal to 1; The i-th spectral angle distance is determined according to the output of the i-th feature extraction sub-network and the output of the i-th feature reconstruction sub-network to obtain the i-th loss value; The feature fusion sub-network after the (i-1)-th training and the feature reconstruction sub-network after the (i-1)-th training are updated according to the i-th loss value to obtain the asymmetric auto-encoding network after the i-th training, and the training is iterated until the asymmetric auto-encoding network after the (i+p)-th training is obtained; p is an integer greater than 1; the asymmetric auto-encoding network after the i-th training comprises the pre-trained feature extraction sub-network, the feature fusion sub-network after the i-th training and the feature reconstruction sub-network after the i-th training; The asymmetric auto-encoding network after the (i+p)-th training is taken as the pre-trained asymmetric auto-encoding network.

5. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 4, characterized in that, The feature extraction sub-network comprises a first convolution processing module and N multi-scale residual convolution blocks; the feature fusion sub-network comprises N residual attention blocks and a second convolution processing module; the feature reconstruction sub-network comprises N residual deconvolution blocks; the output of the feature extraction sub-network comprises the output of the N residual attention blocks; the output of the feature reconstruction sub-network comprises the output of the N residual deconvolution blocks; and the process of selecting mixed target spectra and mixed background spectra for training from the plurality of mixed target spectra and the plurality of mixed background spectra, and inputting the mixed target spectra and the mixed background spectra into the asymmetric auto-encoding network after the (i-1)-th training to obtain the output of the i-th feature extraction sub-network and the output of the i-th feature reconstruction sub-network comprises: In the i-th training, the mixed target spectra and the mixed background spectra for training are input into the first convolution processing module in the asymmetric auto-encoding network after the (i-1)-th training for processing, the first convolution processing module inputs the output into the jth multi-scale residual convolution block in the asymmetric auto-encoding network after the (i-1)-th training, and the jth multi-scale residual convolution block inputs the output into the (j+1)th multi-scale residual convolution block and the N residual attention block after processing the input; j is 1; The (j+1)th multi-scale residual convolution block processes the input and outputs are input into the (j+2)th multi-scale residual convolution block and the (N-1)th residual attention block, respectively; The (j+2)th multi-scale residual convolution block processes the input and outputs are input into the (j+3)th multi-scale residual convolution block and the (N-2)th residual attention block, and so on, until the Nth multi-scale residual convolution block processes the input and outputs are input into the first residual attention block; The N residual attention blocks collectively input the output into the second convolution processing module for processing, the second convolution processing module inputs the output into the kth residual deconvolution block for processing, the kth residual deconvolution block inputs the output into the (k+1)th residual deconvolution block for processing, and so on, until the (N-1)th residual deconvolution block inputs the output into the Nth residual deconvolution block for processing to obtain the output of the Nth residual deconvolution block; k is 1.

6. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 5, characterized in that, The first convolution processing module includes a convolution layer, a batch normalization operation layer and a nonlinear activation layer; each multi-scale residual convolution block includes a first sub-convolution processing module, a second sub-convolution processing module and a third sub-convolution processing module, a fourth sub-convolution processing module and a first summation processing module; wherein the inputs of the first sub-convolution processing module and the second sub-convolution processing module are the same, the output of the first sub-convolution processing module is input into the third sub-convolution processing module, the output of the third sub-convolution processing module is input into the fourth sub-convolution processing module, and the output of the fourth sub-convolution processing module and the output of the second sub-convolution processing module are collectively input into the first summation processing module; The second convolution processing module includes a summation operation layer and a convolution layer; each residual attention block includes, in sequence, a pooling processing layer, a mapping activation layer, the summation operation layer, a multiplication operation layer and the convolution layer; wherein the input of the pooling processing layer and the output of the summation operation layer are collectively input into the multiplication operation layer; Each residual deconvolution block includes a first deconvolution processing module, a second deconvolution processing module, a third deconvolution processing module, a fourth deconvolution processing module and a second summation processing module; wherein the inputs of the first deconvolution processing module and the second deconvolution processing module are the same, the output of the first deconvolution processing module is input into the third deconvolution processing module, the output of the third deconvolution processing module is input into the fourth deconvolution processing module, and the output of the fourth deconvolution processing module and the output of the second deconvolution processing module are collectively input into the second summation processing module.

7. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 1, characterized in that, The method for extracting a background spectrum from the first hyperspectral remote sensing sample image according to the prior spectrum and the automatic endmember extraction method comprises: initializing a number of background spectra to be extracted as using the prior spectra as initial endmember spectra; is an integer greater than 1. In the Kth iteration, an orthogonal subspace projector of the Kth iteration is constructed according to a unit matrix and an endmember spectrum obtained in the (K-1)th iteration; when K=1, the endmember spectrum obtained in the (K-1)th iteration is the initial endmember spectrum; the unit matrix is determined according to the prior spectrum; projecting each pixel in the first hyperspectral remote sensing sample image to an orthogonal subspace through a Kth orthogonal subspace projector to obtain a Kth projected modulus of each pixel; The pixel corresponding to the maximum value in the modulus value of the Kth projection of each pixel is taken as the background spectrum extracted in the Kth iteration and the endmember spectrum obtained in the Kth iteration, respectively. This iteration is continued until the background spectrum extracted in the Kth iteration and the endmember spectrum obtained in the Kth iteration are obtained. ​​ 8.The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 1, characterized in that, the first output comprises N first outputs; the second output comprises N second outputs; the spectral angle distance of the pixel is calculated by the following formula: ； wherein, represents a spectral angular distance of a pixel, is , represents the first output, represents a band dimension, represents all elements in a channel dimension, is , represents the second output, is a transpose symbol, represents a norm symbol.

9. The target-background reconstruction bias based hyperspectral remote sensing image target detection method according to claim 1, characterized in that, the detection result of whether the to-be-detected hyperspectral remote sensing image contains the preset target is obtained through smoothing processing of the spectral angle distance of the pixel, and the detection result comprises: a Gaussian filter is adopted to perform smoothing processing on the spectral angle distance of the pixel to obtain a confidence degree of the pixel; the confidence degree is used to represent whether the preset target is contained; when there is a pixel with a confidence degree greater than or equal to a preset threshold in the pixels contained in the to-be-detected hyperspectral remote sensing image, the detection result of the to-be-detected hyperspectral remote sensing image is that the preset target is contained; when the confidence degrees of the pixels contained in the to-be-detected hyperspectral remote sensing image are all less than the preset threshold, the detection result of the to-be-detected hyperspectral remote sensing image is that the preset target is not contained.

Images