Hyperspectral image anomaly detection method, device, electronic terminal and storage medium

Abstract

The present invention discloses a method, device, electronic terminal and storage medium for detecting anomalies in hyperspectral images in the field of hyperspectral remote sensing image processing technology. The method comprises: obtaining a feature map through an encoding module; suppressing the expression of abnormal information in the feature map by using a self-attention mechanism module; extracting global features through a Transformer encoding module; performing feature fusion on feature maps containing different information; reconstructing the image using a decoding module, and obtaining anomaly detection maps using reconstruction errors. The present invention extracts spatial multi-scale features, suppresses anomalies through a self-attention mechanism, suppresses the contribution of abnormal pixels to the network reconstruction capability, prevents the network from learning the features of abnormal points, and produces a better reconstruction effect on the abnormal points.

Description

Technical Field

[0001] The invention relates to a method, a device, an electronic terminal and a storage medium for detecting anomalies in a hyperspectral image, and belongs to the technical field of hyperspectral remote sensing image processing. Background Art

[0002] Hyperspectral images are different from ordinary color images. Hyperspectral images often have a higher number of spectral bands, and each band corresponds to different material absorption characteristics. Therefore, hyperspectral images can provide richer information than ordinary color images. In hyperspectral images, anomalies are defined as pixels that are significantly different from the background spectrum, and anomaly detection algorithms are methods for detecting these pixels in images. Anomaly detection algorithms are not limited by prior information, so they can detect and identify targets without obtaining the spectral information of the target object in advance.

[0003] Wuhan University proposed a method for anomaly detection in hyperspectral images in its patent application "A method for anomaly detection in hyperspectral images based on a fully convolutional autoencoder" (patent application number: CN 202011508302.1). This method combines the fully convolutional autoencoder model to convert the anomaly detection problem of hyperspectral remote sensing images into a background and anomaly separation problem. The network directly reconstructs the background, and the anomaly manifests as a network reconstruction error, thereby achieving detection. This method combines the moment estimation algorithm with the autoencoder, which can enhance the network's ability to reconstruct the background, making the reconstruction error as an anomaly result more significant. However, the method still has the following shortcomings: the network has great limitations when extracting spatial information and has a small perception range; during reconstruction, the autoencoder may also produce a good reconstruction effect for the anomaly, resulting in reduced detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a hyperspectral image anomaly detection method, which performs spatial multi-scale feature extraction and performs anomaly suppression through a self-attention mechanism, thereby suppressing the contribution of abnormal pixels to the network reconstruction capability, preventing the network from learning the features of the anomalies and producing a better reconstruction effect on the anomalies.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting anomalies in a hyperspectral image, the method comprising:

[0007] Input the original hyperspectral image data to be detected into the first encoding module to obtain a feature map F1;

[0008] Input the feature map F1 into the second encoding module to obtain the feature map F2;

[0009] The feature maps F1 and F2 are processed by the self-attention mechanism module that suppresses the expression of abnormal information, and feature maps F1' and F2' are obtained respectively;

[0010] The global features of feature map F2' are extracted through the Transformer encoding module to obtain feature map F3;

[0011] The feature map F1', the feature map F2' and the feature map F3 are subjected to feature fusion to obtain the feature map F4;

[0012] The feature map F4 is input into the decoding module to restore the size and number of channels of the image and obtain the reconstructed image data;

[0013] The reconstruction error between the reconstructed image data and the original hyperspectral image data is calculated, and the anomaly detection map is obtained using the reconstruction error.

[0014] In combination with the first aspect, optionally, the structure of the first encoding module is the same as the structure of the second encoding module; the two encoding modules respectively include a convolution layer, a first activation layer, a full-dimensional dynamic convolution layer, a second activation layer, and a pooling layer connected in sequence;

[0015] Among them, the full-dimensional dynamic convolution layer focuses on the dynamic weight adjustment of the convolution kernel in four dimensions: space, input channel, output channel, and number of kernels, which is used to optimize the convolution process;

[0016] The first activation layer and the second activation layer have the same structure and both use the ReLU function.

[0017] In combination with the first aspect, optionally, the self-attention mechanism module for suppressing the expression of abnormal information performs feature processing on the feature map F1 and the feature map F2 respectively, including:

[0018] The image data F of the input feature map is convolved with the query convolution kernel, the key convolution kernel, and the value convolution kernel respectively, and the query matrix Q, the key matrix K, and the value matrix V are generated accordingly;

[0019] Reshape and transpose the query matrix Q to obtain a matrix Q';

[0020] Reshape the key matrix K and the value matrix V respectively, and obtain the matrix K' and the matrix V' accordingly;

[0021] Perform matrix multiplication on matrix Q' and matrix K', and perform exponential normalization on the result of matrix multiplication to obtain the attention weight matrix;

[0022] The attention weight matrix is ​​transposed and then matrix multiplication is performed with the matrix V' to obtain the output feature Out;

[0023] Reshape the output feature Out and perform weighted feature fusion with the image data F of the input feature map to obtain the output feature map of the self-attention mechanism module;

[0024] Among them, the expression of weighted feature fusion is as follows:

[0025] ;

[0026] In the formula, Image data of the output feature map of the self-attention mechanism module; is a learnable scalar parameter used to adjust the weight between the original input and the features after attention adjustment; Reshape matrix of output features Out.

[0027] In combination with the first aspect, optionally, before extracting the global features of the feature map F2' through the Transformer encoding module, the image data of the feature map F2' is sequentially subjected to tensor reshaping and dimension conversion so that the feature map F2' conforms to the input format of the Transformer encoding module;

[0028] After extracting the global features of the feature map F2' through the Transformer encoding module, the output tensor of the Transformer encoding module is dimensionally restored, and then the feature map F3 having the same shape as the feature map F2' is obtained by tensor reshaping.

[0029] In combination with the first aspect, optionally, the decoding module includes a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, a fourth deconvolution layer and a third activation layer connected in sequence;

[0030] Among them, the third activation layer adopts the sigmoid activation function to compress the input image pixel output to (0, 1);

[0031] The first deconvolution layer, the second deconvolution layer, the third deconvolution layer, and the fourth deconvolution layer connected in sequence are used to gradually restore the size and number of channels of the image.

[0032] In combination with the first aspect, optionally, the method further includes: adding a coverage perception loss function to assist network training, which is used to adjust the weights and bias items of the first encoding module, the second encoding module, and the decoding module to reduce the error of the model reconstruction background, and the expression of the coverage perception loss function L is as follows:

[0033] ;

[0034] Among them, H and W are the number of rows and columns of the original hyperspectral image data, i is the row index; j is the column index; To reconstruct image data, X i,j is the original hyperspectral image data, It is the image data of the plane coverage map obtained by the anomaly detection map.

[0035] In combination with the first aspect, optionally, a specific method for constructing the coverage perception loss function is:

[0036] Calculate the reconstruction error res between the reconstructed image data and the original hyperspectral image data;

[0037] Calculate the error degree of the reconstruction error res, the formula is:

[0038] ;

[0039] Among them, N is the dimension size of the error, and k is the dimension index;

[0040] Generate an anomaly detection map and reshape it into the shape of the original hyperspectral image to obtain an anomaly reshape map;

[0041] Flatten the anomaly reshape graph and find the top one percent of locations with the highest anomaly scores, which are recorded as anomaly indicators;

[0042] Generate an overlay map with the same size as the abnormal reshaping map, set the overlay map to 1, and the abnormal indicator position to 0, to obtain the image data of the plane overlay map;

[0043] The L1 norm is used as the loss function, and the image data of the plane coverage map is introduced as a coefficient into the loss function to obtain the coverage perception loss function.

[0044] In a second aspect, the present invention provides a hyperspectral image anomaly detection device, the device comprising:

[0045] The first encoding module is used to extract features from the original hyperspectral image to be detected and output a feature map F1;

[0046] The second encoding module is used to extract features from the feature map F1 and output the feature map F2;

[0047] Self-attention mechanism module: used to suppress the abnormal information expression of abnormal points in feature maps F1 and F2 and enhance background features, and output feature maps F1' and F2' accordingly;

[0048] Transformer encoding module: used to extract global information from feature map F2' and output feature map F3;

[0049] Feature fusion module: used to splice the feature map F1', feature map F2' and feature map F3 on the channel, and output the feature map F4;

[0050] Decoding module: used to restore the image size and number of channels and output reconstructed image data;

[0051] Anomaly detection map construction module: used to construct an anomaly detection map based on the reconstruction error between the reconstructed image data and the original hyperspectral image data.

[0052] In a third aspect, the present invention provides an electronic terminal, comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the hyperspectral image anomaly detection method as described in any one of the first aspects are performed.

[0053] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the hyperspectral image anomaly detection method described in any one of the first aspects.

[0054] In a fifth aspect, the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the hyperspectral image anomaly detection method described in any one of the first aspects.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) Design a self-attention mechanism module, use the self-attention mechanism to weaken the weight of pixels that are significantly different from most pixels, process shallow features to obtain feature maps F1' and F2' that suppress abnormal information, suppress the contribution of abnormal pixels to the network reconstruction ability, prevent the network from learning the features of abnormal points, and produce a better reconstruction effect on the abnormal points;

[0057] (2) The first encoding module and the second encoding module extract shallow features of the image, which contain more details. The Transformer encoding module acts as a global information extractor to extract deep features from the feature map F2' that has suppressed abnormal information, and obtains a feature map F3 that contains clearer global information. The feature map F1' and feature map F2' that contain shallow features and suppressed abnormal information, as well as the feature map F3 that contains deep features, are then fused to obtain the feature map F4, which contains spatial multi-scale information, reducing the limitations of the network in extracting spatial information and can be better used for image reconstruction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of a hyperspectral image anomaly detection method according to an embodiment of the present invention.

[0059] Figure 2 This is an RGB pseudo-color image of the airport hyperspectral image according to an embodiment of the present invention.

[0060] Figure 3 This is a real object location map of the comparative example airport hyperspectral image of the present invention.

[0061] Figure 4 This is a graph showing the abnormal target detection results of the comparative airport hyperspectral image of the present invention (Method 1).

[0062] Figure 5 This is a graph showing the abnormal target detection results of the comparative airport hyperspectral image of the present invention (Method 2).

[0063] Figure 6 This is a diagram of abnormal target detection results of the airport hyperspectral image according to an embodiment of the present invention.

[0064] Figure 7 It is a schematic diagram of the internal structure of an electronic terminal provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0065] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application.

[0066] In the description of the present invention, if “first” or “second” is described, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0067] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0068] Embodiment 1:

[0069] like Figure 1 As shown, this embodiment provides a hyperspectral image anomaly detection method, including:

[0070] Step 1: Input the original hyperspectral image data to be detected into the first encoding module for feature extraction to obtain a feature map F1;

[0071] Assume that the original hyperspectral image data X∈RH×W×B , H, W, and B are the number of rows, columns, and bands of the original hyperspectral image data, respectively. In some embodiments, the first encoding module may include a convolution layer, a first activation layer, a full-dimensional dynamic convolution layer, a second activation layer, and a pooling layer connected in sequence; wherein the convolution layer is used to convert the original hyperspectral image data X into a feature map for preliminary feature extraction, and the convolution kernel size used is 3×3, with a total of 64; the full-dimensional dynamic convolution layer focuses on the dynamic weight adjustment of the convolution kernel in four dimensions: space, input channel, output channel, and number of kernels, which is used to optimize the convolution process, and the full-dimensional dynamic convolution layer increases the number of bits of the output feature map to 128; the pooling layer is used for downsampling to reduce the size of the output feature map to half of the original size; the first activation layer and the second activation layer have the same structure, and both use the ReLU function; therefore, the feature map F1 output by the first encoding module can be expressed as: .

[0072] Step 2: Input the feature map F1 into the second encoding module for feature extraction to obtain the feature map F2;

[0073] In some embodiments, the structure of the second encoding module can be the same as that of the first encoding module, both of which can extract shallow features of the image and contain more details. The feature map F2 output by the second encoding module can be expressed as: .

[0074] Step 3: The feature maps F1 and F2 are processed by the self-attention mechanism module that suppresses the expression of abnormal information, and feature maps F1' and F2' are obtained respectively;

[0075] The self-attention mechanism module can suppress the expression of abnormal information in shallow features, thereby suppressing the contribution of abnormal pixels to the network reconstruction ability.

[0076] In some embodiments, step 3 may specifically include the following steps:

[0077] Step 301: Convolve the image data F of the input feature map with the query convolution kernel, the key convolution kernel, and the value convolution kernel respectively, and generate the query matrix Q, the key matrix K, and the value matrix V respectively. The formulas can be expressed as follows:

[0078] ;

[0079] ;

[0080] ;

[0081] In the formula, To query the convolution kernel, is the key convolution kernel, is the value convolution kernel, * is the convolution operation, is the input image data, that is, the image data of the aforementioned feature map F1 or feature map F2.

[0082] Step 302: Reshape and transpose the query matrix Q to obtain the matrix , the formula is: ;

[0083] In the formula, Express Reshape and transpose, The dimension used to represent the image data F, where is the batch size, is the number of channels, is the height, is the width, is the number of channels after convolution.

[0084] Step 303: reshape the key matrix K and the value matrix V respectively, and obtain the matrix K' and the matrix V' respectively. The formulas can be expressed as:

[0085] ;

[0086] ;

[0087] In the formula, To reshape K, To reshape V.

[0088] Step 304: Perform matrix multiplication on the matrix Q' and the matrix K', and perform exponential normalization on the matrix multiplication result to obtain the attention weight matrix, which can be expressed as:

[0089] ;

[0090] In the formula, is the attention weight matrix, Represents a matrix multiplication operation.

[0091] Step 305: Transpose the attention weight matrix and perform matrix multiplication with the matrix V' to obtain the output features , the formula can be expressed as:

[0092] ;

[0093] In the formula, is to transpose the attention weight matrix.

[0094] Step 306: Output features Reshape it and perform weighted feature fusion with the input image data F to obtain the output feature map of the self-attention mechanism module. The formula can be expressed as:

[0095] ;

[0096] Among them, the expression of weighted feature fusion is as follows:

[0097]

[0098] In the formula, Image data of the output feature map of the self-attention mechanism module; is a learnable scalar parameter used to adjust the weight between the original input and the features after attention adjustment; Output feature The reshaped matrix.

[0099] According to step 3 provided in an embodiment of the present invention, the self-attention mechanism module for suppressing the expression of abnormal information utilizes the self-attention mechanism to weaken the weights of pixels in the input feature map that are greatly different from most pixels, thereby reducing the influence of these pixels in subsequent tasks, while strengthening the appearance of background features. The feature map output after the self-attention mechanism module combines the original input features and the features adjusted by self-attention. Therefore, the output feature map retains the important information of the original input features and suppresses the expression of abnormal information.

[0100] Step 4: Extract the global features of feature map F2' through the Transformer encoding module to obtain feature map F3;

[0101] The Transformer encoding module allows the model to take into account the relationship between all elements in the input sequence when making representations. In image processing, this means that each pixel takes into account all other pixels in the image when generating its representation, so the Transformer encoding module can act as a global information extractor.

[0102] In some embodiments, before extracting the global features of the feature map F2' through the Transformer encoding module, the image data of the feature map F2' is sequentially subjected to tensor reshaping and dimension conversion so that the feature map F2' conforms to the input format of the Transformer encoding module;

[0103] After extracting the global features of the feature map F2' through the Transformer encoding module, the output tensor of the Transformer encoding module is dimensionally restored, and then the feature map F3 having the same shape as the feature map F2' is obtained by tensor reshaping.

[0104] Step 5: Fuse the feature maps F1', F2' and F3 to obtain feature map F4;

[0105] Among them, F1' and F2' as shallow features contain more details, while F3 as a deep feature contains global information. The feature map obtained by splicing the three feature maps is It is believed that it contains spatial multi-scale information and can be better used for image reconstruction tasks.

[0106] Step 6: Input the feature map F4 into the decoding module for image reconstruction to obtain reconstructed image data;

[0107] In some embodiments, the decoding module may include a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, a fourth deconvolution layer, and a third activation layer connected in sequence;

[0108] Among them, the first deconvolution layer, the second deconvolution layer, the third deconvolution layer, and the fourth deconvolution layer are used to gradually restore the size and number of channels of the image; the third activation layer adopts the sigmoid activation function to compress the input image pixel output to (0, 1).

[0109] Step 7: Calculate the reconstruction error between the reconstructed image data and the original hyperspectral image data, and use the reconstruction error to obtain the anomaly detection map.

[0110] In summary, the hyperspectral image anomaly detection method provided by the embodiment of the present invention designs a self-attention mechanism module, uses the self-attention mechanism to weaken the weights of pixels that are greatly different from most pixels, processes shallow features to obtain feature maps F1' and feature maps F2' that suppress abnormal information, suppresses the contribution of abnormal pixels to the network reconstruction capability, prevents the network from learning the features of abnormal points and produces a better reconstruction effect on abnormal points; the shallow features of the image are extracted by the first encoding module and the second encoding module, which contain more details, and the Transformer encoding module is used as a global information extractor to extract deep features from the feature map F2' that has suppressed abnormal information to obtain a feature map F3 containing clearer global information, and then the feature map F1' and feature map F2' containing shallow features and suppressed abnormal information and the feature map F3 containing deep features are feature fused to obtain the feature map F4, which contains spatial multi-scale information, reduces the limitations of the network when extracting spatial information, and can be better used for image reconstruction tasks.

[0111] Embodiment 2:

[0112] This embodiment is optimized on the basis of the above embodiment. For details not described in detail in this embodiment, please refer to Embodiment 1. The hyperspectral image anomaly detection method provided in this embodiment also includes:

[0113] In the process of network model training, a coverage-aware loss function is added to assist network training. It is used to adjust the weights and bias items of the encoding module and the decoding module before the original hyperspectral image data to be detected is input into the first encoding module to reduce the error of the model reconstruction background. The formula of the coverage-aware loss function can be expressed as:

[0114]

[0115] Among them, H and W are the number of rows and columns of the original hyperspectral image data, i is the row index; j is the column index; To reconstruct image data, X i,j is the original hyperspectral image data, It is the image data of the plane coverage map obtained by the anomaly detection map.

[0116] In some embodiments, a specific method for constructing a coverage-aware loss function includes:

[0117] Calculate the reconstruction error res between the reconstructed image data and the original hyperspectral image data;

[0118] Calculate the error degree using the formula:

[0119]

[0120] Where N is the dimension size of the error, k is the dimension index, k=1, 2,…, N;

[0121] Generate an anomaly detection map and reshape it into the shape of the original hyperspectral image to obtain an anomaly reshape map;

[0122] Flatten the anomaly reshape graph and find the top one percent of locations with the highest anomaly scores, which are recorded as anomaly indicators;

[0123] Generate an overlay map of the same size as the abnormal reshaping map, set the overlay map to 1, and the abnormal indicator position to 0, and obtain the data of the plane overlay map, that is, ;

[0124] Taking L1 norm as loss function, As a coefficient, we introduce the loss function to obtain the coverage perception loss function. The new loss function will not consider those locations with high anomaly scores. It can be considered that only the loss of the locations that can be regarded as background is calculated. The new loss function is used to optimize the network so that the network has a better reconstruction effect on the background and does not learn to reconstruct anomalies. By training the autoencoder module network, the network can further produce a better reconstruction effect on the background and a poor reconstruction effect on the anomaly.

[0125] It should be noted that the reason why the coverage-aware loss function is added to the embodiment of the present invention is that, in the anomaly detection task, only a small number of pixels are abnormal. During the training process, it is hoped that the network can better reconstruct the background rather than the anomaly. Therefore, in the anomaly detection map, the pixels in the first 1% of the anomaly score are covered, and these covered points will not be calculated when calculating the loss function.

[0126] The method provided by the embodiment of the present invention is further described below in conjunction with an application example, and the technical effect produced by the method provided by the embodiment of the present invention is further verified:

[0127] Using a hyperspectral image of an airport acquired by the Airborne Visible / Infrared Imaging Spectrometer (AVIRIS), such as Figure 2 As shown in Figure 1, the pseudo-color image of the airport hyperspectral image is referred to as the airport hyperspectral image. The spatial resolution of the airport hyperspectral image is 3.4 m and the acquisition time is July 7, 2010. Figure 3 As shown in the figure, the targets in this data set are three planes of different sizes, which are almost impossible to identify with the naked eye in the false color image, and the background is relatively simple. After removing the noise, the entire image contains 100×100 pixels and 191 bands. RXD, SegRX, LRASR, LSMAD, ADLR, LELRP-AD and the present invention are used to detect abnormal targets on the airport hyperspectral image of the embodiment, and the abnormal target detection accuracy is shown in Table 1.

[0128] Table 1 Comparison of hyperspectral abnormal target detection accuracy

[0129] Abnormal target detection method RX LRX CRD LSMAD FWf RGAE The present invention Abnormal target detection accuracy / % 95.26 94.98 96.50 98.73 97.44 88.90 99.92

[0130] It can be seen from Table 1 that the abnormal target detection accuracy of the embodiment of the present invention is as high as 99.92%, which is significantly better than several other methods.

[0131] Assume that the method for detecting using a conventional loss function in the embodiment of the present invention is Method 1. The abnormal target detection result of the airport hyperspectral image obtained by performing abnormal target detection on the airport hyperspectral image by Method 1 is as follows: Figure 4 As shown in the figure, its abnormal target detection accuracy is 99.91%.

[0132] Assuming that the method for removing the designed self-attention mechanism abnormal suppression module is Method2, the abnormal target detection results of the airport hyperspectral image obtained by abnormal target detection on the airport hyperspectral image are as follows: Figure 5 As shown in the figure, its abnormal target detection accuracy is 99.81%.

[0133] Figure 6The abnormal target detection result diagram is obtained by using the method provided by the embodiment of the present invention to detect abnormal targets in the airport hyperspectral image. The abnormal target detection accuracy of the embodiment of the present invention is 0.01% and 0.11% higher than that of Method 1 and Method 2 respectively. Figure 4 , Figure 5 and Figure 6 It can be seen from the clarity of the abnormal target and the degree of separation from the background that the abnormal target detection effect of the embodiment of the present invention is obviously better than that of other methods.

[0134] The above proves that the embodiment of the present invention can fully enable the network model to learn the background features of the hyperspectral image and prevent the network from producing a good reconstruction effect on abnormal pixels through multi-scale feature extraction and fusion, assisted by the method of suppressing abnormal reconstruction, and the improved loss function, thereby effectively improving the accuracy of abnormal target detection. It is feasible in abnormal target detection in hyperspectral images.

[0135] Embodiment three:

[0136] The present invention provides a hyperspectral image anomaly detection device, the device comprising:

[0137] The first encoding module is used to extract features from the original hyperspectral image to be detected and output a feature map F1;

[0138] The second encoding module is used to extract features from the feature map F1 and output the feature map F2;

[0139] Self-attention mechanism module: used to suppress the abnormal information expression of abnormal points in feature maps F1 and F2 and enhance background features, and output feature maps F1' and F2' accordingly;

[0140] Transformer encoding module: used to extract global information from feature map F2' and output feature map F3;

[0141] Feature fusion module: used to splice the feature map F1', feature map F2' and feature map F3 on the channel, and output the feature map F4;

[0142] Decoding module: used to restore the image size and number of channels and output reconstructed image data;

[0143] Anomaly detection map construction module: used to construct an anomaly detection map based on the reconstruction error between the reconstructed image data and the original hyperspectral image data.

[0144] The hyperspectral image anomaly detection device provided in the embodiment of the present invention can execute the hyperspectral image anomaly detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0145] Embodiment 4:

[0146] The embodiment of the present invention further provides an electronic terminal, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data acquired and generated in the method for anomaly detection of hyperspectral images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the method of the aforementioned embodiment one or two is implemented.

[0147] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] The computer device provided in the embodiment of the present invention can execute the hyperspectral image anomaly detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0149] Embodiment five:

[0150] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method of the above-mentioned embodiment 1 or embodiment 2 are first implemented.

[0151] The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0152] Embodiment six:

[0153] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the method of embodiment 1 or embodiment 2 when executed by a processor.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral image anomaly detection method, characterized in that: The method comprises: The original hyperspectral image data to be detected is input into the first encoding module for feature extraction to obtain a feature map F1; Input the feature map F1 to the second encoding module for feature extraction to obtain a feature map F2; The feature maps F1 and F2 are processed by the self-attention mechanism module that suppresses the expression of abnormal information, and feature maps F1' and F2' are obtained respectively; The global features of feature map F2' are extracted through the Transformer encoding module to obtain feature map F3; The feature map F1', the feature map F2' and the feature map F3 are subjected to feature fusion to obtain the feature map F4; The feature map F4 is input into the decoding module to restore the size and number of channels of the image and obtain the reconstructed image data; Calculate the reconstruction error between the reconstructed image data and the original hyperspectral image data, and use the reconstruction error to obtain an anomaly detection map; The self-attention mechanism module for suppressing the expression of abnormal information performs feature processing on the feature map F1 and the feature map F2 respectively, including: The image data F of the input feature map is convolved with the query convolution kernel, the key convolution kernel, and the value convolution kernel respectively, and the query matrix Q, the key matrix K, and the value matrix V are generated accordingly; Reshape and transpose the query matrix Q to obtain a matrix Q'; Reshape the key matrix K and the value matrix V respectively, and obtain the matrix K' and the matrix V' accordingly; Perform matrix multiplication on matrix Q' and matrix K', and perform exponential normalization on the result of matrix multiplication to obtain the attention weight matrix; The attention weight matrix is ​​transposed and then matrix multiplication is performed with the matrix V' to obtain the output feature Out; Reshape the output feature Out and perform weighted feature fusion with the image data F of the input feature map to obtain the output feature map of the self-attention mechanism module; Among them, the expression of weighted feature fusion is as follows: ； In the formula, Image data of the output feature map of the self-attention mechanism module; is a learnable scalar parameter used to adjust the weight between the original input and the features after attention adjustment; Reshape matrix of output features Out.

2. The hyperspectral image anomaly detection method according to claim 1, characterized in that: The structure of the first encoding module is the same as that of the second encoding module; the two encoding modules respectively include a convolution layer, a first activation layer, a full-dimensional dynamic convolution layer, a second activation layer, and a pooling layer connected in sequence; Among them, the full-dimensional dynamic convolution layer focuses on the dynamic weight adjustment of the convolution kernel in four dimensions: space, input channel, output channel, and number of kernels, which is used to optimize the convolution process; The first activation layer and the second activation layer have the same structure and both use the ReLU function.

3. The method for detecting anomalies in hyperspectral images according to claim 1, characterized in that: Before extracting the global features of the feature map F2' through the Transformer encoding module, the image data of the feature map F2' is sequentially subjected to tensor reshaping and dimension conversion so that the feature map F2' conforms to the input format of the Transformer encoding module; After extracting the global features of the feature map F2' through the Transformer encoding module, the output tensor of the Transformer encoding module is dimensionally restored, and then the feature map F3 having the same shape as the feature map F2' is obtained by tensor reshaping.

4. The method for detecting anomalies in hyperspectral images according to claim 1, characterized in that: The decoding module includes a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, a fourth deconvolution layer and a third activation layer connected in sequence; Among them, the third activation layer adopts the sigmoid activation function to compress the input image pixel output to (0,1); The first deconvolution layer, the second deconvolution layer, the third deconvolution layer, and the fourth deconvolution layer connected in sequence are used to gradually restore the size and number of channels of the image.

5. The method for detecting anomalies in hyperspectral images according to any one of claims 1 to 4, characterized in that: The method further includes: adding a coverage perception loss function to assist network training, which is used to adjust the weights and bias items of the first encoding module, the second encoding module, and the decoding module. The expression of the coverage perception loss function L is as follows: ； Among them, H and W are the number of rows and columns of the original hyperspectral image data, i is the row index; j is the column index; To reconstruct image data, X i,j is the original hyperspectral image data, It is the image data of the plane coverage map obtained by the anomaly detection map.

6. The method for detecting anomalies in hyperspectral images according to claim 5, characterized in that: The specific construction method of coverage perception loss function is: Calculate the reconstruction error res between the reconstructed image data and the original hyperspectral image data; Calculate the error degree of the reconstruction error res, the formula is: ； Where N is the dimension size of the error; k is the dimension index; Generate an anomaly detection map and reshape it into the shape of the original hyperspectral image to obtain an anomaly reshape map; Flatten the anomaly reshape graph and find the top one percent of locations with the highest anomaly scores, which are recorded as anomaly indicators; Generate an overlay map with the same size as the abnormal reshaping map, set the overlay map to 1, and the abnormal indicator position to 0, to obtain the image data of the plane overlay map; The L1 norm is used as the loss function, and the image data of the plane coverage map is introduced as the coefficient of the loss function to obtain the coverage perception loss function.

7. A hyperspectral image anomaly detection device, characterized in that: The device comprises: The first encoding module is used to extract features from the original hyperspectral image to be detected and output a feature map F1; The second encoding module is used to extract features from the feature map F1 and output the feature map F2; Self-attention mechanism module: used to suppress the abnormal information expression of abnormal points in feature maps F1 and F2 and enhance background features, and output feature maps F1' and F2' accordingly; Transformer encoding module: used to extract global information from feature map F2' and output feature map F3; Feature fusion module: used to splice the feature map F1', feature map F2' and feature map F3 on the channel, and output the feature map F4; Decoding module: used to restore the image size and number of channels and output reconstructed image data; Anomaly detection map construction module: used to construct anomaly detection map according to the reconstruction error between the reconstructed image data and the original hyperspectral image data; The self-attention mechanism module is specifically used for: The image data F of the input feature map is convolved with the query convolution kernel, the key convolution kernel, and the value convolution kernel respectively, and the query matrix Q, the key matrix K, and the value matrix V are generated accordingly; Reshape and transpose the query matrix Q to obtain a matrix Q'; Reshape the key matrix K and the value matrix V respectively, and obtain the matrix K' and the matrix V' accordingly; Perform matrix multiplication on matrix Q' and matrix K', and perform exponential normalization on the result of matrix multiplication to obtain the attention weight matrix; The attention weight matrix is ​​transposed and then matrix multiplication is performed with the matrix V' to obtain the output feature Out; Reshape the output feature Out and perform weighted feature fusion with the image data F of the input feature map to obtain the output feature map of the self-attention mechanism module; Among them, the expression of weighted feature fusion is as follows: ； In the formula, Image data of the output feature map of the self-attention mechanism module; is a learnable scalar parameter used to adjust the weight between the original input and the features after attention adjustment; Reshape matrix of output features Out.

8. An electronic terminal, characterized in that: The method comprises a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the hyperspectral image anomaly detection method according to any one of claims 1 to 6 are executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the hyperspectral image anomaly detection method according to any one of claims 1 to 6 are implemented.

Images