Anomaly detection method, device and electronic device based on low-rank embedding and multi-scale

Abstract

The present invention relates to the field of information processing technology, and in particular to a method, device and electronic device for anomaly detection based on low-rank embedding and multi-scale. The method comprises: acquiring a hyperspectral image, and preprocessing the hyperspectral image to obtain an original hyperspectral image; performing feature extraction operations on the original hyperspectral image using a preset low-rank background feature extraction model to obtain low-rank background features, and using a pre-built multi-scale fusion feature extraction model to extract global features and local features of the original hyperspectral image; integrating the low-rank background features, global features and local features, and comparing the obtained target reconstructed background image with the hyperspectral image to identify abnormal areas. Therefore, by adopting low-rank background feature extraction and multi-scale fusion feature extraction to realize hyperspectral image disaster monitoring, the problems of long detection process, low efficiency and low precision of the existing detection method are solved, and the method has the advantages of automation, high precision, fastness and high efficiency.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method, device and electronic device for anomaly detection based on low-rank embedding and multi-scale. Background Art

[0002] Hyperspectral images are widely used in military, agriculture, food safety and other fields due to their rich spectral and spatial information. Monitoring natural disasters through hyperspectral images is a hot topic. Anomaly detection in HSI (Hyperspectral Image) is a completely unsupervised object detection task. Its purpose is to detect anomalies that are different from the surrounding environment. It is more in line with practical application scenarios and has attracted more and more attention from researchers in recent years.

[0003] In the related technology, the HSI anomaly detection method can project the original image into a new latent space based on the deep convolutional neural network method, and then use the traditional method to achieve anomaly detection; or it can also use a deep autoencoder to reconstruct the background and directly detect anomalies through the residual image between the original image and the reconstructed background.

[0004] However, the above methods are relatively complicated, the detection process is time-consuming, inefficient, and low in precision, which needs to be solved urgently. Summary of the invention

[0005] The present invention provides an anomaly detection method, device and electronic device based on low-rank embedding and multi-scale to solve the problems of long detection process, low efficiency and low precision of existing detection methods, and has the advantages of automation, high precision, fast and high efficiency.

[0006] To achieve the above objectives, a first embodiment of the present invention proposes an anomaly detection method based on low-rank embedding and multi-scale, comprising the following steps:

[0007] Acquire a hyperspectral image, and preprocess the hyperspectral image to obtain an original hyperspectral image;

[0008] Using a preset low-rank background feature extraction model to perform a feature extraction operation on the original hyperspectral image to obtain low-rank background features, and using a pre-built multi-scale fusion feature extraction model to extract global features and local features of the original hyperspectral image;

[0009] The low-rank background features, the global features and the local features are integrated, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas.

[0010] According to one embodiment of the present invention, the preprocessing of the hyperspectral image includes:

[0011] The global spatial and spectral correlations of the hyperspectral image are determined by using a preset low-rank tensor decomposition denoising algorithm. At the same time, the piecewise smoothing characteristics of the hyperspectral image in the spatial dimension are characterized based on an adaptive weight graph total variation strategy and the edge information of the hyperspectral image is retained.

[0012] According to one embodiment of the present invention, the preset low-rank background feature extraction model is:

[0013]

[0014] in, is an operator that flattens a 3D tensor into a matrix in lexicographic order, is a convolutional layer, is another convolutional layer, is the last convolutional layer, is the input three-dimensional tensor, is the activation function.

[0015] According to one embodiment of the present invention, before using the pre-built multi-scale fusion feature extraction model to extract the global features and local features of the original hyperspectral image, the method further includes:

[0016] Acquire a historical hyperspectral image, and preprocess the historical hyperspectral image to obtain a preprocessed historical hyperspectral image;

[0017] Extracting global context information of the preprocessed historical hyperspectral image using a visual transformer, and extracting local information of the preprocessed historical hyperspectral image using a dilated convolutional neural network;

[0018] The global context information and the local information are fused to obtain fused feature data, and the fused feature data is divided into a training set and a validation set according to a preset division ratio;

[0019] The preset neural network is trained using the training set to obtain an initial neural network model, and the initial neural network model is verified using the verification set until the initial neural network model meets the preset standard, and the iterative training of the preset neural network is terminated to obtain the pre-constructed multi-scale fusion feature extraction model; otherwise, the preset division ratio is adjusted, and iterative training is performed based on the adjusted new training set.

[0020] According to one embodiment of the present invention, the extracting local information of the preprocessed historical hyperspectral image by using the dilated convolutional neural network includes:

[0021] By stacking dilated convolutions with different dilation rates, multi-scale image features of the preprocessed historical hyperspectral image are obtained;

[0022] Using a 3×3 convolution kernel to extract local features of the preprocessed historical hyperspectral image;

[0023] The multi-scale image features and the local features are fused to obtain local information of the preprocessed historical hyperspectral image.

[0024] According to an embodiment of the present invention, the integrating processing of the low-rank background feature, the global feature and the local feature includes:

[0025] Performing 1×1 convolution downscaling processing on the global features and the local features respectively to obtain global low-dimensional features and local low-dimensional features;

[0026] The global low-dimensional features and the local low-dimensional features are fused using a preset pixel-by-pixel weighting strategy to obtain an initial background reconstructed background image;

[0027] The initial background reconstruction background image is upsampled using a preset bilinear interpolation strategy to obtain upsampled features, and the upsampled features are deconvolved using the 3×3 convolution kernel to obtain the target reconstructed background image.

[0028] According to the anomaly detection method based on low-rank embedding and multi-scale proposed in the embodiment of the present invention, the original hyperspectral image can be obtained by preprocessing the hyperspectral image, and the low-rank background feature can be obtained by performing feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model, and then the global and local features of the original hyperspectral image can be extracted by using a pre-built multi-scale fusion feature extraction model; the low-rank background features, global features and local features are integrated and processed, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas. Therefore, by adopting low-rank background feature extraction and multi-scale fusion feature extraction to realize hyperspectral image disaster monitoring, the problems of long detection process, low efficiency and low precision of the existing detection method are solved, and it has the advantages of automation, high precision, fast and high efficiency.

[0029] To achieve the above object, a second embodiment of the present invention proposes an anomaly detection device based on low-rank embedding and multi-scale, comprising:

[0030] An acquisition module is used to acquire a hyperspectral image and preprocess the hyperspectral image to obtain an original hyperspectral image;

[0031] An extraction module is used to perform a feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model to obtain low-rank background features, and to extract global features and local features of the original hyperspectral image using a pre-built multi-scale fusion feature extraction model;

[0032] The identification module is used to integrate the low-rank background features, the global features and the local features, compare the obtained target reconstructed background image with the hyperspectral image, and identify abnormal areas.

[0033] According to one embodiment of the present invention, the obtaining module is specifically used for:

[0034] The global spatial and spectral correlations of the hyperspectral image are determined by using a preset low-rank tensor decomposition denoising algorithm. At the same time, the piecewise smoothing characteristics of the hyperspectral image in the spatial dimension are characterized based on an adaptive weight graph total variation strategy and the edge information of the hyperspectral image is retained.

[0035] According to one embodiment of the present invention, the preset low-rank background feature extraction model is:

[0036]

[0037] in, is an operator that flattens a 3D tensor into a matrix in lexicographic order, is a convolutional layer, To represent another convolutional layer, is the last convolutional layer, is the three-bit tensor of input, is the activation function.

[0038] According to one embodiment of the present invention, before using the pre-built multi-scale fusion feature extraction model to extract the global features and local features of the original hyperspectral image, the extraction module further includes:

[0039] An acquisition unit is used to acquire a historical hyperspectral image and preprocess the historical hyperspectral image to obtain a preprocessed historical hyperspectral image;

[0040] An extraction unit, configured to extract global context information of the preprocessed historical hyperspectral image using a visual transformer, and to extract local information of the preprocessed historical hyperspectral image using a dilated convolutional neural network;

[0041] A fusion unit, used for fusing the global context information and the local information to obtain fused feature data, and dividing the fused feature data into a training set and a validation set according to a preset division ratio;

[0042] A training unit is used to train a preset neural network using the training set to obtain an initial neural network model, and to verify the initial neural network model using the verification set, until the initial neural network model meets the preset standard, and then terminate the iterative training of the preset neural network to obtain the pre-constructed multi-scale fusion feature extraction model; otherwise, adjust the preset division ratio, and perform iterative training based on the adjusted new training set.

[0043] According to one embodiment of the present invention, the extraction unit is specifically used for:

[0044] By stacking dilated convolutions with different dilation rates, multi-scale image features of the preprocessed historical hyperspectral image are obtained;

[0045] Using a 3×3 convolution kernel to extract local features of the preprocessed historical hyperspectral image;

[0046] The multi-scale image features and the local features are fused to obtain local information of the preprocessed historical hyperspectral image.

[0047] According to one embodiment of the present invention, the identification module is specifically used for:

[0048] Performing 1×1 convolution downscaling processing on the global features and the local features respectively to obtain global low-dimensional features and local low-dimensional features;

[0049] The global low-dimensional features and the local low-dimensional features are fused using a preset pixel-by-pixel weighting strategy to obtain an initial background reconstructed background image;

[0050] The initial background reconstruction background image is upsampled using a preset bilinear interpolation strategy to obtain upsampled features, and the upsampled features are deconvolved using the 3×3 convolution kernel to obtain the target reconstructed background image.

[0051] According to the low-rank embedding and multi-scale anomaly detection device proposed in the embodiment of the present invention, the original hyperspectral image can be obtained by preprocessing the hyperspectral image, and the low-rank background feature can be obtained by performing feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model, and then the global and local features of the original hyperspectral image can be extracted by using a pre-built multi-scale fusion feature extraction model; the low-rank background features, global features and local features are integrated and processed, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas. Therefore, by adopting low-rank background feature extraction and multi-scale fusion feature extraction to realize hyperspectral image disaster monitoring, the problems of long detection process, low efficiency and low precision of the existing detection method are solved, and it has the advantages of automation, high precision, fast and high efficiency.

[0052] To achieve the above objectives, the third aspect of the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the low-rank embedding and multi-scale anomaly detection method as described in the above embodiments.

[0053] To achieve the above objectives, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the low-rank embedding and multi-scale anomaly detection method as described in the above embodiments.

[0054] To achieve the above objectives, a fifth aspect of the present invention proposes a computer program product. When the computer program is executed, it is used to implement the low-rank embedding and multi-scale anomaly detection method as described in the above embodiment.

[0055] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0057] Figure 1 A flowchart of a method for anomaly detection based on low-rank embedding and multi-scale according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of the structure of a visual converter in the related art;

[0059] Figure 3 A block diagram of an anomaly detection device based on low-rank embedding and multi-scale according to an embodiment of the present invention;

[0060] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0062] The following describes a low-rank embedding and multi-scale anomaly detection method, device, and electronic device according to an embodiment of the present invention with reference to the accompanying drawings.

[0063] Figure 1 It is a flowchart of an anomaly detection method based on low-rank embedding and multi-scale according to an embodiment of the present invention.

[0064] Before introducing the low-rank embedding and multi-scale anomaly detection method proposed in an embodiment of the present invention, the relevant technical background is first introduced.

[0065] Traditional anomaly detection methods for hyperspectral images are mostly based on statistical methods, and the calculation process is complex. With the development of deep learning technology, there are more and more studies on anomaly detection using deep learning methods, because deep learning methods have powerful feature extraction capabilities. Many methods based on deep convolutional neural networks project the original image into a new latent space and then use traditional methods to achieve anomaly detection. These methods can effectively extract features from images. However, they cannot guarantee that the extracted features can effectively distinguish anomalies from backgrounds, and often require complex post-processing procedures.

[0066] Other methods use deep autoencoders for background reconstruction and directly detect anomalies through the residual image between the original image and the reconstructed background. For example, a discriminative reconstruction constrained generative adversarial network is proposed to estimate the background and detect anomalies; or a fully convolutional autoencoder with a skip connection structure is used to reconstruct the background, and the anomalies appear in the form of reconstruction errors. However, due to the spatial downsampling structure, most methods are not effective in extracting features of small non-abnormal objects, resulting in a high false alarm rate.

[0067] In order to solve the problems of long detection time, low efficiency and low precision in the existing detection methods, the embodiment of the present invention proposes an anomaly detection method based on low-rank embedding and multi-scale, which realizes disaster monitoring of hyperspectral images by adopting low-rank background feature extraction and multi-scale fusion feature extraction, and has the advantages of automation, high precision, fast and high efficiency.

[0068] For example, Figure 1 As shown, the low-rank embedding and multi-scale anomaly detection method includes the following steps:

[0069] In step S101, a hyperspectral image is acquired and preprocessed to obtain an original hyperspectral image.

[0070] Among them, hyperspectral images refer to images with a spectral resolution of 10 -2 Spectral images within the order of magnitude range. In short, hyperspectral images are three-dimensional images. Unlike black and white images, which have a single channel, and color images, which have three channels, the number of channels in hyperspectral images is determined by the number of bands. The length and width of the image are the same as those of a normal image, but it contains information from dozens or even hundreds of bands.

[0071] Specifically, hyperspectral images can be acquired through drones and preprocessed (including normalization and noise reduction). First, the hyperspectral images are normalized to scale the features of all hyperspectral image data to the same range (usually 0 to 1 or -1 to 1). This helps avoid adverse effects on subsequent processing algorithms due to differences in data dimensions and distribution in different bands. Hyperspectral images often contain various noises, such as shot noise, stripe noise, etc. These noises will affect the image quality and interfere with subsequent analysis and processing. Therefore, after the hyperspectral images are normalized, effective noise reduction is particularly important for improving data quality. After normalization and noise reduction of the hyperspectral image data, the original hyperspectral image can be obtained for subsequent analysis.

[0072] It can be understood that normalization ensures that all features of the image data are treated fairly on the same dimension, while denoising eliminates the noise that affects the image quality. Therefore, the quality of the image data and the accuracy of subsequent analysis can be significantly improved by normalizing and denoising the hyperspectral image. The specific normalization method and denoising method can be determined according to the characteristics of the image data and the requirements of the analysis task.

[0073] The following describes in detail how the embodiments of the present invention preprocess the hyperspectral image.

[0074] As a possible implementation method, in some embodiments, the hyperspectral image is preprocessed, including: using a preset low-rank tensor decomposition denoising algorithm to determine the global spatial and spectral correlations of the hyperspectral image, and characterizing the piecewise smoothing characteristics of the hyperspectral image in the spatial dimension based on an adaptive weight graph total variation strategy and retaining the edge information of the hyperspectral image.

[0075] Among them, global spatial correlation mainly involves the spatial position relationship between pixels or regions in hyperspectral images, that is, the degree of their spatial correlation; spectral correlation focuses on the similarity or difference of spectral characteristics of pixels or regions in hyperspectral images. Since hyperspectral images have multiple spectral bands, which record the reflection or emission of electromagnetic radiation by ground objects, the spectral characteristics of different pixels or regions can reflect their material composition, state and other information. Spectral correlation analysis helps to understand the spectral differences between different ground objects, and then perform classification, recognition and other applications; in image processing, total variation is a regularization technique commonly used for denoising, which encourages images to remain smooth in local areas while allowing sharpness to be maintained in edge and texture areas.

[0076] It is understandable that, due to its rich spectral information and spatial resolution, hyperspectral images can usually be regarded as a three-dimensional data cube (or tensor), which contains two spatial dimensions (i.e., width and height) and one spectral dimension (pixel values ​​in different bands), so it can be represented using a tensor (a multidimensional data set). The purpose of low-rank tensor decomposition is to decompose this high-dimensional data cube into multiple lower-rank components (i.e., low-rank tensors) that can capture the main information structure and features in the hyperspectral image while reducing the redundancy and noise of the data. Through low-rank tensor decomposition, the global spatial and spectral correlations of the hyperspectral image can be captured, that is, the dimension and complexity of the data can be reduced without losing important information, making it easier to identify and remove noise. Adaptive weight map total variation can improve the problem of over-smoothing edges or ignoring certain types of noise by introducing a weight map. The weight map can dynamically adjust the strength of regularization according to the local characteristics of the hyperspectral image (such as gradient strength, structural complexity, etc.). In smooth areas, the weight is higher, which can promote smoothness, and in edge or texture areas, the weight is lower, retaining details and sharpness, thereby achieving the important features of the image while removing noise.

[0077] In addition, different types of noise usually require different processing methods. In the preset low-rank tensor decomposition denoising algorithm, by combining the l1-norm and the Frobenius-norm, various types of noise including sparse noise and Gaussian noise can be effectively removed, thereby improving the quality of hyperspectral images. Among them, the l1-norm (also known as the L1 norm or absolute value norm) is often used to process sparse noise, such as streak noise, impulse noise, and dead line noise. These noises generally only appear on a few pixels of the image. Therefore, the l1-norm can effectively identify and remove sparse noise by encouraging sparsity; the Frobenius-norm is a norm used to measure the overall size of a matrix or tensor. In the denoising algorithm, it is usually used to measure the difference between the low-rank tensor and the original image (i.e., the reconstruction error). By minimizing the objective function containing the Frobenius-norm, the algorithm can find the optimal low-rank representation, thereby achieving the removal of a wider range of types of noise including Gaussian noise.

[0078] In step S102, a preset low-rank background feature extraction model is used to perform feature extraction operations on the original hyperspectral image to obtain low-rank background features, and a pre-built multi-scale fusion feature extraction model is used to extract global features and local features of the original hyperspectral image.

[0079] It is understandable that in many practical applications, such as video surveillance and remote sensing image processing, the background often exhibits a low-rank characteristic, which means that the background image can be approximately represented by a linear combination of a few main components (or "basis"), which can capture the main structure and texture information of the background. In contrast, abnormal elements such as moving objects and noise usually do not have this low-rank characteristic. The low-rank background feature extraction model LRM (Low-Rank Matrix) is a specially designed algorithm or network structure for extracting low-rank background features (i.e., pure background representation) from images containing abnormal elements. The model can use matrix decomposition, tensor decomposition or deep learning techniques to achieve low-rank approximation, thereby separating the background and abnormal elements. By improving the low-rank background feature extraction model, that is, removing the skip connection structure, the improved preset low-rank background feature extraction model can focus more on extracting low-rank features of the background. In convolutional neural networks, convolutional layers are used to extract image features. 1×1 convolution is usually used to change the depth (i.e., the number of channels) of the feature map without changing its spatial dimension. In comparison, 3×3 convolution can simultaneously consider adjacent pixels in the spatial dimension, thereby more effectively capturing the spatial information of the image. In the background extraction task, using 3×3 convolution instead of 1×1 convolution helps the network to better utilize the spatial information of the original hyperspectral image, thereby more accurately extracting low-rank background features.

[0080] Specifically, according to the actual task requirements, a deep learning model that includes low-rank approximation technology can be designed, the network is constructed in the deep learning framework, unnecessary jump connection structures are deleted, and 3×3 convolutional layers are used at appropriate locations to extract spatial information; hyperspectral images or video sequences containing pure background and abnormal elements are used as training data to train the network. During the training process, the network parameters are optimized by minimizing the background reconstruction error and anomaly suppression loss. The performance of the network is evaluated on the test set to check whether it can effectively extract low-rank background features and minimize abnormal interference.

[0081] Assumption Matrix It can be approximately decomposed into ,in, , , where F is the data tensor in the hyperspectral image or video sequence, and represents the input pixel data in the hyperspectral image or video frame in an M×N matrix. R refers to the rank of the matrix F, U is an M×K matrix, and V is a K×N matrix. Their product UV approximates the low-rank approximation form of the matrix F. K is the rank in the low-rank approximation, which represents the number of eigenvectors in the U and V matrices and determines the validity and approximation of the matrix UV after approximation. Then, in some embodiments, the preset low-rank background feature extraction model can be:

[0082]

[0083] in, , , is the rank ratio of the low-rank matrix LRM, where w is the width of the image and h is the height of the image, which represent the size of the data tensor in the image space. k is a parameter in the low-rank background feature extraction model, which represents the number of feature vectors or the number of convolution kernels used in the feature extraction process. c is the channel number function in the convolution operation. is an operator that flattens a 3D tensor into a matrix in lexicographic order, It is a convolution layer that applies convolution kernel to the input feature map Perform linear transformation and nonlinear activation, is another convolutional layer to further extract and enhance the high-level features of the input feature map. As the last convolutional layer, it further deepens the nonlinear expression ability of the network and optimizes the extracted features through appropriate activation functions (such as ReLU) and weight adjustment to adapt to the subsequent low-rank feature extraction and background reconstruction process. is the input three-dimensional tensor, which passes through multiple branches , , After processing, the final output is generated .

[0084] In the third branch, the same size matrix can be obtained through the same architecture as above , the middle branch uses a 1×1×1 convolutional layer and a reshaped input feature map, and Multiplying it by , .

[0085] Furthermore, the global features and local features of the original hyperspectral image can be extracted by using the pre-built multi-scale fusion feature extraction model, that is, the original hyperspectral data is input into the pre-built multi-scale fusion feature extraction model, and the global features and local features of the original hyperspectral image are extracted respectively through its internal components.

[0086] For ease of understanding, the following describes in detail how to obtain the pre-built multi-scale fusion feature extraction model.

[0087] As a possible implementation method, in some embodiments, before using a pre-constructed multi-scale fusion feature extraction model to extract the global features and local features of the original hyperspectral image, it also includes: obtaining a historical hyperspectral image, and pre-processing the historical hyperspectral image to obtain a pre-processed historical hyperspectral image; using a visual transformer to extract the global context information of the pre-processed historical hyperspectral image, and using an expanded convolutional neural network to extract the local information of the pre-processed historical hyperspectral image; fusing the global context information and the local information to obtain fused feature data, and dividing the fused feature data into a training set and a verification set according to a preset division ratio; using the training set to train the preset neural network to obtain an initial neural network model, and using the verification set to verify the initial neural network model until the initial neural network model meets the preset standard, ending the iterative training of the preset neural network, and obtaining the pre-constructed multi-scale fusion feature extraction model, otherwise, adjusting the preset division ratio, and performing iterative training based on the adjusted new training set.

[0088] The embodiment of the present invention can use the combination of Vision Transformer (ViT) and dilated convolutional CNN (Convolutional Neural Network) to build a multi-scale fusion feature extraction model. Among them, the Vision Transformer is an architecture that applies the Transformer model to image processing. It divides the image into a series of small blocks, and regards these small blocks as tags in the sequence, and then applies the standard Transformer structure to process these tags, thereby capturing the global context information of the image, such as Figure 2As shown in the figure, the entire network architecture can be divided into three parts, namely, the flattening and linear mapping part (image embedding layer), the overall encoding part (such as Figure 2 The more detailed structure shown on the right) and the classification network output part (i.e. the classification module, the layer structure ultimately used for classification); dilated convolution (also called hole convolution) is a convolution method that introduces "holes" into standard convolution, which can increase the receptive field (i.e. the size of the area of ​​the input image that the network can see) without increasing the number of parameters and the amount of computation, allowing CNN to better capture the local details and texture information of the image.

[0089] Specifically, historical hyperspectral images are obtained from historical databases and preprocessed (such as cropping, normalization, denoising, etc.) to obtain preprocessed historical hyperspectral images. The preprocessed historical hyperspectral images are segmented into a series of small blocks using a visual transformer, each small block is converted into a vector representation, and the Transformer structure is applied to process these vectors, thereby capturing the global context information of the historical hyperspectral images. The preprocessed historical hyperspectral images or a subset thereof are input into a CNN containing dilated convolutions, and the local information of the image is gradually extracted through multi-layer convolution operations. A suitable feature fusion mechanism is designed (which may include simple feature splicing, weighted sum, attention mechanism, etc.) to fuse the global context information and the local information to obtain fused feature data. Then, the fused feature data is divided into a training set and a validation set according to a preset division ratio (such as 70% training set and 30% validation set), a suitable neural network architecture is selected as the preset neural network, the preset neural network is iteratively trained using the training set, the model parameters are optimized, and the model performance is evaluated using the validation set. If the initial neural network model does not meet the preset standards (such as accuracy, recall rate, etc.), you can try to adjust the division ratio of the training set and the validation set, as well as the model parameters, optimization algorithm, etc., and continue to perform iterative training to improve the model performance until the initial neural network model meets the preset standards. End the iterative training and obtain the pre-built multi-scale fusion feature extraction model.

[0090] Among them, in some embodiments, a dilated convolutional neural network is used to extract local information of the preprocessed historical hyperspectral image, including: obtaining multi-scale image features of the preprocessed historical hyperspectral image by stacking dilated convolutions with different dilation rates; extracting local features of the preprocessed historical hyperspectral image by using a 3×3 convolution kernel; and fusing the multi-scale image features and the local features to obtain the local information of the preprocessed historical hyperspectral image.

[0091] Specifically, the embodiment of the present invention improves the CNN process of capturing detailed local information to a certain extent, that is, firstly, using dilated convolutions of different sizes to expand the network receptive field to obtain richer fine features (such as roads) and high-level information, and obtain multi-scale image features of the pre-processed historical hyperspectral image, that is, designing multiple parallel convolutional layers, each of which uses a different dilation rate (the dilation rate determines the distance between elements in the convolution kernel, thereby controlling the size of the receptive field), and each dilated convolutional layer can output feature maps of different scales, which together constitute a multi-scale image feature set; in order to ensure that the image detail features are extracted more fully, after the dilated convolutional layer, multiple 3×3 convolutional layers are stacked, and each convolutional layer can usually be followed by an activation function (such as ReLU) and a possible normalization layer (such as BatchNorm) to deeply extract the local features of the pre-processed historical hyperspectral image, and fuse the local features with the multi-scale image features, so as to further enrich the features extracted by the network; in addition, in the entire feature extraction process, the spatial resolution of the image feature space is kept constant, and more detailed depth features in the image are retained as much as possible, and finally the local information of the pre-processed historical hyperspectral image is obtained.

[0092] In step S103, the low-rank background features, global features and local features are integrated, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas.

[0093] That is to say, after obtaining the low-rank background features, global features and local features, these features are integrated to obtain the target reconstructed background image (close to the scene when there is no anomaly (such as natural disasters)). By constructing a lookup table of abnormal features corresponding to different natural disasters in advance, this lookup table can be constructed based on historical data, expert knowledge or simulation experiments. It is used to associate the detected anomalies with specific natural disaster types, and compare the target reconstructed background image with the initial hyperspectral image. For example, the difference map between the two is calculated (such as absolute value, square difference, etc.), and the comparison result is compared with the abnormal feature lookup table through a suitable matching algorithm (such as nearest neighbor search, template matching, etc.) to quickly evaluate the most likely type of natural disaster.

[0094] Next, we will explain in detail how to integrate low-rank background features, global features, and local features.

[0095] As a possible implementation method, in some embodiments, low-rank background features, global features and local features are integrated, including: performing 1×1 convolution downscaling processing on the global features and local features respectively to obtain global low-dimensional features and local low-dimensional features; using a preset pixel-by-pixel weighting strategy to fuse the global low-dimensional features and the local low-dimensional features to obtain an initial background reconstructed background image; using a preset bilinear interpolation strategy to upsample the initial background reconstructed background image to obtain upsampled features, and using a 3×3 convolution kernel to perform deconvolution on the upsampled features to obtain a target reconstructed background image.

[0096] Specifically, the global features and local features are downscaled by 1×1 convolution respectively, that is, for the input feature map F in Perform 1×1 convolution downscaling processing, the formula can be expressed as:

[0097] F out =W×F in +b;

[0098] Among them, W is the 1×1 convolution kernel weight matrix, b is the bias term, and F out is the output feature map after downscaling.

[0099] In this way, the dimension of the feature map can be reduced, reducing the computational complexity while maintaining the integrity of the original information. Through this convolution operation, the global features and local features can be compressed into a low-dimensional feature map with the same dimension, generating global low-dimensional features and local low-dimensional features. In order to ensure the effective combination of global low-dimensional features and local low-dimensional features, the two feature maps can be fused using a preset pixel-by-pixel weighting strategy to obtain the initial background and reconstruct the background image. In this way, the relationship between global information and local details can be balanced to ensure that the reconstructed background retains both the global structure and the local details.

[0100] Furthermore, after the feature downscaling and fusion are completed, the initial background reconstruction background image can be upsampled to the same resolution as the original hyperspectral image using the preset bilinear interpolation strategy to obtain the upsampled features (i.e., the upsampled feature map), so as to smoothly restore the image resolution and avoid introducing artifacts during the upsampling process. Upsampled feature map F up The preset bilinear interpolation strategy can be expressed as:

[0101]

[0102] in, is the interpolation weight, is the location of the input feature map, is the position of the upsampled feature map, is a natural number.

[0103] In addition, in order to further enhance the accuracy of background reconstruction, a deconvolution operation (also called transposed convolution) is applied to the upsampled feature map. The deconvolution uses a 3×3 convolution kernel and a convolution operation with a stride of 1 to further improve the resolution and detail expression ability of the upsampled feature map. This operation can gradually supplement and optimize high-frequency detail information in the process of restoring the image, and finally obtain the target reconstructed background image.

[0104] In order to optimize the background reconstruction process, the embodiment of the present invention designs a loss function based on mean square error (MSE):

[0105]

[0106] in, Reconstruct the pixel values ​​of the background image for the target, is the actual background pixel value, is the total number of pixels.

[0107] The loss function calculates the difference between the target reconstructed background image and the actual background image, guiding the model to continuously adjust parameters during the training process to minimize the background reconstruction error. Through multiple rounds of iterative training, the model can gradually optimize the accuracy of background reconstruction.

[0108] According to the anomaly detection method based on low-rank embedding and multi-scale proposed in the embodiment of the present invention, the original hyperspectral image can be obtained by preprocessing the hyperspectral image, and the low-rank background feature can be obtained by performing feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model, and then the global and local features of the original hyperspectral image can be extracted by using a pre-built multi-scale fusion feature extraction model; the low-rank background features, global features and local features are integrated and processed, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas. Therefore, by adopting low-rank background feature extraction and multi-scale fusion feature extraction to realize hyperspectral image disaster monitoring, the problems of long detection process, low efficiency and low precision of the existing detection method are solved, and it has the advantages of automation, high precision, fast and high efficiency.

[0109] Next, a low-rank embedding and multi-scale anomaly detection device proposed according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0110] Figure 3 It is a block diagram of an anomaly detection device based on low-rank embedding and multi-scale according to an embodiment of the present invention.

[0111] like Figure 3As shown, the low-rank embedding and multi-scale anomaly detection device 10 includes: an acquisition module 100, an extraction module 200 and an identification module 300.

[0112] The acquisition module 100 is used to acquire a hyperspectral image and preprocess the hyperspectral image to obtain an original hyperspectral image;

[0113] The extraction module 200 is used to perform feature extraction operations on the original hyperspectral image using a preset low-rank background feature extraction model to obtain low-rank background features, and to extract global features and local features of the original hyperspectral image using a pre-built multi-scale fusion feature extraction model;

[0114] The identification module 300 is used to integrate the low-rank background features, global features and local features, compare the obtained target reconstructed background image with the hyperspectral image, and identify abnormal areas.

[0115] Further, in some embodiments, the obtaining module 100 is specifically used for:

[0116] The global spatial and spectral correlations of the hyperspectral image are determined by using a preset low-rank tensor decomposition denoising algorithm. At the same time, the piecewise smoothing characteristics of the hyperspectral image in the spatial dimension are characterized based on the adaptive weight graph total variation strategy and the edge information of the hyperspectral image is retained.

[0117] Furthermore, in some embodiments, the preset low-rank background feature extraction model is:

[0118]

[0119] in, is an operator that flattens a 3D tensor into a matrix in lexicographic order, is a convolutional layer, is another convolutional layer, is the last convolutional layer, is the input three-dimensional tensor, is the activation function.

[0120] Furthermore, in some embodiments, before extracting the global features and local features of the original hyperspectral image using the pre-built multi-scale fusion feature extraction model, the extraction module 200 further includes:

[0121] An acquisition unit is used to acquire a historical hyperspectral image and preprocess the historical hyperspectral image to obtain a preprocessed historical hyperspectral image;

[0122] An extraction unit, for extracting global context information of the preprocessed historical hyperspectral image using a visual transformer, and extracting local information of the preprocessed historical hyperspectral image using a dilated convolutional neural network;

[0123] A fusion unit is used to fuse the global context information and the local information to obtain fused feature data, and divide the fused feature data into a training set and a validation set according to a preset division ratio;

[0124] The training unit is used to train the preset neural network using the training set to obtain an initial neural network model, and to verify the initial neural network model using the verification set until the initial neural network model meets the preset standard, thereby terminating the iterative training of the preset neural network and obtaining a pre-constructed multi-scale fusion feature extraction model; otherwise, adjusting the preset division ratio, and performing iterative training based on the adjusted new training set.

[0125] Further, in some embodiments, the extraction unit is specifically used to:

[0126] By stacking dilated convolutions with different dilation rates, multi-scale image features of the preprocessed historical hyperspectral images are obtained;

[0127] The local features of the preprocessed historical hyperspectral image are extracted using a 3×3 convolution kernel;

[0128] The multi-scale image features and local features are fused to obtain the local information of the preprocessed historical hyperspectral image.

[0129] Furthermore, in some embodiments, the identification module 300 is specifically configured to:

[0130] Perform 1×1 convolution downscaling on the global features and local features respectively to obtain global low-dimensional features and local low-dimensional features;

[0131] The global low-dimensional features and local low-dimensional features are fused using a preset pixel-by-pixel weighting strategy to obtain an initial background reconstructed background image;

[0132] The preset bilinear interpolation strategy is used to upsample the initial background reconstructed background image to obtain upsampled features, and a 3×3 convolution kernel is used to perform a deconvolution operation on the upsampled features to obtain the target reconstructed background image.

[0133] It should be noted that the aforementioned explanation of the embodiment of the anomaly detection method based on low-rank embedding and multi-scale is also applicable to the anomaly detection device based on low-rank embedding and multi-scale of this embodiment, and will not be repeated here.

[0134] According to the low-rank embedding and multi-scale anomaly detection device proposed in the embodiment of the present invention, the original hyperspectral image can be obtained by preprocessing the hyperspectral image, and the low-rank background feature can be obtained by performing feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model, and then the global and local features of the original hyperspectral image can be extracted by using a pre-built multi-scale fusion feature extraction model; the low-rank background features, global features and local features are integrated and processed, and the obtained target reconstructed background image is compared with the hyperspectral image to identify abnormal areas. Therefore, by adopting low-rank background feature extraction and multi-scale fusion feature extraction to realize hyperspectral image disaster monitoring, the problems of long detection process, low efficiency and low precision of the existing detection method are solved, and it has the advantages of automation, high precision, fast and high efficiency.

[0135] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0136] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0137] When the processor 402 executes the program, the low-rank embedding and multi-scale anomaly detection method provided in the above embodiment is implemented.

[0138] Furthermore, the electronic device further comprises:

[0139] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0140] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0141] The memory 401 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0142] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0143] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0144] The processor 402 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0145] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned low-rank embedding and multi-scale anomaly detection method.

[0146] An embodiment of the present invention also provides a computer program product. When the computer program is executed, the above-mentioned anomaly detection method based on low-rank embedding and multi-scale is implemented.

[0147] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0148] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0149] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A low-rank embedding and multi-scale anomaly detection method, characterized in that: The following steps are involved: Acquire a hyperspectral image, and preprocess the hyperspectral image to obtain an original hyperspectral image; Using a preset low-rank background feature extraction model to perform a feature extraction operation on the original hyperspectral image to obtain low-rank background features, and using a pre-built multi-scale fusion feature extraction model to extract global features and local features of the original hyperspectral image; Integrate the low-rank background features, the global features and the local features, compare the obtained target reconstructed background image with the hyperspectral image, and identify abnormal areas; Wherein, before using the pre-constructed multi-scale fusion feature extraction model to extract the global features and local features of the original hyperspectral image, it also includes: obtaining a historical hyperspectral image, and pre-processing the historical hyperspectral image to obtain a pre-processed historical hyperspectral image, using a visual transformer to extract the global context information of the pre-processed historical hyperspectral image, and using an extended convolutional neural network to extract the local information of the pre-processed historical hyperspectral image, fusing the global context information and the local information to obtain fused feature data, dividing the fused feature data into a training set and a verification set according to a preset division ratio, using the training set to train the preset neural network to obtain an initial neural network model, and using the verification set to verify the initial neural network model until the initial neural network model meets the preset standard, ending the iterative training of the preset neural network to obtain the pre-constructed multi-scale fusion feature extraction model, otherwise, adjusting the preset division ratio, and performing iterative training based on the adjusted new training set; Among them, the integration processing of the low-rank background features, the global features and the local features includes: performing 1×1 convolution downscaling processing on the global features and the local features respectively to obtain global low-dimensional features and local low-dimensional features, adopting a preset pixel-by-pixel weighting strategy to fuse the global low-dimensional features and the local low-dimensional features to obtain an initial background reconstructed background image, adopting a preset bilinear interpolation strategy to upsample the initial background reconstructed background image to obtain upsampled features, and using the 3×3 convolution kernel to perform a deconvolution operation on the upsampled features to obtain the target reconstructed background image.

2. The low-rank embedding and multi-scale anomaly detection method according to claim 1, characterized in that: The preprocessing of the hyperspectral image comprises: The global spatial and spectral correlations of the hyperspectral image are determined by using a preset low-rank tensor decomposition denoising algorithm. At the same time, the piecewise smoothing characteristics of the hyperspectral image in the spatial dimension are characterized based on an adaptive weight graph total variation strategy and the edge information of the hyperspectral image is retained.

3. The low-rank embedding and multi-scale anomaly detection method according to claim 1, characterized in that: The preset low-rank background feature extraction model is: in, is an operator that flattens a 3D tensor into a matrix in lexicographic order, is a convolutional layer, is another convolutional layer, is the last convolutional layer, is the input three-dimensional tensor, is the activation function.

4. The low-rank embedding and multi-scale anomaly detection method according to claim 1, characterized in that: The extracting local information of the preprocessed historical hyperspectral image by using the dilated convolutional neural network includes: By stacking dilated convolutions with different dilation rates, multi-scale image features of the preprocessed historical hyperspectral image are obtained; Using a 3×3 convolution kernel to extract local features of the preprocessed historical hyperspectral image; The multi-scale image features and the local features are fused to obtain local information of the preprocessed historical hyperspectral image.

5. An anomaly detection device based on low-rank embedding and multi-scale, characterized in that: include: An acquisition module is used to acquire a hyperspectral image and preprocess the hyperspectral image to obtain an original hyperspectral image; An extraction module is used to perform a feature extraction operation on the original hyperspectral image using a preset low-rank background feature extraction model to obtain low-rank background features, and to extract global features and local features of the original hyperspectral image using a pre-built multi-scale fusion feature extraction model; An identification module is used to integrate the low-rank background features, the global features and the local features, compare the obtained target reconstructed background image with the hyperspectral image, and identify abnormal areas; Wherein, before using the pre-constructed multi-scale fusion feature extraction model to extract the global features and local features of the original hyperspectral image, the extraction module is also used to: obtain a historical hyperspectral image, and pre-process the historical hyperspectral image to obtain a pre-processed historical hyperspectral image, use a visual transformer to extract the global context information of the pre-processed historical hyperspectral image, and use an extended convolutional neural network to extract the local information of the pre-processed historical hyperspectral image, fuse the global context information and the local information to obtain fused feature data, divide the fused feature data into a training set and a verification set according to a preset division ratio, use the training set to train the preset neural network to obtain an initial neural network model, and use the verification set to verify the initial neural network model until the initial neural network model meets the preset standard, end the iterative training of the preset neural network, and obtain the pre-constructed multi-scale fusion feature extraction model; otherwise, adjust the preset division ratio, and perform iterative training based on the adjusted new training set; Among them, the recognition module is specifically used to: perform 1×1 convolution downscaling processing on the global features and the local features respectively to obtain global low-dimensional features and local low-dimensional features, use a preset pixel-by-pixel weighted strategy to fuse the global low-dimensional features and the local low-dimensional features to obtain an initial background reconstructed background image, use a preset bilinear interpolation strategy to upsample the initial background reconstructed background image to obtain upsampled features, and use the 3×3 convolution kernel to perform a deconvolution operation on the upsampled features to obtain the target reconstructed background image.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the low-rank embedding and multi-scale anomaly detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the low-rank embedding and multi-scale anomaly detection method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that When the computer program is executed, it is used to implement the low-rank embedding and multi-scale anomaly detection method according to any one of claims 1 to 4.

Images