Intelligent imaging method and system based on high-resolution spectrum

Abstract

The invention discloses an intelligent imaging method and system based on a high-resolution spectrum, and the method comprises the steps: carrying out the image collection of a target in a preset region, and obtaining a to-be-detected high-resolution spectrum image; pixel shuffling down-sampling is carried out on the to-be-detected high-resolution spectral image to obtain a plurality of sub-images; performing image reconstruction on the plurality of sub-images based on a reconstruction network to obtain a reconstructed image; performing pixel shuffling inverse sampling on the reconstructed image to obtain a to-be-analyzed image corresponding to the to-be-detected high-resolution spectral image; and calculating an abnormal score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the to-be-analyzed image, and determining an image abnormal detection result according to the abnormal score of each pixel. According to the method, large-scale field background information and global information are fully utilized for reconstruction, and pixel shuffling downsampling operation is combined, so that anomaly detection of a large-scale abnormal target is realized, and the high-resolution spectral image anomaly detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an intelligent imaging method and system based on high-resolution spectrum. Background Art

[0002] As one of the important links in the hyperspectral image processing process, anomaly detection of hyperspectral image (HSI) can detect abnormal targets in the scene without using prior spectral information. It has high application value and a wide range of applications.

[0003] In the related technology, the abnormal targets in the image anomaly detection task usually present the following characteristics compared with most of the background: small size, small proportion and certain differences in spectral characteristics; and the commonly used detection algorithms generally reconstruct the image through the neighborhood background information first. Since the spectral characteristics of the abnormal targets are quite different from those of the background, the reconstruction results often only retain the background, and the anomalies are manifested as reconstruction errors, resulting in inaccurate detection results; in addition, the anomaly detection algorithm based on the blind spot network adopts a small-area reconstruction strategy, but since the target size of the high-resolution hyperspectral image HRHSI is much larger than the target size of the medium and low-resolution HSI, relying only on the information of a small area can easily cause the reconstruction of large targets, resulting in missed detections, and then resulting in inaccurate image anomaly detection results. Summary of the invention

[0004] The present invention provides an intelligent imaging method and system based on high-resolution spectrum, which are used to improve the accuracy of image anomaly detection results.

[0005] In a first aspect, the present invention provides an intelligent imaging method based on high-resolution spectroscopy, comprising:

[0006] The high-resolution hyperspectral instrument is used to collect images of the target in the preset area to obtain a high-resolution spectral image to be detected;

[0007] Performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images;

[0008] Reconstructing the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of the potential abnormal pixel in the sample high-resolution spectral image;

[0009] Perform pixel shuffling and reverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected;

[0010] The anomaly score of each pixel is calculated according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and the image anomaly detection result is determined according to the anomaly score of each pixel.

[0011] According to the intelligent imaging method based on high-resolution spectroscopy provided by the present invention, the reconstruction network is trained by the following steps:

[0012] Perform pixel shuffling down-sampling on the sample high-resolution spectral image to obtain multiple sample sub-images;

[0013] Input the multiple sample sub-images into the reconstruction network, use the Adam algorithm and update the network weights of the generator according to the optimization objective function, and obtain the reconstruction network when the number of updates reaches the maximum number of iterations or the generator converges;

[0014] The optimization objective function is determined by the network weights, network parameters, multiple sample sub-image sets and reconstructed sample sub-image sets of the generator.

[0015] According to the intelligent imaging method based on high-resolution spectrum provided by the present invention, the generator includes a large-kernel center block mask convolution module, a channel attention mechanism ECA module and a dilated convolution reconstruction module;

[0016] The large-core central block mask convolution module is used to reconstruct pixels of large-scale surrounding background information obtained from the high-resolution spectral image to obtain a background feature map;

[0017] The ECA module is used to perform one-dimensional convolution on the background feature map to obtain a new feature map, and the information of each band in the new feature map is a linear combination of the bands and adjacent bands at the same position in the background feature map; the dilated convolution reconstruction module is used to fit the background information using pixels at positions corresponding to the high-resolution spectral image to obtain a reconstructed image when reconstructing the image according to the new feature map.

[0018] According to the intelligent imaging method based on high-resolution spectroscopy provided by the present invention, the optimization objective function is expressed by the following formula:

[0019]

[0020] in, represents the value of the optimization objective function; G(W G ,X) represents the generator, W G represents the network parameters of the generator, W nrepresents the network weight of the generator, X represents a set of multiple sample sub-images, N represents the number of pixels in the sample sub-image, L represents the number of bands, and P represents the spatial size of the sample sub-image. represents the reconstructed sample sub-image set; || ||1 represents the L1 norm operation, and AeB represents the element-by-element multiplication operation between A and B.

[0021] According to the intelligent imaging method based on high-resolution spectrum provided by the present invention, the abnormal score of each pixel is calculated according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, including:

[0022] Performing differential processing on the high-resolution spectral image to be detected and the image to be analyzed to obtain the differential image;

[0023] Calculating the mean and covariance of all pixels in the differential image, and calculating the L2 distance and Mahalanobis distance of each pixel according to the mean and the covariance;

[0024] The anomaly score is determined by calculating the L2 distance of each pixel and the average of the Mahalanobis distance.

[0025] According to the intelligent imaging method based on high-resolution spectrum provided by the present invention, the pixel shuffling and down-sampling of the high-resolution spectrum image to be detected to obtain multiple sub-images includes:

[0026] The high-resolution spectral image to be detected is split according to the area of ​​the abnormal target to obtain the multiple sub-images; different sub-images include different parts of the abnormal target.

[0027] In a second aspect, the present invention further provides an intelligent imaging system based on high-resolution spectroscopy, comprising:

[0028] An image acquisition module is used to acquire images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectral image to be detected;

[0029] A first image sampling module, used for performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images;

[0030] An image reconstruction module, used to reconstruct the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of potential abnormal pixels in the sample high-resolution spectral image;

[0031] A second image sampling module is used to perform pixel shuffling and inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected;

[0032] The detection module is used to calculate the anomaly score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and determine the image anomaly detection result according to the anomaly score of each pixel.

[0033] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing the intelligent imaging method based on high-resolution spectroscopy as described in the first aspect above.

[0034] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, wherein the storage medium stores a computer software program, and when the computer software program is executed by a processor, the intelligent imaging method based on high-resolution spectroscopy as described in the first aspect above is implemented.

[0035] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the intelligent imaging method based on high-resolution spectroscopy as described in the first aspect above.

[0036] The intelligent imaging method based on high-resolution spectrum provided by the present invention collects images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectrum image to be detected; performs pixel shuffling down-sampling on the high-resolution spectrum image to be detected to obtain a plurality of sub-images; performs image reconstruction on the plurality of sub-images based on a reconstruction network to obtain a reconstructed image; performs pixel shuffling inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectrum image to be detected; calculates the anomaly score of each pixel according to statistical data corresponding to all pixels in a differential image corresponding to the image to be analyzed, and determines the image anomaly detection result according to the anomaly score of each pixel, thereby making full use of large-scale field background information and global information for reconstruction, and combining the pixel shuffling down-sampling operation, thereby realizing anomaly detection of large-scale abnormal targets and improving the accuracy of anomaly detection of high-resolution spectrum images. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the process of the intelligent imaging method based on high-resolution spectrum provided by the present invention;

[0038] Figure 2 It is a schematic diagram of pixel shuffling downsampling provided by the present invention;

[0039] Figure 3 It is a structural schematic diagram of the reconstruction network provided by the present invention;

[0040] Figure 4 It is a structural schematic diagram of an intelligent imaging system based on high-resolution spectroscopy provided by the present invention;

[0041] Figure 5 A schematic diagram of an electronic device according to an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0044] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0045] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0046] Optional, see Figure 1 , Figure 1 : is a flow chart of the intelligent imaging method based on high-resolution spectroscopy provided by the present invention. The execution subject of the intelligent imaging method based on high-resolution spectroscopy provided by the present invention can be an intelligent imaging system. Therefore, the intelligent imaging method based on high-resolution spectroscopy includes the following steps:

[0047] Step 10: collect images of the target in the preset area by a high-resolution hyperspectral instrument to obtain a high-resolution spectral image to be detected. Step 20: perform pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain multiple sub-images.

[0048] In this embodiment, the high-resolution hyperspectral instrument is used to obtain a high-resolution spectral image. Therefore, when it is necessary to obtain a high-resolution spectral image of a target in a certain area, an image acquisition instruction can be sent to the intelligent imaging system. The intelligent imaging system calls the high-resolution hyperspectral instrument to acquire an image of the target in the preset area according to the image acquisition instruction to obtain a high-resolution spectral image to be detected.

[0049] This embodiment uses a pixel shuffling method to preprocess the high-resolution spectral image to be detected, and reduces the spatial correlation between abnormal pixels and background pixels by splitting the original image into several sub-images.

[0050] Specifically, performing pixel shuffling and down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images includes: splitting the high-resolution spectral image to be detected according to the area of ​​the abnormal target to obtain a plurality of sub-images; different sub-images include different parts of the abnormal target.

[0051] In this embodiment, by splitting the original image (the high-resolution spectral image to be detected), and then splitting the abnormal target in the original image, the scale of the abnormal target in each sub-image can be reduced; it should be noted that the above pixel shuffling downsampling operation will not lose the information of the original image

[0052] See also Figure 2 , Figure 2 is a schematic diagram of pixel shuffling downsampling provided by the present invention. Figure 2 In the embodiment shown, the high-resolution spectrum image to be detected is divided into 9 sub-images, wherein each sub-image may include 4 sub-areas (the sub-image corresponding to the upper right corner of the left image includes a 11 、b 11 、c 11 d 11 , the corresponding sub-image in the upper right corner of the right image includes a 11 、a 12 、a 21 、a 22 ); Then, in the process of downsampling PD, the first reference sub-image corresponding to the position of the first sub-image in the downsampled noise image is obtained as follows: the first sub-regions of the first sub-image to the ninth sub-image are sequentially obtained from the left image to obtain the right image; in the process of inverse sampling PD -1 In the process of downsampling PD, the right image sequentially obtains the first sub-regions of the first sub-image to the ninth sub-image and combines them to obtain the left image (which is actually the inverse process of downsampling PD).

[0053] Step 30: reconstruct the multiple sub-images based on the reconstruction network to obtain a reconstructed image.

[0054] Among them, the reconstruction network is obtained by iteratively training the generator using the sample high-resolution spectral image as the training sample and the optimization objective function as the loss function; the optimization objective function is used to reduce the weight of the location of potential abnormal pixels in the sample high-resolution spectral image.

[0055] The generator in this embodiment can refer to the generator G in the Generative Adversarial Networks (GAN).

[0056] In this embodiment, the generator includes a large-kernel center block mask convolution module, a channel attention mechanism ECA module, and a dilated convolution reconstruction module.

[0057] For example, the large-kernel central block mask convolution module is used to obtain large-scale surrounding background information from the original image for pixel reconstruction, and the ECA module uses the spectral difference between the abnormal target and the background to suppress abnormal pixels. The dilated convolution reconstruction module uses the dilated convolution of the residual structure to use the information of the pixel at the same position in the original image to fit the background during reconstruction.

[0058] In this embodiment, the optimization objective function is expressed by the following formula:

[0059]

[0060] in, represents the value of the optimization objective function; G(W G , X) represents the generator, W G represents the network parameters of the generator, W n represents the network weight of the generator, X represents a set of multiple sample sub-images, N represents the number of pixels in the sample sub-image, L represents the number of bands, and P represents the spatial size of the sample sub-image. represents the reconstructed sample sub-image set; || ||1 represents the L1 norm operation, and AeB represents the element-by-element multiplication operation between A and B.

[0061] In this embodiment, the dataset used is first defined as a set of image blocks of the original image after preprocessing. in, represents P rows, P columns, and L channels, N represents the number of pixels, L represents the number of bands, and P represents the spatial size of the image block. The output image block set after the reconstruction network The optimization goal of the reconstruction network G is to make the information without abnormal pixels close to X, to ensure that G can focus on restoring background pixels and prevent the network from learning the identity mapping between input and output.

[0062] In this embodiment, when designing the objective function, firstly, X and Do the difference Then let the difference between the two and the maximum value of each channel form a new three-dimensional matrix Δ max , and then make the difference between the two W = Δ max -Δ, let W n =(WW min ) / (W max -W min The optimization goal of the network is to minimize the following objective function based on L1 loss, thereby optimizing the network parameters (W G ).

[0063] Input multiple sub-images into the above-trained reconstruction network to obtain reconstructed images.

[0064] Step 40, performing pixel shuffling and inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected.

[0065] Step 50, calculating the abnormality score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and determining the image abnormality detection result according to the abnormality score of each pixel.

[0066] Specifically, calculating the abnormal score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed includes: performing differential processing on the high-resolution spectral image to be detected and the imaging to be analyzed to obtain a differential image; calculating the mean and covariance of all pixels in the differential image, and calculating the L2 distance and Mahalanobis distance of each pixel according to the mean and covariance; calculating the mean of the L2 distance and Mahalanobis distance of each pixel to determine the abnormal score.

[0067] In this embodiment, the steps of implementing anomaly detection on the hyperspectral image include:

[0068] Step 1. Input: hyperspectral image X, image downsampling factor n, number of iterations n_iter, initial number of iterations n 0- iter; It is a three-dimensional cube-shaped hyperspectral image with R rows, C columns and L channels. L is the number of bands, R and C are the rows and columns of the spatial dimension, N = R × C represents the number of pixels in the image, and X can also be regarded as a pixel set X = {x1, ..., x i , ..., x N},

[0069] Step 2: Perform pixel shuffling downsampling operation PD(X).

[0070] Step 3: Input PD(X) into the trained reconstruction network G t In the image to be analyzed, in,

[0071] Step 4: Perform pixel shuffling and inverse sampling on the reconstructed image to obtain the reconstruction result of the original image.

[0072] Step 5: Subtract the original image from the reconstructed image to obtain a differential image.

[0073] Step 6: Calculate all pixels of the difference image R The mean value u r and the covariance matrix C r ,in,

[0074]

[0075] At the same time, calculate the pixel r in the difference image R i The L2 distance LD(r i )=‖|r i |‖2 and Mahalanobis distance The mean of the two is taken as the abnormal score D(r i )=LD(r i )+MD(r i ).

[0076] Step 8. Output: Final anomaly detection result D of the entire image f ; Among them, D f It can be expressed as:

[0077] D f = {D(r1), ..., D(r i ), ..., D(r N )}.

[0078] The embodiment of the present invention collects images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectral image to be detected; performs pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain multiple sub-images; performs image reconstruction on the multiple sub-images based on a reconstruction network to obtain a reconstructed image; performs pixel shuffling inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected; calculates the anomaly score of each pixel according to statistical data corresponding to all pixels in a differential image corresponding to the image to be analyzed, and determines the image anomaly detection result according to the anomaly score of each pixel. Therefore, full use is made of large-scale field background information and global information for reconstruction, and combined with the pixel shuffling down-sampling operation, anomaly detection of large-scale abnormal targets is achieved, and the accuracy of anomaly detection of high-resolution spectral images is improved.

[0079] In one embodiment, the reconstruction network is trained by the following steps: pixel shuffling and downsampling of a sample high-resolution spectral image to obtain a plurality of sample sub-images; the plurality of sample sub-images are input into a generator, the Adam algorithm is used to update the network weights of the generator according to the optimization objective function, and the reconstruction network is obtained when the number of updates reaches the maximum number of iterations or the generator converges; the reconstruction network optimization objective function is determined by the network weights of the generator, the network parameters, the plurality of sample sub-image sets and the reconstructed sample sub-image set.

[0080] In one embodiment, from the hyperspectral image anomaly detection stage, step 1, input the hyperspectral image; step 2, perform pixel shuffling downsampling operation on the original image; step 2, obtain the reconstruction network trained in the reconstruction network training stage; step 4, input the downsampled image into the trained network to obtain a reconstructed image (the reconstruction here is the same operation as the above reconstruction); step 5, perform pixel shuffling inverse sampling operation on the reconstructed image to obtain a reconstructed image of the original image; step 6, subtract the original image from the inversely sampled image to be analyzed to obtain a differential image; step 7, combine the L2 distance of each pixel of the differential image and the Mahalanobis distance distance, determine the anomaly score; step 8, output the detection result of the whole image; in the reconstruction network training stage, step 1, input the training sample; step 2, build the reconstruction network and design the optimization objective function; step 3, the current number of iterations is equal to the initial number of iterations; step 4, use the Adam algorithm to update the network weight according to the objective function of the network; step 5, update the current number of iterations (that is, the number of iterations + 1); step 6, determine whether the current number of iterations is equal to the set number of iterations; step 7, when the current number of iterations is equal to the set number of iterations, output the trained network, that is, the reconstruction network.

[0081] The embodiment of the present invention inputs multiple sample sub-images obtained by pixel shuffling and downsampling the sample high-resolution spectral image into a generator, adopts the Adam algorithm and updates the network weights of the generator according to the optimization objective function, and obtains a reconstructed network when the number of updates reaches the maximum number of iterations or the generator converges, thereby improving the detection accuracy and robustness of the reconstructed network.

[0082] In one embodiment, the generator includes a large kernel center block mask convolution module, a channel attention mechanism ECA module and an expanded convolution reconstruction module; the large kernel center block mask convolution module is used to reconstruct pixels of large-scale surrounding background information obtained from a high-resolution spectral image to obtain a background feature map; the ECA module is used to perform a one-dimensional convolution on the background feature map to obtain a new feature map, and the information of each band in the new feature map is a linear combination of the bands and adjacent bands at the same position in the background feature map; the expanded convolution reconstruction module is used to fit the background information using pixels at positions corresponding to the high-resolution spectral image when reconstructing the image according to the new feature map to obtain a reconstructed image.

[0083] In this embodiment, the loss function during network training can reduce the weight of the location of potential abnormal pixels during the training process, thereby suppressing the reconstruction of abnormal pixels during network training.

[0084] Furthermore, the operations required in this embodiment are described.

[0085] 1. Two-dimensional convolution operation; using the two-dimensional convolution function: use a convolution layer w1 composed of n convolution kernels with spatial size t×t×C to perform convolution on the M rows, N columns and C channels image The convolution operation is recorded as: F Conv2D () represents a two-dimensional convolution operation, where It is the M-row, N-column, C-channel output image, and d represents the dilation rate of the convolution.

[0086] 2. One-dimensional convolution operation; the one-dimensional convolution function uses a convolution layer w2 composed of n convolution kernels with spatial size of 1×1×t to perform an M-row, N-column, C-channel image The convolution operation is recorded as:

[0087] F Conv1D (w2, X2), Y2 = F Conv1D (w2,X2),F Conv1D () represents a one-dimensional convolution operation, where It is an M-row, N-column, C-channel output image.

[0088] 3. Exponential linear unit (ELU) function:

[0089] (1) For the value x, the ELU function is defined as: y = fE LU (x) = max{α(e x -1), x}, where e is the base of natural logarithms, approximately equal to 2.71828.

[0090] (2) Vector The ELU function is defined as:

[0091] y=F ELU (x), where the vector And y n =F ELU (x n ).

[0092] 4. For M rows, N columns, C channel image An image with M rows, N columns and C channels The element-by-element summation operation between the two (i.e., pixel-by-pixel summation) is recorded as The result of the operation (pixel-by-pixel sum) It is still an image with spatial size of M rows, N columns and C channels, where z ijc =x ijc +y ijc .

[0093] 5. Element-by-element product: for M rows, N columns, C channel image An image with M rows, N columns and C channels The element-by-element multiplication operation (i.e., pixel-by-pixel product) between the two is recorded as z=Xe Y, and the result of the operation (pixel-by-pixel product) is It is still an image with spatial size of M rows, N columns and C channels, where z ijc =x ijc ×y ijc .

[0094] 6. Efficient Channel Attention (ECA); ECA is a one-dimensional convolution that adaptively adjusts the convolution kernel size according to the number of channels in the input feature map. The convolution kernel size is:

[0095]

[0096] The parameters b and γ are set to 1 and 2 respectively, and the output is: Y3 = F Conv1D (w3, X3) k , where k represents the convolution kernel size is k.

[0097] See also Figure 3 , Figure 3 is a schematic diagram of the structure of the reconstruction network provided by the present invention. Figure 3 In the embodiment shown, the network structure of the generator G is as follows Figure 3 shown.

[0098] The first 6 layers are composed of convolutional layers (Conv2D) and ELU units stacked together, where the third layer is a 23×23 convolution. The large kernel center block mask convolution module is represented as:

[0099]

[0100] Among them, w BC and w C ={w C , w' C} are the learnable weight parameters of the module and convolutional layer, m represents the y mask with a center size of 5×5, and m t ∈{0, 1} 23×23×L ; ECA[1] is a one-dimensional convolution with adaptive convolution kernel size, and the output is expressed as:

[0101]

[0102] The dilated convolution reconstruction module consists of 9 dilated convolutions with residual structures, followed by ELU units and 1×1 convolutions. The module is represented as:

[0103]

[0104] Among them, w RC is the learnable weight parameter of the module, and the output is:

[0105]

[0106] If the input of the reconstruction network G is a set of image blocks The image block X in i , the output can be expressed as:

[0107]

[0108] In this embodiment, the whole process is recorded as:

[0109] in, is all the learnable weight parameters of the reconstructed network G, is the set of output image patches.

[0110] The embodiment of the present invention designs a generator through a large-core central block mask convolution module, a channel attention mechanism ECA module and a dilated convolution reconstruction module, which can obtain large-scale surrounding background information for pixel reconstruction and suppress the reconstruction of abnormal pixels during network training, thereby improving the accuracy of the network output results.

[0111] The intelligent imaging system based on high-resolution spectroscopy provided by the present invention is described below. The intelligent imaging system based on high-resolution spectroscopy described below and the intelligent imaging method based on high-resolution spectroscopy described above can be referred to each other.

[0112] See also Figure 4 , Figure 4 A schematic diagram of the structure of an intelligent imaging system based on high-resolution spectroscopy provided in an embodiment of the present invention, wherein the intelligent imaging system based on high-resolution spectroscopy includes:

[0113] An image acquisition module 410 is used to acquire images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectral image to be detected;

[0114] A first image sampling module 420 is used to perform pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images;

[0115] An image reconstruction module 430 is used to reconstruct the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of the potential abnormal pixel in the sample high-resolution spectral image;

[0116] The second image sampling module 440 is used to perform pixel shuffling inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected;

[0117] The detection module 450 is used to calculate the anomaly score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and determine the image anomaly detection result according to the anomaly score of each pixel.

[0118] The intelligent imaging system based on high-resolution spectrum provided by the present invention collects images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectrum image to be detected; performs pixel shuffling down-sampling on the high-resolution spectrum image to be detected to obtain a plurality of sub-images; performs image reconstruction on the plurality of sub-images based on a reconstruction network to obtain a reconstructed image; performs pixel shuffling inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectrum image to be detected; calculates the anomaly score of each pixel according to statistical data corresponding to all pixels in a differential image corresponding to the image to be analyzed, and determines the image anomaly detection result according to the anomaly score of each pixel, thereby making full use of large-scale field background information and global information for reconstruction, and combining the pixel shuffling down-sampling operation, thereby realizing anomaly detection of large-scale abnormal targets and improving the accuracy of anomaly detection of high-resolution spectrum images.

[0119] See also Figure 5 , Figure 5 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented:

[0120] The high-resolution hyperspectral instrument is used to collect images of the target in the preset area to obtain a high-resolution spectral image to be detected;

[0121] Performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images;

[0122] Reconstructing the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of the potential abnormal pixel in the sample high-resolution spectral image;

[0123] Perform pixel shuffling and reverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected;

[0124] The anomaly score of each pixel is calculated according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and the image anomaly detection result is determined according to the anomaly score of each pixel.

[0125] See also Figure 6 , Figure 6 Schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented:

[0126] The high-resolution hyperspectral instrument is used to collect images of the target in the preset area to obtain a high-resolution spectral image to be detected;

[0127] Performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images;

[0128] Reconstructing the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of the potential abnormal pixel in the sample high-resolution spectral image;

[0129] Perform pixel shuffling and reverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected;

[0130] The anomaly score of each pixel is calculated according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and the image anomaly detection result is determined according to the anomaly score of each pixel.

[0131] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent imaging method based on high-resolution spectroscopy, characterized in that: include: The high-resolution hyperspectral instrument is used to collect images of the target in the preset area to obtain a high-resolution spectral image to be detected; Performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images; Reconstructing the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of the potential abnormal pixel in the sample high-resolution spectral image; Perform pixel shuffling and reverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected; The anomaly score of each pixel is calculated according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and the image anomaly detection result is determined according to the anomaly score of each pixel.

2. The intelligent imaging method based on high-resolution spectrum according to claim 1, characterized in that: The reconstruction network is trained by the following steps: Perform pixel shuffling down-sampling on the sample high-resolution spectral image to obtain multiple sample sub-images; Input the multiple sample sub-images into the reconstruction network, use the Adam algorithm and update the network weights of the generator according to the optimization objective function, and obtain the reconstruction network when the number of updates reaches the maximum number of iterations or the generator converges; The optimization objective function is determined by the network weights, network parameters, multiple sample sub-image sets and reconstructed sample sub-image sets of the generator.

3. The intelligent imaging method based on high-resolution spectroscopy according to claim 1 or 2, characterized in that: The generator includes a large core center block mask convolution module, a channel attention mechanism ECA module and a dilated convolution reconstruction module; The large-core central block mask convolution module is used to reconstruct pixels of large-scale surrounding background information obtained from the high-resolution spectral image to obtain a background feature map; The ECA module is used to perform one-dimensional convolution on the background feature map to obtain a new feature map, and the information of each band in the new feature map is a linear combination of the bands and adjacent bands at the same position in the background feature map; the dilated convolution reconstruction module is used to fit the background information using pixels at positions corresponding to the high-resolution spectral image to obtain a reconstructed image when reconstructing the image according to the new feature map.

4. The intelligent imaging method based on high-resolution spectroscopy according to claim 1 or 2, characterized in that: The optimization objective function is expressed by the following formula: in, represents the value of the optimization objective function; G(W G ,X) represents the generator, W G represents the network parameters of the generator, W n represents the network weight of the generator, X represents a set of multiple sample sub-images, N represents the number of pixels in the sample sub-image, L represents the number of bands, and P represents the spatial size of the sample sub-image. represents the reconstructed sample sub-image set; || ||1 represents the L1 norm operation, and AeB represents the element-by-element multiplication operation between A and B.

5. The intelligent imaging method based on high-resolution spectrum according to claim 1, characterized in that: The step of calculating the abnormal score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed includes: Performing differential processing on the high-resolution spectral image to be detected and the image to be analyzed to obtain the differential image; Calculating the mean and covariance of all pixels in the differential image, and calculating the L2 distance and Mahalanobis distance of each pixel according to the mean and the covariance; The anomaly score is determined by calculating the L2 distance of each pixel and the average of the Mahalanobis distance.

6. The intelligent imaging method based on high-resolution spectroscopy according to claim 1, characterized in that: The pixel shuffling and downsampling of the high-resolution spectral image to be detected to obtain multiple sub-images includes: The high-resolution spectral image to be detected is split according to the area of ​​the abnormal target to obtain the multiple sub-images; different sub-images include different parts of the abnormal target.

7. An intelligent imaging system based on high-resolution spectroscopy, characterized in that: include: An image acquisition module is used to acquire images of targets in a preset area through a high-resolution hyperspectral instrument to obtain a high-resolution spectral image to be detected; A first image sampling module, used for performing pixel shuffling down-sampling on the high-resolution spectral image to be detected to obtain a plurality of sub-images; An image reconstruction module, used to reconstruct the multiple sub-images based on a reconstruction network to obtain a reconstructed image; wherein the reconstruction network is obtained by iteratively training a generator using a sample high-resolution spectral image as a training sample and an optimization objective function as a loss function; the optimization objective function is used to reduce the weight of the position of potential abnormal pixels in the sample high-resolution spectral image; A second image sampling module is used to perform pixel shuffling and inverse sampling on the reconstructed image to obtain an image to be analyzed corresponding to the high-resolution spectral image to be detected; The detection module is used to calculate the anomaly score of each pixel according to the statistical data corresponding to all pixels in the differential image corresponding to the imaging to be analyzed, and determine the image anomaly detection result according to the anomaly score of each pixel.

8. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the intelligent imaging method based on high-resolution spectroscopy as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, and when the computer software program is executed by a processor, the intelligent imaging method based on high-resolution spectroscopy as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the intelligent imaging method based on high-resolution spectroscopy as claimed in any one of claims 1 to 6 is implemented.

Images