A Hyperspectral Change Detection Method Based on Prototype Analysis and Convolutional Neural Network

Abstract

The present invention discloses a hyperspectral change detection method based on prototype analysis and convolutional neural network. The method includes: preprocessing two hyperspectral images of different time phases to be detected to obtain observed data; estimating the virtual dimension of the observed data based on noise whitening processing to obtain an estimated value; determining the ground object change situation according to the first estimated value and the second estimated value; performing spectral unmixing on the observed data through a relaxation factor to obtain a target abundance matrix; calculating an affinity matrix based on the target abundance matrix; and obtaining a change detection result through a trained network model based on the affinity matrix. By introducing a relaxation factor for spectral unmixing, the present invention improves the generality, and pre-estimates the ground object categories of the image by the virtual dimension method, adding the analysis of the ground object category change to the traditional change detection process, improving the interpretability of the change detection result, and can be widely applied to the field of change detection technology.

Description

Technical Field

[0001] The present invention relates to the technical field of change detection, and in particular to a hyperspectral change detection method based on prototype analysis and convolutional neural network. Background Art

[0002] As an important application of remote sensing image analysis, change detection specifically refers to the process of analyzing and determining the ground objects that have changed by using two remote sensing images of the same location at different times. The changes in ground objects include changes in spectral characteristics and changes in the spatial position of objects. It is an effective technical means for monitoring land cover changes, urban changes, etc. In remote sensing images, hyperspectral images have become an important data source for ground object observation due to their rich spectral information. Since hyperspectral imaging technology is relatively new, the change detection methods for hyperspectral remote sensing images are not yet mature.

[0003] Due to the low spatial resolution of hyperspectral images, in complex ground object scenarios, it is easy to have multiple ground objects in the same pixel. Spectral unmixing technology can decompose mixed pixels into different endmembers and calculate the proportion of each endmember, that is, abundance. However, traditional spectral unmixing technology has poor adaptability to data. Therefore, how to achieve a spectral unmixing method with better generality is an urgent problem to be solved.

[0004] Moreover, in the existing change detection technologies, most of them only focus on whether there are changes in ground objects in the observation scene, but rarely focus on the categories of ground object changes. Summary of the Invention

[0005] In view of this, the embodiments of the present invention provide a hyperspectral change detection method based on prototype analysis and convolutional neural network, which can effectively solve the problems existing in the prior art.

[0006] On the one hand, the embodiments of the present invention provide a hyperspectral change detection method based on prototype analysis and convolutional neural network, including:

[0007] Preprocess two hyperspectral images of different time phases to be detected to obtain observation data;

[0008] Based on noise whitening processing, estimate the virtual dimension of the observation data to obtain an estimated value; wherein, the estimated value includes a first estimated value and a second estimated value;

[0009] Determine the ground object change situation according to the first estimated value and the second estimated value;

[0010] Perform spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix;

[0011] Calculate an affinity matrix based on the target abundance matrix;

[0012] Based on the affinity matrix, the change detection result is obtained through the trained network model.

[0013] Optionally, preprocessing the hyperspectral images of the two temporal phases to be detected to obtain observation data, including:

[0014] Preprocessing the hyperspectral images of the two temporal phases to be detected through ENVI remote sensing image processing software to obtain observation data;

[0015] Among them, the observation data includes first observation data and second observation data; the preprocessing includes radiometric correction, geometric correction, and image registration.

[0016] Optionally, based on noise whitening processing, estimating the virtual dimension of the observation data to obtain an estimated value, including:

[0017] Determine the noise covariance matrix according to the observation data;

[0018] Based on the eigenvalues and eigenvectors obtained by eigenvalue decomposition of the noise covariance matrix, perform noise whitening processing on the observation data to obtain whitened observation data;

[0019] Determine the correlation matrix of the whitened observation data and the whitened noise covariance matrix according to the whitened observation data to obtain the whitened signal correlation matrix;

[0020] According to the whitened signal correlation matrix, obtain the estimated value through the Hysime model;

[0021] Among them, the estimated value includes the subspace eigenvector and the arrangement order of the target eigenvectors; by estimating the virtual dimension of the observation data corresponding to the hyperspectral images of the two temporal phases to be detected respectively, the first estimated value and the second estimated value are obtained.

[0022] Optionally, the determining the correlation matrix of the whitened observation data and the whitened noise covariance matrix according to the whitened observation data to obtain the whitened signal correlation matrix includes:

[0023] Determine the correlation matrix of the whitened observation data and the whitened noise covariance matrix according to the whitened observation data, and calculate the whitened signal correlation matrix through the signal correlation matrix formula;

[0024] Among them, the signal correlation matrix formula is:

[0025]

[0026] In the formula, represents the whitened signal correlation matrix, R yw represents the correlation matrix of the whitened observation signal, denotes the covariance matrix of white noise, and I denotes the identity matrix.

[0027] Optionally, determining the ground object change situation according to the first estimation value and the second estimation value includes:

[0028] Taking the difference between the first estimation value and the second estimation value to obtain the changed ground object category and the unchanged ground object category of the hyperspectral images of two time phases;

[0029] Based on the changed ground object category and the unchanged ground object category, organizing to obtain the ground object change situation.

[0030] Optionally, the spectral unmixing of the observation data by the relaxation factor to obtain the target abundance matrix includes:

[0031] Initializing the prototype vector and the abundance matrix based on the observation data;

[0032] Iteratively updating the variable factor by the projection gradient method and continuously adjusting the relaxation factor until a preset error threshold is reached to obtain the target abundance matrix;

[0033] Wherein, the target abundance matrix includes a first target abundance matrix and a second target abundance matrix; the first target abundance matrix and the second target abundance matrix are obtained by performing spectral unmixing on the observation data corresponding to the hyperspectral images of two time phases to be detected respectively.

[0034] Optionally, the target abundance matrix includes a first target abundance matrix and a second target abundance matrix, and calculating the affinity matrix based on the target abundance matrix includes:

[0035] According to the first target abundance matrix, combining with the pixels of the hyperspectral image of the first time phase to obtain a first data block set;

[0036] According to the second target abundance matrix, combining with the pixels of the hyperspectral image of the second time phase to obtain a second data block set;

[0037] According to the first data block set and the second data block set, calculating the affinity matrix of each pixel corresponding to the hyperspectral images of two time phases through the affinity matrix formula;

[0038] Wherein, the affinity matrix formula is:

[0039] K = 1 - (R1 - R2) / R2

[0040] Wherein, K represents the affinity matrix, R1 represents the data block in the first data block set, R2 represents the data block in the second data block set, and the corresponding pixels of R1 and R2 are in the same positions in the hyperspectral images of two time phases.

[0041] Optionally, it further includes:

[0042] Train the convolutional neural network to obtain a trained network model.

[0043] Optionally, the training the convolutional neural network to obtain a trained network model includes:

[0044] Based on a preset ratio, divide the pixels in the hyperspectral image into a training data set and a test data set;

[0045] Calculate the affinity matrix for each pixel in the training data set and input it into the convolutional neural network;

[0046] Optimize the loss function by the stochastic gradient descent method and perform supervised training on the convolutional neural network based on a preset period;

[0047] When the model metric class average accuracy and mean intersection over union meet the preset conditions, obtain a trained network model.

[0048] On the other hand, an embodiment of the present invention provides a hyperspectral change detection system based on prototype analysis and convolutional neural network, including:

[0049] A first module for preprocessing the hyperspectral images of two time phases to be detected to obtain observation data;

[0050] A second module for performing virtual dimensionality estimation on the observation data based on noise whitening processing to obtain estimated values; wherein, the estimated values include a first estimated value and a second estimated value;

[0051] A third module for determining the ground object change situation according to the first estimated value and the second estimated value;

[0052] A fourth module for performing spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix;

[0053] A fifth module for calculating an affinity matrix based on the target abundance matrix;

[0054] A sixth module for obtaining a change detection result through the trained network model based on the affinity matrix.

[0055] On the other hand, an embodiment of the present invention provides a hyperspectral change detection system based on prototype analysis and convolutional neural network, including a processor and a memory;

[0056] The memory is used to store programs;

[0057] The processor executes the program to implement the method as described above.

[0058] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, and the storage medium stores a program, and the program is executed by a processor to implement the method as described above.

[0059] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the foregoing method.

[0060] In an embodiment of the present invention, first, hyperspectral images of two time phases to be detected are preprocessed to obtain observation data; based on noise whitening processing, virtual dimension estimation is performed on the observation data to obtain an estimated value; wherein the estimated value includes a first estimated value and a second estimated value; according to the first estimated value and the second estimated value, the ground object change situation is determined; spectral unmixing is performed on the observation data through a relaxation factor to obtain a target abundance matrix; based on the target abundance matrix, an affinity matrix is calculated; based on the affinity matrix, a change detection result is obtained through a trained network model. By introducing a relaxation factor for spectral unmixing, the present invention improves the generality, and pre-estimates the ground object categories of the image by the virtual dimension method, and adds the analysis of the change of the ground object categories to the traditional change detection process, thereby improving the interpretability of the change detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0062] Figure 1 It is a schematic diagram of the overall process of a hyperspectral change detection method based on prototype analysis and convolutional neural network provided by an embodiment of the present invention;

[0063] Figure 2 It is a schematic diagram of the overall process of a hyperspectral change detection method based on prototype analysis and convolutional neural network provided by an embodiment of the present invention;

[0064] Figure 3Schematic flowchart of virtual dimensionality estimation provided by an embodiment of the present invention;

[0065] Figure 4 Schematic flowchart of spectral unmixing provided by an embodiment of the present invention. Detailed implementation manners

[0066] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] First of all, it should be noted that for the pixel unmixing technology, prototype analysis is used as a convex optimization algorithm to extract the main convex set of the dataset. Under the condition that the number of given prototype samples is equal to the number of endmembers, prototype analysis extracts the pure sample set from the hyperspectral image data, and the obtained coefficient matrix is the abundance matrix S of the mixed pixels. Its objective equation is as follows:

[0068]

[0069] s.t. 1 - δ ≤ α d ≤ 1 + δ

[0070] C ≥ 0, S ≥ 0

[0071] |c d | = 1, |s n | = 1

[0072] The constraints of C ≥ 0 and |c d | = 1 are for the weighted combination conditions of data samples, and S ≥ 0 and |s n | = 1 ensure the requirements that the data samples are the weighted combinations of prototype samples.

[0073] Compared with the traditional spectral unmixing model, the prototype analysis model can provide more characteristic information about the mixed data in addition to the endmembers and abundances, but it has poor effects when dealing with the problem of no pure pixels, and there is still room for optimization in universality.

[0074] Virtual Dimensionality (VD) refers to the number of different spectral features in a hyperspectral image. In a hyperspectral image, the number of signal sources is usually much smaller than the number of spectral bands, and one type of ground object usually corresponds to several bands. Through this technology, the change situation of ground object categories can be understood, which adds interpretability to the change detection results.

[0075] For virtual dimension estimation, Jos é M. Bioucas et al. proposed the Hymise algorithm based on the principle of least squares. This algorithm first estimates the correlation matrix of the signal and noise, and then selects a subset of eigenvectors that optimally represents the signal subspace in the form of minimum mean square error. This model can be expressed by the formula:

[0076]

[0077] where is a constant, c is the dimension, k is the dimension, and L is the optimal eigenvector arrangement order that satisfies the best estimation criterion of the signal subspace. is the binomial expression of the correlation matrix of the observed signal and noise, that is:

[0078]

[0079] It can be seen from the above formula that only by finding all negative values can the corresponding (k, L) be obtained.

[0080] In the Hysime algorithm, noise estimation is to estimate the noise for each pixel. After obtaining the noise estimation value of each pixel, the original data observation value is subtracted from the noise estimation value to obtain an estimated value of the approximate true signal, and further the estimated value of the signal correlation matrix can be calculated. In practical applications, it is very difficult to estimate and calculate the noise value of each pixel of the image, so the accuracy of the results of noise estimation and removal using the Hysime algorithm is not ideal.

[0081] Assume that the observed data is white noise and the noise covariance matrix is Signal correlation matrix The eigenvalues λ of and the corresponding eigenvectors x satisfy the following equation relationship.

[0082]

[0083] It can be found from the above equation that when the noise is white noise, regardless of how the noise size changes, the eigenvectors obtained by the Hysime algorithm remain unchanged, and the subspace determined by the eigenvectors cannot be adjusted according to the noise change.

[0084] In view of the problems existing in the prior art, on the one hand, referring to Figure 1 and Figure 2 Embodiments of the present invention provide a hyperspectral change detection method based on prototype analysis and convolutional neural network, including:

[0085] S100. Preprocess the hyperspectral images of two time phases to be detected to obtain observed data;

[0086] It should be noted that, in some embodiments, the hyperspectral images of the two time phases to be detected are preprocessed by ENVI remote sensing image processing software to obtain observation data; wherein the observation data includes first observation data and second observation data; the preprocessing includes radiation correction, geometric correction and image registration.

[0087] Specifically, if Figure 2 First, image preprocessing is performed on the input hyperspectral images of the two time phases. Raw hyperspectral images generally contain considerable noise, so preprocessing procedures such as radiometric correction, geometric correction, and image registration are performed on the two time phases to be detected. This preprocessing can be performed using ENVI remote sensing image processing software. By preprocessing the hyperspectral images of the first and second time phases, the first and second observation data are obtained.

[0088] S200, performing virtual dimension estimation on the observed data based on noise whitening processing to obtain an estimated value;

[0089] It should be noted that the estimated value includes a first estimated value and a second estimated value; in some embodiments, a noise covariance matrix is determined based on the observation data; based on the eigenvalues and eigenvectors obtained by eigenvalue decomposition of the noise covariance matrix, the observation data is subjected to noise whitening processing to obtain whitened observation data; based on the whitened observation data, a whitened observation data correlation matrix and a whitened noise covariance matrix are determined to obtain a whitened signal correlation matrix; based on the whitened signal correlation matrix, an estimated value is obtained through a Hysim model; wherein the estimated value includes the arrangement order of the subspace eigenvectors and the target eigenvectors; the first estimated value and the second estimated value are obtained by performing virtual dimension estimation on the observation data corresponding to the hyperspectral images of the two time phases to be detected.

[0090] Specifically, if Figure 2 , the virtual dimension of the hyperspectral image is estimated for the observed data after image preprocessing, such as Figure 3 , the specific steps are as follows:

[0091] Assuming that the original observation data consists of signal x and noise n, the observation data can be represented by a vector:

[0092] y=x+n

[0093] Let the estimated value of the signal correlation matrix be Assuming the noise mean is 0, the noise covariance matrix is recorded as Signal Correlation Matrix It can be expressed as:

[0094]

[0095] In the formula, represents the Nth signal vector, where N represents the total number of signal vectors, and T represents the transpose operation.

[0096] For the noise covariance matrix perform eigenvalue decomposition to obtain the eigenvalues Λ n and eigenvectors A. Then, through the matrix whiten the noise of the original observed data y, and the whitened observed data is denoted as y w (i.e., the whitened observed data), and the noise covariance matrix is denoted as (i.e., the whitened noise covariance matrix). At this time, the noise is white noise, and the noise covariance matrix is the identity matrix, that is:

[0097]

[0098] The correlation matrix of the whitened observed data is denoted as R yw (i.e., the correlation matrix of the whitened observed data), and the signal correlation matrix is denoted as (i.e., the whitened signal correlation matrix), which is the observed data minus the noise data. The formula for the signal correlation matrix is:

[0099]

[0100] Next, perform virtual dimension estimation according to the Hysime algorithm, and denote the eigenvectors as Due to the binomial expression of the noise correlation matrix:

[0101]

[0102] The expression of the Hysime model can be represented as:

[0103]

[0104] where is the binomial expression of the observed signal matrix. After noise whitening, the (k, L) corresponding to all negative values of (2 - P ij ) (i.e., the estimated value) is the required subspace dimension.

[0105] S300. Determine the ground object change situation according to the first estimated value and the second estimated value;

[0106] It should be noted that in some embodiments, by taking the difference between the first estimated value and the second estimated value, the changed ground object categories and the unchanged ground object categories of the hyperspectral images of two time phases are obtained; based on the changed ground object categories and the unchanged ground object categories, the ground object change situation is sorted out.

[0107] Specifically, by taking the difference of (k, L) of two images, the changed and unchanged ground object categories can be obtained, and the change situation of the ground objects can be obtained by outputting this result.

[0108] S400. Spectral unmixing of the observed data is performed through a relaxation factor to obtain the target abundance matrix.

[0109] It should be noted that in some embodiments, based on the observed data, the prototype vector and the abundance matrix are initialized; the variable factors are iteratively updated through the projection gradient method, and the relaxation factor is continuously adjusted until a preset error threshold is reached to obtain the target abundance matrix; wherein, the target abundance matrix includes the first target abundance matrix and the second target abundance matrix; the first target abundance matrix and the second target abundance matrix are obtained by performing spectral unmixing on the observed data corresponding to the hyperspectral images of two time phases to be detected respectively.

[0110] Specifically, spectral unmixing is performed on the hyperspectral image to obtain the abundance map of each pixel. As Figure 4 shown, the specific steps are as follows:

[0111] First, the hyperspectral data X to be processed is read in, X ∈ R M×N , where M is the dimension of the spectral vector and N is the number of all pixels of the data. The number of endmembers D is determined according to existing experience, and the PCA dimensionality reduction algorithm is used to extract the first D - 1 principal components. The preprocessed hyperspectral data is denoted as X' ∈ R (D-1)×N .

[0112] The prototype vector is initialized with the principle that the differences between samples are as large as possible and the distances are as far as possible. The principle can be expressed as:

[0113]

[0114] where x i ∈ X', j new is the position serial number of the newly selected point, and C is the position serial number of all currently selected points.

[0115] Then the abundance matrix S is initialized and a is set to 1. The projection gradient method is used to iteratively update the variables a, C, S (i.e., the variable factors).

[0116] 1. Update a. First, calculate the gradient Then update in the gradient direction, specifically: a ← a - u a g a .

[0117] 2. Update C. First, calculate the gradient Then update in the gradient direction, specifically: Simplex

[0118] 3. Update S. First, calculate the gradient Then update in the gradient direction, specifically Simplex

[0119] In the above formula, d and n are the column numbers represented by D and N respectively, T is the matrix transpose operation, u a , is the step size factor of the linear change in each iteration, and the prototype vector matrix X′C ∈ R (D-1)×N is the endmember matrix in spectral unmixing.

[0120] By setting the relaxation factor δ and the error threshold h, iterate the above process in the model When reaching the threshold h, the result of pixel unmixing can be calculated: that is, the endmember matrix E = X′C and the target abundance matrix S.

[0121] S500. Calculate the affinity matrix based on the target abundance matrix;

[0122] It should be noted that according to the first target abundance matrix and combined with the pixels of the hyperspectral image in the first time phase, the first data block set is obtained; according to the second target abundance matrix and combined with the pixels of the hyperspectral image in the second time phase, the second data block set is obtained; according to the first data block set and the second data block set, the affinity matrix of each pixel corresponding to the hyperspectral images in the two time phases is calculated through the affinity matrix formula; where the affinity matrix formula is:

[0123] K = 1 - (R1 - R2) / R2

[0124] In the formula, represents the affinity matrix, represents the data block in the first data block set, represents the data block in the second data block set, and the corresponding pixels have the same positions in the hyperspectral images in the two time phases.

[0125] Specifically, calculate the affinity matrix of each pixel according to K = 1 - (R1 - R2) / R2, and then input the affinity matrix into the previously trained network model, and the network output is the obtained change detection result.

[0126] S600. Obtain the change detection result through the trained network model based on the affinity matrix.

[0127] It should be noted that in some embodiments, it further includes: training a convolutional neural network to obtain a trained network model. Among them, in some embodiments: based on a preset ratio, the pixels in the hyperspectral image are divided into a training data set and a test data set; an affinity matrix is calculated for each pixel in the training data set and input into the convolutional neural network; the loss function is optimized by the stochastic gradient descent method, and the convolutional neural network is supervised trained based on a preset period; when the model metrics class average accuracy and mean intersection over union meet the preset conditions, a trained network model is obtained.

[0128] Specifically, 60% of the pixels are randomly selected from the hyperspectral image as the training data set, and the remaining pixels form the test data set. For each pixel K = 1 - (R1 - R2) / R2 in the training data set, its affinity matrix K is calculated, where R1 is the new data block obtained by directly adding the pixel in the first-phase hyperspectral image and its abundance map, and R2 is the new data block obtained by directly adding the pixel at the same position in the second-phase hyperspectral image and its abundance map; then, the obtained affinity matrix is used as the input of the convolutional neural network for change detection, and the stochastic gradient descent method is used to minimize the loss function to perform supervised training on the network. The training period is 100 cycles. When the model metrics class average accuracy (oAcc) reaches 95% and the mean intersection over union (mIoU) reaches 95%, the model training is completed.

[0129] In summary, in view of the problems of low generality of traditional prototype analysis and poor estimation accuracy of virtual dimension, corresponding improvement schemes are proposed respectively. At the same time, in view of the low interpretability of the result map in the traditional change detection task, the present invention combines the two technologies of prototype analysis and virtual dimension estimation to realize not only understanding the change position of two successive images, but also understanding the change of ground object categories in the images. Compared with the existing hyperspectral change detection methods, the present invention introduces a relaxation factor to improve the generality of the algorithm in solving practical problems for the problem that the traditional spectral unmixing method has poor generality due to different degrees of pixel mixing in hyperspectral data. In addition, for the result maps obtained by other change detection methods which are mostly binary maps, the present invention uses the virtual dimension estimation method Hys ime to estimate the dimensions of the two-phase images, analyze and obtain the ground object change situation, and improve the interpretability of the change detection results.

[0130] On the other hand, an embodiment of the present invention provides a hyperspectral change detection system based on prototype analysis and convolutional neural network, including: a first module for preprocessing hyperspectral images of two time phases to be detected to obtain observation data; a second module for estimating the virtual dimension of the observation data based on noise whitening processing to obtain an estimated value, where the estimated value includes a first estimated value and a second estimated value; a third module for determining the ground object change situation according to the first estimated value and the second estimated value; a fourth module for performing spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix; a fifth module for calculating an affinity matrix based on the target abundance matrix; and a sixth module for obtaining a change detection result through a trained network model based on the affinity matrix.

[0131] The content of the method embodiment of the present invention is applicable to the system embodiment of the present invention. The functions specifically implemented by the system embodiment of the present invention are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those of the above method.

[0132] On the other hand, an embodiment of the present invention also provides a hyperspectral change detection system based on prototype analysis and convolutional neural network, including a processor and a memory;

[0133] The memory is used to store a program;

[0134] The processor executes the program to implement the method as described above.

[0135] The content of the method embodiment of the present invention is applicable to the system embodiment of the present invention. The functions specifically implemented by the system embodiment of the present invention are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those of the above method.

[0136] On the other hand, an embodiment of the present invention also provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the method as described above.

[0137] The content of the method embodiment of the present invention is applicable to the computer-readable storage medium embodiment of the present invention. The functions specifically implemented by the computer-readable storage medium embodiment of the present invention are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those of the above method.

[0138] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method as described above.

[0139] In some alternative embodiments, the functions / operations recited in the block diagrams may not occur in the order presented in the operational illustrations. For example, depending on the functions / operations involved, two consecutive blocks shown may actually be executed substantially simultaneously or the blocks may sometimes be executed in the reverse order. Further, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are performed independently.

[0140] In addition, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It should also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skill of an engineer. Thus, those of ordinary skill in the art can implement the present invention as set forth in the claims without undue experimentation. It should also be understood that the particular concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0141] If the described functions are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definitional sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution apparatus, apparatus, or device (such as a computer-based device, a device including a processor, or other devices that can fetch and execute instructions from the instruction execution apparatus, apparatus, or device), or used in combination with these instruction execution apparatuses, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution apparatus, apparatus, or device.

[0143] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0144] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution apparatus. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0145] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0146] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

[0147] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without violating the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of the present invention.

Claims

1. A hyperspectral change detection method based on prototype analysis and convolutional neural network, characterized in that Including: Preprocess the hyperspectral images of two temporal phases to be detected to obtain observation data; Based on noise whitening processing, perform virtual dimensionality estimation on the observation data to obtain estimated values; wherein, the estimated values include a first estimated value and a second estimated value; Determine the ground object change situation according to the first estimated value and the second estimated value; Perform spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix; Calculate an affinity matrix based on the target abundance matrix; Based on the affinity matrix, obtain a change detection result through a trained network model.

2. The hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, characterized in that The preprocessing of the hyperspectral images of two temporal phases to be detected to obtain observation data includes: Preprocess the hyperspectral images of two temporal phases to be detected through ENVI remote sensing image processing software to obtain observation data; Wherein, the observation data includes first observation data and second observation data; the preprocessing includes radiometric correction, geometric correction, and image registration.

3. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, characterized in that The performing virtual dimensionality estimation on the observation data based on noise whitening processing to obtain estimated values includes: Determine a noise covariance matrix according to the observation data; Perform noise whitening processing on the observation data based on the eigenvalues and eigenvectors obtained by eigenvalue decomposition of the noise covariance matrix to obtain whitened observation data; Determine a whitened observation data correlation matrix and a whitened noise covariance matrix according to the whitened observation data to obtain a whitened signal correlation matrix; Obtain estimated values through the Hysime model according to the whitened signal correlation matrix; Wherein, the estimated values include subspace eigenvectors and the arrangement order of target eigenvectors; the first estimated value and the second estimated value are obtained by performing virtual dimensionality estimation on the observation data corresponding to the hyperspectral images of two temporal phases to be detected respectively.

4. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 3, characterized in that, The determining a whitened observation data correlation matrix and a whitened noise covariance matrix according to the whitened observation data to obtain a whitened signal correlation matrix includes: Determine a whitened observation data correlation matrix and a whitened noise covariance matrix according to the whitened observation data, and calculate a whitened signal correlation matrix through a signal correlation matrix formula; Wherein, the signal correlation matrix formula is: In the formula, represents the correlation matrix of the whitened signal, R yw represents the correlation matrix of the whitened observation signal, represents the covariance matrix of the whitened noise, and I represents the identity matrix, 5. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, characterized in that The determining the ground object change situation according to the first estimated value and the second estimated value includes: Obtain the changed ground object categories and unchanged ground object categories of the hyperspectral images of two temporal phases by taking the difference between the first estimated value and the second estimated value; Sort out the ground object change situation based on the changed ground object categories and the unchanged ground object categories.

6. The hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, wherein The performing spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix includes: Initialize the prototype vectors and the abundance matrix based on the observation data; Iteratively update the variable factors through the projection gradient method and continuously adjust the relaxation factor until a preset error threshold is reached to obtain a target abundance matrix; Among them, the target abundance matrix includes a first target abundance matrix and a second target abundance matrix; the first target abundance matrix and the second target abundance matrix are obtained by performing spectral unmixing on the observation data corresponding to the hyperspectral images of two time phases to be detected respectively.

7. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, characterized in that, The target abundance matrix includes a first target abundance matrix and a second target abundance matrix. Calculating an affinity matrix based on the target abundance matrix includes: According to the first target abundance matrix, combining with the pixels of the hyperspectral image of the first time phase to obtain a first data block set; According to the second target abundance matrix, combining with the pixels of the hyperspectral image of the second time phase to obtain a second data block set; According to the first data block set and the second data block set, calculating the affinity matrix of each pixel corresponding to the hyperspectral images of the two time phases through the affinity matrix formula; Among them, the affinity matrix formula is: K = 1 - (R1 - R2) / R3 In the formula, K represents the affinity matrix, R1 represents the data block in the first data block set, R3 represents the data block in the second data block set, and the pixels corresponding to R1 and R3 are in the same positions in the hyperspectral images of the two time phases.

8. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 1, characterized in that, It also includes: Training a convolutional neural network to obtain a trained network model.

9. A hyperspectral change detection method based on prototype analysis and convolutional neural network according to claim 8, characterized in that, Training the convolutional neural network to obtain a trained network model includes: Based on a preset ratio, dividing the pixels in the hyperspectral image into a training data set and a test data set; Calculating the affinity matrix for each pixel in the training data set and inputting it into the convolutional neural network; Optimizing the loss function by the stochastic gradient descent method and performing supervised training on the convolutional neural network based on a preset number of epochs; When the model metric class average accuracy and average intersection over union meet the preset conditions, obtaining a trained network model.

10. A hyperspectral change detection system based on prototype analysis and convolutional neural network, characterized in that, It includes: A first module for preprocessing the hyperspectral images of two time phases to be detected to obtain observation data; A second module for performing virtual dimensionality estimation on the observation data based on noise whitening processing to obtain estimated values; among them, the estimated values include a first estimated value and a second estimated value; A third module for determining the ground object change situation according to the first estimated value and the second estimated value; A fourth module for performing spectral unmixing on the observation data through a relaxation factor to obtain a target abundance matrix; A fifth module for calculating an affinity matrix based on the target abundance matrix; A sixth module for obtaining a change detection result through the trained network model based on the affinity matrix.

Images