Heterogeneous change detection method and device of interpretable neural network copula

Abstract

The application provides a heterogeneous change detection method and device of an interpretable neural network Copula, and the method comprises the following steps: performing superpixel segmentation on a double-time heterogeneous remote sensing image to obtain a double-time superpixel set; performing feature extraction on each superpixel in the double-time superpixel set to obtain a double-time superpixel feature value set; estimating the cumulative distribution function value of each superpixel feature value based on a kernel density estimation method to obtain a double-time cumulative distribution function value set; inputting the cumulative distribution function value in the double-time cumulative distribution function value set into a pre-trained neural network Copula model to solve the gradient and obtain a probability density function value representing the correlation of superpixels; and performing clustering segmentation on the probability density function value to obtain a remote sensing image change detection result. The application combines the characteristics of neural networks and Copula theory, and improves the interpretability and robustness of the existing heterogeneous change detection method.

Description

Technical Field

[0001] This application relates to the field of heterogeneous remote sensing image analysis, and in particular to a method and apparatus for detecting heterogeneous changes in an interpretable neural network Copula. Background Technology

[0002] Remote sensing change detection refers to identifying changes in surface features within a geographic area by comparing co-registered images (i.e., bimodal remote sensing images) taken at two different times. Based on the selected remote sensing image modalities, remote sensing change detection is divided into isomorphic change detection and heteromorphic change detection. Isomorphic change detection identifies changes in ground features or the environment in remote sensing images acquired by the same type of sensor at different times; heteromorphic change detection identifies changes in ground features or the environment in remote sensing images acquired by different types of sensors at different times. Because heteromorphic change detection can compare remote sensing images acquired by different sensors, it offers higher real-time performance compared to isomorphic change detection. In disaster monitoring and emergency response, heteromorphic change detection can utilize multi-source data for real-time monitoring and analysis, providing more accurate and timely monitoring results, and contributing to improved efficiency and accuracy in rescue and disaster relief.

[0003] The powerful representational capabilities of deep neural networks (DNNs) have greatly facilitated the development of heterogeneous change detection. However, data-driven DNNs often lack interpretability, which limits their reliability and controllability in practical change detection applications. Furthermore, Copula theory, as a knowledge-driven tool for measuring the correlation between random variables, has been introduced into change detection, but existing Copula-based change detection algorithms exhibit unrobust performance without manual selection of the Copula function. Therefore, improving the interpretability and robustness of change detection is a crucial technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a method and apparatus for detecting heterogeneous changes in an interpretable neural network Copula, so as to overcome the above problems or at least partially solve the above problems.

[0005] A first aspect of this application discloses a method for detecting heterogeneous changes in an interpretable neural network, Copula, the method comprising:

[0006] Superpixel segmentation is performed on dual-temporal heterogeneous remote sensing images to obtain dual-temporal superpixel sets. The dual-temporal superpixel sets include: a first temporal superpixel set and a second temporal superpixel set. The corresponding superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs.

[0007] Feature extraction is performed on each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set;

[0008] Based on the kernel density estimation method, the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set is estimated to obtain the dual-temporal cumulative distribution function value set;

[0009] The cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​are input into a pre-trained neural network Copula model to solve for the gradient, thereby obtaining the probability density function values ​​that characterize the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function.

[0010] Clustering and segmentation are performed on the probability density function values ​​to obtain the remote sensing image change detection results.

[0011] Optionally, the neural network Copula model is trained according to the following steps:

[0012] Construct a bitemporal training superpixel set using unchanged superpixel pairs as training samples;

[0013] Extract the feature values ​​of each training superpixel from the dual-temporal training sample set to obtain the dual-temporal training superpixel feature value set;

[0014] The training superpixel feature values ​​in the dual-temporal training superpixel feature value set are processed to obtain the dual-temporal training superpixel cumulative distribution function value set;

[0015] The training cumulative distribution function values ​​in the dual-temporal training superpixel cumulative distribution function value set are input into the neural network Copula model for training to obtain the predicted cumulative distribution function value and the predicted probability density function value.

[0016] Based on the predicted cumulative distribution function value and the predicted probability density function value, the loss function of the neural network Copula model is calculated. The training parameters of the neural network Copula model are updated based on the loss function. After the training termination condition is met, a satisfactory neural network Copula model is obtained.

[0017] Optionally, constructing a training superpixel set using unchanged superpixel pairs as training samples includes:

[0018] After processing the training dual-temporal heterogeneous remote sensing images into a single-channel form, superpixel segmentation was performed to obtain a dual-temporal training dual-temporal superpixel set.

[0019] Select an unchanging training region, determine a pair of superpixels to be selected from the dual-temporal training dual-temporal superpixel set, and determine whether the cross-union ratio between the pair of superpixels to be selected and the selected training region is greater than the ratio threshold.

[0020] If the cross-union ratio is greater than the ratio threshold, the candidate superpixel pairs are included in the dual-temporal training superpixel set.

[0021] Optionally, processing the training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the dual-temporal training superpixel cumulative distribution function value set includes:

[0022] Based on the kernel density estimation method, the distribution of the training superpixel feature values ​​fitted in the dual-temporal training superpixel feature value set is obtained;

[0023] Based on the distribution, the cumulative distribution function value of the training superpixel feature value is estimated by using a lookup table method to obtain a set of cumulative distribution function values ​​of the dual-temporal training superpixel. The table used by the lookup table method consists of cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values.

[0024] Optionally, the table is pre-generated according to the following steps:

[0025] The distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set is fitted to obtain the distribution of the training superpixel feature values.

[0026] Based on the aforementioned distribution, the cumulative distribution function values ​​of the remote sensing image pixel values ​​from 0 to 255 are estimated respectively, and used as constituent elements of the table;

[0027] Based on the distribution, the cumulative distribution function value of the trained superpixel feature values ​​is estimated using a lookup table method, including:

[0028] For each training superpixel feature value, the corresponding cumulative distribution function value is found in the table and used as the cumulative distribution function value of that training superpixel feature value.

[0029] Optionally, calculating the loss function of the neural network Copula model based on the predicted cumulative distribution function value and the predicted probability density function value includes:

[0030] Based on the predicted cumulative distribution function value and the predicted probability density function value, the boundary loss, probability density function integral loss, probability density function non-negativity loss, maximum likelihood estimation loss, and joint distribution function observation loss are calculated respectively.

[0031] According to preset weights, the calculated boundary loss, the probability density function integral loss, the probability density function nonnegativity loss, the maximum likelihood estimation loss, and the joint distribution function observation loss are linearly combined to obtain the loss function of the Copula neural network model.

[0032] Optionally, the predicted cumulative distribution function value needs to satisfy the boundary defined by the Copula function, and the boundary loss is expressed as:

[0033]

[0034] in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Representing vectors The i-th value in Representing vectors The i-th value in and All are linear intervals between 0 and 1. The vector.

[0035] Optionally, the integral of the predicted probability density function value over the domain of the Copula function is 1, and the integral loss of the probability density function is expressed as:

[0036]

[0037] in, This represents the trainable parameters of the Copula neural network model. Indicates the linear step size. This represents the predicted probability density function value after non-negativity processing. It represents the range between 0 and 1. Linear interval vector of points The i-th point, It represents the range between 0 and 1. Linear interval vector of points The i-th point.

[0038] Optionally, the predicted probability density function value is non-negative within the domain of the Copula function, and the non-negativity loss of the probability density function is expressed as:

[0039]

[0040] in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value. Representing vectors The i-th value, Representing vectors The j-th value, and All are linear intervals between 0 and 1. The vector.

[0041] Optionally, the sampling maximum likelihood estimation method evaluates the goodness of fit of the neural network Copula model to the cumulative distribution function values ​​of the training superpixels, wherein the maximum likelihood estimation loss is expressed as:

[0042]

[0043] in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value after non-negativity processing. Indicates the preset evaluation coefficient. This represents the set of cumulative distribution function values ​​of superpixel features trained in the first time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This represents the set of cumulative distribution function values ​​for superpixel features trained in the second time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This indicates the number of training superpixels.

[0044] Optionally, the joint distribution function observation loss is used to optimize the difference between the observed cumulative distribution function value and the empirical cumulative distribution function value, and the joint distribution function observation loss is expressed as:

[0045]

[0046] in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Indicates inclusion The linear interval vector of observation points The i-th value in Indicates inclusion The linear interval vector of observation points The j-th value in This represents the m-th feature value in the first-temporal training superpixel feature value set. This represents the m-th feature value in the second-temporal training superpixel feature value set. Indicates an indicator function, This indicates that the superpixel feature set generated in the first time-series training is generated from the distribution of the superpixel feature set. The cumulative distribution function value, This indicates that the superpixel feature set generated in the second time-series training is generated in the distribution of the superpixel feature set. The cumulative distribution function value.

[0047] Optionally, the step of clustering and segmenting the probability density function values ​​to obtain the remote sensing image change detection results includes:

[0048] The probability density function value is normalized by performing a negative logarithm operation to obtain a normalized probability density function value.

[0049] The normalized probability density function values ​​are divided into two categories using the fuzzy C-means clustering algorithm, and then assigned to the corresponding pixels according to the change category to which the superpixel belongs, thus obtaining the remote sensing image change detection results.

[0050] A second aspect of this application discloses a heterogeneous change detection device for an interpretable neural network Copula, the device comprising:

[0051] The segmentation module is used to perform superpixel segmentation on the dual-temporal heterogeneous remote sensing image to obtain a dual-temporal superpixel set. The dual-temporal superpixel set includes a first temporal superpixel set and a second temporal superpixel set. The superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs.

[0052] The extraction module is used to extract features from each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set;

[0053] The estimation module is used to estimate the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set based on the kernel density estimation method, so as to obtain the dual-temporal cumulative distribution function value set;

[0054] The solution module is used to input the cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​into a pre-trained neural network Copula model to solve for the gradient, thereby obtaining the probability density function value characterizing the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function.

[0055] The results module is used to cluster and segment the probability density function values ​​to obtain the remote sensing image change detection results.

[0056] The embodiments of this application have the following advantages:

[0057] In this embodiment, combining the robustness of neural networks and the interpretability of Copula theory, a heterogeneous change detection method based on an interpretable Copula neural network is proposed. First, superpixel segmentation is performed on the dual-temporal heterogeneous remote sensing image. Then, feature extraction is performed on each superpixel in the resulting dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set. Next, based on kernel density estimation, the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set is estimated to obtain a dual-temporal cumulative distribution function value set. Then, the cumulative distribution function values ​​in the dual-temporal cumulative distribution function value set are input into a pre-trained Copula neural network model to solve for the gradient, obtaining the probability density function value representing the correlation between superpixel pairs. Finally, the probability density function values ​​are clustered to obtain the remote sensing image change detection result.

[0058] Because the loss function of the neural network Copula model is designed by combining data-driven neural networks and knowledge-driven Copula theory, this loss function can regulate the output of the neural network Copula model to approximate Copula theory. This allows the trained neural network Copula model to implement Copula theory using neural networks. Furthermore, the loss function makes the training process of the neural network Copula model interpretable, achieving robustness while ensuring interpretability. Therefore, the embodiments of this application organically combine data-driven neural networks and knowledge-driven Copula theory, significantly improving the interpretability and robustness of existing heterogeneous change detection methods based on deep neural networks and Copula theory. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating the steps of a method for detecting heterogeneous changes in an interpretable neural network, Copula, provided in an embodiment of this application.

[0061] Figure 2 This is a flowchart illustrating a dual-temporal training superpixel selection method provided in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of the overall framework of a heterogeneous change detection method for an interpretable neural network Copula provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram illustrating the training process of a Copula neural network model provided in an embodiment of this application;

[0064] Figure 5 This is a schematic diagram of the structure of a heterogeneous change detection device for an interpretable neural network Copula provided in an embodiment of this application. Detailed Implementation

[0065] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] Reference Figure 1 As shown, Figure 1 This document illustrates a flowchart of the steps involved in a method for detecting heterogeneous changes in an interpretable neural network, Copula, according to an embodiment of this application. Figure 1 As shown in the embodiments of this application, a method for detecting heterogeneous changes in an interpretable neural network Copula may specifically include steps S110 and S150:

[0067] Step S110: Perform superpixel segmentation on the dual-temporal heterogeneous remote sensing image to obtain a dual-temporal superpixel set. The dual-temporal superpixel set includes a first temporal superpixel set and a second temporal superpixel set. The superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs.

[0068] In this embodiment, a dual-temporal heterogeneous remote sensing image refers to two remote sensing images of the same geographical area taken at two different times using different types of sensors. Specifically, the dual-temporal heterogeneous remote sensing image includes a first temporal remote sensing image (the image before the change) and a second temporal remote sensing image (the image after the change). The types of sensors include radar sensors, optical sensors, multispectral sensors, etc.; correspondingly, the types of remote sensing images include radar images, optical images, multispectral images, etc.

[0069] Generally, before performing superpixel segmentation on dual-temporal heterogeneous remote sensing images, preprocessing is required. This includes: first, checking if the dual-temporal heterogeneous remote sensing image is multi-channel; if not, no preprocessing is needed; if it is multi-channel, it needs to be preprocessed to convert it into a single-channel format. Specifically, if the dual-temporal heterogeneous remote sensing image is an optical image, it is converted into a grayscale single-channel image; if it is a multispectral image, principal component analysis is used for dimensionality reduction, and the extracted first principal component is used to reconstruct the single-channel image for subsequent processing. Finally, after preprocessing, the size of the dual-temporal heterogeneous remote sensing image should be reduced to [value missing]. (H is the length of the remote sensing image, W is the width of the remote sensing image, and 1 represents the number of channels in the remote sensing image).

[0070] In practice, superpixel segmentation is performed on the dual-temporal heterogeneous remote sensing images. Specifically, a cooperative simple linear iterative clustering algorithm (Co-SLIC) is used to segment the dual-temporal heterogeneous remote sensing images into dual-temporal superpixel sets. The mathematical expression for superpixel segmentation can be expressed as follows:

[0071]

[0072] in, This represents the first-temporal superpixel set, which is a first-temporal remote sensing image. The segmented set of superpixels This represents the second-temporal superpixel set, which is a second-temporal remote sensing image. The segmented set of superpixels Indicates the number of superpixels in the specified segment. and They are and The i-th superpixel in the array.

[0073] Step S120: Extract features from each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set.

[0074] In this embodiment, the dual-temporal superpixel feature value set includes a first temporal superpixel feature value set and a second temporal superpixel feature value set. Each superpixel contains multiple pixels, and feature extraction for each superpixel specifically uses the average pixel value of all pixels within that superpixel as its feature. For example, if superpixel A has 10 pixels, then the average pixel value of these 10 pixels is calculated as the feature value of superpixel A.

[0075] Specifically, for the first temporal superpixel set Second temporal superpixel set Feature extraction is performed to obtain the first temporal superpixel feature value set. Second temporal superpixel feature set Specifically, it is expressed as:

[0076]

[0077] in, This represents a function that calculates the average value of all pixels in a given superpixel.

[0078] Step S130: Based on the kernel density estimation method, estimate the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set to obtain the dual-temporal cumulative distribution function value set.

[0079] In this embodiment, to transform the feature values ​​in the first and second temporal superpixel feature value sets into the probability space, it is necessary to estimate the cumulative distribution function value of each superpixel feature value. Specifically, the kernel density estimation method (KDE) is used to estimate the cumulative distribution function value of each superpixel feature value. Simultaneously, to maintain consistency of the marginal distribution during training and testing, the cumulative distribution function value of each superpixel feature value in the two temporal superpixel feature value sets is estimated based on the marginal distribution fitted to the training set (the training set used to train the Copula neural network model).

[0080] Furthermore, considering that directly using the kernel density estimation method to estimate the cumulative distribution function values ​​of a large number of points is very slow, this embodiment adopts a lookup table method to accelerate the cumulative distribution function value estimation process. The table used by the lookup table method consists of the cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values. The table is generated during the model training stage based on the distribution fitted to the training set. Then, the corresponding cumulative distribution function value is looked up in the table for each superpixel feature value as the cumulative distribution function value of that superpixel feature value.

[0081] Specifically, assuming This means that the cumulative distribution function value of each superpixel feature value in set Y is estimated based on the distribution fitted to the samples in set X. Then, the first temporal superpixel feature value set... Second temporal superpixel feature set The estimate of the cumulative distribution function value of each superpixel feature value can be expressed as:

[0082]

[0083] in, Represents the set of superpixel features based on the first temporal state. The estimated set of first-temporal cumulative distribution function values, Represents the set of superpixel features based on the second temporal context. The estimated set of values ​​for the second temporal cumulative distribution function. This represents the set of superpixel feature values ​​trained in the first time phase. This represents the set of superpixel feature values ​​trained in the second temporal phase.

[0084] Step S140: Input the cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​into the pre-trained neural network Copula model to solve for the gradient, and obtain the probability density function values ​​representing the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function.

[0085] In this embodiment, a neural network Copula model is used to learn correlation information from training dual-temporal remote sensing images in advance. Then, based on the neural network Copula model, the correlation of each superpixel corresponding to the dual-temporal remote sensing images can be calculated. Specifically, the neural network Copula model is trained using unchanged superpixel pairs as training samples. That is, the neural network Copula model learns the fitting relationship between unchanged superpixel pairs (i.e., learns the correlation between unchanged superpixel pairs). Therefore, when using this neural network Copula model to solve for the gradient of the cumulative distribution function values ​​in the dual-temporal cumulative distribution function value set, the neural network Copula model, based on the fitting relationship between unchanged superpixel pairs, has a better fitting effect for unchanged superpixels, but a poorer fitting effect for changed superpixels. Thus, the neural network Copula model ultimately outputs two types of probability density function values ​​with significant differences, representing changed and unchanged superpixel pairs respectively. This allows for the segmentation of changed and unchanged regions based on the probability density function values ​​in subsequent steps, thereby achieving a semi-supervised heterogeneous change detection.

[0086] For example, the probability density function value Represented as:

[0087]

[0088] in, This indicates the predicted cumulative distribution function value. It is a modified linear unit function. To represent a tiny positive value, generally Set to 1e-9.

[0089] Furthermore, since the loss function of the neural network Copula model is designed by combining data-driven neural networks and knowledge-driven Copula theory, this loss function can regulate the output of the neural network Copula model to approximate Copula theory. This allows the trained neural network Copula model to implement Copula theory using neural networks. Consequently, the loss function makes the training process of the neural network Copula model interpretable, extending the robustness of the neural network Copula model while ensuring interpretability. Therefore, the probability density function values ​​obtained based on this neural network Copula model are highly reliable, further ensuring the reliability of subsequent remote sensing image change detection results.

[0090] Step S150: Cluster the probability density function values ​​to obtain the remote sensing image change detection results.

[0091] In this embodiment, the probability density function value characterizes the correlation of superpixel pairs (i.e., the corresponding superpixels in the first temporal superpixel set and the second temporal superpixel set). A larger probability density function value indicates greater similarity between the two superpixels in the pair, meaning the geographic region corresponding to the pair has not changed. Conversely, a smaller probability density function value indicates lower similarity between the two superpixels in the pair, meaning the geographic region corresponding to the pair has changed. Therefore, the remote sensing image change detection result can be determined based on the probability density function value.

[0092] In one optional embodiment, the step of clustering and segmenting the probability density function values ​​to obtain remote sensing image change detection results includes:

[0093] The probability density function value is normalized by performing a negative logarithmic operation to obtain a normalized probability density function value. The normalized probability density function value is then divided into two categories using a fuzzy C-means clustering algorithm, and assigned to the corresponding pixels according to the change category to which the superpixel belongs, thus obtaining the remote sensing image change detection result.

[0094] In this embodiment, the probability density function values ​​of different superpixel pairs may differ significantly in step S150. Therefore, for ease of analysis, a negative logarithmic operation is used to normalize the probability density function values ​​to a number between 0 and 1. Then, based on the fuzzy C-means clustering algorithm, the normalized probability density function values ​​are divided into two categories: all superpixel pairs are classified as changed and unchanged superpixel pairs. The change detection result is then obtained by assigning the change category of the superpixel to the corresponding pixel.

[0095] For example, after normalizing the probability density function value through negative logarithmic operation, two distinct types of data are obtained: one part of the data is around 0.7 and the other part is around 0.2. The superpixel pairs corresponding to the data around 0.7 are classified as unchanged, and the superpixel pairs corresponding to the data around 0.2 are classified as changed, thereby determining the change detection result of the remote sensing image.

[0096] In one optional embodiment, the Copula neural network model is a fully connected neural network comprising three parts: an input layer, hidden layers, and an output layer. This Copula model contains a total of five hidden layers, each consisting of twenty neurons. Specifically, the Copula neural network model is trained according to the following steps, including steps A1 to A6:

[0097] Step A1: Construct a bitemporal training superpixel set using unchanged superpixel pairs as training samples.

[0098] In this embodiment, an unchanged superpixel pair refers to a superpixel whose corresponding geographic region has not changed. For example... Figure 2 As shown, the specific steps include: processing the training dual-temporal heterogeneous remote sensing image into a single-channel format and then performing superpixel segmentation to obtain a dual-temporal training dual-temporal superpixel set; selecting an unchanged training region and determining candidate superpixel pairs from the dual-temporal training dual-temporal superpixel set, and determining whether the intersection-union ratio (IU) between the candidate superpixel pair and the selected training region is greater than a ratio threshold; if the IU is greater than the ratio threshold, the candidate superpixel pair is included in the dual-temporal training superpixel set. The training dual-temporal heterogeneous remote sensing image includes a first-temporal training heterogeneous remote sensing image. Second temporal training heterogeneous remote sensing images The unchanging training regions are roughly selected from the training bitemporal heterogeneous remote sensing images, and the regions that have not changed are selected.

[0099] Specifically, based on a cooperative simple linear iterative clustering algorithm (Co-SLIC), the training dual-temporal heterogeneous remote sensing images are segmented into superpixel pairs. The mathematical expression for superpixel segmentation can be expressed as follows:

[0100]

[0101] in, This represents the first-temporal training superpixel set, which is the first-temporal training remote sensing image. The segmented set of superpixels This represents the second-temporal training superpixel set, which is the second-temporal training remote sensing image. The segmented set of superpixels Indicates the number of superpixels in the specified segment. and They are and The i-th superpixel in the array.

[0102] Assumption Let be the selected training region, and let the proportion threshold be 0.5. Then the training sample selection process can be formally represented as:

[0103]

[0104] in, It is a function for calculating the number of pixels. This represents the set of superpixels selected from the first-temporal training remote sensing images. This represents the set of superpixels selected from the training remote sensing images in the second timeframe.

[0105] Step A2: Extract the feature value of each training superpixel in the dual-temporal training sample set to obtain the dual-temporal training superpixel feature value set.

[0106] In this embodiment, we directly use the average value of all pixels within each training superpixel as its feature. Specifically, this is expressed as follows:

[0107]

[0108] in, It is a function that calculates the average value of all pixels in a corresponding superpixel. This represents the set of superpixel feature values ​​trained in the first time phase. This represents the set of superpixel feature values ​​trained in the second temporal phase. This represents the k-th feature value in the first-temporal training superpixel feature value set. This represents the k-th feature value in the second-temporal training superpixel feature value set.

[0109] Step A3: Process the training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the dual-temporal training superpixel cumulative distribution function value set.

[0110] In this embodiment, a neural network Copula model is used to directly learn correlation information from remote sensing data. The first step of the neural network Copula model is to perform a probability integral transformation on the marginal distribution of each random variable, that is, to estimate the cumulative distribution function value of each training superpixel feature value in the dual-temporal training superpixel feature value set.

[0111] In one optional embodiment, processing the training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the dual-temporal training superpixel cumulative distribution function value set includes: based on the kernel density estimation method, fitting the distribution of the training superpixel feature values ​​in the dual-temporal training superpixel feature value set; and estimating the cumulative distribution function values ​​of the training superpixel feature values ​​according to the distribution using a lookup table method to obtain the dual-temporal training superpixel cumulative distribution function value set, wherein the table used in the lookup table method consists of cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values.

[0112] Specifically, the table is pre-generated according to the following steps: The distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set is fitted to obtain the distribution of the training superpixel feature values; based on the distribution, the cumulative distribution function values ​​of remote sensing image pixel values ​​from 0 to 255 are estimated respectively, and used as constituent elements of the table. Then, for each training superpixel feature value, the corresponding cumulative distribution function value is searched in the table as the cumulative distribution function value of that training superpixel feature value.

[0113] Since the pixel values ​​in remote sensing images only vary between 0 and 255, and these pixel values ​​are integers, it is only necessary to calculate the cumulative distribution function values ​​for pixel values ​​between 0 and 255. Then, the cumulative distribution function values ​​for the trained superpixel features can be obtained directly through a lookup table operation.

[0114] For example, to obtain the cumulative distribution function value for 10,000 superpixels, 10,000 calculations are required. However, the pixel values ​​of the 10,000 superpixels in the remote sensing image are all integers between 0 and 255. Therefore, it is only necessary to solve for the cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values ​​to form a table. Then, the table is queried sequentially according to the pixel values ​​of the 10,000 superpixels to obtain the cumulative distribution function value corresponding to the 10,000 superpixels.

[0115] For example, suppose Let X represent the method of estimating the cumulative distribution function value of each superpixel feature value in set Y based on the distribution fitted to the samples in set X. Then, the estimate of the cumulative distribution function value of each training superpixel feature value in the first time-state training superpixel feature value set and the second time-state training superpixel feature value set can be expressed as:

[0116]

[0117] in, This represents the set of cumulative distribution function values ​​for superpixel features during the first time-series training. This represents the set of cumulative distribution function values ​​for superpixel features during the first time-series training.

[0118] Step A4: Input the training cumulative distribution function values ​​from the dual-temporal training superpixel cumulative distribution function value set into the neural network Copula model for training, and obtain the predicted cumulative distribution function value and the predicted probability density function value.

[0119] In this embodiment, after estimating the set of cumulative distribution function values ​​for the two-temporal training superpixels, training of the Copula neural network model can be initiated. The expression for the Copula neural network model is defined as follows: , This represents the trainable parameters of the Copula neural network model, and the cumulative distribution function value input to the Copula neural network model is... ,in and These are vectors representing the training cumulative distribution function values ​​of N sampling points following distributions U and V, respectively, and are the outputs of the Copula neural network model. The cumulative distribution function value (i.e., the predicted cumulative distribution function value) of the trained Copula neural network model, and the gradient of the Copula neural network model. This represents the probability density function value of the trained Copula neural network model. To prevent the cumulative distribution function value of the Copula model from becoming negative, the gradient of the Copula model needs to be non-negative. This can be expressed as:

[0120]

[0121] in, It is a modified linear unit function. To represent a tiny positive value, generally Set to 1e-9.

[0122] Step A5: Calculate the loss function of the Copula neural network model based on the predicted cumulative distribution function value and the predicted probability density function value. Update the training parameters of the Copula neural network model based on the loss function. After the training termination condition is met, a satisfactory Copula neural network model is obtained.

[0123] In this embodiment, Copula theory is an interpretable theory. Interpretability refers to the clear, symbolic representation of the process and results of a method or theory. To fully leverage the potential of this fully connected neural network (neural network Copula model), the loss function is calculated using Copula theory and the neural network to facilitate the convergence of the neural network Copula model to the form of the Copula function. This allows the trained neural network Copula model to implement Copula theory using the neural network, achieving robustness of the extended neural network Copula model while ensuring interpretability.

[0124] In an optional embodiment, calculating the loss function of the neural network Copula model based on the predicted cumulative distribution function value and the predicted probability density function value includes steps B1 and B2:

[0125] Step B1: Based on the predicted cumulative distribution function value and the predicted probability density function value, calculate the boundary loss, probability density function integral loss, probability density function non-negativity loss, maximum likelihood estimation loss, and joint distribution function observation loss, respectively.

[0126] Among them, boundary loss, probability density function integral loss, and probability density function nonnegativity loss are losses designed based on Copula theory. Boundary loss constrains the output boundary of the neural network Copula model, ensuring that the predicted cumulative distribution function value is within the range defined by the Copula function. Probability density function integral loss constrains the integral of the predicted probability density function value of the neural network Copula model over its domain to be 1. Probability density function nonnegativity loss constrains the predicted probability density function value of the neural network Copula model to be positive. Thus, based on boundary loss, probability density function integral loss, and probability density function nonnegativity loss, the output of the neural network Copula model can be made closer to Copula theory, enabling the neural network Copula model to implement Copula theory using a neural network, ensuring the interpretability of the neural network. Meanwhile, maximum likelihood estimation loss and joint distribution function observation loss are losses designed based on the neural network. Maximum likelihood estimation loss constrains the predicted probability density function value of the neural network Copula model to have a good fit, and joint distribution function observation loss constrains the minimization of the difference between the predicted cumulative distribution function value and the empirical cumulative distribution function value of the neural network Copula model, thereby improving the accuracy of the neural network Copula model output. Therefore, this loss function can standardize the output of the neural network Copula model to approximate Copula theory, thereby extending the robustness of the neural network Copula model while ensuring interpretability.

[0127] (1) Predicting the cumulative distribution function value requires satisfying the boundary defined by the Copula function. Specifically, the boundary defined by the Copula function is expressed as:

[0128] ,

[0129]

[0130] Therefore, the boundary loss is expressed as:

[0131]

[0132] in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Representing vectors The i-th value in Representing vectors The i-th value in and All are linear intervals between 0 and 1. The vector.

[0133] (2) The integral of the predicted probability density function value over the domain of the Copula function is 1. When the probability density function value is continuous, the integral loss of the probability density function is expressed as:

[0134]

[0135] However, in practical applications, it is impossible to sample continuous probability density function values. Therefore, it is necessary to rewrite the integral loss of the probability density function in its discrete approximation form. Assume... and All are between 0 and 1. A linear interval vector of points. Represents the linear step size, equal to Then, the integral loss of the probability density function in the discrete case can be approximated as:

[0136]

[0137] in, This represents the trainable parameters of the Copula neural network model. Indicates the linear step size. This represents the predicted probability density function value after non-negativity processing. It represents the range between 0 and 1. Linear interval vector of points The i-th point, It represents the range between 0 and 1. Linear interval vector of points The i-th point.

[0138] (3) The predicted probability density function value is non-negative within the domain of the Copula function. During training, it is necessary to penalize cases where the probability density function value is negative. When the probability density function value is continuous, the non-negativity constraint of the probability density function can be expressed as:

[0139]

[0140] Considering the inability to sample continuous probability density function values, the integral loss of the probability density function is rewritten in its discrete approximation form, assuming... and All are between 0 and 1. A linear interval vector of points. The nonnegativity loss of the probability density function in the discrete case is expressed as:

[0141]

[0142] in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value. Representing vectors The i-th value, Representing vectors The j-th value, and All are linear intervals between 0 and 1. The vector.

[0143] (4) The maximum likelihood estimation method is used to evaluate the fit of the neural network Copula model to the cumulative distribution function value of the training superpixel. The maximum likelihood estimation loss is expressed as:

[0144]

[0145] in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value after non-negativity processing. Indicates the preset evaluation coefficient. This represents the set of cumulative distribution function values ​​of superpixel features trained in the first time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This represents the set of cumulative distribution function values ​​for superpixel features trained in the second time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This indicates the number of training superpixels.

[0146] In theory It should be positive infinity, but in practical applications, it is usually taken as... That should be applicable to most situations.

[0147] (5) Joint distribution function observation loss is used to optimize the gap between the observed cumulative distribution function value and the empirical cumulative distribution function value. In order to ensure the consistency between the cumulative distribution function value of Copula and the cumulative distribution function value of the original data, it is necessary to sample some observation points within the domain of Copula function to reduce the gap between the cumulative distribution function value generated by the neural network Copula model and the empirical cumulative distribution function value at the observation points.

[0148] Assumption and It is a collection generated between 0 and 1 The two linear margin vectors of the observation points are generated from the distributions of the first-temporal training superpixel feature set and the second-temporal training superpixel feature set. and The cumulative distribution function values ​​are respectively and Therefore, the observation loss of the joint distribution function is expressed as:

[0149]

[0150] in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Indicates inclusion The linear interval vector of observation points The i-th value in Indicates inclusion The linear interval vector of observation points The j-th value in This represents the m-th feature value in the first-temporal training superpixel feature value set. This represents the m-th feature value in the second-temporal training superpixel feature value set. Indicates an indicator function, This indicates that the superpixel feature set generated in the first time-series training is generated from the distribution of the superpixel feature set. The cumulative distribution function value, This indicates that the superpixel feature set generated in the second time-series training is generated in the distribution of the superpixel feature set. The cumulative distribution function value.

[0151] Among them, indicator function The definition of is:

[0152]

[0153] Step B2: According to preset weights, linearly combine the calculated boundary loss, the probability density function integral loss, the probability density function nonnegativity loss, the maximum likelihood estimation loss, and the joint distribution function observation loss to obtain the loss function of the Copula neural network model. The loss function of the Copula neural network model is expressed as:

[0154]

[0155] in, , , , , These represent the weights of the boundary loss, the integral loss of the probability density function, the nonnegativity loss of the probability density function, the maximum likelihood estimation loss, and the observation loss of the joint distribution function, respectively. In practical applications, the corresponding weights are set according to the actual situation.

[0156] In this embodiment, the loss function based on Copula theory and the definition of neural networks makes the training of the Copula model of the neural network interpretable. The training process of the Copula model is explained using Copula theory, specifically by constraining the output boundary, the integral of the output result within the domain, the positive or negative value of the output result within the domain, the fit of the output result, and the difference between the output result and empirical data. In contrast, general neural networks are data-driven, directly using data labels to forcibly constrain the output result during training, resulting in a lack of interpretability. Therefore, this embodiment organically combines data-driven neural networks with knowledge-driven Copula theory, significantly improving the interpretability and robustness of existing heterogeneous change detection methods based on deep neural networks and Copula theory.

[0157] Figure 3 This is a schematic diagram of the overall framework of a heterogeneous change detection method using the interpretable neural network Copula, provided in an embodiment of this application. During the Copula model training phase, the remote sensing images before and after the change (i.e., dual-temporal training images) are first segmented into superpixels to obtain the segmentation results of the remote sensing images before and after the change (i.e., dual-temporal training dual-temporal superpixel sets). Then, unchanging training regions are selected, and unchanging training superpixel pairs are formed based on these regions to constitute a dual-temporal training superpixel set. The feature values ​​of the training superpixels in the dual-temporal training superpixel feature value set are extracted to obtain the cumulative distribution function value of the training superpixels. This cumulative distribution function value is then input into the Copula neural network model for training to obtain a satisfactory Copula neural network model. When detecting heterogeneous changes based on the Copula neural network model, the first step is to perform superpixel segmentation on the dual-temporal heterogeneous remote sensing image. Feature extraction is then performed on each superpixel in the dual-temporal superpixel set. Simultaneously, based on the kernel density estimation method, the cumulative distribution function value of each superpixel feature value is estimated to obtain a set of dual-temporal cumulative distribution function values. The cumulative distribution function values ​​in the dual-temporal cumulative distribution function value set are then input into the Copula neural network model to solve for the gradient, obtaining the probability density function value representing the correlation between superpixel pairs. Finally, fuzzy C-means clustering is performed on the probability density function values ​​to obtain the remote sensing image change detection result.

[0158] Figure 4 The diagram illustrates the training process of the Copula neural network model. Specifically, the training cumulative distribution function (CDF) values ​​from the set of dual-temporal training superpixel CDF values ​​obtained through kernel density estimation are input into the Copula neural network model for training, resulting in predicted CDF values ​​and predicted probability density function (PDF) values. Then, based on these values, boundary loss, integral loss of probability density function, non-negativity loss of probability density function, maximum likelihood estimation loss, and joint distribution function observation loss are calculated. These losses are then linearly combined according to preset weights to obtain the loss function of the Copula neural network model. Finally, the training parameters of the Copula neural network model are updated based on this loss function.

[0159] Reference Figure 5 The diagram shows a schematic representation of a heterogeneous change detection device for an interpretable neural network Copula, according to an embodiment of this application. Figure 5 As shown, the device includes:

[0160] Segmentation module 51 is used to perform superpixel segmentation on dual-temporal heterogeneous remote sensing images to obtain dual-temporal superpixel sets. The dual-temporal superpixel sets include: a first temporal superpixel set and a second temporal superpixel set. The superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs.

[0161] Extraction module 52 is used to extract features from each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set;

[0162] Estimation module 53 is used to estimate the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set based on the kernel density estimation method, so as to obtain the dual-temporal cumulative distribution function value set;

[0163] The solution module 54 is used to input the cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​into a pre-trained neural network Copula model to solve for the gradient, thereby obtaining the probability density function values ​​characterizing the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function.

[0164] The result module 55 is used to perform clustering and segmentation on the probability density function values ​​to obtain remote sensing image change detection results.

[0165] In one optional embodiment, the apparatus includes a training module for training the neural network Copula model, the training module comprising:

[0166] The sample construction module is used to construct a bitemporal training superpixel set with unchanged superpixel pairs as training samples;

[0167] The feature extraction module is used to extract the feature value of each training superpixel in the dual-temporal training sample set to obtain the dual-temporal training superpixel feature value set;

[0168] The feature processing module is used to process the training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the dual-temporal training superpixel cumulative distribution function value set;

[0169] The training prediction module is used to input the training cumulative distribution function values ​​from the dual-temporal training superpixel cumulative distribution function value set into the neural network Copula model for training, and to obtain the predicted cumulative distribution function value and the predicted probability density function value.

[0170] The loss calculation module is used to calculate the loss function of the neural network Copula model based on the predicted cumulative distribution function value and the predicted probability density function value, update the training parameters of the neural network Copula model based on the loss function, and obtain a satisfactory neural network Copula model after the training termination condition is met.

[0171] In one optional embodiment, the sample construction module includes:

[0172] The image segmentation module is used to process the training dual-temporal heterogeneous remote sensing image into a single-channel form and then perform superpixel segmentation to obtain a dual-temporal training dual-temporal superpixel set.

[0173] The pixel selection module is used to select the training region that does not change, and to determine the superpixel pair to be selected from the dual-temporal training dual-temporal superpixel set, and to determine whether the cross-union ratio between the superpixel pair to be selected and the selected training region is greater than the ratio threshold.

[0174] The pixel determination module is used to classify the candidate superpixel pairs into the dual-temporal training superpixel set when the cross-union ratio is greater than the ratio threshold.

[0175] In one optional embodiment, the feature processing module includes:

[0176] The distribution estimation module is used to fit the distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set based on the kernel density estimation method.

[0177] The function calculation module is used to estimate the cumulative distribution function value of the training superpixel feature value according to the distribution and by using a lookup table method, so as to obtain a set of cumulative distribution function values ​​of the dual-temporal training superpixel. The table used by the lookup table method consists of cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values.

[0178] In an optional embodiment, the function calculation module further includes a table generation module, the table generation module comprising:

[0179] The distribution fitting module is used to fit the distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the distribution of the training superpixel feature values.

[0180] The original generation module is used to estimate the cumulative distribution function values ​​of remote sensing image pixel values ​​from 0 to 255 according to the distribution, so as to serve as the constituent elements of the table;

[0181] The lookup module is used to search for the corresponding cumulative distribution function value in the table for each training superpixel feature value, and use it as the cumulative distribution function value of the training superpixel feature value.

[0182] In one optional embodiment, the loss calculation module includes:

[0183] The first loss calculation submodule is used to calculate the boundary loss, probability density function integral loss, probability density function non-negativity loss, maximum likelihood estimation loss, and joint distribution function observation loss based on the predicted cumulative distribution function value and the predicted probability density function value, respectively.

[0184] The second loss calculation submodule is used to linearly combine the calculated boundary loss, the probability density function integral loss, the probability density function non-negativity loss, the maximum likelihood estimation loss, and the joint distribution function observation loss according to preset weights to obtain the loss function of the neural network Copula model.

[0185] In one optional embodiment, the segmentation module includes:

[0186] The normalization module is used to normalize the probability density function value through negative logarithm operation to obtain the normalized probability density function value.

[0187] The pixel classification module is used to divide the normalized probability density function value into two categories using the fuzzy C-means clustering algorithm, and assign the superpixel to the corresponding pixel according to the change category to obtain the remote sensing image change detection result.

[0188] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0189] This application describes embodiments of methods and apparatus according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0193] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0194] The above provides a detailed description of the heterogeneous change detection method and apparatus for the interpretable neural network Copula provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting heterogeneous changes in an interpretable neural network Copula, characterized in that, The method includes: Superpixel segmentation is performed on dual-temporal heterogeneous remote sensing images to obtain dual-temporal superpixel sets. The dual-temporal superpixel sets include: a first temporal superpixel set and a second temporal superpixel set. The corresponding superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs. Feature extraction is performed on each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set; Based on the kernel density estimation method, the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set is estimated to obtain the dual-temporal cumulative distribution function value set; The cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​are input into a pre-trained neural network Copula model to solve for the gradient, thereby obtaining the probability density function values ​​that characterize the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function. Clustering and segmentation are performed on the probability density function values ​​to obtain the remote sensing image change detection results.

2. The method according to claim 1, characterized in that, The Copula neural network model is trained according to the following steps: Construct a bitemporal training superpixel set using unchanged superpixel pairs as training samples; Extract the feature values ​​of each training superpixel from the dual-temporal training sample set to obtain the dual-temporal training superpixel feature value set; The training superpixel feature values ​​in the dual-temporal training superpixel feature value set are processed to obtain the dual-temporal training superpixel cumulative distribution function value set; The training cumulative distribution function values ​​in the dual-temporal training superpixel cumulative distribution function value set are input into the neural network Copula model for training to obtain the predicted cumulative distribution function value and the predicted probability density function value. Based on the predicted cumulative distribution function value and the predicted probability density function value, the loss function of the neural network Copula model is calculated. The training parameters of the neural network Copula model are updated based on the loss function. After the training termination condition is met, the trained neural network Copula model is obtained.

3. The method according to claim 2, characterized in that, The construction of the training superpixel set using unchanged superpixel pairs as training samples includes: After processing the training dual-temporal heterogeneous remote sensing images into a single-channel form, superpixel segmentation was performed to obtain a dual-temporal training dual-temporal superpixel set. Select an unchanging training region, determine a pair of superpixels to be selected from the dual-temporal training dual-temporal superpixel set, and determine whether the cross-union ratio between the pair of superpixels to be selected and the selected training region is greater than the ratio threshold. If the cross-union ratio is greater than the ratio threshold, the candidate superpixel pairs are included in the dual-temporal training superpixel set.

4. The method according to claim 2, characterized in that, The step of processing the training superpixel feature values ​​in the dual-temporal training superpixel feature value set to obtain the dual-temporal training superpixel cumulative distribution function value set includes: Based on the kernel density estimation method, the distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set is fitted to obtain the distribution of training superpixel feature values. Based on the distribution, the cumulative distribution function value of the training superpixel feature value is estimated by using a lookup table method to obtain a set of cumulative distribution function values ​​of the dual-temporal training superpixel. The table used by the lookup table method consists of cumulative distribution function values ​​corresponding to 255 remote sensing image pixel values.

5. The method according to claim 4, characterized in that, The table is pre-generated according to the following steps: The distribution of training superpixel feature values ​​in the dual-temporal training superpixel feature value set is fitted to obtain the distribution of the training superpixel feature values. Based on the aforementioned distribution, the cumulative distribution function values ​​of the remote sensing image pixel values ​​from 0 to 255 are estimated respectively, and used as constituent elements of the table; Based on the distribution, the cumulative distribution function value of the trained superpixel feature values ​​is estimated using a lookup table method, including: For each training superpixel feature value, the corresponding cumulative distribution function value is found in the table and used as the cumulative distribution function value of that training superpixel feature value.

6. The method according to claim 2, characterized in that, The step of calculating the loss function of the Copula neural network model based on the predicted cumulative distribution function value and the predicted probability density function value includes: Based on the predicted cumulative distribution function value and the predicted probability density function value, the boundary loss, probability density function integral loss, probability density function non-negativity loss, maximum likelihood estimation loss, and joint distribution function observation loss are calculated respectively. According to preset weights, the calculated boundary loss, the probability density function integral loss, the probability density function nonnegativity loss, the maximum likelihood estimation loss, and the joint distribution function observation loss are linearly combined to obtain the loss function of the Copula neural network model.

7. The method according to claim 6, characterized in that, The predicted cumulative distribution function value needs to satisfy the boundary defined by the Copula function, and the boundary loss is expressed as: in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Represents the boundary point set The i-th point in Representing vectors The i-th value in Representing vectors The i-th value in and All are linear intervals between 0 and 1. The vector.

8. The method according to claim 6, characterized in that, The integral of the predicted probability density function over the domain of the Copula function is 1, and the integral loss of the probability density function is expressed as: in, This represents the trainable parameters of the Copula neural network model. Indicates the linear step size. This represents the predicted probability density function value after non-negativity processing. It represents the range between 0 and 1. Linear interval vector of points The i-th point, It represents the range between 0 and 1. Linear interval vector of points The i-th point.

9. The method according to claim 6, characterized in that, The predicted probability density function value is non-negative within the domain of the Copula function, and the non-negativity loss of the probability density function is expressed as: in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value. Representing vectors The i-th value, Representing vectors The j-th value, and All are linear intervals between 0 and 1. The vector.

10. The method according to claim 6, characterized in that, The sampling maximum likelihood estimation method evaluates the goodness of fit of the neural network Copula model to the cumulative distribution function values ​​of the training superpixels. The maximum likelihood estimation loss is expressed as: in, This represents the trainable parameters of the Copula neural network model. This represents the predicted probability density function value after non-negativity processing. Indicates the preset evaluation coefficient. This represents the set of cumulative distribution function values ​​of superpixel features trained in the first time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This represents the set of cumulative distribution function values ​​for superpixel features trained in the second time series. The cumulative distribution function value of the i-th training superpixel in the dataset. This indicates the number of training superpixels.

11. The method according to claim 6, characterized in that, The joint distribution function observation loss is used to optimize the gap between the predicted cumulative distribution function value and the empirical cumulative distribution function value. The joint distribution function observation loss is expressed as follows: in, This represents the trainable parameters of the Copula neural network model. This indicates the predicted cumulative distribution function value. Indicates inclusion The linear interval vector of observation points The i-th value in Indicates inclusion The linear interval vector of observation points The j-th value in This represents the m-th feature value in the first-temporal training superpixel feature value set. This represents the m-th feature value in the second-temporal training superpixel feature value set. Indicates an indicator function, This indicates that the superpixel feature set generated in the first time-series training is generated from the distribution of the superpixel feature set. The cumulative distribution function value, This indicates that the superpixel feature set generated in the second time-series training is generated in the distribution of the superpixel feature set. The cumulative distribution function value.

12. The method according to claim 1, characterized in that, The process of clustering and segmenting the probability density function values ​​to obtain remote sensing image change detection results includes: The probability density function value is normalized by performing a negative logarithm operation to obtain a normalized probability density function value. The normalized probability density function values ​​are divided into two categories using the fuzzy C-means clustering algorithm, and then assigned to the corresponding pixels according to the change category to which the superpixel belongs, thus obtaining the remote sensing image change detection results.

13. A device for detecting heterogeneous changes in an interpretable neural network Copula, characterized in that, The device includes: The segmentation module is used to perform superpixel segmentation on dual-temporal heterogeneous remote sensing images to obtain dual-temporal superpixel sets. The dual-temporal superpixel sets include: a first temporal superpixel set and a second temporal superpixel set. The corresponding superpixels in the first temporal superpixel set and the second temporal superpixel set constitute superpixel pairs. The extraction module is used to extract features from each superpixel in the dual-temporal superpixel set to obtain a dual-temporal superpixel feature value set; The estimation module is used to estimate the cumulative distribution function value of each superpixel feature value in the dual-temporal superpixel feature value set based on the kernel density estimation method, so as to obtain the dual-temporal cumulative distribution function value set; The solution module is used to input the cumulative distribution function values ​​in the set of dual-temporal cumulative distribution function values ​​into a pre-trained neural network Copula model to solve for the gradient, thereby obtaining the probability density function value characterizing the correlation of superpixel pairs. The neural network Copula model uses unchanged superpixel pairs as training samples and combines a loss function designed based on data-driven neural networks and knowledge-driven Copula theory to promote the convergence of the neural network Copula model to the form of a Copula function. The results module is used to cluster and segment the probability density function values ​​to obtain the remote sensing image change detection results.

Images