Remote sensing image building change detection method based on conditional adversarial network

Abstract

The present invention relates to the technical field of remote sensing image building detection, and to a remote sensing image building change detection method based on a conditional adversarial network, including: 1) using a fully convolutional network to study semantic analysis and adaptive propagation methods of multi-source high-resolution image buildings, and studying multi-temporal building morphological feature perception methods and implicit probability distribution; 2) using a loss function to optimize and train the network, carry out self-learning, analysis and generation of a polymorphic simulation set of buildings, and simultaneously perform authenticity determination, optimization and confrontation, study the confrontation and optimization method between a generation model and a discriminant model, and extract the probability distribution of building changes; 3) studying a domain-invariant feature analysis method of a building, adding a classifier model, and obtaining a building change detection result; 4) using real surface change data to perform qualitative and quantitative evaluation in multiple ways. The present invention uses a conditional adversarial network to achieve effective determination and extraction of building changes.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent change detection of remote sensing image buildings, and in particular to a remote sensing image building change detection method based on a conditional adversarial network. Background Art

[0002] Dynamic monitoring of the spatial structure of buildings provides reliable decision-making information and technical support for urbanization management, planning and construction. High-resolution remote sensing images are an important means of dynamic monitoring of deep urbanization because of their rich information on detailed representation of objects, spatial structure, and neighboring relationships. They are of great significance for understanding the relationship and interaction between social development and human activities. However, the accelerated development of urbanization and the heterogeneity between multi-source high-resolution images have increased the difficulty of dynamic monitoring of buildings, resulting in polymorphism of buildings in high-resolution images in different phases, which greatly limits the development of automatic interpretation of building change information. Therefore, under the influence of imaging conditions, buildings themselves and background factors, it is of great research significance and application value to develop effective building change information extraction methods for the polymorphism of multi-phase buildings, improve the accuracy of building detection and maintain its integrity.

[0003] Existing methods for building change detection can be roughly divided into the following three categories:

[0004] 1) Change detection based on pixel spectral information: that is, obtaining the intensity of change by calculating the spectral difference information before and after the phase. Typical methods include algebraic operation, fuzzy clustering, feature transformation, etc. This type of method can effectively detect the change target in the change detection of low- and medium-resolution (above 10 meters) remote sensing images. However, when applied to high-resolution images with different spectral resolutions (especially multi-source remote sensing images), its accuracy is significantly reduced. However, this type of method only considers the spectral information of a single pixel and assumes that the spatial features of adjacent pixels are independent. In the actual application of high-resolution images, the building features are more refined and there is a feature dependency relationship between adjacent pixels.

[0005] 2) Change detection that takes into account spatial information: that is, the image is divided into processing units, and the change detection is performed by combining the spectrum and space. It mainly includes: (1) Change detection methods based on fixed-size processing units such as texture information extraction and Markov random field spatial information extraction; (2) Object-oriented change detection methods, which segment pixel clusters with spectral and spatial feature homogeneity as basic units. Its representatives are eCognition's object-oriented methods and segmentation strategies; (3) Morphological methods, which extract ground object structure information by analyzing image morphological components, such as building index, morphological filtering, region growing method, morphological attribute analysis, etc. This type of method can take into account the spatial distribution characteristics of buildings and improve the accuracy of change detection in high-resolution remote sensing images. However, this type of method regards multi-temporal building targets as homogeneous objects for comparison, which has the phenomenon of building distribution distortion and cannot effectively utilize the detailed information inside the target object; At the same time, the spectral differences between multi-source high-resolution images make the object-oriented method of spectral feature analysis difficult to apply, which in turn affects the change detection results.

[0006] 3) Change detection based on deep learning: that is, based on the statistical consistency of the observed data, it can automatically and multi-levelly identify and extract nonlinear features, greatly improving the accuracy of image feature extraction, including generative adversarial networks, joint sparse expressions, spatial structure extraction based on neural networks and random forests, etc. This type of method can extract spatial and spectral integrated features, establish nonlinear correlations between target features, and to a certain extent suppress the feature differences between buildings in multi-source remote sensing images, thereby improving detection accuracy. However, this type of method still has difficulty in taking into account the correlation and complementarity of semantic features of multi-phase buildings, the polymorphism of buildings has not been effectively utilized, and the sample library is difficult to build, which limits the development of deep learning change detection.

[0007] In summary, the research on building change detection based on high-resolution remote sensing images has made great progress. In particular, the detection method based on deep learning has shown great application potential. However, there are still many defects, which are mainly reflected in:

[0008] 1) It is difficult to express the polymorphism of buildings in multi-temporal images. Target diversity, complex structure, shadow occlusion and other phenomena are the pain points and difficulties of building change detection. In high-resolution remote sensing images, even if the target has not changed, it will cause a magnification effect of feature differences, such as factory buildings in industrial areas, low-rise buildings in residential areas, etc.; at the same time, complex background problems such as differences in building spatial structure and landform categories are prominent. For example, high-rise buildings are densely populated in the city center, and their spatial position and distribution are also very different due to imaging conditions. It is difficult to take into account the integrity of the building's morphological characteristics, and thus it is impossible to accurately calculate and determine the building change information.

[0009] 2) It is difficult to construct a training set for deep learning change detection methods. The current technical development direction is to combine artificial feature prior knowledge with deep learning models, and several deep learning change detection methods have been proposed. However, the diversity and complexity of buildings and the heterogeneity of multi-source remote sensing images objectively exist, which puts extremely high demands on change detection training sets. For example, complex conditions such as imaging systems, observation conditions, regional types, and background environments make it difficult to obtain deep learning training samples, lack public sample data sets, and have few related studies, which seriously affects the development of intelligent interpretation of remote sensing images. Summary of the invention

[0010] The present invention provides a remote sensing image building change detection method based on conditional adversarial network, which can overcome certain or some defects of the prior art.

[0011] According to the present invention, a remote sensing image building change detection method based on a conditional adversarial network comprises the following steps:

[0012] 1) In the conditional adversarial network, the semantic analysis and adaptive propagation methods of multi-source high-resolution image buildings are studied by using the fully convolutional network. The noise vector is constructed by cross-fusion to obtain its potential relationship. The domain adaptive feature space is introduced to study the multi-temporal building morphological feature perception method and implicit probability distribution, providing a reliable data source for the self-learning and training of the conditional adversarial network;

[0013] 2) Use the loss function to optimize and train the network, combine the building morphological characteristics, and use the generative model to carry out self-learning, analysis and generation of the building polymorphic simulation set. At the same time, perform authenticity judgment, optimization and confrontation between the data of the discriminant model. Combined with the multi-level semantic features and potential spatial relationships of the building, study the confrontation and optimization methods between the generative model and the discriminant model to accurately extract the change probability distribution of the building;

[0014] 3) Research the domain-invariant feature analysis method of buildings, add the classifier model with Softmax cross entropy loss function, and combine it with decision rules to obtain high-precision building change detection results;

[0015] 4) Use real surface change data to conduct qualitative and quantitative evaluation of change detection results in various ways.

[0016] As a preferred method, the multi-layer network structure of the fully convolutional network is used to study the adaptive learning and analysis methods from shallow local features to deep abstract features, design multi-source data batch normalization and nonlinear activation functions, improve the stability and convergence speed of the network, and realize the multi-level semantic feature analysis of buildings, as follows:

[0017] Each layer of the convolutional network is a three-dimensional data consisting of h×w spatial dimensions and d feature dimensions. It performs convolution operations and feature extraction on high-resolution remote sensing images. In the convolutional layer, the input pixel coordinates (i, j) are x ij vector, whose output vector is y ij , and its expression formula is:

[0018] y ij =f Ks ({x si+δi,sj+δj}0≤δi,δj≤K) (11)

[0019] Among them, K is the convolution kernel size, s is the stride length, and finally the feature image is obtained through the activation function f. When the convolution kernel size and stride length follow the conversion rule, the activation function is expressed as:

[0020]

[0021] in, Represents Hadamard product; when the ordinary deep network g calculates the nonlinear function and only calculates the nonlinear filtering for each layer, it can be called a fully convolutional network; it calculates through input data of any size and generates the corresponding spatial dimension output. If the loss function is the sum of the spatial dimensions of the last layer, it is expressed as:

[0022]

[0023] Ultimately, the gradient l will be the sum of the gradients of its spatial components at each layer.

[0024] As a preferred method, the Leaky ReLU (Rectified Linear Unit) activation function is used, which is a specialization of the rectified linear unit, so that the model can adapt to spatial propagation, maintain the essential characteristics and stability of high-resolution images, and have sparse activation characteristics; when the Leaky ReLU is not activated, it still outputs a non-zero value, which is expressed as:

[0025]

[0026] Among them, α is a floating point number greater than 0;

[0027] Batch normalization is used in each layer of the fully convolutional network, and its expression formula is:

[0028]

[0029] in, β ij =E[x ij ].

[0030] Preferably, the T1 phase image is used as the input data of the generative model, and the T2 phase image is used as the input data of the discriminative model. The noise vector is constructed by cross-fusion. The perception method of building morphological features is studied. Taking into account the structural differences of land object categories, a domain adaptive space is constructed. The generator automatically generates a polymorphic simulation set of buildings to realize adaptive learning and training of the network, discover high-level expressions in the image, and obtain its change probability distribution based on the building morphology, thereby suppressing the representation differences between multi-phase buildings.

[0031] As a preference, is a dataset from the T1 phase image domain, The data set from the T2 phase, the generator function G(x s ,z1;θ G )→x f , by the parameter θ G It means that the generator G(x s ,z1;θ G )→x f The generated image x of the image under T1 phase s and noise vector z is mapped to the domain adaptive feature image x f , the generator function G can generate a new data set X of any size f = {G(x s ,z),y s}; The generator noise vector is extracted by cross-fusion of the panchromatic image at T1 phase and the multispectral image at T2 phase using Gram-Schimidt Adaptive, so that the features of the unchanged targets remain unchanged, the pseudo-changed targets show similar spatial features, and the changed targets show larger spectral distortion and fuzzy quantization;

[0032] The discriminant model in the conditional adversarial network consists of the discriminator function D(x; θ D ), the discriminator is composed of the target domain X from the T2 phase image x t For the feature image X generated by the generator f To make a judgment, the likelihood estimation d is performed between the image output by the generative model and the real image x at the T2 phase; in addition, the classifier Join the conditional adversarial network and assign the discrimination result label to the image x∈{X f ,X t};

[0033] The conditional adversarial network can be expressed as:

[0034]

[0035] Among them, α, β are the weights that control the interaction of loss functions, L d It represents the domain loss, and its expression formula is:

[0036]

[0037] L t The typical Softmax cross entropy loss function is used for classification label calculation:

[0038]

[0039] In the formula, y s For x s One-hot encoding of image category labels; when training the classifier T, the original image and simulated image are used to train T, but only the domain adaptive feature image x is used f When different initialization values ​​are used to maintain the stability of the model, the generative model can change the classification task, optimize and train the building simulation set, and learn the implicit probability distribution of the real data without the source of training data.

[0040] As a preferred method, a morphological feature perception method is used to describe building features, that is, the relationship between the morphologies is expressed without losing the building morphological information; in the process of expressing the building morphological features of high-resolution images, a fast morphological decomposition method is used to obtain the set of building morphologies in the image, and the relationship between the morphologies is used to establish multi-level morphological features; wherein, an image level set decomposition is established, and the image i:Ω→R is decomposed into an upper level set and a lower level set, which are respectively expressed as: I μ (i)={p∈Ω;i(p)≥μ} and I μ (i) = {p∈Ω; i(p)≤μ}, μ∈R is a grayscale value; the level set is used as the complete expression of the image, and the building morphology is defined as the set of all pixels within the horizontal contour line. Then each node under the morphological feature set represents a morphology, which describes the morphological characteristics of the building in the image;

[0041] In the process of minimax optimization of conditional adversarial network, under the premise of perception of building morphological features, the multi-level joint expression of multi-temporal images is extracted, and the multi-state simulation set of buildings is automatically generated according to the self-learning and simulation of the generative model and the confrontation and optimization of the discriminant model, and the self-training and analysis of the network are realized to obtain high-precision probability distribution of changes; first, keep the parameters of the generative model unchanged, and perform discriminant function θ on the discriminant model. D Update and classifier function θ T Optimize; secondly, keep θ D and θ T No change, update the generation function θ of the generation modelG ; Secondly, the building features learned from multi-temporal high-resolution images are projected into the domain adaptive space for nonlinear feature recognition. In this space, the high-level features of unchanged buildings tend to be consistent, while the features of changed buildings have large differences, thus showing high separability; finally, through the optimization and confrontation of the conditional adversarial network, the polymorphic simulation set of buildings is continuously generated and expanded, X s and X t As the input of the discriminator D, the discriminator estimates the probability distribution of data changes; in addition, the discriminator will output a scalar value, which is derived using the maximum likelihood estimation formula when assuming that the probability of building features in each phase depends only on the characteristics of the building itself, to represent the maximum likelihood probability of the generated model output data and the real data.

[0042] As a preferred method, the similarity loss function is studied through the processing method of consistency training, and the domain invariant feature analysis method of scale invariant mask is designed to eliminate the huge difference between the real image and the simulated image before and after the change, and maintain the real characteristics of the building; at the same time, the similarity loss function based on the real image helps to stabilize the optimization process of the conditional adversarial network, and the collaborative decision rule further improves the change detection accuracy; therefore, the optimization problem can be implemented using the following formula:

[0043]

[0044] Among them, α, β, γ are the weights that control the interaction of loss functions, L c is the similarity loss; in the similarity loss function, a scale-invariant error mask is used to eliminate the feature difference between the real image and the simulated image, rather than the absolute difference between the input and the output. When there is an error between the simulated image and the real image, the similarity loss function gives the mask m∈R k Expressed as:

[0045]

[0046] Where k represents the number of pixels in the input image x. The square of the regularized L2 norm is the sum of the squares of the elements of the vector and then the square root is taken; the loss function allows the model to learn and reproduce the overall morphological characteristics of the building without operating on the spectrum or grayscale of the input data; for data sources of different time phases, adversarial training is allowed to be performed in a consistent manner.

[0047] Preferably, after acquiring the semantic features and change probability distribution of the building, the building change information is optimized according to the decision rules.

[0048] The present invention takes the simulation set and self-learning of building polymorphism as the core, based on the idea of ​​conditional adversarial network, carries out feature self-learning of the generative model and automatically generates a training set, and simultaneously conducts confrontation, optimization and judgment of the discriminant model, combines morphological feature perception and domain adaptive space methods, analyzes and simulates the polymorphism of multi-phase buildings, and studies the method of self-learning and identifying the essential characteristics of buildings by automatically generating a polymorphic simulation set of buildings, and obtains the probability distribution of building changes; in addition, it is proposed to introduce domain invariant feature analysis and decision rules into the conditional adversarial network to further optimize the building change detection results. The present invention uses conditional adversarial network to realize the effective judgment and extraction of building changes, in order to improve the accuracy and integrity of intelligent change detection of buildings, explore the application potential of high-resolution image change detection, and has method innovation and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flowchart of a remote sensing image building change detection method based on conditional adversarial network in Example 1. DETAILED DESCRIPTION

[0050] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0051] Example 1

[0052] like Figure 1 As shown, this embodiment provides a remote sensing image building change detection method based on a conditional adversarial network, which includes the following steps:

[0053] 1) In the conditional adversarial network, the semantic analysis and adaptive propagation methods of multi-source high-resolution image buildings are studied by using the fully convolutional network. The noise vector is constructed by cross-fusion to obtain its potential relationship. The domain adaptive feature space is introduced to study the multi-temporal building morphological feature perception method and implicit probability distribution, providing a reliable data source for the self-learning and training of the conditional adversarial network;

[0054] 2) Use the loss function to optimize and train the network, combine the building morphological characteristics, and use the generative model to carry out self-learning, analysis and generation of the building polymorphic simulation set. At the same time, perform authenticity judgment, optimization and confrontation between the data of the discriminant model. Combined with the multi-level semantic features and potential spatial relationships of the building, study the confrontation and optimization methods between the generative model and the discriminant model to accurately extract the change probability distribution of the building;

[0055] 3) Research the domain-invariant feature analysis method of buildings, add the classifier model with Softmax cross entropy loss function, and combine it with decision rules to obtain high-precision building change detection results;

[0056] 4) Use real surface change data to conduct qualitative and quantitative evaluation of change detection results in various ways.

[0057] The background of buildings in multi-temporal high-resolution images is complex and the hierarchical structure is diverse. In order to effectively extract them, it is necessary to distinguish the polymorphism of buildings at multiple levels so as to realize the subsequent processing of multi-temporal buildings. Based on this, the multi-layer network structure of the fully convolutional network is used to study the adaptive learning and analysis methods from shallow local features to deep abstract features, and multi-source data batch normalization and nonlinear activation functions are designed to improve the stability and convergence speed of the network, so as to realize the multi-level semantic feature analysis of buildings, as follows:

[0058] Each layer of the convolutional network is a three-dimensional data consisting of h×w spatial dimensions and d feature dimensions. It performs convolution operations and feature extraction on high-resolution remote sensing images. In the convolutional layer, the input pixel coordinates (i, j) are x ij vector, whose output vector is y ij , and its expression formula is:

[0059] y ij =f Ks ({x si+δi,sj+δj}0≤δi,δj≤K) (21)

[0060] Among them, K is the convolution kernel size, s is the stride length, and finally the feature image is obtained through the activation function f. When the convolution kernel size and stride length follow the conversion rule, the activation function is expressed as:

[0061]

[0062] in, Represents Hadamard product; when a normal deep network g calculates a nonlinear function and only calculates nonlinear filtering for each layer in this way, it can be called a fully convolutional network; it calculates with input data of any size and generates corresponding spatial dimension output. If the loss function is the sum of the spatial dimensions of the last layer, it is expressed as:

[0063]

[0064] Ultimately, the gradient l will be the sum of the gradients of each spatial component of each layer. So in the image, the stochastic gradient descent calculation based on l is consistent with the gradient descent based on l', and the network processes all received features in batches in the last layer.

[0065] In order to provide effective building feature input and learning for the conditional adversarial network and maintain the original spatial structure and potential features, the Leaky ReLU (Rectified Linear Unit) activation function is used, which is a specialization of the rectified linear unit, so that the model can adapt to spatial propagation, maintain the essential characteristics and stability of high-resolution images, and have sparse activation characteristics; when the Leaky ReLU is not activated, it still outputs a non-zero value, which is expressed as:

[0066]

[0067] Among them, α is a floating point number greater than 0;

[0068] In order to ensure the stability of the network gradient during the training and transmission process, speed up the convergence speed, prevent the gradient diffusion effect, and enable the conditional adversarial network to perform nonlinear feature recognition and analysis on high-resolution image buildings, "batch normalization" (BN) is used in each layer of the full convolutional network. The expression formula is:

[0069]

[0070] in, β ij =E[x ij ].

[0071] At present, convolutional networks have achieved remarkable results in the field of deep learning and pattern recognition. It has been proven that layer-by-layer abstraction of data through deep networks can greatly improve the accuracy of image feature recognition. Therefore, the image features are spatially abstracted by the maximum pooling method, and sparsely expressed in subsequent processing. At the same time, in the process of multi-layer network expression and feature abstraction, the network adaptive spatial propagation mechanism and the conditional adversarial network learning and training layer-by-layer abstraction feature method are studied to provide reliable input data for subsequent building change detection.

[0072] Generally speaking, the polymorphism of buildings in multi-temporal high-resolution images is very sensitive to subtle changes in the external environment, and the changes are extremely frequent. This phenomenon is more obvious under different sensors. However, if the building itself has not changed, there will be statistical consistency in the observed target in the data. Therefore, multi-temporal high-resolution images have high-level expressions and potential distributions of buildings. Based on this principle, this project proposes a change detection method for conditional adversarial networks, using T1 phase images as input data for the generative model and T2 phase images as input data for the discriminative model. A noise vector is constructed by cross-fusion, and the perception method of building morphological features is studied. Taking into account the structural differences of the land object categories, a domain adaptive space is constructed. The generator automatically generates a polymorphic simulation set of buildings to achieve adaptive learning and training of the network, discover high-level expressions in the image, and obtain its change probability distribution based on the building morphology, thereby suppressing the representation differences between multi-temporal buildings.

[0073] is a dataset from the T1 phase image domain, The data set from the T2 phase, the generator function G(x s ,z1;θ G )→x f , by the parameter θ G It means that the generator G(x s ,z1;θ G )→x f The generated image x of the image under T1 phase s and noise vector z is mapped to the domain adaptive feature image x f , the generator function G can generate a new data set X of any size f = {G(x s ,z),y s}; The generator noise vector is extracted by cross-fusion of the panchromatic image at T1 phase and the multispectral image at T2 phase using Gram-Schimidt Adaptive, so that the features of the unchanged targets remain unchanged, the pseudo-changed targets show similar spatial features, and the changed targets show larger spectral distortion and fuzzy quantization;

[0074] The discriminant model in the conditional adversarial network consists of the discriminator function D(x; θ D ), the discriminator is composed of the target domain X from the T2 phase image x t For the feature image X generated by the generator f To make a judgment, the likelihood estimation d is performed between the image output by the generative model and the real image x at the T2 phase; in addition, the classifier Join the conditional adversarial network and assign the discrimination result label to the image x∈{Xf ,X t};

[0075] The conditional adversarial network can be expressed as:

[0076]

[0077] Among them, α, β are the weights that control the interaction of loss functions, L d It represents the domain loss, and its expression formula is:

[0078]

[0079] L t The typical Softmax cross entropy loss function is used for classification label calculation:

[0080]

[0081] In the formula, y s For x s One-hot encoding of image category labels; when training the classifier T, the original image and simulated image are used to train T, but only the domain adaptive feature image x is used f When different initialization values ​​are used to maintain the stability of the model, the generative model can change the classification task, optimize and train the building simulation set, and learn the implicit probability distribution of real data without the source of training data. In addition, while the generative model trains the classifier based on the image of the source domain and the domain adaptive image, the noise vector function z and the classifier function θ T The combined hidden layer representation greatly increases the stability of the model, and uses different spatial distribution feature markers for building features to reduce the impact of factors such as shadows and occlusions on high-resolution images under different imaging conditions.

[0082] In order to accurately and effectively express the polymorphism of high-resolution image buildings, the morphological feature perception method is used to describe the building features, that is, the relationship between the morphologies is expressed without losing the building morphological information, and the overall morphological characteristics of the building are reflected to a certain extent. In the process of expressing the morphological features of high-resolution image buildings, the fast morphological decomposition method is used to obtain the set of building morphologies in the image, and the relationship between the morphologies is used to establish multi-level morphological features. Among them, the image level set decomposition is established, and the image i:Ω→R is decomposed into an upper level set and a lower level set, which are expressed as: I μ (i)={p∈Ω;i(p)≥μ} and I μ(i) = {p∈Ω; i(p)≤μ}, μ∈R is a certain grayscale value; the level set is used as the complete expression of the image, and the building morphology is defined as the set of all pixels within the horizontal contour line, then each node under the morphological feature set represents a morphology, and the morphological characteristics of the building in the image are described; based on the topological map expression and building semantic analysis, this method realizes the perception of multi-level morphological characteristics of buildings, describes the relationship between building morphologies, effectively expresses the local information of buildings, and has local contrast invariance, which has great advantages in the accurate extraction and recognition of building morphological features in high-resolution images.

[0083] In the process of minimax optimization of conditional adversarial network, under the premise of perception of building morphological features, the multi-level joint expression of multi-temporal images is extracted, and the multi-state simulation set of buildings is automatically generated according to the self-learning and simulation of the generative model and the confrontation and optimization of the discriminant model, and the self-training and analysis of the network are realized to obtain high-precision probability distribution of changes; first, keep the parameters of the generative model unchanged, and perform discriminant function θ on the discriminant model. D Update and classifier function θ T Optimize; secondly, keep θ D and θ T No change, update the generation function θ of the generation model G ; Secondly, the building features learned from multi-temporal high-resolution images are projected into the domain adaptive space for nonlinear feature recognition. In this space, the high-level features of unchanged buildings tend to be consistent, while the features of changed buildings have large differences, thus showing high separability; finally, through the optimization and confrontation of the conditional adversarial network, the polymorphic simulation set of buildings is continuously generated and expanded, X s and X t As the input of the discriminator D, the discriminator estimates the probability distribution of the data change; in addition, the discriminator will output a scalar value. When assuming that the probability of building features in each phase depends only on the characteristics of the building itself, the maximum likelihood estimation formula is used for derivation to represent the maximum likelihood probability of the generated model output data and the real data.

[0084] In order to ensure the integrity and accuracy of the building change detection results, the similarity loss function is studied through the consistency training processing method, and the domain invariant feature analysis method of the scale invariant mask is designed to eliminate the huge difference between the real image and the simulated image before and after the change, and maintain the real characteristics of the building; at the same time, the similarity loss function based on the real image helps to stabilize the optimization process of the conditional adversarial network, and the collaborative decision-making rule further improves the change detection accuracy; therefore, the optimization problem can be implemented using the following formula:

[0085]

[0086] Among them, α, β, γ are the weights that control the interaction of loss functions, L c is the similarity loss; in the similarity loss function, a scale-invariant error mask is used to eliminate the feature difference between the real image and the simulated image, rather than the absolute difference between the input and the output. When the mask has an error between the simulated image and the real image, the similarity loss function gives the mask m∈R k Expressed as:

[0087]

[0088] Where k represents the number of pixels in the input image x. The square of the regularized L2 norm is the sum of the squares of the elements of the vector and then the square root is taken; the loss function allows the model to learn and reproduce the overall morphological characteristics of the building without operating on the spectrum or grayscale of the input data; for data sources in different time phases, adversarial training is allowed to be performed in a consistent manner, so that the similarity loss function can effectively maintain the semantic characteristics and integrity of the building, reduce the impact of building polymorphism in different time phases, and improve the accuracy of building change detection.

[0089] After obtaining the semantic features and change probability distribution of the building, the building change information is optimized according to the decision rules. If the change probability of the multi-temporal image building is very small, then it should belong to the same target. Therefore, several decision rules are proposed. If the changed building extracted according to the change probability distribution meets the rule, the multi-temporal building will be revised to an unchanged building, such as unfinished, house renovation, etc. The unchanged building can be determined by the change probability obtained by the conditional adversarial network analysis method. For example, a building with a change probability of less than 50% is considered to have not changed. The purpose of the decision rule is to eliminate possible false detection problems. On the basis of maintaining the original accuracy, the change detection information is further integrated to improve the recognition effect to ensure the reliability of building change detection.

[0090] This embodiment uses the conditional adversarial network of the Nash equilibrium of game theory as a framework, and conducts confrontation and optimization under the minimax game by learning the generative model of data features and the discriminant model for determining the authenticity of data features, multi-level calculations to solve, adaptively analyze and extract feature information in the image, reconstruct, simulate and restore the true characteristics of the data. This method has achieved remarkable and excellent processing effects in scene classification, domain adaptation, target detection and other aspects. In the detection of building changes in high-resolution remote sensing images, this method can reveal the mapping relationship between multi-phase data, self-learn and analyze the polymorphism of buildings and automatically generate training sets, thereby eliminating the representation differences caused by external factors and highlighting the real change information. Therefore, this framework provides a new way to achieve high-precision building change detection.

[0091] In view of this, based on the polymorphic simulation conditional adversarial network, we develop accurate models and methods suitable for high-resolution remote sensing image building change detection. The main research ideas are: (1) Develop adaptive spatial propagation and mapping methods based on building semantic analysis of fully convolutional networks, study the multi-level building morphological feature perception and latent spatial relationship, and realize high-precision building semantic feature joint extraction; (2) Using the idea of ​​conditional adversarial network, combined with building morphological feature perception and domain adaptive feature space, study the automatic generation method of building polymorphic simulation set, realize network adaptive learning and training, and obtain the probability distribution of building changes through multi-level network confrontation and optimization; (3) With the support of building polymorphic simulation set, study the domain loss function of building domain invariant feature analysis, design building change decision rules, develop multi-temporal building change information optimization and extraction methods, and further improve the accuracy of change detection.

[0092] This embodiment proposes a remote sensing image building change detection method based on a polymorphic simulation conditional adversarial network, in order to propose a high-precision deep learning change detection method that supports polymorphic learning and semantic feature analysis of buildings in the field of remote sensing intelligent interpretation. The implementation of this method has important practical significance for further promoting the intelligence and automation level of remote sensing technology in the fields of geographic national conditions monitoring, national land surveys, and urban research.

[0093] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A remote sensing image building change detection method based on conditional adversarial network, characterized by: The following steps are involved: 1) In the conditional adversarial network, the semantic analysis and adaptive propagation methods of multi-source high-resolution image buildings are studied by using the fully convolutional network. The noise vector is constructed by cross-fusion to obtain its potential relationship. The domain adaptive feature space is introduced to study the multi-temporal building morphological feature perception method and implicit probability distribution, providing a reliable data source for the self-learning and training of the conditional adversarial network; 2) Use the loss function to optimize and train the network, combine the building morphological characteristics, and use the generative model to carry out self-learning, analysis and generation of the building polymorphic simulation set. At the same time, the authenticity judgment, optimization and confrontation between the data of the discriminant model are carried out. Combined with the multi-level semantic features and latent spatial relationships of the building, the confrontation and optimization methods between the generative model and the discriminant model are studied to accurately extract the change probability distribution of the building; 3) Research the domain-invariant feature analysis method of buildings, add the classifier model with Softmax cross entropy loss function, and combine it with decision rules to obtain high-precision building change detection results; 4) Use the real surface change data to conduct qualitative and quantitative evaluation of the change detection results in various ways; By using the multi-layer network structure of the fully convolutional network, we study the adaptive learning and analysis methods from shallow local features to deep abstract features, design multi-source data batch normalization and nonlinear activation functions, improve the stability and convergence speed of the network, and realize the multi-level semantic feature analysis of buildings. The details are as follows: Each layer of the convolutional network is a three-dimensional data consisting of h×w spatial dimensions and d feature dimensions. It performs convolution operations and feature extraction on high-resolution remote sensing images. In the convolutional layer, the x value at the pixel coordinate (i, j) is input. ij vector, whose output vector is y ij , and its expression formula is: y ij =f Ks ({x si+δi,sj+δj }0≤δi,δj≤K) (1) Among them, K is the convolution kernel size, s represents the stride length, and finally the feature image is obtained through the activation function f; When the convolution kernel size and stride length follow the conversion rule, the activation function is expressed as: in, Represents Hadamard product; when ordinary deep network g calculates nonlinear functions, it only calculates nonlinear filtering for each layer, which is called a fully convolutional network; it calculates through input data of any size and generates corresponding spatial dimension output. If the loss function is the sum of the spatial dimensions of the last layer, it is expressed as: Ultimately, the gradient l will be the sum of the gradients of its spatial components at each layer.

2. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 1, characterized in that: The Leaky ReLU activation function, which is a specialization of the corrected linear unit, enables the model to adapt to spatial propagation, maintain the essential characteristics and stability of high-resolution images, and have sparse activation characteristics; when the Leaky ReLU is not activated, it still outputs a non-zero value, which is expressed as: Among them, α is a floating point number greater than 0; Batch normalization is used in each layer of the fully convolutional network, and its expression formula is: in, β ij =E[x ij ].

3. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 2, characterized in that: The T1 phase image is used as the input data of the generative model, and the T2 phase image is used as the input data of the discriminative model. The noise vector is constructed by cross fusion. The perception method of building morphological characteristics is studied. Taking into account the structural differences of land object categories, a domain adaptive space is constructed. The generator automatically generates a polymorphic simulation set of buildings to realize adaptive learning and training of the network, discover the high-level expression in the image, and obtain its change probability distribution based on the building morphology, thereby suppressing the representation differences between multi-phase buildings.

4. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 3 is characterized in that: is a dataset from the T1 phase image domain, The data set from the T2 phase, the generator function G(x s ,z1;θ G )→x f , by the parameter θ G It means that the generator G(x s ,z1;θ G )→x f The generated image x of the image under T1 phase s and noise vector z is mapped to the domain adaptive feature image x f , the generator function G can generate a new data set X of any size f = {G(x s ,z),y s }; The generator noise vector is extracted by cross-fusion of panchromatic images at T1 phase and multispectral images at T2 phase using Gram-Schimidt Adaptive, so that the features of unchanged targets remain unchanged, pseudo-changed targets show similar spatial features, and changed targets show large spectral distortion and fuzzy quantization. The discriminant model in the conditional adversarial network consists of the discriminator function D(x; θ D ), the discriminator is composed of the target domain X from the T2 phase image x t For the feature image X generated by the generator f To make a judgment, the likelihood estimation d is performed between the image output by the generative model and the real image x at the T2 phase; in addition, the classifier Join the conditional adversarial network and assign the discrimination result label to the image x∈{X f ,X t }; The conditional adversarial network is expressed as: Among them, α, β are the weights that control the interaction of loss functions. It represents the domain loss, and its expression formula is: The typical Softmax cross entropy loss function is used for classification label calculation: In the formula, y s For x s One-hot encoding of image category labels; when training the classifier T, the original image and simulated image are used to train T, but only the domain adaptive feature image x is used f When different initialization values ​​are used to maintain the stability of the model, the generative model can change the classification task, optimize and train the building simulation set, and learn the implicit probability distribution of real data without the source of training data.

5. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 4 is characterized in that: The method of morphological feature perception is used to describe the building features, that is, the relationship between the morphologies is expressed without losing the building morphological information; in the process of expressing the building morphological features of high-resolution images, the fast morphological decomposition method is used to obtain the set of building morphologies in the image, and the relationship between the morphologies is used to establish multi-level morphological features; among them, the image level set decomposition is established, and the image i:Ω→R is decomposed into an upper level set and a lower level set, which are expressed as: I μ (i)={p∈Ω;i(p)≥μ} and I μ (i) = {p∈Ω; i(p)≤μ}, μ∈R is a grayscale value; the level set is used as the complete expression of the image, and the building morphology is defined as the set of all pixels within the horizontal contour line. Then each node under the morphological feature set represents a morphology, which describes the morphological characteristics of the building in the image; In the process of minimax optimization of conditional adversarial network, under the premise of perception of building morphological features, the multi-level joint expression of multi-temporal images is extracted. According to the self-learning and simulation of the generative model and the confrontation and optimization of the discriminant model, the polymorphic simulation set of buildings is automatically generated, and the self-training and analysis of the network are realized, so as to obtain high-precision probability distribution of changes. First, the parameters of the generative model are kept unchanged, and the discriminant function θ is used for the discriminant model. D Update and classifier function θ T Optimize; secondly, keep θ D and θ T No change, update the generation function θ of the generation model G ; Secondly, the building features learned from multi-temporal high-resolution images are projected into the domain adaptive space for nonlinear feature recognition. In this space, the high-level features of unchanged buildings tend to be consistent, while the features of changed buildings have great differences, thus showing high separability; finally, through the optimization and confrontation of the conditional adversarial network, the polymorphic simulation set of buildings is continuously generated and expanded, X s and X t As the input of the discriminator D, the discriminator estimates the probability distribution of data changes; in addition, the discriminator will output a scalar value. When assuming that the probability of building features in each phase depends only on the characteristics of the building itself, the maximum likelihood estimation formula is used to derive to represent the maximum likelihood probability of the generated model output data and the real data.

6. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 5, characterized in that: Through the consistency training processing method, the similarity loss function is studied, and the domain invariant feature analysis method of the scale invariant mask is designed to eliminate the huge difference between the real image and the simulated image before and after the change, and maintain the real characteristics of the building; at the same time, the similarity loss function based on the real image helps to stabilize the optimization process of the conditional adversarial network, and the collaborative decision rule further improves the change detection accuracy; therefore, the following formula is used to implement the optimization problem: Among them, α, β, γ are the weights that control the interaction of loss functions. is the similarity loss; in the similarity loss function, a scale-invariant error mask is used to eliminate the feature difference between the real image and the simulated image, rather than the absolute difference between the input and the output. When there is an error between the simulated image and the real image, the similarity loss function gives the mask Expressed as: Where k represents the number of pixels in the input image x. The square of the regularized L2 norm is the sum of the squares of the elements of the vector and then the square root is taken; the loss function allows the model to learn and reproduce the overall morphological characteristics of the building without operating on the spectrum or grayscale of the input data; for data sources of different time phases, adversarial training is allowed to be performed in a consistent manner.

7. The method for detecting building changes in remote sensing images based on conditional adversarial networks according to claim 6, characterized in that: After obtaining the semantic features and change probability distribution of buildings, the building change information is optimized according to the decision rules.

Images