A method and system for detecting and classifying geospatial information changes based on deep learning

Abstract

The present invention discloses a method for detecting and classifying geographic spatial information changes based on deep learning, comprising the following steps: generating a multi-level and multi-dimensional network model of a compilation-interpretation structure based on a deep learning semantic segmentation and change detection method; constructing a method for data cleaning preprocessing and segmentation clustering to generate a sample data set; constructing a migration learning framework based on transfer learning technology, applying a new data set to the basic model to automatically and quickly update the correction model through training, and generating a fusion model for application in new geographic spatial information change detection and classification fields. The present invention also discloses a system for detecting and classifying geographic spatial information changes based on deep learning. The present invention uses transfer learning technology to retrain the basic model based on the basic model and sample data, and obtains a dynamic growth model for detecting and classifying geographic spatial information changes that is suitable for different fields and scenarios, thereby improving accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of satellite remote sensing image technology, and in particular to a method and system for detecting and classifying geographic space information changes based on deep learning. Background Art

[0002] Geographic information data surveys based on remote sensing image data have been widely used in various fields. However, due to the diversity, complexity, and volatility of geographic spatial information, data information changes are difficult to extract accurately, and there are many influencing factors. Regarding the classification of changes in remote sensing images, traditional remote sensing technologies mostly analyze the spectral and texture information of remote sensing images, extract each classification feature, and use supervised classification methods for classification. However, due to the neglect of geographic spatial correlation and complex internal composition structure, there are many problems and poor accuracy; or rely on manual detection and classification, which relies on human resources and has low production efficiency. With the continuous increase of remote sensing image data, it is obvious that traditional remote sensing technology is no longer applicable. Therefore, relying on the continuous improvement of computing power today, it is of great significance to process remote sensing big data based on deep learning semantic segmentation architecture models and extract change features for accurate classification. Summary of the invention

[0003] In view of the shortcomings of the existing technology, the purpose of this invention is to propose a method for detecting and classifying geographic spatial information changes based on deep learning, so as to solve the problem that the chaotic and fragmented multi-source remote sensing image data and the relationship between geographic spatial changes are difficult to detect and classify systematically. Based on deep learning technology, a basic dynamic model is established based on deep learning technology, which has learning ability, and based on transfer learning, the model is retrained to expand the model's cognition, thereby performing systematic and high-precision change detection and classification on different remote sensing images.

[0004] The present invention discloses a method for detecting and classifying geographic spatial information changes based on deep learning, comprising the following steps:

[0005] Generate a network model of a compilation-interpretation structure based on a deep learning semantic segmentation and change detection method, and train and test the network model of the compilation-interpretation structure to obtain a basic model;

[0006] Based on the transfer learning technology and the basic model, a transfer learning framework is constructed; a sample data set is used to perform transfer learning on the transfer learning framework to construct a new model for transfer learning;

[0007] The new model of transfer learning is integrated with the basic model to form a fusion model;

[0008] The remote sensing image to be detected is detected and classified according to the fusion model.

[0009] Preferably, the method for acquiring the sample data set is: cleaning the historical remote sensing image data and producing a sample data set for geographic spatial information change detection and classification.

[0010] Preferably, the data is cleaned and a sample data set for geospatial information change detection and classification is prepared, and the specific steps are as follows:

[0011] Acquire historical remote sensing image data, pre-process the historical remote sensing image data, and obtain pre-processed image data;

[0012] The preprocessed image data is segmented to extract texture feature vectors; the texture feature vectors are clustered using a Gaussian mixture clustering method to obtain a clustered image with a feature dictionary;

[0013] The clustered images are sliced ​​to obtain a sample data set.

[0014] Preferably, a network model of a compilation-interpretation structure is generated based on a deep learning semantic segmentation and change detection method, and the network model of the compilation-interpretation structure is trained and tested to obtain a basic model. The specific steps are:

[0015] Constructing a network model of a compilation-interpretation structure of a deep learning semantic segmentation and change detection method, wherein the network model of the compilation-interpretation structure of the deep learning semantic segmentation and change detection method includes a basic extraction layer of geospatial information features, a basic mapping layer of geospatial information features, and a basic output layer of geospatial information features that are distributed in a cross-layer combination;

[0016] The network model of the compile-interpret structure is trained and tested to obtain a basic model.

[0017] Preferably, a network model of a compilation-interpretation structure of a deep learning semantic segmentation and change detection method is constructed, including a basic extraction layer of geospatial information features, a basic mapping layer of geospatial information features, and a basic output layer of geospatial information features distributed across layers, including:

[0018] Constructing a geospatial information feature extraction layer to generate a geospatial information feature relationship matrix;

[0019] Constructing geospatial information feature mapping layer for learning clustering feature vectors;

[0020] Construct a geospatial information feature output layer to output geospatial information changes and detection and classification results;

[0021] A network model is defined based on the geospatial information feature extraction layer, the geospatial information feature mapping layer and the geospatial information feature output layer.

[0022] Preferably, the basic model is trained and tested, and the specific steps are as follows:

[0023] Obtain a public image dataset, which is divided into training data, verification data, and test data;

[0024] A loss function is introduced into the network model of the compilation-interpretation structure to evaluate the degree of modeling of the data set and to quantitatively evaluate the deviation between the model prediction value and the actual target value;

[0025] Use the training data to train the model, then use the test data set to test the model, and then test the model with the test data to obtain the basic model.

[0026] Preferably, based on the transfer learning technology and the basic model, a transfer learning framework is constructed; and a sample data set is used to perform transfer learning on the transfer learning framework to construct a new model for transfer learning, and the specific steps are as follows:

[0027] Go to the base model and freeze all layers, use the average of all output feature maps of the base model as input weights, and add a new pooling layer on top of the frozen layers;

[0028] Add the Softmax function to the new pooling layer;

[0029] Define the modified optimization method on the new pooling layer and compile the model;

[0030] Train the model on the new pooling layer using the training and test samples from the sample dataset;

[0031] Adjust the modified optimization method and use the test samples in the sample data set to test the model to obtain a new model for transfer learning.

[0032] Preferably, the historical remote sensing image data is preprocessed to obtain preprocessed image data, and the specific steps include:

[0033] Clean and preprocess the signal intensity or reflectivity of remote sensing image data to make it close to the true value;

[0034] The formula for calculating the minimized mean square error Error is as follows:

[0035]

[0036] In the formula, a and b represent coefficients, s represents the actual received signal strength or reflectivity, represents the average value of s, and μ represents the real signal strength or reflectivity;

[0037]

[0038] In the formula, represents the variance, s represents the actual received signal strength or reflectivity, Indicates the real signal strength or reflectivity, m indicates the parameter, μ, Different symbols represent the same meaning.

[0039] Preferably, a geospatial information feature extraction layer is constructed to generate a geospatial information feature relationship matrix, and the specific steps are as follows:

[0040] Construct a global geospatial information feature extraction unit, apply convolution calculation and activation function to form a feature extraction unit, and generate a geospatial information feature relationship matrix of the current unit;

[0041] Based on the discretization process of samples, calculation and activation functions are applied to extract the maximum value of the input area overlapping with the kernel, forming a local secondary feature extraction unit, and generating the local geographic spatial information feature relationship matrix of the current unit;

[0042] Inject noise into the local secondary feature extraction unit through regularization technology to increase the difficulty of data in training and improve robustness;

[0043] The feature extraction unit and the local secondary feature extraction unit constitute a basic feature extraction layer; the input of the geospatial information feature extraction layer is the output of the previous geospatial information feature extraction layer, and the input of the first geospatial information feature extraction layer is a sample; the geospatial information feature extraction layer is multi-layer and distributed in cross-layer combination, the input of the feature extraction layer is the output of the previous geospatial information feature extraction layer, the output of the feature extraction layer is the input of the next geospatial information feature extraction layer, the input of the first geospatial information extraction layer is a sample, and the output of the last geospatial information feature extraction layer is the input of the first geospatial information feature mapping layer;

[0044] Construct a geospatial information feature mapping layer to learn cluster feature vectors; the specific steps are:

[0045] The mapping layer learns the nonlinear combination of high-level features output by the basic feature extraction unit, maps the distributed feature representation of the extracted features to the sample label multi-dimensional space, and generates a multi-dimensional feature classification;

[0046] Applying activation functions introduces nonlinearity;

[0047] Regularization techniques are applied to reduce overfitting of neural networks. The regularization techniques use discrete probability distributions, and the discrete probability distributions are selected as Bernoulli distributions. Samples from the Bernoulli distribution are used to randomly reset some connections of the input tensor to zero with probability p.

[0048] The geospatial information feature mapping layer is multi-layer and distributed in combination across layers, the input of the feature mapping layer is the output of the previous geospatial information feature mapping layer, the output of the feature mapping layer is the input of the next geospatial information feature mapping layer, the input of the first geospatial information mapping layer is the output of the last geospatial information feature extraction layer, and the output of the last geospatial information feature mapping layer is the input of the geospatial information feature output layer;

[0049] Construct a geospatial information feature output layer to output geospatial information changes and detection and classification results; the specific steps are:

[0050] The output layer applies the normalized exponential function and finally outputs the geospatial information changes and detection classification results;

[0051] The geospatial information output layer is a layer, the input of the output layer is the output of the last geospatial information mapping layer, and the output of the output layer is the geospatial information change and detection classification result.

[0052] The present invention also discloses a system for implementing the above-mentioned deep learning-based geospatial information change detection and classification method, comprising:

[0053] Sample data set module: used to clean data, extract features, build feature dictionaries, and create sample data sets;

[0054] Detection, recognition and classification model module: used to build a network model of the compilation-interpretation structure based on deep learning semantic segmentation and change detection methods, and train, test and study the generation of basic models;

[0055] Transfer learning module: used to transfer learning framework based on transfer learning technology and basic model architecture, and to update and modify the basic model to generate a new model according to the needs of different fields;

[0056] Fusion model module: the new model used for the transfer learning is integrated with the basic model to form a fusion model;

[0057] Detection classification and identification classification module: used to input the detection remote sensing image into the fusion model, and then detect and classify the remote sensing image to be detected according to the fusion model.

[0058] Compared with the prior art, the beneficial effects of the present invention are:

[0059] The present invention generates a network basic model of compilation-interpretation structure based on deep learning semantic segmentation and change detection methods. The model adopts multi-layer cross-layer combined distribution and multi-dimensional feature classification, so that geographic spatial information features are transmitted in multiple dimensions in the network. By abstracting feature representation, calculations are reduced, computing power is guaranteed, and overfitting is avoided. The basic model completed after training, verification, and testing by injecting noise through regularization technology has better robustness and higher precision detection and recognition.

[0060] The present invention is based on transfer learning technology. According to different images and different scene categories, the model is automatically and quickly updated and corrected based on the basic model. New data sets are applied to the top layer of the basic model for modification and partial retraining. New typical geographic spatial information feature relationships are deeply learned. The model is updated and reapplied to new change detection classification fields such as basic geographic information data and remote sensing image change detection.

[0061] The fusion model of the present invention is applicable to the detection needs of geographic spatial changes in different fields, such as surface cover classification, land landscape distribution, forest monitoring, fire point detection, agricultural crop distribution, cultivated land distribution, land use planning and other precise monitoring and identification. It generates intelligent detection models suitable for different scenarios based on transfer learning. Compared with traditional detection methods, the method of the present invention can automatically learn deep geographic spatial relationship characteristics from sample data sets based on the model, adapt to different remote sensing image detection and identification classification, save manpower and time, and greatly improve efficiency; compared with other deep learning methods, the method of the present invention can update the basic model based on transfer learning technology, and automatically transfer learning according to scene categories for model training and interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 It is a flow chart of a method for detecting and classifying geographic spatial information changes based on deep learning according to the present invention;

[0064] Figure 2 It is a schematic diagram of the calculation method for extracting cluster feature vectors of the present invention;

[0065] Figure 3 It is a schematic diagram of a network model method for constructing a compilation-interpretation structure based on a deep learning semantic segmentation and change detection method of the present invention;

[0066] Figure 4It is a schematic diagram of the calculation method of compiling and extracting the characteristic structure network model of geographic space information in the present invention;

[0067] Figure 5 This is a schematic diagram of the architecture transfer learning framework of the present invention;

[0068] Figure 6 This is an example diagram of historical remote sensing image data;

[0069] Figure 7 This is an example diagram of the results obtained after the existing technology detection and classification;

[0070] Figure 8 It is an example diagram of the geographic space information change and detection classification results of the present invention. DETAILED DESCRIPTION

[0071] The specific implementation manner and working principle of the present invention are further described in detail below with reference to the accompanying drawings.

[0072] like Figure 1 As shown, the present invention proposes a classification method for detecting changes in geographic spatial information based on deep learning, and the specific steps are as follows:

[0073] Step 1: Data cleaning and preprocessing, feature extraction, feature dictionary construction, and production of geographic spatial information change detection classification sample data set. The specific steps are as follows:

[0074] Step 1.1, obtain historical remote sensing image data, perform cleaning preprocessing, i.e., filter noise, and improve the quality of feature extraction; obtain preprocessed image data;

[0075] The formula for calculating the minimized mean square error Error is as follows:

[0076]

[0077] In the formula, a and b represent coefficients, s represents the actual received signal strength or reflectivity, represents the average value of s, μ represents the real signal strength or reflectivity, μ, Different symbols represent the same meaning.

[0078] Calculate true signal strength or reflectivity The formula is as follows:

[0079]

[0080] In the formula, represents the variance, s represents the actual received signal strength or reflectivity, represents the real signal strength or reflectivity, and m means the parameter is set to 4.7.

[0081] Step 1.2, pre-process the image data into several (n*n) unit blocks, use the local binary pattern (LBP) to extract the texture features of the blocks, count the LBP histogram of the blocks, normalize the histogram by normalization, and generate the texture feature vector;

[0082]

[0083]

[0084] Where P is the number of neighborhood pixels, n i represents the intensity of the ith neighborhood pixel, c represents the center pixel, G c represents the intensity of the central pixel, and x represents the difference between the grayscale value of the surrounding pixels of the central pixel and the grayscale value of the central pixel.

[0085] Step 1.3, using the Gaussian mixture clustering method based on Bayesian theorem and expectation maximization algorithm to classify the texture feature vectors, construct a feature dictionary, and obtain a clustered image with a feature dictionary;

[0086] Calculation example: Figure 2 As shown, the steps are as follows:

[0087] Step A1, based on Bayes’ theorem, calculate z j The posterior distribution probability p is:

[0088]

[0089] Where: μ i represents the average value of vector i, ∑ i The covariance of vector i, K represents the number of Gaussian distributions, z represents the hypothesis, x represents the data sample, α represents the coefficient, j represents the indicator variable, l represents the indicator variable, α i represents the coefficient corresponding to i, α l represents the coefficient corresponding to l, p(x j |μ i ,∑ i ) represents the maximum likelihood probability.

[0090] Step A2, iterate based on the expectation maximization algorithm:

[0091] Calculate the mean vector μ i :

[0092]

[0093] Where: m represents the sample size, and j represents the indicator variable.

[0094] Calculate the covariance matrix∑ i :

[0095]

[0096] Where: T represents the maximum number of iterations.

[0097] Calculate the mixing coefficient Φ i :

[0098]

[0099] Step A3, repeating steps A1 and A2 until the termination condition is met; the termination condition is when the likelihood function value (ie, the maximum likelihood probability) no longer increases.

[0100] Step A4, based on the mean vector μ i , covariance matrix∑ i , mixing coefficient Φ i Define Gaussian mixture clustering:

[0101]

[0102] Based on the feature vector, the samples are clustered into K clusters that conform to the Gaussian probability density function. The class with the largest mixing coefficient in each sample is used as the label. The label is the feature dictionary, and the clustered image with the feature dictionary is obtained.

[0103] The feature dictionary may adopt residential buildings, farmland, industrial buildings, forests, etc.

[0104] Step 1.4, slice the clustered images to generate standard size samples, which are used as annotations to generate sample data sets.

[0105] The slicing method is: based on the center point pixels of the clustered images, for each image, the image slices are cut at its center point with a size of 64*64 pixels.

[0106] The sample data set is randomly split into training samples, verification samples, and test samples in the ratio of (0.75∶0.15∶0.1);

[0107] Step 2: Build a network model of the compilation-interpretation structure based on deep learning semantic segmentation and change detection methods. The network model of the compilation-interpretation structure is trained, tested, and studied to form a basic model. The calculation example is as follows Figure 3 As shown, the specific steps are as follows:

[0108] Step 2.1, compile the structural network model, extract the geospatial information features, and calculate the example as follows Figure 4 As shown, the specific steps are:

[0109] Step 2.1.1, constructing a geospatial information feature extraction layer, which is used to generate a geospatial information feature relationship matrix, which is the compilation structure part;

[0110] Apply convolution (Conv) calculation and regularized linear unit ReLu activation function to form a feature extraction unit and generate the global geospatial information feature relationship matrix of the current unit;

[0111] The convolution is calculated as follows:

[0112] H=I*K

[0113] In the formula, H means that the output matrix (i.e., feature) has three dimensions, I means that the input matrix has four dimensions, and K means that the weight matrix has four dimensions.

[0114] The ReLu activation function formula is as follows:

[0115]

[0116] Calculate the unit feature matrix F i :

[0117]

[0118] In the formula, n represents the n-th layer network, i represents the i-th matrix unit of the n-th layer, Ker represents the weight matrix parameter, Bias represents the bias parameter of the convolution calculation, Represents the output matrix of the i-th matrix unit of the n-1 layer network.

[0119] Based on the discretization process of samples, convolution calculation and activation function are applied to extract the maximum value of the input area overlapping with the kernel, forming a local secondary feature extraction unit, and generating the local geographic spatial information feature relationship matrix of the current unit; this can reduce the size of the convolution feature space, reduce overfitting by abstractly representing the convolution feature, further reduce the dimension of the extracted information to reduce the amount of calculation, strengthen the invariance of image features, increase the robustness of image offset, rotation, etc., and reduce the offset of the estimated mean caused by parameter errors;

[0120] The Sigmoid activation function formula is as follows:

[0121] f(x) = max(0, x)

[0122] Calculate the unit feature matrix F p :

[0123]

[0124] Inject noise into the local secondary feature extraction unit through regularization technology to increase the difficulty of data in training and improve robustness. Regularization technology uses discrete probability distribution, and the discrete probability distribution selects Bernoulli distribution. The noise is applied to the weight. The noise injected through the weight is a random value obtained by sampling the random variable. The weight is an uncertain random variable that conforms to the probability p distribution;

[0125] The Bernoulli distribution formula is as follows:

[0126]

[0127]

[0128]

[0129] Where p represents probability; q = 1-p; f X (x) represents the probability mass function, E[X] represents the expected value, and var[X] represents the variance.

[0130] The feature extraction unit and the local secondary feature extraction unit are combined across multiple layers to form a geographic spatial information feature extraction layer;

[0131] The input of the geospatial information feature extraction layer is the output of the previous geospatial information feature extraction layer, and the input of the first geospatial information feature extraction layer is the sample.

[0132] The geospatial information feature extraction layer is multi-layer and distributed across layers. The input of the feature extraction layer is the output of the previous geospatial information feature extraction layer, and the output of the feature extraction layer is the input of the next geospatial information feature extraction layer. The input of the first geospatial information extraction layer is the sample, and the output of the last geospatial information feature extraction layer is the input of the first geospatial information feature mapping layer.

[0133] Step 2.1.2, constructing a geospatial information feature mapping layer for learning clustering feature vectors; this is the compilation structure part;

[0134] The Sigmoid activation function and the ReLu activation function are applied again; nonlinear factors are added when clustering feature vectors in the feature mapping layer to solve the defect of insufficient expression ability of the linear model;

[0135] Regularization techniques are applied to reduce overfitting of neural networks. Some connections of the input tensor are randomly reset to zero with probability p based on samples from the Bernoulli distribution, and some network units are discarded. The output of the integrated network is given by the average of the outputs of all sub-networks.

[0136] Weighted average of all subnetwork outputs:

[0137]

[0138] The geometric mean of all subnetwork outputs:

[0139]

[0140] Geometric mean normalization ensures that multiple probability distributions are one probability distribution. The probability normalization formula is:

[0141]

[0142] In the formula, represents the mask vector, represents the probability distribution of the output of each network, express The probability distribution of , d represents the number of units that can be discarded, y∈{1, 2, 3, …, N} represents the classification category, Represents the input vector.

[0143] Step 2.2, construct the geospatial information feature output layer, which is used to output geospatial information changes and detection and classification results; for the interpretation structure part:

[0144] Add a feature output layer after the feature output layer and apply the normalized exponential (Softmax) function to output the geospatial information changes and detection classification results;

[0145] The Softmax function formula is as follows:

[0146]

[0147] In the formula, σ represents the Softmax function; z represents the input vector; represents the standard exponential function of the input vector; K represents the number of classes in the multi-class classifier; represents the standard exponential function of the output vector; j represents the indicator variable.

[0148] Step 2.3, based on the feature extraction layer, feature mapping layer and feature output layer of step 2.2, define a network model as a network model of the compilation-decoding structure;

[0149] The geospatial information output layer is one layer, the input of the output layer is the output of the final geospatial information mapping layer, and the output of the output layer is the geospatial information change and detection classification results.

[0150] Step 2.4, compile and interpret the network model of the structure for training and testing. After completion, the basic model is obtained. The steps are as follows:

[0151] Step B1, download the public image dataset S1&2 Image Pairs Dataset, and randomly split the public image dataset into training dataset, verification dataset, and test dataset in the ratio of (0.8:0.15:0.5);

[0152] Step B2, using the training data set to train the network model of the compilation-interpretation structure, setting the cross entropy function to perform gradient descent in the deep learning back propagation, correcting the direction and size of each parameter of the update function, so that each parameter after the update makes the loss function value constantly approach the global minimum;

[0153] The formula is as follows:

[0154]

[0155] In the formula, p represents the probability, M represents the number of clusters, and t i Indicates the true value of sample i, which is either 0 or 1, p i represents the normalized predicted probability of sample i.

[0156] The following formula is derived:

[0157]

[0158] Where: loss represents the cross entropy loss function, x represents the sample, class represents the cluster, j represents a cluster, and exp represents the expected value that the sample x belongs to cluster j.

[0159] In this step, the test data set is used to continuously test the model error during the model training process. The error will first decrease and then increase as the number of training times increases. The training is completed and the output model is output when the error is minimized.

[0160] Step B3, applying the test data set to test the model. After each training data set is used to train the model, the output model is tested using the test data after the training is completed to complete the basic model;

[0161] In this step, after the optimal model is trained using the test data set and the verification data set, the test data set is used to perform model prediction to measure the performance and prediction accuracy of the model.

[0162] In step 3, based on the transfer learning technology, the basic model architecture obtained in step 2 is used to transfer the learning framework, such as Figure 5 As shown in the figure, the construction of the transfer learning framework includes the following steps:

[0163] Step 3.1, freeze the layers of the base model, use the weighted average of all output feature maps of the base model as input weights, and add a new pooling layer on top of the frozen layer;

[0164] Step 3.2, add an output layer that applies the Softmax function to the new pooling layer and contains the corresponding classification categories;

[0165] Step 3.3, define Stochastic Gradient Descent as the correction optimization method on the new pooling layer, set the learning rate to 0.01, the momentum parameter to 0.6, the decay value to 0.01, and compile;

[0166] Step 3.4, based on the training samples and test samples in the sample data set, train the model on the new pooling layer to learn change detection and prediction of the sample data set;

[0167] Step 3.5, fine-tune other weights in the Stochastic Gradient Descent optimization method, adjust the learning rate to 0.005, the momentum parameter to 0.6, the decay value to 0.001, and set the cross entropy;

[0168] Step 3.6, testing the new model, including the frozen part or the entire new model, based on the test samples in the sample data set;

[0169] In step 4, the sample data set in step 1 is applied to the transfer learning framework in step 3 for transfer learning, and the model fusion is used to detect and classify changes in geographic spatial information to generate corresponding products;

[0170] Step 4.1, based on the sample data set and the transfer learning framework obtained in step 3, transfer learning generates a new model for transfer learning;

[0171] Step 4.2, the new model of transfer learning is integrated with the basic model obtained in step 2 to form a fusion model;

[0172] Output geospatial information changes and detection and classification results;

[0173] Step 4.3, inputting the remote sensing image to be detected into the fusion model to generate geographic spatial information changes and detection classification results.

[0174] Corresponding to the above method, the present invention also proposes a geographic spatial information change detection and classification system based on deep learning, the system comprising:

[0175] Sample data set module: extract features from historical remote sensing image data, build feature dictionaries, and create sample data sets;

[0176] Detection, recognition and classification model module: build a network model based on the compilation-interpretation structure of deep learning semantic segmentation and change detection methods, train, test and study the generation of basic models;

[0177] Transfer learning module: Based on transfer learning technology and basic model architecture transfer learning framework, a new model is generated based on the basic model updated and modified according to the needs of different fields;

[0178] Fusion model module: Fusion of new model with basic model based on deep learning network;

[0179] Detection classification and identification classification module: input the image to be detected into the fusion model to generate geographic spatial information changes and detection classification results.

[0180] The technical solution provided by the present invention is described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0181] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0182] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0183] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0185] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0186] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for detecting and classifying geographic spatial information changes based on deep learning, characterized in that: The following steps are involved: Generate a network model of a compilation-interpretation structure based on a deep learning semantic segmentation and change detection method, and train and test the network model of the compilation-interpretation structure to obtain a basic model; the network model of the compilation-interpretation structure includes a basic extraction layer of geospatial information features, a basic mapping layer of geospatial information features, and a basic output layer of geospatial information features that are distributed in a cross-layer combination; The construction of the basic extraction layer of geographic spatial information features is used to generate a geographic spatial information feature relationship matrix, and the specific steps of the construction are: constructing a global geographic spatial information feature extraction unit, applying convolution calculation and activation function, forming a feature extraction unit, and generating a geographic spatial information feature relationship matrix of the current unit; Based on the discretization process of samples, the maximum value of the input area overlapping with the kernel is extracted by applying calculation and activation functions to form a local secondary feature extraction unit, and the local geographic spatial information feature relationship matrix of the current unit is generated; noise is injected into the local secondary feature extraction unit through regularization technology; the feature extraction unit and the local secondary feature extraction unit form a basic extraction layer of geographic spatial information features; The basic mapping layer of geographic spatial information features is constructed to learn cluster feature vectors, and the specific steps of the construction are as follows: The mapping layer learns the nonlinear combination of high-level features output by the basic feature extraction unit, maps the distributed feature representation of the extracted features to the sample label multi-dimensional space, and generates a multi-dimensional feature classification; Applying activation functions introduces nonlinearity; Regularization technology is applied to reduce overfitting of neural networks. Regularization technology uses discrete probability distribution, and the discrete probability distribution selects Bernoulli distribution; samples of Bernoulli distribution are used with probability Randomly zero out some of the connections of the input tensor; The basic output layer of geographic spatial information features is used to output geographic spatial information changes and detection and classification results. The specific steps of its construction are: The output layer applies the normalized exponential function and finally outputs the geospatial information changes and detection classification results; The geospatial information output layer is a layer, the input of the output layer is the output of the last layer of geospatial information mapping layer, and the output of the output layer is the geospatial information change and detection classification results; The basic extraction layer of geographic spatial information features is multi-layer and distributed in cross-layer combination; The basic mapping layer of geographic spatial information features is multi-layer and distributed in combination across layers; Based on the transfer learning technology and the basic model, a transfer learning framework is constructed; Use the sample data set to perform transfer learning on the transfer learning framework to build a new model for transfer learning; The new model of transfer learning is integrated with the basic model to form a fusion model; The remote sensing image to be detected is detected and classified according to the fusion model.

2. The method for detecting and classifying geographic spatial information changes based on deep learning according to claim 1, characterized in that: The method for obtaining the sample data set is as follows: cleaning the historical remote sensing image data and making a sample data set for geospatial information change detection and classification.

3. According to claim 2, a method for detecting and classifying geographic spatial information changes based on deep learning is characterized in that: Clean historical remote sensing image data and create a sample dataset for geospatial information change detection and classification. The specific steps are: Acquire historical remote sensing image data, pre-process the historical remote sensing image data, and obtain pre-processed image data; The preprocessed image data is segmented to extract texture feature vectors; the texture feature vectors are clustered using a Gaussian mixture clustering method to obtain a clustered image with a feature dictionary; The clustered images are sliced ​​to obtain a sample data set.

4. According to claim 3, a method for detecting and classifying geographic spatial information changes based on deep learning is characterized in that: Training and testing complete the basic model. The specific steps are as follows: Obtain a public image dataset, which is divided into training data, verification data, and test data; A loss function is introduced into the network model of the compilation-interpretation structure to evaluate the degree of modeling of the data set and to quantitatively evaluate the deviation between the model prediction value and the actual target value; Use the training data to train the model, then use the test data set to test the model, and then test the model with the test data to obtain the basic model.

5. According to the method for detecting and classifying geographic spatial information changes based on deep learning in claim 3, it is characterized in that: Based on the transfer learning technology and the basic model, a transfer learning framework is constructed; a sample data set is used to perform transfer learning on the transfer learning framework to construct a new model for transfer learning. The specific steps are as follows: Go to the base model and freeze all layers, use the average of all output feature maps of the base model as input weights, and add a new pooling layer on top of the frozen layers; Add the new pooling layer function; Define the modified optimization method on the new pooling layer and compile the model; Train the model on the new pooling layer using the training and test samples from the sample dataset; Adjust the modified optimization method and use the test samples in the sample data set to test the model to obtain a new model for transfer learning.

6. The method for detecting and classifying geographic spatial information changes based on deep learning according to claim 3 is characterized in that: Preprocess the historical remote sensing image data to obtain preprocessed image data. The specific steps include: Clean and preprocess the signal intensity or reflectivity of remote sensing image data to make it close to the true value; Calculate the minimized mean square error The formula is as follows: In the formula, represents the coefficient, Indicates the actual received signal strength or reflectivity. express The average value of Indicates true signal strength or reflectivity; In the formula, represents the variance, Indicates the actual received signal strength or reflectivity. Indicates the real signal strength or reflectivity, Indicates the parameters, , Different symbols represent the same meaning.

7. The method for detecting and classifying geographic spatial information changes based on deep learning according to claim 3, characterized in that: The basic extraction layer of geospatial information features is multi-layer and distributed in combination across layers. The input of the basic extraction layer of geospatial information features is the output of the basic extraction layer of geospatial information features in the previous layer. The input of the first basic extraction layer of geospatial information features is the sample. The output of the last basic extraction layer of geospatial information features is the input of the first basic mapping layer of geospatial information features. The geospatial information feature basic mapping layer is multi-layer and distributed in cross-layer combination. The input of the feature mapping layer is the output of the geospatial information feature basic mapping layer of the previous layer, and the output of the feature mapping layer is the input of the geospatial information feature basic mapping layer of the next layer. The input of the first layer of geospatial information mapping layer is the output of the last layer of geospatial information feature basic extraction layer, and the output of the last layer of geospatial information feature basic mapping layer is the input of the geospatial information feature basic output layer.

8. A system for implementing a deep learning-based geospatial information change detection and classification method as claimed in any one of claims 1 to 7, characterized in that: include: Sample data set module: used to clean data, extract features, build feature dictionaries, and create sample data sets; Detection, recognition and classification model module: used to build a network model of the compilation-interpretation structure based on deep learning semantic segmentation and change detection methods, and train, test and study the generation of basic models; Transfer learning module: used to transfer learning framework based on transfer learning technology and basic model architecture, and to update and modify the basic model to generate a new model according to the needs of different fields; Fusion model module: the new model used for the transfer learning is integrated with the basic model to form a fusion model; Detection classification and identification classification module: used to input the detection remote sensing image into the fusion model, and then detect and classify the remote sensing image to be detected according to the fusion model.

Images