Method and device for extracting land use type change information

Abstract

The application provides a land use type change information extraction method and device, comprising: preparing remote sensing image data of a target area at two time phases and completing spatial calibration; designing a parameter-shared twin convolutional neural network CD-Net, the network being of an encoding-correlation layer-decoding structure; sending the remote sensing image data at the two time phases into the encoder of the CD-Net respectively, extracting deep features at the two time phases, building a connection between the two deep features in the correlation layer, inputting the extracted features and the correlation features into the decoder of the CD-Net in sequence according to the time phases, obtaining restoration features of the respective time phases, superimposing the restoration features of the two time phases, and obtaining change features containing land use type change information; and inputting the change features into the classifier of the CD-Net network, and outputting land use type change information of the target area. The application realizes land use type change information extraction based on deep learning, and improves classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image data processing technology, and in particular relates to a method and apparatus for extracting land use type change information. Background Technology

[0002] Land cover refers to the collective term for vegetation and artificial coverings on the Earth's surface. It is a comprehensive reflection of the various elements of the Earth's surface covered by natural vegetation, natural formations, and man-made structures. Land cover is essential information for humankind to understand nature and grasp its laws, and it is also the most basic data required for various resource management and geographic information services. Changes in land cover reflect the changes and development of a region. Therefore, the acquisition, analysis, and updating of land cover change information are extremely important.

[0003] Remote sensing imagery data, with its macroscopic and real-time characteristics, has always been an important tool for land cover change detection. Currently, land cover change detection methods based on remote sensing data are generally divided into two types: one is the traditional method based on visual interpretation of raw remote sensing images; the other utilizes deep learning concepts to detect changes in remote sensing data products, including primary and more advanced products. Summary of the Invention

[0004] To overcome the problem that existing land cover change detection methods cannot meet the accuracy requirements of land cover change detection, or at least partially solve the above problems, embodiments of the present invention provide a method and apparatus for extracting land use type change information.

[0005] According to a first aspect of the present invention, a method for extracting land use type change information is provided, comprising the following steps:

[0006] (1) Prepare remote sensing image data of the target area in two time phases and complete spatial calibration;

[0007] (2) Design a parameter-sharing Siamese convolutional neural network CD-Net, which has an encoder-cross-correlation layer-decoder structure;

[0008] (3) The remote sensing image data of the two time phases are sent to the encoder of CD-Net respectively to extract the depth features of the two time phases. The connection between the two depth features is constructed in the cross-correlation layer. The extracted features and associated features are input into the decoder of CD-Net in sequence according to the time phase to obtain the restored features of each time phase. The restored features of the two time phases are superimposed to obtain the change features containing land use type change information.

[0009] (4) Input the change features into the classifier of the CD-Net network and output the land use type change information of the target area.

[0010] The steps preceding step (3) also include:

[0011] The encoder of the CD-Net network extracts the depth features of two temporal remote sensing images, then constructs the relationship between the two depth features in the cross-correlation layer, and finally inputs the extracted features and associated features into the CD-Net decoder in sequence according to the temporal phase to obtain the restored features of each temporal phase. The restored features of the two temporal phases are superimposed to obtain the change features containing land use type change information.

[0012] Accordingly, step (4) specifically includes:

[0013] The change features are input into the classifier of the CD-Net network, which outputs land use type change information for the target area.

[0014] Furthermore, the following formula expresses how the encoder of the CD-Net network extracts depth features from two temporal remote sensing images, then constructs the relationship between the two depth features in the cross-correlation layer, and finally inputs the extracted features and associated features sequentially into the CD-Net decoder according to time phase to obtain the reconstructed features of each time phase. The reconstructed features of the two time phases are then superimposed to obtain the change features containing land use type change information:

[0015]

[0016] Among them, T1 and T2 are remote sensing image data from two different time periods. This is a feature overlay operation; φ and φ are the encoder and decoder of the CD-Net network, respectively. The encoder extracts the depth features X1 and X2 of the remote sensing image data, and the decoder restores the intermediate features to obtain the restored features Y1 and Y2. ψ is the cross-correlation layer of the CD-Net network, which is used to connect the depth features of two time phases to obtain the associated features X′1 and X′2. The variation feature Y is obtained by superimposing Y1 and Y2.

[0017] Preferably, step (4) is represented by the following formula:

[0018] Res=σ(Y)

[0019] Among them, the classifier σ of the CD-Net network extracts land use type change information Res from the change feature Y.

[0020] In this invention, the remote sensing image data includes high-resolution satellite imagery and high-resolution aerial imagery.

[0021] Before step (4) of inputting remote sensing image data from two time phases into the CD-Net network, the method further includes: training the CD-Net network based on pre-acquired remote sensing image data samples and preset category labels corresponding to each remote sensing image data sample.

[0022] According to a second aspect of the present invention, a land use type change information extraction device is provided, comprising:

[0023] The hidden module is used to extract change features. Remote sensing image data from two different time phases are fed into the CD-Net encoder to extract depth features from the two time phases. A relationship between the two depth features is established in the cross-correlation layer. The extracted features and associated features are then input into the CD-Net decoder sequentially according to time phase to obtain the reconstructed features of each time phase. The reconstructed features of the two time phases are superimposed to obtain change features containing land use type change information.

[0024] A classification module is used to identify change information. The change features are input into the classifier of the CD-Net network, which outputs land use type change information for the target area.

[0025] According to a third aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor invokes program instructions to execute a land use type change information extraction method provided by any of the various possible implementations of the first aspect.

[0026] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is also provided, the non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform a land use type change information extraction method provided by any of the various possible implementations of the first aspect.

[0027] This invention provides a method and apparatus for extracting land use type change information. The method improves existing convolutional neural networks. The improved CD-Net network includes an encoder, a cross-correlation layer, and a decoder. Two temporal remote sensing image data are input into the encoder to extract depth features from both time phases. A connection between the two depth features is established in the cross-correlation layer. The extracted features and associated features are sequentially input into the decoder according to time phase to obtain the reconstructed features for each time phase. The reconstructed features from both time phases are then superimposed to obtain change features containing land use type change information. A Softmax classifier classifies the change features, obtaining land use type change information for the target area. This achieves land use type change information extraction based on deep learning, improving classification accuracy. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the overall process of the land use type change information extraction method provided in an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the cross-correlation layer structure provided in an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the overall structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] One embodiment of the present invention provides a method for extracting land use type change information. Figure 1 This is a schematic diagram of the overall process of the land use type change information extraction method provided in an embodiment of the present invention. The method includes steps S1, S2, S3, and S4:

[0034] S1, depth feature encoding of remote sensing image.

[0035] The target area is the region where land use change information needs to be extracted, and the data consists of two high-resolution remote sensing images from different time periods. The CD-Net is a Siamese network with shared weight parameters. The remote sensing images from the two time periods are input into the CD-Net encoder to obtain depth features. The encoder consists of four convolutional blocks, each including a convolutional layer (Conv), a max-pooling layer (MaxPooling), an attention mechanism layer (Att), and a ReLU nonlinear activation function. The formulas for each convolutional block are as follows:

[0036] 0≤i≤i

[0037]

[0038] T1 and T2 represent remote sensing images from two different time periods. This represents the output of the j-th time phase in the i-th convolutional block (j∈{1,2}). The Conv convolutional kernel size is 3×3, and the MaxP is 2×2. Each convolutional block contains two (ReLU, Conv) combinations. After passing through the encoder's four convolutional blocks, the remote sensing image data is encoded into depth features. and

[0039] S2, dual-temporal feature correlation.

[0040] The cross-correlation layer is used to establish the correlation between two temporal deep features, providing the decoder with comprehensive information on the two temporal deep features at the high-order semantic level, thereby enhancing the CD-Net network's ability to distinguish changing information. Figure 2 This is a schematic diagram of the cross-correlation layer of the CD-Net network provided in an embodiment of the present invention. The CD-Net network cross-correlation layer includes two cross-correlation blocks with identical structures, each cross-correlation block focusing on the depth features of a single temporal phase. The flow of the cross-correlation block is shown in the following formula:

[0041]

[0042] in, For overlay operation, and These are the corresponding correlation features at two different times.

[0043] S3, Feature Decoding and Land Use Type Change Information Extraction.

[0044] The CD-Net network encoder is responsible for restoring depth and correlation features to the original remote sensing image size to recover detailed information. The encoder consists of four deconvolutional blocks, each including a deconvolutional layer (DeConv), an attention mechanism layer (Att), and a ReLU nonlinear activation function. The formulas for each convolutional block are as follows:

[0045] 0≤i≤4

[0046]

[0047] Res = Softmax(Y)

[0048] In this process, the initial input feature Y0 of the decoder is superimposed with the deep feature X4 and the associated feature X. t The decoder superimposes the input features Y into each deconvolution block. i and the output features X of the corresponding encoder convolutional block 4-i Finally, the restored features from the two time phases are superimposed. and The restored features Y are obtained, and the classifier Softmax combines the restored features Y to perform classification, thereby obtaining the land use type change information Res.

[0049] S4, train the CD-Net network.

[0050] Before extracting land use change information, the CD-Net network needs to be trained to enable it to distinguish change information. The CD-Net network is trained using pre-acquired remote sensing image data samples and their corresponding preset category labels. The overall loss is calculated based on the cross-entropy loss function, and the deep semantic segmentation network is optimized using the backpropagation algorithm and stochastic gradient descent method.

[0051] L=-(y0log(Res0)+y1log(Res1))

[0052] y is an indicator variable (0 or 1). It is 1 if the category is the same as the sample category, and 0 otherwise. Res0 represents a prediction of no change, and Res1 represents a prediction of a change.

[0053]

[0054] Where θ represents the parameters of CD-Net. The parameters θ in the current iteration... i+1 The parameter θ from the previous iteration i The learning rate α and J(θ) represent the total loss.

[0055] This embodiment provides an electronic device. Figure 3 This is a schematic diagram of the overall structure of an electronic device provided in an embodiment of the present invention. The device includes: at least one processor 301, at least one memory 302, and a bus 303; wherein,

[0056] The processor 301 and the memory 302 communicate with each other via the bus 303;

[0057] The memory 302 stores program instructions that can be executed by the processor 301. The processor can execute the methods provided in the various method embodiments by calling the program instructions.

[0058] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions. These instructions instruct a computer to execute the methods provided in the various method embodiments, including, for example: preparing remote sensing image data of two time phases of a target area and performing spatial calibration; designing a parameter-shared Siamese convolutional neural network (CD-Net), which has an encoder-cross-correlation layer-decoder structure; feeding the remote sensing image data of the two time phases into the encoder of the CD-Net to extract depth features of the two time phases, constructing a connection between the two depth features in the cross-correlation layer, sequentially inputting the extracted features and associated features into the decoder of the CD-Net according to time phase to obtain the restored features of each time phase, superimposing the restored features of the two time phases to obtain change features containing land use type change information; inputting the change features into the classifier of the CD-Net network to output the land use type change information of the target area.

[0059] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0060] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0061] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting land use type change information, characterized in that, Includes the following steps: (1) Prepare remote sensing image data of the target area in two time phases and complete spatial calibration; (2) Design a parameter-sharing Siamese convolutional neural network CD-Net, which has an encoder-cross-correlation layer-decoder structure; (3) The remote sensing image data of the two time phases are sent to the encoder of CD-Net respectively to extract the depth features of the two time phases. The connection between the two depth features is constructed in the cross-correlation layer. The extracted features and associated features are input into the decoder of CD-Net in sequence according to the time phase to obtain the restored features of each time phase. The restored features of the two time phases are superimposed to obtain the change features containing land use type change information. (4) Input the change features into the classifier of the CD-Net network and output the land use type change information of the target area.

2. The method for extracting land use type change information according to claim 1, characterized in that, The following formula expresses how the encoder of the CD-Net network extracts depth features from two temporal remote sensing images, then establishes the relationship between the two depth features in the cross-correlation layer, and finally inputs the extracted features and associated features sequentially into the CD-Net decoder according to time phase to obtain the reconstructed features of each time phase. The reconstructed features of the two time phases are then superimposed to obtain the change features containing land use type change information: Among them, T1 and T2 are remote sensing image data from two different time periods. This is a feature overlay operation; φ and φ are the encoder and decoder of the CD-Net network, respectively. The encoder extracts the depth features X1 and X2 of the remote sensing image data, and the decoder restores the intermediate features to obtain the restored features Y1 and Y2. Ψ is the cross-correlation layer of the CD-Net network, which is used to connect the depth features of two time phases to obtain the associated features X′1 and X′2. The variation feature Y is obtained by superimposing Y1 and Y2.

3. The method for extracting land use type change information according to claim 1, characterized in that, Step (4) is represented by the following formula: Res=σ(Y) Among them, the classifier σ of the CD-Net network extracts land use type change information Res from the change feature Y.

4. The method for extracting land use type change information according to claim 1, characterized in that, The remote sensing image data includes high-resolution satellite imagery and high-resolution aerial imagery.

5. The method for extracting land use type change information according to any one of claims 1-4, characterized in that, Before step (4), which involves inputting remote sensing image data from two different time phases into the CD-Net network, the following steps are also included: The CD-Net network is trained based on pre-acquired remote sensing image data samples and preset category labels corresponding to each remote sensing image data sample.

6. A land use type change information extraction device, characterized in that, include: The hidden module is used to extract change features. Remote sensing image data from two time phases are fed into the CD-NET encoder to extract the depth features of the two time phases. The relationship between the two depth features is constructed in the cross-correlation layer. The extracted features and associated features are input into the CD-Net decoder in sequence according to the time phase to obtain the restored features of each time phase. The restored features of the two time phases are superimposed to obtain the change features containing land use type change information. The classification module is used to identify change information; the change features are input into the softmax classifier of the CD-Net network, and the land use type change information of the target area is output.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the land use type change information extraction method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the land use type change information extraction method as described in any one of claims 1 to 5.

Images