Land utilization type change information extraction method and device

Abstract

The invention provides a land utilization type change information extraction method and device. The method comprises the following steps: preparing remote sensing image data of two time phases of a target area and completing space calibration; the method comprises the following steps: designing a parameter-shared twinborn convolutional neural network CD-Net, wherein the network is of a coding-cross-correlation layer-decoding structure; remote sensing image data of the two time phases are respectively sent to an encoder of the CD-Net, depth features of the two time phases are extracted, a relation between the two depth features is constructed in a cross-correlation layer, the extracted features and the correlation features are successively input to the decoder of the CD-Net according to the time phases, reduction features of the respective time phases are obtained, the reduction features of the two time phases are superposed, and the CD-Net is constructed. Obtaining change characteristics containing land utilization type change information; and inputting the change characteristics into a classifier of a CD-Net network, and outputting land utilization type change information of the target area. According to the method, land utilization type change information extraction based on deep learning is realized, and the classification precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image data processing, and particularly relates to a method and device for extracting land use type change information. Background Art

[0002] Land cover refers to the general term for the vegetation cover and artificial cover on the earth's surface, and is a comprehensive reflection of various elements on the earth's surface covered by natural vegetation, natural constructs, and artificial buildings. Land cover is essential information for humans to understand nature and master the laws of nature, and is also the most basic data required for various resource management and geographic information services. Changes in land cover reflect the changes and development of the area. Therefore, the acquisition, analysis, and update of land cover change information are extremely important.

[0003] Remote sensing image data has always been an important means for land cover change detection due to its macroscopic and real-time characteristics. Currently, land cover change detection methods based on remote sensing data are generally divided into two types. One is the traditional method based on visual interpretation of remote sensing original images; the other is the change detection based on remote sensing data products using the idea of deep learning, including primary products and more advanced products. Summary of the Invention

[0004] To overcome the problem that the existing land cover change detection methods cannot meet the accuracy requirements of land cover change detection or at least partially solve the above problems, embodiments of the present invention provide a method and device for extracting land use type change information.

[0005] According to the first aspect of the embodiments of the present invention, a method for extracting land use type change information is provided, including the following steps:

[0006] (1) Prepare remote sensing image data of two time phases for the target area and complete spatial calibration;

[0007] (2) Design a twin convolutional neural network CD-Net with shared parameters, and the network has an encoding-cross-correlation layer-decoding structure;

[0008] (3) Send the remote sensing image data of the two time phases into the encoder of CD-Net respectively, extract the depth features of the two time phases, establish the connection between the two depth features in the cross-correlation layer, and input the extracted features and associated features into the decoder of CD-Net in sequence according to the time phase to obtain the restored features of each time phase, and superimpose the restored features of the two time phases to obtain the change features containing land use type change information;

[0009] (4) Input the change features into the classifier of the CD-Net network to output the land use type change information of the target area.

[0010] Before step (3), it further includes:

[0011] The encoder of the CD-Net network extracts the depth features of two-phase remote sensing images, then constructs the connection between the two depth features in the cross-correlation layer, and finally inputs the extracted features and associated features into the decoder of the CD-Net successively according to the phase to obtain the restored features of each phase, and superimposes the restored features of the two phases to obtain the change features containing land use type change information;

[0012] Correspondingly, step (4) specifically includes:

[0013] Input the change features into the classifier of the CD-Net network to output the land use type change information of the target area.

[0014] Further, the encoder of the CD-Net network extracts the depth features of two-phase remote sensing images, then constructs the connection between the two depth features in the cross-correlation layer, and finally inputs the extracted features and associated features into the decoder of the CD-Net successively according to the phase to obtain the restored features of each phase, and superimposes the restored features of the two phases to obtain the change features containing land use type change information, which are expressed by the following formula:

[0015]

[0016] Among them, T1 and T2 are the remote sensing image data of two phases, is the feature superposition operation; and φ are the encoder and decoder of the CD-Net network respectively. The encoder extracts the depth features X1 and X2 of the remote sensing image data, and the decoder restores the intermediate features to obtain the restored features Y1 and Y1; ψ is the cross-correlation layer of the CD-Net network, which is used to establish the connection between the depth features of the two phases and obtain the associated features X′1 and X′2; the change feature Y is composed of the superposition of Y1 and Y2.

[0017] Preferably, step (4) is expressed by the following formula:

[0018] Res = σ(Y)

[0019] Among them, the land use type change information Res is extracted from the change feature Y by the classifier σ of the CD-Net network.

[0020] In the present invention, the remote sensing image data includes high-resolution satellite images and high-resolution aerial images.

[0021] Before the step of inputting the remote sensing image data of two time phases into the CD-Net network in step (4), it further includes: training the CD-Net network according to the pre-acquired remote sensing image data samples and the preset class labels corresponding to each of the remote sensing image data samples.

[0022] According to a second aspect of an embodiment of the present invention, there is provided a device for extracting land use type change information, including:

[0023] A hidden module, configured to extract change features. The remote sensing image data of two time phases are respectively sent into the encoder of the CD-Net to extract the depth features of the two time phases, establish the connection between the two depth features in the cross-correlation layer, and input the extracted features and associated features into the decoder of the CD-Net in sequence according to the time phase to obtain the restored features of their respective time phases, and superimpose the restored features of the two time phases to obtain change features including land use type change information.

[0024] A classification module, configured to identify change information. The change features are input into the classifier of the CD-Net network, and the land use type change information of the target area is output.

[0025] According to a third aspect of an embodiment of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor can execute the land use type change information extraction method provided by any one of the various possible implementation manners of the first aspect by invoking the program instructions.

[0026] According to a fourth aspect of an embodiment of the present invention, there is also provided a non-transitory computer-readable storage medium, and the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the land use type change information extraction method provided by any one of the various possible implementation manners of the first aspect.

[0027] An embodiment of the present invention provides a method and device for extracting land use type change information. The method improves the existing convolutional neural network. The improved CD-Net network includes an encoder, a cross-correlation layer, and a decoder. The remote sensing image data of two time phases are respectively input into the encoder to extract the depth features of the two time phases, establish the connection between the two depth features in the cross-correlation layer, and input the extracted features and associated features into the decoder in sequence according to the time phase to obtain the restored features of their respective time phases, and superimpose the restored features of the two time phases to obtain change features including land use type change information. The Softmax classifier completes the classification of the change features to obtain the land use type change information of the target area, realizes the extraction of land use type change information based on deep learning, and improves the classification accuracy. Description of the Drawings

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0029] Figure 1 It is a schematic diagram of the overall process of the method for extracting land use type change information provided by the embodiments of the present invention;

[0030] Figure 2 It is a schematic diagram of the cross-correlation layer structure provided by the embodiments of the present invention;

[0031] Figure 3 It is a schematic diagram of the overall structure of the electronic device provided by the embodiments of the present invention. Specific Embodiments

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0033] In an embodiment of the present invention, a method for extracting land use type change information is provided. Figure 1 It is a schematic diagram of the overall process of the method for extracting land use type change information provided by the embodiments of the present invention. The method includes steps S1, S2, S3, and S4:

[0034] S1, Deep feature encoding of remote sensing images.

[0035] The target area is the area where land use type change information needs to be extracted, and the data are two-phase high-resolution remote sensing images belonging to different time phases. The CD-Net is a siamese network with shared weight parameters inside the network. The remote sensing images of the two time phases are respectively input into the encoder of the CD-Net to obtain deep features. The encoder consists of four convolutional blocks, and each convolutional block includes a convolutional layer (Conv), a max pooling layer (MaxPooling), an attention mechanism layer (Att), and a ReLU non-linear activation function. Among them, each convolutional block is shown by the following formula:

[0036] 0 ≤ i ≤ i

[0037]

[0038] Among them, T1 and T2 respectively represent remote sensing images of two phases. represents the output of the j-th phase in the i-th convolutional block (j ∈ {1, 2}). The size of the Conv convolutional kernel is 3×3, and the size of MaxP is 2×2. Each convolutional block contains two (ReLU, Conv) combinations. After passing through the four convolutional blocks of the encoder, the remote sensing image data is encoded into deep features and

[0039] S2, dual-phase feature correlation.

[0040] The cross-correlation layer is used to establish the correlation between the deep features of the two phases, providing the decoder with the comprehensive information of the deep features of the two phases at the high-order semantic level, and enhancing the ability of the CD-Net network to distinguish change information. Figure 2 This is the schematic diagram of the cross-correlation layer of the CD-Net network provided by the embodiment of the present invention. The cross-correlation layer of the CD-Net network includes two cross-correlation blocks with the same structure. One cross-correlation block focuses on the deep features of one phase. The process of the cross-correlation block is shown in the following formula:

[0041]

[0042] Among them, is the stacking operation, and are the corresponding correlation features of the two phases respectively.

[0043] S3, feature decoding and extraction of land use type change information.

[0044] The encoder of the CD-Net network is responsible for restoring the deep features and correlation features to the restored features of the original remote sensing image size to restore the detailed information. The encoder consists of four deconvolution blocks. Each deconvolution block includes a deconvolution layer (DeConv), an attention mechanism layer (Att), and a ReLU non-linear activation function. Among them, each convolutional block is shown in the following formula:

[0045] 0 ≤ i ≤ 4

[0046]

[0047] Res = Softmax(Y)

[0048] Among them, the initial input feature Y0 of the decoder is superimposed with the deep feature X4 and the correlation feature X t . Each deconvolution block of the decoder is superimposed with the input feature Y i and the output feature X of the corresponding encoder convolutional block 4-i . Finally, the restored features of the two phases are superimposed and The restored feature Y is obtained, and the classifier Softmax combines the restored feature Y for classification to obtain the land use type change information Res.

[0049] S4. Train the CD-Net network.

[0050] Before extracting the land use type change information, the CD-Net network needs to be trained first so that the CD-Net network can learn the ability to distinguish change information. According to the pre-acquired remote sensing image data samples and the preset class labels corresponding to each of the remote sensing image data samples, the CD-Net network is trained, and the overall loss is calculated based on the cross-entropy loss function. The deep semantic segmentation network is optimized by the backpropagation algorithm and the stochastic gradient descent method:

[0051] L = -(y0log(Res0)+y1log(Res1))

[0052] y is an indicator variable (0 or 1). If the category is the same as the sample category, it is 1; otherwise, it is 0. Res0 represents the result predicted as unchanged, and Res1 represents the result predicted as changed.

[0053]

[0054] Among them, θ is the parameter of the CD-Net. The parameter θ in the current iteration i+1 , the parameter θ in the previous iteration i , the learning rate α, and J(θ) is the overall loss.

[0055] This embodiment provides an electronic device. Figure 3 This is a schematic diagram of the overall structure of the electronic device provided by the embodiment of the present invention. The device includes: at least one processor 301, at least one memory 302, and a bus 303; among them,

[0056] The processor 301 and the memory 302 complete communication with each other through the bus 303;

[0057] The memory 302 stores program instructions executable by the processor 301, and the processor can execute the methods provided by the respective method embodiments by calling the program instructions.

[0058] This embodiment provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided by the respective method embodiments. For example, it includes: preparing remote sensing image data of two time phases of the target area and completing spatial calibration; designing a twin convolutional neural network CD-Net with shared parameters, which has an encoding-cross correlation layer-decoding structure; respectively inputting the remote sensing image data of the two time phases into the encoder of CD-Net to extract the depth features of the two time phases, establishing the connection between the two depth features in the cross correlation layer, and successively inputting the extracted features and associated features into the decoder of CD-Net according to the time phase to obtain the restored features of their respective time phases, superimposing the restored features of the two time phases to obtain the change features containing land use type change information; inputting the change features into the classifier of the CD-Net network to output the land use type change information of the target area.

[0059] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes.

[0060] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0061] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions to cause a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for extracting land use type change information, characterized in that: The following steps are involved: (1) Prepare remote sensing image data of the target area at two time phases and complete spatial calibration; (2) Design a parameter-sharing twin convolutional neural network CD-Net, which has an encoding-cross-correlation layer-decoding structure; (3) The remote sensing image data of the two time phases are respectively sent to the encoder of CD-Net to extract the deep features of the two time phases, and the connection between the two deep features is established in the cross-correlation layer. The extracted features and related features are successively input into the decoder of CD-Net according to the time phase to obtain the restored features of each time phase, and the restored features of the two time phases are superimposed to obtain the change features containing the land use type change information; (4) Inputting the change characteristics into the classifier of the CD-Net network, and outputting the land use type change information of the target area.

2. The method for extracting land use type change information according to claim 1, characterized in that: Before step (3), the method further includes: The encoder of the CD-Net network extracts the depth features of the remote sensing images of the two time phases, then builds the connection between the two depth features in the cross-correlation layer, and finally inputs the extracted features and the associated features into the decoder of the CD-Net in sequence according to the time phase, obtains the restored features of each time phase, superimposes the restored features of the two time phases, and obtains the change features containing the land use type change information; Accordingly, step (4) specifically includes: The change characteristics are input into the classifier of the CD-Net network, and the land use type change information of the target area is output.

3. The method for extracting land use type change information according to claim 2, characterized in that: The following formula is used to express that the encoder of the CD-Net network extracts the deep features of the remote sensing images of the two time phases, and then builds the connection between the two deep features in the cross-correlation layer. Finally, the extracted features and related features are input into the decoder of the CD-Net in sequence according to the time phase to obtain the restored features of each time phase, and the restored features of the two time phases are superimposed to obtain the change features containing the land use type change information: Among them, T1 and T2 are remote sensing image data of two phases. It is a feature superposition operation; and φ are the encoder and decoder of the CD-Net network respectively. The encoder extracts the deep features X1 and X2 of the remote sensing image data, and the decoder restores the intermediate features to obtain the restored features Y1 and Y1; Ψ is the cross-correlation layer of the CD-Net network, which is used to connect the deep features of the two phases to obtain the associated features X′1 and X′2; the change feature Y is the superposition of Y1 and Y2.

4. The method for extracting land use type change information according to claim 1, characterized in that: Step (4) is expressed by the following formula: Res=σ(Y) Among them, the classifier σ of the CD-Net network extracts the land use type change information Res from the change feature Y.

5. The method for extracting land use type change information according to claim 1, characterized in that: The remote sensing image data includes high-resolution satellite images and high-resolution aerial images.

6. The method for extracting land use type change information according to any one of claims 1 to 5, characterized in that: Before the step (4) of inputting the remote sensing image data of the two time phases into the CD-Net network, the method further includes: The CD-Net network is trained according to the pre-acquired remote sensing image data samples and the preset category labels corresponding to each of the remote sensing image data samples.

7. A device for extracting information on changes in land use types, characterized in that: include: Hidden module, used to extract change features; send the remote sensing image data of the two phases to the encoder of CD-NET respectively, extract the deep features of the two phases, build the connection between the two deep features in the cross-correlation layer, input the extracted features and related features into the decoder of CD-Net in turn according to the phase, obtain the restored features of each phase, superimpose the restored features of the two phases, and obtain the change features containing the land use type change information; The classification module is used to identify change information; input the change characteristics into the softmax classifier of the CD-Net network, and output the land use type change information of the target area.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for extracting land use type change information as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for extracting land use type change information as described in any one of claims 1 to 6 are implemented.

Images