Multi-task building extraction and change detection method based on deep learning

Abstract

The invention provides a multi-task building extraction and change detection method based on deep learning, which combines building extraction and change detection tasks through a deep neural network architecture, and comprises an encoder network, a building extraction decoder network and a building change decoder network, so as to realize an identification task of a building in a remote sensing image. Detecting a building change area task; a multi-task building extraction and change detection neural network is adopted to preprocess satellite remote sensing image data based on multiple time phases and generate image pairs of different time phases in a data set as data input of two channels of an image T1 and an image T2, and deep features of buildings in the images are extracted through an encoder network; the features extracted by the image T1 channel pass through a building extraction decoder network to obtain a building extraction result, and the features extracted by the image T1 channel and the features extracted by the image T2 channel pass through a building change decoder network to obtain a building change detection result.

Description

Technical Field

[0001] The present invention relates to technical fields such as deep learning, computer vision, building extraction, change detection, and more particularly to a multi-task building extraction and change detection method based on deep learning. Background Art

[0002] In the field of power grids, building extraction and change detection technology based on remote sensing images has wide application value, especially in power facility monitoring, line inspection and post-disaster assessment. Traditional building extraction and change detection methods usually rely on manually designed feature extraction and classifiers, which are inefficient and inaccurate when processing large-scale remote sensing data. Deep learning algorithms, with their powerful feature learning capabilities and adaptability, have become an effective tool for solving building extraction and change detection problems. Deep learning-based methods can automatically learn feature representations in images without manually designing features, and can optimize models through end-to-end training. Such advantages have enabled deep learning-based methods to achieve significant improvements in building extraction and change detection tasks.

[0003] However, existing deep learning-based building extraction and change detection methods still have some problems. Traditional building extraction and change detection methods usually treat them as two independent tasks, ignoring the correlation between them. Summary of the invention

[0004] By adopting the deep learning method and constructing a deep neural network architecture, the building extraction and change detection tasks are combined to achieve efficient processing of multiple tasks, so as to solve the problem that the existing technology cannot simultaneously achieve the two tasks of building extraction and its change detection. The present invention designs a multi-task building extraction and change detection neural network, including an encoder network, a building extraction decoder network and a building change decoder network, which can not only realize the task of accurately identifying buildings in remote sensing images, but also effectively detect the task of building change areas. This multi-task learning method not only improves the detection accuracy and efficiency, but also can make full use of the complementary information between tasks and enhance the generalization ability of the model. It is suitable for practical application scenarios such as power transmission line channel environment monitoring, urban planning, disaster assessment and land use monitoring.

[0005] The present invention specifically adopts the following technical solutions:

[0006] A multi-task building extraction and change detection method based on deep learning: The building extraction and change detection tasks are combined through a deep neural network architecture, including: an encoder network, a building extraction decoder network and a building change decoder network, to achieve the task of identifying buildings in remote sensing images and detecting building change areas;

[0007] A multi-task building extraction and change detection neural network is used to preprocess the multi-temporal satellite remote sensing image data and generate image pairs of different phases in the dataset as data input for the two channels of image T1 and image T2 respectively. The deep features of the buildings in the image are extracted through the encoder network respectively, and the features extracted from the image T1 channel are passed through the building extraction decoder network to obtain the building extraction result. The features extracted from the image T1 channel and the features extracted from the image T2 channel are passed through the building change decoder network to obtain the building change detection result.

[0008] Furthermore, data preprocessing includes: collecting multi-temporal satellite remote sensing image data, performing radiation correction, atmospheric correction and geometric correction on the images; then cropping the images according to the target area range to remove irrelevant areas; and then registering the multi-temporal images to have the same spatial reference system to ensure that the same location is aligned in images of different temporal phases;

[0009] Establishing a building extraction and change detection data set includes annotating buildings in preprocessed images to generate building outline data; comparing images of different phases, annotating the increase and decrease of buildings and the changed areas, and generating annotation data for change detection.

[0010] Further, the decoder network includes a CNN feature extraction module and a multi-scale feature extraction module;

[0011] The CNN feature extraction module consists of a 7×7 convolution, 64 filters, a convolution layer with a stride of 2, a 3×3 maximum pooling, a maximum pooling layer with a stride of 2, 3 convolution blocks 1, 4 convolution blocks 2, and 6 convolution blocks 3; wherein, convolution block 1 contains 3 residual blocks, each residual block contains 3 convolution layers, namely, 1×1 convolution of 64 filters, 3×3 convolution of 64 filters, and 1×1 convolution of 256 filters; convolution block 2 Contains 4 residual blocks, each of which contains 3 convolutional layers, namely 1×1 convolution of 256 filters, 3×3 convolution of 256 filters, and 1×1 convolution of 1024 filters; Convolution block 3 contains 6 residual blocks, each of which contains 3 convolutional layers, namely 1×1 convolution of 256 filters, 3×3 convolution of 256 filters, and 1×1 convolution of 1024 filters; Convolution block 2 and convolution block 3 output low-dimensional features L respectively. low and high-dimensional features L high , image T1 and image T2 channels are respectively obtained after passing through the CNN feature extraction module. low1 , L high2 and feature L low2 , L high2 ;

[0012] The multi-scale feature extraction module processes objects of different scales in building extraction and change detection tasks through a 3×3 convolution structure; the module first performs dimensionality reduction through 1×1 convolution and 3×3 convolution to obtain a 3×3 receptive field; the right branch then undergoes 1×1 convolution and 3×3 convolution to obtain a 5×5 receptive field; the left branch then undergoes 1×1 convolution and 3×3 convolution to obtain a 7×7 receptive field; the three receptive fields of different sizes are fused to capture feature information of different scales.

[0013] Furthermore, the building extraction decoder network extracts the low-dimensional features L output by the CNN feature extraction module of the image T1 channel. low1 Next, the low-dimensional feature L low1 The results obtained through 1×1 convolution, batch normalization and relu activation function layers are consistent with the high-dimensional features L output by the CNN feature extraction module. high1 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code1 , the feature L code1 After 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling, the result of building extraction is obtained.

[0014] Furthermore, the building change decoder network converts the low-dimensional features L output by the CNN feature extraction module of the image T2 channel low2 , the result obtained after 1×1 convolution, batch normalization and relu activation function layer, and the high-dimensional feature L output by the CNN feature extraction module high2 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code2 , then, the feature L code2 With the above-obtained feature L code1 The connection is made, and the result is obtained through 3×3 convolution, batch normalization and relu activation function layer to obtain the fusion feature L F1 ; The feature L high1 and L high2 The features after the multi-scale feature extraction module are connected, and the results are combined with the fusion feature L after 3×3 convolution, batch normalization, relu activation function layer, and 4 times upsampling. F2 ; The feature L F1 and feature L F2 After connection, 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling are performed to obtain the result of building change detection.

[0015] Furthermore, the loss function of the multi-task building extraction and change detection neural network includes a building extraction loss function and a change detection loss function:

[0016] The loss for building extraction is calculated using the binary cross entropy loss function:

[0017]

[0018] Among them, y i is the true label of the i-th pixel, p i is the probability that the i-th pixel is predicted to be a building, and N is the total number of pixels;

[0019] The loss for change detection is calculated using the binary cross entropy loss function:

[0020]

[0021] Among them, y i ′ is the true label of the change of the i-th pixel, p i ′ is the probability that the i-th pixel is predicted as a building change, and N is the total number of pixels;

[0022] The loss functions of the two tasks are weighted and summed; the weights λ1 and λ2 are used to balance the losses of the two tasks:

[0023] L total =λ1L extract +λ2L change .

[0024] Furthermore, the process of training the multi-task building extraction and change detection neural network is as follows: using the building extraction and change detection dataset as the input of the multi-task building extraction and change detection neural network, and using the labeled building contour data and the labeled building change area as the output to train the multi-task building extraction and change detection neural network; using pytorch as the deep neural network learning framework to train the model, using the Adam optimizer, setting the initial learning rate to 0.01, and adopting the hierarchical descent learning rate method, and the number of training epochs is 200; two satellite images of different phases that have been registered are input into the trained neural network to obtain the results of the building area and its changes in the satellite image.

[0025] A multi-task building extraction and change detection system based on deep learning, including:

[0026] The preprocessing module is used to preprocess the multi-temporal satellite remote sensing image data and generate a data set;

[0027] And, a multi-task building extraction and change detection neural network, including: an encoder network, a building extraction decoder network and a building change decoder network, which are used to realize the task of identifying buildings in remote sensing images and detecting building change areas;

[0028] A multi-task building extraction and change detection neural network is used to preprocess the multi-temporal satellite remote sensing image data and generate image pairs of different phases in the dataset as data input for the two channels of image T1 and image T2 respectively. The deep features of the buildings in the image are extracted through the encoder network respectively, and the features extracted from the image T1 channel are passed through the building extraction decoder network to obtain the building extraction result. The features extracted from the image T1 channel and the features extracted from the image T2 channel are passed through the building change decoder network to obtain the building change detection result.

[0029] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a multi-task building extraction and change detection method based on deep learning as described above are implemented.

[0030] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a multi-task building extraction and change detection method based on deep learning as described above.

[0031] Compared with the prior art, the present invention and its preferred solution adopt a deep learning method, and by constructing a deep neural network architecture, the building extraction and change detection tasks are combined to achieve efficient processing of multiple tasks, so as to solve the problem that the prior art cannot simultaneously achieve the two tasks of building extraction and its change detection. The present invention designs a multi-task building extraction and change detection neural network, including an encoder network, a building extraction decoder network and a building change decoder network, which can not only realize the task of accurately identifying buildings in remote sensing images, but also effectively detect the task of building change areas. This multi-task learning method not only improves the detection accuracy and efficiency, but also can make full use of the complementary information between tasks and enhance the generalization ability of the model. It is suitable for practical application scenarios such as power transmission line channel environment monitoring, urban planning, disaster assessment and land use monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0033] Figure 1 This is a neural network structure diagram of multi-task building extraction and change detection according to an embodiment of the present invention;

[0034] Figure 2 This is a structural diagram of a CNN feature extraction module according to an embodiment of the present invention;

[0035] Figure 3 A structural diagram of a multi-scale feature extraction module according to an embodiment of the present invention;

[0036] Figure 4 It is a schematic diagram of the implementation process of the embodiment of the present invention. DETAILED DESCRIPTION

[0037] Hereinafter, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without violating the principles of the present application, the features in the various embodiments can be combined to obtain new implementations, or certain features in certain embodiments can be replaced to obtain other preferred implementations.

[0038] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail with the accompanying drawings as follows:

[0039] like Figure 4 As shown, the embodiment of the present invention proposes a multi-task building extraction and change detection method based on deep learning, which can simultaneously complete the two tasks of building extraction and change detection. By designing a deep neural network architecture, the building extraction and change detection tasks are combined, and the correlation between the two is fully utilized to improve the accuracy and robustness of the model.

[0040] The implementation process can refer to the following steps:

[0041] Step S1: Data preprocessing. Collect multi-temporal satellite remote sensing image data, perform radiation correction, atmospheric correction and geometric correction on the image, eliminate sensor noise, atmospheric effects and geometric distortion, and make the image have uniform spectral and geometric characteristics. Then, according to the scope of the target area, crop the image, remove irrelevant areas, reduce the amount of data, and align the multi-temporal images to have the same spatial reference system, ensuring that the same location is aligned in images of different phases to facilitate change detection.

[0042] Step S2: Establish a building extraction and change detection data set. Manually annotate the buildings in the images collected and processed in step S1 to generate building outline data; compare images of different phases, annotate the increase and decrease of buildings and the changed areas, generate annotated data for change detection, check the matching degree between the annotated data and the images, and ensure the accuracy and consistency of the annotation.

[0043] Step S3: Construct a multi-task building extraction and change detection neural network. Figure 1As shown, the multi-task building extraction and change detection neural network includes an encoder network, a building extraction decoder network, and a building change decoder network. The image pairs obtained in different phases in step S2 are respectively used as data inputs of the two channels of image T1 and image T2, and the deep features of the buildings in the images are extracted through the encoder network respectively. The features extracted from the image T1 channel are passed through the building extraction decoder network to obtain the building extraction result. The features extracted from the image T1 channel and the features extracted from the image T2 channel are passed through the building change decoder network to obtain the building change detection result.

[0044] The decoder network in step S3 is further constructed. The decoder network includes a CNN feature extraction module and a multi-scale feature extraction module.

[0045] CNN feature extraction module such as Figure 2 As shown, it contains a 7×7 convolution, 64 filters, a convolution layer with a stride of 2, a 3×3 maximum pooling, a maximum pooling layer with a stride of 2, 3 convolution blocks 1, 4 convolution blocks 2, and 6 convolution blocks 3. Convolution block 1 contains 3 residual blocks, each of which contains 3 convolution layers, namely 1×1 convolution (64 filters), 3×3 convolution (64 filters), and 1×1 convolution (256 filters); convolution block 2 contains 4 residual blocks, each of which contains 3 convolution layers, namely 1×1 convolution (256 filters), 3×3 convolution (256 filters), and 1×1 convolution (1024 filters); convolution block 3 contains 6 residual blocks, each of which contains 3 convolution layers, namely 1×1 convolution (256 filters), 3×3 convolution (256 filters), and 1×1 convolution (1024 filters). Convolutional blocks 2 and 3 output low-dimensional features L respectively. low and high-dimensional features L high , image T1 and image T2 channels are respectively obtained after passing through the CNN feature extraction module. low1 , L high2 and feature L low2 , L high2 .

[0046] The core of the multi-scale feature extraction module is the 3×3 convolution structure. Figure 3 As shown in the figure, this is beneficial for processing objects of different scales in building extraction and change detection tasks. This module can be well aggregated into receptive fields of size 3×3, 5×5, and 7×7. First, 1×1 convolution is used to reduce the dimension and 3×3 convolution is used to obtain a 3×3 receptive field; the right branch is then subjected to 1×1 convolution and 3×3 convolution to obtain a 5×5 receptive field; the left branch is then subjected to 1×1 convolution and 3×3 convolution to obtain a 7×7 receptive field; the three receptive fields of different sizes are fused to capture feature information of different scales.

[0047] Further construct the building extraction decoder network described in step S3. Figure 1 As shown, the low-dimensional feature L output by the CNN feature extraction module of the image T1 channel is low1 Next, the low-dimensional feature L low1 The results obtained through 1×1 convolution, batch normalization and relu activation function layers are consistent with the high-dimensional features L output by the CNN feature extraction module. high1 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code1 , the feature L code1 After 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling, the result of building extraction is obtained.

[0048] Further construct the building change decoder network described in step S3. Figure 1 As shown, the low-dimensional feature L output by the CNN feature extraction module of the image T2 channel is low2 , the result obtained after 1×1 convolution, batch normalization and relu activation function layer is compared with the high-dimensional feature L output by the CNN feature extraction module. high2 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code2 , then, the feature L code2 With the above-obtained feature L code1 The result is connected through 3×3 convolution, batch normalization and relu activation function layer to obtain the fusion feature L F1 ; The feature L high1 and L high2 The features after the multi-scale feature extraction module are connected, and the results are combined with the fusion feature L obtained after 3×3 convolution, batch normalization, relu activation function layer, and 4 times upsampling. F2 ; The feature L F1 and feature L F2 After connection, 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling are performed to obtain the result of building change detection.

[0049] Step S4: construct a multi-task building extraction and change detection neural network loss function. The multi-task building extraction and change detection neural network loss function includes a building extraction loss function and a change detection loss function.

[0050] Building extraction is a binary classification problem (buildings and non-buildings). The loss of building extraction can be calculated using the binary cross entropy loss function:

[0051]

[0052] Among them, y i is the true label of the i-th pixel, p i is the probability that the i-th pixel is predicted to be a building, and N is the total number of pixels.

[0053] The change detection loss function is also a binary classification problem (change and no change). A similar binary cross entropy loss function can be used to calculate the loss of change detection:

[0054]

[0055] Among them, y i ′ is the true label of the change of the i-th pixel, p i ′ is the probability that the i-th pixel is predicted to be a building change, and N is the total number of pixels.

[0056] In order to construct a joint loss function for multiple tasks, the loss functions of the two tasks can be weighted and summed. The weights λ1 and λ2 are used to balance the losses of the two tasks:

[0057] L total =λ1L extract +λ2L change

[0058] Step S5: training a multi-task building extraction and change detection neural network. The building extraction and change detection dataset established in step S2 is used as the input of the multi-task building extraction and change detection neural network, and the manually labeled building outline data and the labeled building change area are used as output to train the multi-task building extraction and change detection neural network; pytorch is used as the deep neural network learning framework, and NVDIA GeForce RTX4090 is used as the GPU for model training, using the Adam optimizer, the initial learning rate is set to 0.01, and the hierarchical descent learning rate method is used, and the number of training epochs is 200 (all training data completes 1 operation).

[0059] Step S6: Identify the building area and its changes in the satellite image. Input the two registered satellite images of different phases into the neural network trained in step S5 to obtain the building area and its changes in the satellite image.

[0060] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0061] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0066] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive various other forms of a multi-task building extraction and change detection method based on deep learning under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A multi-task building extraction and change detection method based on deep learning, characterized by: Combine building extraction and change detection tasks through a deep neural network architecture, including: encoder network, building extraction decoder network and building change decoder network, to achieve the task of identifying buildings in remote sensing images and detecting building change areas; A multi-task building extraction and change detection neural network is used to preprocess the multi-temporal satellite remote sensing image data and generate image pairs of different phases in the dataset as data input for the two channels of image T1 and image T2 respectively. The deep features of the buildings in the image are extracted through the encoder network respectively, and the features extracted from the image T1 channel are passed through the building extraction decoder network to obtain the building extraction result. The features extracted from the image T1 channel and the features extracted from the image T2 channel are passed through the building change decoder network to obtain the building change detection result.

2. The multi-task building extraction and change detection method based on deep learning according to claim 1 is characterized in that: Data preprocessing includes: collecting multi-temporal satellite remote sensing image data, performing radiation correction, atmospheric correction and geometric correction on the images; then cropping the images according to the target area range to remove irrelevant areas; and then registering the multi-temporal images to have the same spatial reference system to ensure that the same location is aligned in images of different phases; Establishing a building extraction and change detection data set includes annotating buildings in preprocessed images to generate building outline data; comparing images of different phases, annotating the increase and decrease of buildings and the changed areas, and generating annotation data for change detection.

3. The multi-task building extraction and change detection method based on deep learning according to claim 1, characterized in that: The decoder network includes a CNN feature extraction module and a multi-scale feature extraction module; The CNN feature extraction module consists of a 7×7 convolution, 64 filters, a convolution layer with a stride of 2, a 3×3 maximum pooling, a maximum pooling layer with a stride of 2, 3 convolution blocks 1, 4 convolution blocks 2, and 6 convolution blocks 3; wherein, convolution block 1 contains 3 residual blocks, each residual block contains 3 convolution layers, namely, 1×1 convolution of 64 filters, 3×3 convolution of 64 filters, and 1×1 convolution of 256 filters; convolution block 2 Contains 4 residual blocks, each of which contains 3 convolutional layers, namely 1×1 convolution of 256 filters, 3×3 convolution of 256 filters, and 1×1 convolution of 1024 filters; Convolution block 3 contains 6 residual blocks, each of which contains 3 convolutional layers, namely 1×1 convolution of 256 filters, 3×3 convolution of 256 filters, and 1×1 convolution of 1024 filters; Convolution block 2 and convolution block 3 output low-dimensional features L respectively. low and high-dimensional features L high After the image T1 and image T2 channels pass through the CNN feature extraction module, the features L low1 , L high2 and feature L low2 , L high2 ; The multi-scale feature extraction module processes objects of different scales in building extraction and change detection tasks through a 3×3 convolution structure; the module first performs dimensionality reduction through 1×1 convolution and 3×3 convolution to obtain a 3×3 receptive field; the right branch then undergoes 1×1 convolution and 3×3 convolution to obtain a 5×5 receptive field; the left branch then undergoes 1×1 convolution and 3×3 convolution to obtain a 7×7 receptive field; the three receptive fields of different sizes are fused to capture feature information of different scales.

4. The multi-task building extraction and change detection method based on deep learning according to claim 3 is characterized in that: The building extraction decoder network converts the image T1 channel into a low-dimensional feature L output by the CNN feature extraction module low1 Next, the low-dimensional feature L low1 The results obtained through 1×1 convolution, batch normalization and relu activation function layers are consistent with the high-dimensional features L output by the CNN feature extraction module. high1 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code1 , the feature L code1 After 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling, the result of building extraction is obtained.

5. The multi-task building extraction and change detection method based on deep learning according to claim 3 is characterized in that: The architectural variation decoder network converts the low-dimensional features L output by the CNN feature extraction module of the image T2 channel low2 , the result obtained after 1×1 convolution, batch normalization and relu activation function layer, and the high-dimensional feature L output by the CNN feature extraction module high2 After the multi-scale feature extraction module and 4 times upsampling, the result is superimposed to obtain the feature L code2 , then, the feature L code2 With the above-obtained feature L code1 The connection is made, and the result is obtained through 3×3 convolution, batch normalization and relu activation function layer to obtain the fusion feature L F1 ; The feature L high1 and L high2 The features after the multi-scale feature extraction module are connected, and the results are combined with the fusion feature L after 3×3 convolution, batch normalization, relu activation function layer, and 4 times upsampling. F2 ; The feature L F1 and feature L F2 After connection, 3×3 convolution, batch normalization, relu activation function layer and 4 times upsampling are performed to obtain the result of building change detection.

6. The multi-task building extraction and change detection method based on deep learning according to claim 1, characterized in that: The loss function of the multi-task building extraction and change detection neural network includes a building extraction loss function and a change detection loss function: The loss for building extraction is calculated using the binary cross entropy loss function: Among them, y i is the true label of the i-th pixel, p i is the probability that the i-th pixel is predicted to be a building, and N is the total number of pixels; The loss for change detection is calculated using the binary cross entropy loss function: Among them, y i ′ is the true label of the change of the i-th pixel, p i ′ is the probability that the i-th pixel is predicted as a building change, and N is the total number of pixels; The loss functions of the two tasks are weighted and summed; the weights λ1 and λ2 are used to balance the losses of the two tasks: L total =λ1L extract +λ2L change 。 7. The multi-task building extraction and change detection method based on deep learning according to claim 1, characterized in that: The process of training the multi-task building extraction and change detection neural network is as follows: using the building extraction and change detection dataset as the input of the multi-task building extraction and change detection neural network, using the labeled building contour data and the labeled building change area as the output, and training the multi-task building extraction and change detection neural network; Pytorch is used as the deep neural network learning framework for model training. The Adam optimizer is used, the initial learning rate is set to 0.01, and the hierarchical descent learning rate method is adopted. The training times are 200 epochs. Two registered satellite images of different phases are input into the trained neural network to obtain the results of the building areas and their changes in the satellite images.

8. A multi-task building extraction and change detection system based on deep learning, characterized in that: include: The preprocessing module is used to preprocess the multi-temporal satellite remote sensing image data and generate a data set; And, a multi-task building extraction and change detection neural network, including: an encoder network, a building extraction decoder network and a building change decoder network, which are used to realize the task of identifying buildings in remote sensing images and detecting building change areas; A multi-task building extraction and change detection neural network is used to preprocess the multi-temporal satellite remote sensing image data and generate image pairs of different phases in the dataset as data input for the two channels of image T1 and image T2 respectively. The deep features of the buildings in the image are extracted through the encoder network respectively, and the features extracted from the image T1 channel are passed through the building extraction decoder network to obtain the building extraction result. The features extracted from the image T1 channel and the features extracted from the image T2 channel are passed through the building change decoder network to obtain the building change detection result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a multi-task building extraction and change detection method based on deep learning are implemented as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a multi-task building extraction and change detection method based on deep learning are implemented as described in any one of claims 1 to 7.

Images