A method, system, device and medium for detecting a building change area

Abstract

The present invention provides a method, system, device and medium for detecting building change areas, including the following steps: obtaining a first phase remote sensing image and a second phase remote sensing image; the first phase remote sensing image and the second phase remote sensing image are both remote sensing images of the target area; respectively inputting the first phase remote sensing image and the second phase remote sensing image into a building change detection model to obtain the output result of the building change detection model; based on the output result, obtaining the probability distribution of building changes; based on the probability distribution of building changes, obtaining the detection result of the building change area. The method, system, device and medium for detecting building change areas of the present invention can accurately extract the detail change areas of large-area and / or irregular buildings in complex scenes, thereby improving the detection accuracy of building change areas.

Description

Technical Field

[0001] The present application belongs to the field of surveying and mapping remote sensing detection technology, and in particular, relates to a detection method, system, device and medium for a building change area. Background Art

[0002] Building change detection provides important data support for urban planning, helping managers understand the development dynamics of the city so that managers can carry out effective urban planning and land resource management.

[0003] With the development of deep learning technology, more and more scholars use remote sensing images and deep learning technology to detect building changes in order to improve the automation level and detection speed of building change detection. The existing building change detection method based on deep learning usually adopts the DeepLabV3Plus model, which combines modules such as dilated convolution and pyramid pooling. While encoding multi-scale background information, it retains the information of the target boundary and realizes the detection of building change areas. Although the DeepLabV3Plus model uses dilated convolution to expand its receptive field and improve the globality and semantic level of extracted features. However, since the convolution results of the current convolution layer of the dilated convolution come from an independent set in the previous convolution layer, there is no correlation between the convolution results of the current convolution layer of the dilated convolution, and it is easy to lose local information; and the sampling input signal of the dilated convolution is sparse, resulting in no correlation between the features obtained by long-distance convolution, thereby reducing the accuracy of classification, especially in complex natural environments, it is easy to lose the detailed features of the building, and the detection accuracy of large-area or irregular-shaped buildings is low.

[0004] Based on this, how to achieve high-accuracy detection of building change areas in complex backgrounds is an important issue that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a method for detecting building change areas, which is used to solve the problem that the current building change detection method is difficult to accurately detect building change areas under complex backgrounds.

[0006] To achieve the above object and other related objects, the present invention provides a method for detecting a changed area of ​​a building, comprising the following steps:

[0007] Acquire a first temporal phase remote sensing image and a second temporal phase remote sensing image; the first temporal phase remote sensing image and the second temporal phase remote sensing image are both remote sensing images of the target area;

[0008] respectively inputting the first temporal phase remote sensing image and the second temporal phase remote sensing image into a building change detection model to obtain an output result of the building change detection model;

[0009] Based on the output result, a building change probability distribution is obtained; based on the building change probability distribution, a detection result of a building change area is obtained;

[0010] Among them, the building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptive module; the multi-view channel adaptive module is used to obtain a weighted feature map; the weighted feature map contains multi-scale global features, local features and edge features of the building change area.

[0011] In one embodiment of the present invention, the method for obtaining the weighted feature map includes:

[0012] The encoder extracts features from the first temporal phase remote sensing image and the second temporal phase remote sensing image respectively, and obtains a first feature map and a second feature map accordingly; performs feature similarity calculation on the first feature map and the second feature map to obtain a change feature map as an encoder feature map;

[0013] The decoder upsamples the encoder feature map to obtain a decoder feature map;

[0014] The multi-view channel adaptive module performs multi-scale and multi-view feature extraction on the decoder feature map to obtain the weighted feature map, including:

[0015] Performing multi-scale feature extraction on the decoder feature map to obtain scale feature maps at different scales;

[0016] Performing multi-view feature extraction on each of the scale feature maps to obtain multi-view feature maps corresponding to each of the scale feature maps; a single multi-view feature map contains global features, local features and edge features;

[0017] Fusing the multi-view feature maps to obtain a fused gain feature map; the image height, image width and channel dimension of the gain feature map are the same as the image height, image width and channel dimension of the decoder feature map;

[0018] The gain feature map is multiplied pixel by pixel by the decoder feature map to obtain the weighted feature map.

[0019] In one embodiment of the present invention, performing multi-scale feature extraction on the decoder feature map to obtain scale feature maps at different scales includes:

[0020] Using a plurality of first convolutional layers, features of different scales are extracted from the decoder feature map to obtain scale feature maps of corresponding scales; the convolution kernel sizes of the first convolutional layers are different.

[0021] In one embodiment of the present invention, the method for acquiring the multi-view feature map includes:

[0022] Performing global average pooling, local perception pooling and edge feature extraction on the scale feature map, respectively, to obtain a global average feature map, several local perception feature maps and an edge feature map;

[0023] In the channel dimension, the global average feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map;

[0024] Among them, the method for obtaining the plurality of local perception feature maps includes: using convolution kernels of several different scales to extract features from the scale feature maps respectively, and correspondingly obtaining a plurality of local perception feature maps.

[0025] In one embodiment of the present invention, after obtaining the scale feature map, the method further includes: performing global maximum pooling on the scale feature map to obtain a global maximum feature map; then the method for obtaining the multi-view feature map includes:

[0026] In the channel dimension, the global average feature map, the global maximum feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map.

[0027] In one embodiment of the present invention, after obtaining the weighted feature map, the method further includes:

[0028] In the channel dimension, average pooling and maximum pooling are performed on the weighted feature map respectively, and a channel average pooling map and a channel maximum pooling map are obtained correspondingly;

[0029] In the spatial dimension, the channel average pooling map and the channel maximum pooling map are spliced ​​to obtain a spliced ​​pooling map; the second convolutional layer is used to extract features of the spliced ​​pooling map to obtain a spatial attention feature map;

[0030] The spatial attention feature map is normalized to obtain a spatial attention weight matrix; the multi-view feature map and the spatial attention weight matrix are multiplied pixel by pixel to obtain a spatial enhancement feature map as an output result of the building change detection model.

[0031] In one embodiment of the present invention, obtaining a detection result of a building change area based on the building change probability distribution includes:

[0032] Based on the building change probability distribution, obtaining a building change probability distribution map as a detection result of the building change area; or,

[0033] Based on the building change probability distribution, a building change probability distribution map is obtained; and threshold segmentation is performed on the building change probability distribution map to obtain a detection result of the building change area.

[0034] Correspondingly, the present invention provides a detection system for a building change area, the system comprising:

[0035] An acquisition module is used to acquire a first temporal phase remote sensing image and a second temporal phase remote sensing image; the first temporal phase remote sensing image and the second temporal phase remote sensing image are both remote sensing images of the target area;

[0036] A detection module, used for inputting the first temporal phase remote sensing image and the second temporal phase remote sensing image into a building change detection model respectively, and obtaining an output result of the building change detection model; the building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptive module; the multi-view channel adaptive module is used for obtaining a weighted feature map; the weighted feature map contains multi-scale global features, local features and edge features of the building change area;

[0037] The detection result output module is used to obtain the building change probability distribution based on the output result; and obtain the detection result of the building change area based on the building change probability distribution.

[0038] Correspondingly, the present invention provides a computer device, the device comprising:

[0039] Memory for storing computer programs;

[0040] The processor is used to execute the computer program stored in the memory so that the device executes the method for detecting the building change area as described above.

[0041] Correspondingly, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting a changed area of ​​a building as described above is implemented.

[0042] As described above, the present application provides a method, system, device and medium for detecting a building change area, which have at least the following beneficial effects:

[0043] The output result of the building change detection model is obtained by respectively inputting a first phase remote sensing image and a second phase remote sensing image into a pre-constructed building change detection model; the output result at least includes multi-scale global features, local features and edge features of the building change area; based on the output result, a building change probability distribution is obtained; based on the building change probability distribution, a detection result of the building change area is obtained. The method improves the comprehensiveness and details of the feature extraction of the building change area by performing multi-scale and multi-perspective feature extraction on the building change area in the image, and realizes high-accuracy extraction of large-area and / or irregular building change areas in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic flow chart of a method for detecting a changed area of ​​a building provided by the present application in one embodiment is shown.

[0045] Figure 2 A schematic diagram of the structure of a building change detection model provided by the present application in a first embodiment is shown.

[0046] Figure 3 A schematic diagram of the structure of a multi-view channel adaptation module in the building change detection model provided in the present application in a first embodiment is shown.

[0047] Figure 4 A schematic diagram of the structure of a multi-view channel adaptation module in the building change detection model provided in the present application in a second embodiment is shown.

[0048] Figure 5 A schematic diagram of the structure of a multi-view channel adaptation module in the building change detection model provided in the present application in a third embodiment is shown.

[0049] Figure 6 A schematic diagram of the structure of a multi-view channel adaptation module in the building change detection model provided in the present application in a fourth embodiment is shown.

[0050] Figure 7 A schematic diagram of the structure of a multi-view channel adaptation module in the building change detection model provided in the present application in a fifth embodiment is shown.

[0051] Figure 8 A schematic diagram of the structure of the building change detection model provided in the present application in the second embodiment is shown.

[0052] Fig. 9 A schematic flow chart of another embodiment of a method for detecting a changed area of ​​a building provided by the present application is shown.

[0053] Fig.10 A module structure diagram of a building change area detection system provided by the present application in one embodiment is shown.

[0054] Fig.11 A schematic diagram of the structure of a computer device provided by the present application in one embodiment is shown.

[0055] Description of Reference Numerals

[0056] S1~S3, steps; S21~S24, steps; 200, detection system; 201, acquisition module; 202, detection module; 203, detection result output module; 300, device; 301, memory; 302, processor. DETAILED DESCRIPTION

[0057] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0058] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0059] To facilitate the understanding of the technical solutions provided by the present application, the relevant terms in the present application are explained before the specific embodiments, as follows:

[0060] Self-attention decoder: Improves the expressiveness and flexibility of the model when processing sequential data by parallel processing and capturing long-distance dependencies.

[0061] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0062] The following embodiments of the present application provide a method for detecting building change areas, by respectively inputting a first phase remote sensing image and a second phase remote sensing image into a pre-constructed building change detection model to obtain an output result of the building change detection model; the output result includes at least multi-scale global features, local features and edge features of the building change area; based on the output result, a building change probability distribution is obtained; based on the building change probability distribution, a detection result of the building change area is obtained, and the method improves the comprehensiveness and detail of the feature extraction of the building change area by performing multi-scale and multi-perspective feature extraction on the building change area in the image, and realizes high-accuracy extraction of large-area and / or irregular building change areas in complex scenes.

[0063] See also Figure 1 , showing a flow chart of a method for detecting a building change area provided by the present invention in one embodiment.

[0064] like Figure 1 As shown, in this embodiment, the method for detecting the building change area provided by the present invention comprises the following steps:

[0065] Step S1, acquiring a first temporal phase remote sensing image and a second temporal phase remote sensing image; the first temporal phase remote sensing image and the second temporal phase remote sensing image are both optical remote sensing images of the target area;

[0066] The first temporal phase remote sensing image and the second temporal phase remote sensing image are remote sensing images after image preprocessing.

[0067] Specifically, original remote sensing images of the target area in a first phase and a second phase are respectively obtained, image preprocessing is performed on each of the original remote sensing images, and the original remote sensing images in the first phase and the second phase after image preprocessing are used as the first phase remote sensing image and the second phase remote sensing image respectively.

[0068] Optionally, the image preprocessing includes: image registration and / or geometric correction, etc.

[0069] Step S2, respectively inputting the first temporal phase remote sensing image and the second temporal phase remote sensing image into a building change detection model to obtain an output result of the building change detection model;

[0070] The output result includes: the predicted value of each pixel belonging to the corresponding category; the predicted value is not normalized; the category includes building changes and building no changes;

[0071] The building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptive module connected in sequence; the encoder is used to output an encoder feature map; the encoder feature map contains semantic features and multi-scale information of changes between the first phase remote sensing image and the second phase remote sensing image; the decoder is used to output a decoder feature map; the decoder feature map highlights the change features of the building; the multi-view channel adaptive module is used to perform multi-scale and multi-view feature extraction on the encoder feature map to obtain a weighted feature map; the weighted feature map contains multi-scale global features, local features and edge features of building changes.

[0072] Specifically, the first phase remote sensing image and the second phase remote sensing image are respectively input into the building change detection model; the encoder performs feature extraction on the first phase remote sensing image and the second phase remote sensing image respectively to output the encoder feature map; the decoder upsamples the encoder feature map to obtain the upsampled decoder feature map; the size of the decoder feature map is consistent with the size of the first phase remote sensing image or the second phase remote sensing image; the multi-view channel adaptive module performs multi-scale and multi-view feature extraction on the decoder feature map to obtain the weighted feature map, and the weighted feature map is used as the output result of the building change detection model.

[0073] Optionally, the encoder is a twin network structure to respectively extract features from the first phase remote sensing image and the second phase remote sensing image; based on the features in the first phase remote sensing image and the second phase remote sensing image, feature similarity calculation is performed to obtain the change features between the first phase remote sensing image and the second phase remote sensing image.

[0074] Optional, such as Figure 2 As shown, the encoder includes a first branch network and a second branch network; the first branch network and the second branch network share weights; the first branch network is used to extract the first feature in the first phase remote sensing image and output a first feature map; the second branch network is used to extract the second feature in the second phase remote sensing image and output a second feature map; then the method for obtaining the encoder feature map includes:

[0075] The first feature map output by the first branch network and the second feature map output by the second branch network are respectively obtained; the feature similarity between the first feature in the first feature map and the second feature in the second feature map is calculated using a preset similarity measurement algorithm; based on the feature similarity, a change feature between the first feature map and the second feature map is obtained to obtain a change feature map as the encoder feature map.

[0076] Optionally, the preset similarity measurement algorithm includes: existing similarity measurement algorithms such as Euclidean distance, Euclidean distance or cosine similarity.

[0077] Optionally, obtaining the change feature between the first feature map and the second feature map based on the feature similarity includes: based on a preset similarity threshold, performing change feature judgment on the feature similarity value corresponding to each pixel between the first feature map and the second feature map, judging pixels whose feature similarity value is less than the preset similarity threshold as pixels with changes, and obtaining the change feature between the first feature map and the second feature map.

[0078] Optionally, the first branch network and the second branch network respectively adopt the backbone network structure of DeepLabV3Plus; the backbone network structure includes ResNet and a multi-scale dilated spatial pyramid pooling module; the ResNet is used to extract semantic features; and the multi-scale dilated spatial pyramid pooling module is used to extract multi-scale contextual information.

[0079] Optionally, the decoder includes: a self-attention mechanism decoder, which dynamically assigns different attention weights to capture the long-distance dependencies between features in the encoder feature map, thereby enhancing the building change detection model's ability to extract complex features and improving the building change detection model's ability to suppress the background.

[0080] Optionally, the self-attention mechanism decoder includes: a multi-head attention mechanism unit; the multi-head attention mechanism unit is used to segment the input features into multiple independent segmentation features, input each of the segmentation features into the corresponding head respectively, so as to learn the feature representation of different subspaces, obtain the single-head attention feature map corresponding to the output of each head respectively, splice each of the single-head attention feature maps to obtain the splicing result, perform linear transformation on the splicing result, and obtain the multi-head attention feature map as the decoder feature map.

[0081] In this embodiment, by setting the multi-head attention mechanism unit in the self-attention mechanism decoder, the feature representation of the encoder feature map is learned in parallel in different representation subspaces, and the dependencies of the features in the encoder feature map in different aspects are extracted, thereby reducing the misextraction of the changing background area by the building change detection model, effectively suppressing background interference, and enhancing the extraction accuracy of the building change detection model for the building change area in complex scenes.

[0082] Exemplarily, the method of obtaining the single-head attention feature map is as follows:

[0083]

[0084] In the above formula, Attention(Q,K,V) represents the single-head attention feature map; Q represents the query weight matrix of the i-th head; K represents the key weight matrix of the i-th head; V represents the value weight matrix of the i-th head; softmax() is used to normalize the input vector to obtain the normalized weight matrix; d k Indicates the dimension size of the key weight matrix.

[0085] Exemplarily, the method of obtaining the multi-head attention feature map is as follows:

[0086] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O

[0087] In the above formula, MultiHead(Q, K, V) represents the multi-head attention feature map; Concat() is used to concatenate the single-head attention feature maps; W 0 A weight matrix representing a linear transformation.

[0088] Optionally, the method for obtaining the weighted feature map includes:

[0089] The encoder extracts features from the first temporal phase remote sensing image and the second temporal phase remote sensing image respectively, and obtains a first feature map and a second feature map accordingly; performs feature similarity calculation on the first feature map and the second feature map to obtain a change feature map as an encoder feature map;

[0090] The decoder upsamples the encoder feature map to obtain a decoder feature map;

[0091] The multi-view channel adaptive module performs multi-scale and multi-view feature extraction on the decoder feature map to obtain the weighted feature map, including:

[0092] Performing multi-scale feature extraction on the decoder feature map to obtain scale feature maps at different scales;

[0093] Performing multi-view feature extraction on each of the scale feature maps to obtain multi-view feature maps corresponding to each of the scale feature maps; a single multi-view feature map contains global features, local features and edge features;

[0094] Fusing the multi-view feature maps to obtain a fused gain feature map; the image height, image width and channel dimension of the gain feature map are the same as the image height, image width and channel dimension of the decoder feature map;

[0095] The gain feature map is multiplied pixel by pixel by the decoder feature map to obtain the weighted feature map.

[0096] Optionally, the method for acquiring the gain characteristic map includes:

[0097] Each of the multi-view feature maps is spliced ​​along the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is feature transformed so that the image height, image width and channel dimension of the spliced ​​feature map after the feature transformation are the same as the image height, image width and channel dimension of the decoder feature map; the spliced ​​feature map after the feature transformation is used as the gain feature map.

[0098] Optionally, when the first temporal phase remote sensing image and the second temporal phase remote sensing image are RGB images, the channel dimension of each feature map includes three channel dimensions of R, G and B.

[0099] Optionally, after obtaining the spliced ​​feature map after the feature transformation, the method further includes:

[0100] The spliced ​​feature map after the feature transformation is normalized to obtain the normalized spliced ​​feature map as the gain feature map.

[0101] For example, Figure 3 As shown, the multi-view channel adaptive module includes: a multi-scale feature extraction unit, a pooling unit, a feature splicing unit, a fully connected layer and an activation layer connected in sequence; then the method for obtaining the weighted feature map includes:

[0102] The decoder feature map is input into the multi-scale feature extraction unit; the multi-scale feature extraction unit performs multi-scale feature extraction on the decoder feature map to obtain scale feature maps at different scales; the pooling unit performs multi-view feature extraction on each scale feature map to obtain multi-view feature maps corresponding to each scale feature map; the multi-view feature map includes global features, local features and edge features; each multi-view feature map is input into the feature splicing unit, and the feature splicing unit splices each multi-view feature map in the channel dimension. Obtain a spliced ​​feature map; use the fully connected layer to perform feature transformation on the spliced ​​feature map so that the image height, image width and channel dimension of the spliced ​​feature map obtained after the feature transformation are consistent with the image height, image width and channel dimension of the decoder feature map; input the spliced ​​feature map after the feature transformation into the activation layer, and the activation layer normalizes the spliced ​​feature map after the feature transformation to obtain the normalized spliced ​​feature map as the gain feature map; multiply the gain feature map by the decoder feature map pixel by pixel to obtain the weighted feature map.

[0103] Optional, such as Figure 4 As shown, the multi-scale feature extraction unit includes: a plurality of convolution branches; the receptive fields corresponding to the convolution branches are different in size and are respectively connected to the pooling units; the multi-scale feature extraction unit performs multi-scale feature extraction on the decoder feature map to obtain scale feature maps at different scales, including: inputting the decoder feature map into each of the convolution branches respectively; each of the convolution branches performs feature extraction of the corresponding scale on the decoder feature map to obtain the scale feature map output by each of the convolution branches.

[0104] In this embodiment, the convolution branches with different receptive field sizes are used to perform feature extraction on the decoder feature map respectively, so as to extract data features from different scales in the decoder feature map and obtain scale feature maps with different scale details.

[0105] For example, Figure 5 As shown, the multi-scale feature extraction unit includes 4 convolution branches; each of the convolution branches includes at least one first convolution layer; the convolution kernel sizes corresponding to each of the convolution branches are: 1×1, 3×3, 5×5 and 7×7.

[0106] Optionally, the method for acquiring the multi-view feature map includes:

[0107] Performing global average pooling, local perception pooling and edge feature extraction on the scale feature map, respectively, to obtain a global average feature map, several local perception feature maps and an edge feature map;

[0108] In the channel dimension, the global average feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map;

[0109] Among them, the method for obtaining the plurality of local perception feature maps includes: using convolution kernels of several different scales to extract features from the scale feature maps respectively, and correspondingly obtaining a plurality of local perception feature maps.

[0110] For example, Figure 6 As shown, the pooling unit at least includes: a global average pooling layer, a local perception pooling layer and an edge feature extraction operator; then the pooling unit is used to perform multi-view feature extraction on each scale feature map, respectively, to obtain multi-view feature maps corresponding to each scale feature map, and when multi-view feature extraction is performed on a single scale feature map to obtain the corresponding multi-view feature map, it includes:

[0111] The scale feature map is respectively input into the global average pooling layer, the local perception pooling layer and the edge feature extraction operator; the global average pooling layer performs global average pooling on the scale feature map to extract the global average features in the scale feature map to obtain a global average feature map; the local perception pooling layer uses convolution kernels of several different scales to perform local feature extraction on the scale feature map at different scales, and obtains several local perception feature maps accordingly; the edge feature extraction operator performs edge feature extraction on the scale feature map to obtain an edge feature map; in the channel dimension, the global average feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map.

[0112] In this embodiment, the global average pooling, local perception pooling and edge feature extraction are performed on each of the scale feature maps respectively, so as to extract the global average features, local features and edge features at different scales from the scale feature maps at different scales, and obtain the global average feature map, the local perception feature map and the edge feature map; the global average feature map, the local perception feature map and the edge feature map are spliced ​​to obtain a multi-view feature map including the global average feature, local features at different scales and edge detail features, so as to improve the comprehensiveness and detail of the model feature extraction, and improve the accuracy of the model in extracting fine-grained change features of buildings in complex backgrounds.

[0113] Optionally, the edge feature extraction operator includes: an existing edge feature extraction operator such as a Sobel operator.

[0114] Optionally, after obtaining the scale feature map, the method for detecting building change areas further includes: performing global maximum pooling on the scale feature map to obtain a global maximum feature map; then the method for obtaining the multi-view feature map includes:

[0115] In the channel dimension, the global average feature map, the global maximum feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map.

[0116] For example, Figure 7 As shown, the pooling unit also includes: a global maximum pooling layer; then the pooling unit is used to perform multi-view feature extraction on each of the scale feature maps to obtain multi-view feature maps corresponding to each of the scale feature maps, and when multi-view feature extraction is performed on a single scale feature map to obtain the corresponding multi-view feature map, it includes:

[0117] The scale feature map is respectively input into the global average pooling layer, the global maximum pooling layer, the local perception pooling layer and the edge feature extraction operator; the global average pooling layer performs global average pooling on the scale feature map to extract the global average feature in the scale feature map to obtain the global average feature map; the global maximum pooling layer performs global maximum pooling on the global average feature map to extract the global maximum feature in the scale feature map to obtain the global maximum feature map; the local perception pooling layer uses convolution kernels of several different scales to perform feature extraction on the scale feature map respectively, and obtains several local perception feature maps accordingly; the edge feature extraction operator performs edge feature enhancement on the scale feature map to obtain an edge feature map; in the channel dimension, the global average feature map, the global maximum feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map.

[0118] In this embodiment, the global maximum pooling layer is set in the pooling unit to extract the global maximum feature from each of the scale feature maps, respectively, to obtain a global maximum feature map containing the most significant features in the scale feature map; the global average feature map, the global maximum feature map, the local perception feature map and the edge feature map corresponding to each of the scale feature maps are spliced ​​to obtain a multi-view feature map containing the global average feature, the global maximum feature, the local features at different scales and the edge detail features, thereby further improving the comprehensiveness of the model feature extraction and the accuracy of the model in extracting the global change features of the building.

[0119] Optionally, the activation layer adopts a sigmoid activation function.

[0120] Exemplarily, when the activation layer adopts the sigmoid activation function, the gain feature map is obtained as follows:

[0121] ω=sigmoid(WF cat +b)

[0122] In the above formula, ω represents the gain feature map; sigmoid() is used to map the feature value to the range of [0,1]; W represents the weight matrix of the fully connected layer; F cat represents the concatenated feature map after the feature transformation; b represents the bias term.

[0123] Exemplarily, the method of obtaining the weighted feature map is as follows:

[0124]

[0125] In the above formula, represents the weighted feature map; ω represents the gain feature map; e represents element-by-element multiplication; X represents the decoder feature map.

[0126] Exemplarily, the method of obtaining the scale feature map is as follows:

[0127] F i =f i (X)

[0128] In the above formula, F i represents the scale feature map output by the i-th convolution branch; f i (X) represents the i-th convolution branch; X represents the decoder feature map.

[0129] Exemplarily, the method for obtaining the global average feature map is as follows:

[0130]

[0131] In the above formula, GAP i represents the global average feature map obtained based on the scale feature map output by the i-th convolution branch; h represents the height of the scale feature map; w represents the width of the scale feature map; F i (x, y) represents the scale feature map output by the i-th convolution branch.

[0132] Exemplarily, the method for obtaining the global maximum feature map is as follows:

[0133]

[0134] In the above formula, GMP irepresents the global maximum feature map obtained based on the scale feature map output by the i-th convolution branch; h represents the height of the scale feature map; w represents the width of the scale feature map; F i (x, y) represents the scale feature map output by the i-th convolution branch.

[0135] Exemplarily, the method of acquiring the local perception feature map is as follows:

[0136] LPP i，j =Pool sj (F i ), j∈{1,...,k}

[0137] In the above formula, LPP i,j represents the j-th scale convolution kernel in the local perception pooling layer, and the local perception feature map is obtained based on the scale feature map output by the i-th convolution branch; sj represents the scale of the convolution kernel; Pool sj () indicates that the convolution kernel of scale sj is used to extract features from the scale feature map; F i represents the scale feature map output by the i-th convolution branch; k represents the total number of convolution kernels in the local perception pooling layer.

[0138] Exemplarily, the method of obtaining the edge feature map is as follows:

[0139] EEFE i =|G x *F i |+|G y *F i |

[0140] In the above formula, EEFE i represents the edge feature map obtained based on the scale feature map output by the i-th convolution branch; G x represents the edge feature extraction operator in the height direction of the scale feature map; G y represents the edge feature extraction operator in the width direction of the scale feature map; F i Represents the scale feature map output by the i-th convolution branch.

[0141] Exemplarily, the method of acquiring the multi-view feature map is as follows:

[0142] F cat，i =Concat(GAP i , GMP i , LPP i，1 , ..., LPP i，k ,EEFE i )

[0143] In the above formula, F cat,i It represents the multi-view feature map obtained based on the scale feature map output by the i-th convolution branch; Concat() represents splicing in the channel dimension.

[0144] Exemplarily, the method for obtaining the splicing feature map is as follows:

[0145] F cat =Concat(F cat,1 , …, F cat，n )

[0146] In the above formula, F cat represents the concatenated feature map; Concat() represents concatenation in the channel dimension; F cat,n represents the multi-view feature map obtained based on the scale feature map output by the nth convolution branch; n represents the total number of the convolution branches.

[0147] Optional, such as Figure 8 As shown, the building change detection model also includes: a weighted spatial attention mechanism module connected to the multi-view channel adaptation module; the weighted spatial attention mechanism module includes: a channel average pooling layer, a channel maximum pooling layer and a second convolutional layer; then the method for obtaining the output result of the building change detection model includes:

[0148] The weighted feature map is input into the channel average pooling layer and the channel maximum pooling layer respectively; the channel average pooling layer performs average pooling on the weighted feature map in the channel dimension to obtain a channel average pooling map; the channel maximum pooling layer performs maximum pooling on the weighted feature map in the channel dimension to obtain a channel maximum pooling map; in the spatial dimension, the channel average pooling map and the channel maximum pooling map are spliced ​​to obtain a spliced ​​pooling map; the second convolutional layer is used to extract features from the spliced ​​pooling map to obtain a spatial attention feature map; the size of the spatial attention feature map is consistent with the size of the multi-view feature map; the spatial attention feature map is normalized to obtain a spatial attention weight matrix; each weight value in the attention weight matrix is ​​in the range of 0 to 1; each pixel in the multi-view feature map is multiplied by the corresponding weight value in the spatial attention weight matrix to obtain a spatial enhanced feature map as the output result of the building change detection model.

[0149] Optionally, the convolution kernel size of the second convolution layer includes: 7×7.

[0150] Optionally, the spatial dimension is an image height direction or an image width direction.

[0151] Exemplarily, the spatial attention weight matrix is ​​obtained as follows:

[0152] M att =sigmoid(W 7×7 *Concat(CAP, CMP)+b 7×7 )

[0153] In the above formula, M att represents the spatial attention weight matrix; Concat() represents concatenation on the channel dimension; CAP represents the channel average pooling map; CMP represents the channel maximum pooling map; W 7×7 represents the weight matrix of the second convolutional layer; b 7×7 Represents the bias matrix of the second convolutional layer.

[0154] Exemplarily, the method of obtaining the channel average pooling map is as follows:

[0155]

[0156] In the above formula, CAP represents the channel average pooling map; D i represents the i-th channel of the multi-view feature map D; C represents the total number of channels of the multi-view feature map.

[0157] Exemplarily, the method of obtaining the channel maximum pooling map is as follows:

[0158]

[0159] In the above formula, CMP represents the maximum pooling map of the channel; D i represents the i-th channel of the multi-view feature map; C represents the total number of channels of the multi-view feature map.

[0160] Exemplarily, the method for acquiring the spatial enhancement feature map includes:

[0161]

[0162] In the above formula, represents the i-th channel of the spatial enhancement feature map; M att represents the spatial attention weight matrix; e represents pixel-by-pixel multiplication; D i represents the i-th channel of the multi-view feature map; C represents the total number of channels of the multi-view feature map.

[0163] In this embodiment, a weighted spatial attention mechanism module connected to the multi-view channel adaptive module is set in the building change detection model. The weighted spatial attention mechanism module performs average pooling and maximum pooling on the weighted feature map in the channel dimension, respectively, to obtain a channel average pooling map and a channel maximum pooling map correspondingly. In the spatial dimension, the channel average pooling map and the channel maximum pooling map are spliced ​​to obtain a spliced ​​pooling map containing spatial information; feature extraction is performed on the spliced ​​pooling map to generate a spatial attention weight matrix; the multi-view feature map and the spatial attention weight matrix are multiplied pixel by pixel to enhance the expression of spatial information in the multi-view feature map to obtain a spatial enhanced feature map, thereby enhancing the model's ability to extract spatial information and further improving the model's accuracy in extracting building change features.

[0164] Optional, such as Fig. 9 As shown, the training method of the building change detection model includes:

[0165] Step S21, initializing the building change detection model;

[0166] The encoder in the building change detection model adopts the backbone network structure of DeepLabV3Plus; the input size of the building change detection model is 512×512; the batch size is 8; the initial learning rate is 0.01; the total number of training rounds is 100 epochs; and the loss function adopts the cross entropy function. Specifically, the model parameters in the pre-trained ResNet34 are used as the initialization parameters of the backbone network; and the parameters of the decoder are initialized using the He initialization method.

[0167] Step S22: using the Adam optimizer to train the building change detection model, and using a polynomial decay adjustment strategy to dynamically adjust the learning rate of the building change detection model;

[0168] Step S23: after each epoch, the model performance of the building change detection model is evaluated on the validation set, and the model weight with the best performance is saved;

[0169] Step S24, parameter update is implemented through back propagation. When the building change detection model converges or reaches a preset epoch number, the training of the building change detection model is terminated.

[0170] Step S3: based on the output result, obtain a building change probability distribution; based on the building change probability distribution, obtain a detection result of a building change area.

[0171] Specifically, the output result is normalized so that the predicted value corresponding to each pixel in the output result is converted into a probability value, respectively, to obtain the building change probability distribution; based on the building change probability distribution, the detection result of the building change area is obtained.

[0172] Optionally, the probability value is in the range of [0, 1].

[0173] Optionally, the output result is normalized using a Sofmax() function.

[0174] Optionally, obtaining a detection result of a building change area based on the building change probability distribution includes:

[0175] Based on the building change probability distribution, obtaining the building change probability distribution map as the detection result of the building change area; or,

[0176] Based on the building change probability distribution, the building change probability distribution map is obtained; the building change probability distribution map is binarized by using a threshold segmentation method to obtain a building change detection result map as a detection result of the building change area.

[0177] Exemplarily, the method for obtaining the building change probability distribution map is as follows:

[0178]

[0179] In the above formula, P represents the probability distribution diagram of building changes; W cls represents the weight of the classifier; represents the spatial enhancement feature map; b cls represents the bias parameter of the classifier.

[0180] Exemplarily, a method for obtaining the building change detection result corresponding to each pixel in the building change detection result map is as follows:

[0181]

[0182] In the above formula, B(i, j) represents the building change detection result corresponding to pixel (i, j); P(i, j) represents the change probability value corresponding to pixel (i, j); θ represents the preset change probability threshold; 1 represents the building change; 0 represents the building no change.

[0183] Optionally, after obtaining the building change detection result map, the method further includes:

[0184] The building change detection result map is subjected to morphological post-processing to remove noise in the building change detection result map, smooth the building change area boundary in the building change detection result map, and obtain the building change detection result map after morphological processing as the detection result of the building change area.

[0185] Optionally, the morphological post-processing includes: a morphological opening operation and / or a morphological closing operation.

[0186] Exemplarily, the morphological opening operation includes: corroding the building change detection result map to obtain a corroded building change detection result map; and dilating the corroded building change detection result map to obtain a dilated building change detection result map.

[0187] Exemplarily, the morphological closing operation includes: dilating the building change detection result map to obtain an expanded building change detection result map; and corroding the dilated building change detection result map to obtain an corroded building change detection result map.

[0188] In order to verify the performance of the building change area detection method provided in this application in the building change area detection problem, the following experiments were conducted:

[0189] Image preprocessing is performed on each image in the WHU-CD-256 data set, and the WHU-CD-256 data set after image preprocessing is divided into a training set, a test set and a validation set, wherein the training set, the validation set and the test set respectively contain 5947, 743, 744 images of 256×256 size; under the deep learning framework of PyTorch, the building change detection model is trained, tested and validated based on the training set, the test set and the validation set, and the Adam optimizer is used to optimize the training process of the building change detection model, the iterative training is 52 epochs, the initial learning rate is 0.0001, the learning rate in each epoch is decayed by 0.000001, and the batch size is 8.

[0190] The precision, recall and F1 scores were used to evaluate the performance of the detection method based on the building change detection model provided in this application, the detection method based on Unet and the detection method based on DeepLabV3Plus, and the evaluation results corresponding to each method were obtained, as shown in Table 1:

[0191] Table 1

[0192]

[0193] In the above table, precision is the probability that all samples predicted as positive are actually positive samples, which measures the accuracy of the prediction results; recall is the probability that the actual positive samples are predicted as positive samples, which measures how many of the actual positive samples are correctly predicted by the model; F1 score is the harmonic mean of precision and recall, and the higher the value, the better the performance.

[0194] It can be seen from the above table that the detection method for building change areas provided in this application has achieved the best results in the evaluation indicators of precision, recall and F1 score. Compared with the existing Unet-based detection method and DeepLabV3Plus-based detection method, the evaluation indicators of this method have been significantly improved, which fully proves the effectiveness and superiority of this method in the detection problem of building change areas.

[0195] A method for detecting building change areas provided in the above embodiment comprises the following steps: inputting a first phase remote sensing image and a second phase remote sensing image into a pre-constructed building change detection model respectively to obtain an output result of the building change detection model; the output result at least includes multi-scale global features, local features and edge features of the building change area; based on the output result, a building change probability distribution is obtained; based on the building change probability distribution, a detection result of the building change area is obtained. The method performs multi-scale and multi-perspective feature extraction on the building change area in the image, thereby improving the comprehensiveness and detail of feature extraction on the building change area, and realizing high-accuracy extraction of large-area and / or irregular building change areas in complex scenes.

[0196] See also Fig.10 , showing a module structure diagram of a building change area detection system provided by the present invention in one embodiment.

[0197] like Fig.10 As shown, in this embodiment, the detection system 200 of the building change area provided by the present invention includes:

[0198] An acquisition module 201 is used to acquire a first temporal phase remote sensing image and a second temporal phase remote sensing image; the first temporal phase remote sensing image and the second temporal phase remote sensing image are both remote sensing images of a target area;

[0199] The detection module 202 is used to input the first phase remote sensing image and the second phase remote sensing image into a building change detection model respectively to obtain an output result of the building change detection model; the building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptation module; the encoder is a twin network structure; the multi-view channel adaptation module is used to obtain a weighted feature map; the weighted feature map contains multi-scale global features, local features and edge features of the building change area;

[0200] The detection result output module 203 is used to obtain the building change probability distribution based on the output result; and obtain the detection result of the building change area based on the building change probability distribution.

[0201] See also Fig.11 , showing a schematic structural diagram of a computer device provided by the present invention in one embodiment;

[0202] like Fig.11 As shown, in this embodiment, the device 300 provided by the present invention includes a memory 301 and a processor 302, the memory 301 is used to store a computer program; the processor 302 is used to execute the computer program stored in the memory 301, so that the device 300 performs the building change area detection method of any of the above embodiments of the present application. Since the specific implementation process of the steps of the building change area detection method has been described in detail in the above embodiments, it will not be repeated here.

[0203] The memory 301 includes: ROM (Read Only Memory image), RAM (Random Access Memory), a disk, a USB flash drive, a memory card, or an optical disk, etc., which can store program codes.

[0204] The processor 302 is connected to the memory 301 and is used to execute the computer program stored in the memory 301 so that the device 300 executes the above-mentioned method for detecting the changed area of ​​the building.

[0205] A computer-readable storage medium is also provided in the embodiment of the present application. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0206] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0207] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0208] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0209] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for detecting a building change area, characterized in that: The method comprises: Acquire a first temporal phase remote sensing image and a second temporal phase remote sensing image; the first temporal phase remote sensing image and the second temporal phase remote sensing image are both remote sensing images of the target area; respectively inputting the first temporal phase remote sensing image and the second temporal phase remote sensing image into a building change detection model to obtain an output result of the building change detection model; Based on the output result, a building change probability distribution is obtained; based on the building change probability distribution, a detection result of a building change area is obtained; The building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptive module; the multi-view channel adaptive module is used to extract multi-scale features from the decoder feature map output by the decoder to obtain scale feature maps at different scales; Performing multi-view feature extraction on each of the scale feature maps to obtain multi-view feature maps corresponding to each of the scale feature maps; a single multi-view feature map contains global features, local features and edge features; Fusing the multi-view feature maps to obtain a fused gain feature map; the image height, image width and channel dimension of the gain feature map are the same as the image height, image width and channel dimension of the decoder feature map; The gain feature map is multiplied pixel by pixel by the decoder feature map to obtain a weighted feature map.

2. The method according to claim 1, characterized in that The method for obtaining the decoder feature map includes: The encoder extracts features from the first temporal phase remote sensing image and the second temporal phase remote sensing image respectively, and obtains a first feature map and a second feature map accordingly; performs feature similarity calculation on the first feature map and the second feature map to obtain a change feature map as an encoder feature map; The decoder upsamples the encoder feature map to obtain a decoder feature map.

3. The method according to claim 1, characterized in that The step of performing multi-scale feature extraction on the decoder feature map output by the decoder to obtain scale feature maps at different scales includes: Using a plurality of first convolutional layers, features of different scales are extracted from the decoder feature map to obtain scale feature maps of corresponding scales; the convolution kernel sizes of the first convolutional layers are different.

4. The method according to claim 1, characterized in that The method for obtaining the multi-view feature map includes: Performing global average pooling, local perception pooling and edge feature extraction on the scale feature map, respectively, to obtain a global average feature map, several local perception feature maps and an edge feature map; In the channel dimension, the global average feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map; Among them, the method for obtaining the plurality of local perception feature maps includes: using convolution kernels of several different scales to extract features from the scale feature maps respectively, and correspondingly obtaining a plurality of local perception feature maps.

5. The method according to claim 4, characterized in that After obtaining the scale feature map, the method further includes: performing global maximum pooling on the scale feature map to obtain a global maximum feature map; then the method for obtaining the multi-view feature map includes: In the channel dimension, the global average feature map, the global maximum feature map, each of the local perception feature maps and the edge feature map are spliced ​​to obtain the multi-view feature map.

6. The method according to claim 1, characterized in that After obtaining the weighted feature map, the method further includes: In the channel dimension, average pooling and maximum pooling are performed on the weighted feature map respectively, and a channel average pooling map and a channel maximum pooling map are obtained correspondingly; In the spatial dimension, the channel average pooling map and the channel maximum pooling map are spliced ​​to obtain a spliced ​​pooling map; using the second convolutional layer, feature extraction is performed on the spliced ​​pooling map to obtain a spatial attention feature map; The spatial attention feature map is normalized to obtain a spatial attention weight matrix; the multi-view feature map and the spatial attention weight matrix are multiplied pixel by pixel to obtain a spatial enhancement feature map as an output result of the building change detection model.

7. The method according to claim 1, characterized in that The obtaining the detection result of the building change area based on the building change probability distribution includes: Based on the building change probability distribution, obtaining a building change probability distribution map as a detection result of the building change area; or, Based on the building change probability distribution, a building change probability distribution map is obtained; and threshold segmentation is performed on the building change probability distribution map to obtain a detection result of the building change area.

8. A detection system for building change areas, characterized in that: include: An acquisition module, used for acquiring a remote sensing image of a first time phase and a remote sensing image of a second time phase; The first temporal phase remote sensing image and the second temporal phase remote sensing image are both remote sensing images of the target area; A detection module, used to input the first temporal phase remote sensing image and the second temporal phase remote sensing image into a building change detection model respectively, to obtain an output result of the building change detection model; The building change detection model at least includes: an encoder, a decoder and a multi-view channel adaptive module; the multi-view channel adaptive module is used to perform multi-scale feature extraction on the decoder feature map output by the decoder to obtain scale feature maps at different scales; multi-view feature extraction is performed on each scale feature map to obtain multi-view feature maps corresponding to each scale feature map; a single multi-view feature map contains global features, local features and edge features; each multi-view feature map is fused to obtain a fused gain feature map; the image height, image width and channel dimension of the gain feature map are the same as the image height, image width and channel dimension of the decoder feature map; the gain feature map is multiplied pixel by pixel with the decoder feature map to obtain a weighted feature map; The detection result output module is used to obtain the building change probability distribution based on the output result; and obtain the detection result of the building change area based on the building change probability distribution.

9. A computer device, characterized in that: The device comprises: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory so that the device performs the method for detecting a building change area according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the method for detecting a building change area according to any one of claims 1 to 7 is implemented.

Images