A time-domain attention-based time-series remote sensing image building change monitoring method and system

Abstract

The present application provides a kind of time domain attention-based time sequence remote sensing image building change monitoring method and system.For the problem that the apparent style difference between different time images is large and the target changes a lot, the method introduces time domain attention in remote sensing single image segmentation model, and replaces normalization layer to alleviate the problem that style difference leads to insufficient fusion of time sequence information.At the same time, change detection auxiliary loss is added to inject additional supervision information to guide the model to learn the change pattern of buildings in remote sensing time sequence images.After the model completes semantic segmentation, a post-processing module is added to realize building instance segmentation and change monitoring.The instance segmentation is realized by improved watershed algorithm, and the change monitoring is realized by sliding average sequence.The method can effectively utilize time sequence information, thereby realizing more accurate remote sensing time sequence building extraction and change monitoring.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a time-series remote sensing image building change monitoring method and system based on time domain attention. BACKGROUND

[0002] With the continuous growth of the world population and the continuous expansion of the economic scale, increasing human activities are driving changes in land cover, among which building land is an important type of land use. Since buildings are closely related to human activities, timely and accurate understanding of the changes of buildings such as new construction, demolition and expansion is of great significance to urban planning, urban management, urban security, population estimation and disaster rescue. Although field surveys can accurately obtain the information of buildings, they are time-consuming, labor-intensive and costly, and it is difficult to achieve full coverage. In contrast, satellite remote sensing images are a reliable, efficient and full-coverage means of building change monitoring, which can support rapid extraction of buildings and monitoring and analysis.

[0003] The current time-series remote sensing image building extraction and change monitoring faces the following challenges: in time-series remote sensing images, the apparent feature differences between different time phases are large, and the target changes are various, which makes it difficult to fully utilize the information of the time dimension. SUMMARY

[0004] To solve the above problems, the application provides an improved time domain attention method, which uses an instance normalization layer to eliminate the apparent style differences between time phases, and introduces a change detection auxiliary task to provide additional supervision information, thereby improving the prediction accuracy of the model.

[0005] In view of the deficiencies in the prior art, the application aims to provide a time-series remote sensing image building change monitoring method based on time domain attention, which can eliminate the apparent style differences between time phases to some extent and effectively utilize the time-series information, thereby improving the accuracy of building extraction.

[0006] The time-series remote sensing image building change monitoring method based on time domain attention provided by the application comprises the following specific steps:

[0007] Step 1, data set preparation, including registered time-series remote sensing images and manually annotated building masks;

[0008] Step 2, training a time-series building segmentation model with the prepared data set, the input of the model is a time-series segment, each segment is composed of a plurality of registered multi-time phase remote sensing images, and the building segmentation model is divided into three parts, namely a backbone network, a time domain Transformer and a segmentation head;

[0009] The backbone network extracts features of each image in the time sequence segment, the time domain Transformer performs time domain self-attention on all images in the time sequence segment, and fuses the features of all time phases, and the segmentation head processes each image in the time sequence segment respectively to output the final building semantic segmentation result.

[0010] Step 3, calculate the loss function of the model, which includes two parts: semantic segmentation loss and change detection loss.

[0011] Step 4, after training the time sequence building segmentation model, predict the building probability map of all time phases in the time sequence using the model, and compress all the predicted results of all time phases into a probability map S ij .

[0012] Step 5, convert the compressed building probability map S ij into building instance segmentation results to determine the shape and position of the building.

[0013] Step 6, after obtaining the shape and position of each building instance in step 5, determine the probability of the building appearing in each time phase.

[0014] First, calculate the appearance probability of each building instance in each time phase wherein represents the prediction result of the position with horizontal and vertical coordinates (i, j) in the image of the t-th time phase, represents the l-th building instance, and |·| represents the number of elements in the set, represents the appearance probability of the l-th building instance in the t-th time phase.

[0015] Step 7, calculate the forward and reverse sliding average sequences of the appearance probability of the building in each time phase.

[0016] Step 8, determine whether the state of the building in the time sequence changes according to the two sliding average sequences calculated in step 7.

[0017] Further, the backbone network is ResNet18-V1c, the number of attention modules in the time domain Transformer is 1, the number of multi-heads is 1, and the segmentation head is the Atrous Spatial Pyramid Pooling (ASPP) module used by DeepLab v3. In addition, the first batch normalization layer in the backbone network and the first layer normalization layer in the time domain Transformer are replaced by an instance normalization layer.

[0018] Further, the formula of the semantic segmentation loss is wherein H and W represent the height and width of the image; C represents the number of semantic segmentation categories; P and Y represent the prediction result of the model and the labeled true value respectively, Pij denotes the predicted class probability of pixel with coordinate (i, j) in the image, and the prediction result is the class probability; ω ij denotes the weight of pixel with coordinate (i, j), ω ij takes 2, otherwise takes 1; meanwhile, the change detection loss is calculated for each two images in the time series, and the prediction results of the two time phases are denoted as P 1 and P 2 , respectively, then the change detection loss is where θ ij denotes the weight of pixel with coordinate (i, j), θ ij takes -10, otherwise takes 1; ||·||1 denotes the L1 norm.

[0019] Further, the calculation formula of the probability map is where denotes the prediction result of the position with coordinate (i, j) in the image of the t-th time phase; is an indicator function, which takes 1 when the condition in the parentheses is true, otherwise 0; ∈ is used to prevent division by zero error, and α is the prediction probability threshold, t = 1, 2, … N t denotes the time phase coordinate, where N t denotes the total number of time phases.

[0020] Further, M 0 in step 5 denotes the region where S ij > β h , that is, if S ij > β h , otherwise Similarly, M 1 denotes the region where S ij > β l , where β h and β l are constants, and based on the opposite number -S of the probability map S, M 0 is used as a seed, and the watershed algorithm is used to segment each M 1 = 1 region to obtain the shape and position of all building instances.

[0021] Further, the calculation formula of the forward moving average sequence in step 7 is The calculation formula of the reverse moving average sequence is

[0022] Further, the specific implementation manner of determining whether the state of the building in the time series changes in step 8 is as follows;

[0023] if determine whether the state of the building in the time series has changed, 0 indicating that the building does not exist, and 1 indicating that the building exists, the state of the lth building in the tth time phase The formula can be used determined; if determine whether the state of the building in the time series has changed, 0 indicating that the building does not exist, and 1 indicating that the building exists, the state of the lth building in the tth time phase The formula can be used determined, wherein is the time point at which the building appears, inf represents the minimum value, and γ d and γ m are constants.

[0024] Further, it further comprises the step of using precision, recall, IoU, Boundary IoU and SCOT as evaluation indexes to evaluate the detection effect.

[0025] The application also provides a time-domain attention-based time-series remote sensing image building change monitoring system, comprising the following modules.

[0026] A data preparation module, a data set is prepared, comprising registered time-series remote sensing images and artificially annotated building masks;

[0027] A model construction module, a time-series building segmentation model is trained using the prepared data set, the input of the model is a time-series segment, each segment is composed of a plurality of registered multi-temporal remote sensing images, the building segmentation model is divided into three parts, namely a backbone network, a time-domain Transformer and a segmentation head;

[0028] The backbone network extracts the features of each image in the time-series segment, respectively; the time-domain Transformer performs time-domain self-attention on all images in the time-series segment, and fuses the features of all time phases; the segmentation head processes each image in the time-series segment again, and outputs the final building semantic segmentation result;

[0029] A loss function calculation module, a loss function of the model is calculated, the loss function comprises two parts: a semantic segmentation loss and a change detection loss;

[0030] A probability map calculation module, after the time-series building segmentation model is trained, the model is used to predict the building probability map of all time phases in the time series, and all predicted results of the time phases are compressed into a probability map S ij ;

[0031] A building shape and position determination module, the compressed building probability map S ij is converted into a building instance segmentation result, and the shape and position of the building are determined;

[0032] a probability determination module, which determines the probability of the appearance of the building in each time phase after obtaining the shape and position of each building instance;

[0033] First, the probability of the appearance of each building instance in each time phase is calculated wherein represents the prediction result of the position with the horizontal and vertical coordinates of (i, j) in the image of the t-th time phase, represents the l-th building instance, and |·| represents the number of elements in the set, represents the probability of the appearance of the l-th building instance in the t-th time phase;

[0034] an average sequence calculation module, which calculates the forward sliding average sequence and the reverse sliding average sequence of the probability of the appearance of the building in each time phase;

[0035] a change monitoring module, which determines whether the state of the building in the time sequence changes according to the two calculated sliding average sequences.

[0036] Further, the formula of the semantic segmentation loss is wherein H and W represent the height and width of the image; C represents the number of semantic segmentation categories; P and Y represent the prediction result and the labeled true value of the model respectively, P ij represents the category probability predicted by the model for the pixel with the coordinates of (i, j) in the image, and the prediction result is the category probability; ω ij represents the weight of the pixel with the coordinates of (i, j), when there is a change in the time sequence, ω ij takes 2, otherwise takes 1; at the same time, the change detection loss is calculated for each two images in the time sequence segment, and the prediction results of the two time phases are respectively P 1 and P 2 , then the change detection loss is wherein θ ij represents the weight of the pixel with the coordinates of (i, j), when the pixel belongs to the change category, θ ij takes -10, otherwise takes 1; ||·||1 represents the L1 norm.

[0037] The principle of the present application is to use improved timing attention to fuse and enhance the features extracted from each multi-temporal image, and to introduce a change detection auxiliary task to guide the model to learn the change pattern prior in remote sensing time series, thereby improving the accuracy of the model in predicting buildings. The present application provides a time series remote sensing image building change monitoring method based on time domain attention. In view of the large difference in image appearance style between different time phases and the problem of multiple target changes, the method introduces time domain attention in the remote sensing single image segmentation model, and replaces the normalization layer to alleviate the problem of insufficient fusion of timing information caused by style difference. At the same time, a change detection auxiliary loss is added to inject additional supervision information to guide the model to learn the change pattern of buildings in remote sensing time series images. After the model completes semantic segmentation, a post-processing module is added to realize instance segmentation and change monitoring of buildings. The instance segmentation is realized by an improved watershed algorithm, and the change monitoring is realized by a sliding average sequence. The method can effectively utilize the timing information to realize more accurate remote sensing time series building extraction and change monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 For an example of registered satellite remote sensing images and building masks, in the building mask diagram, white represents buildings, and black represents non-buildings.

[0039] Figure 2 For a structure diagram of a time series building segmentation model. DETAILED DESCRIPTION

[0040] The present application provides a time series remote sensing image building extraction and change monitoring method based on time domain attention. In order to make the purpose, technical scheme and effect of the present application more clear and explicit, the present application will be further described in detail below with reference to the drawings.

[0041] The time series remote sensing image building change monitoring method based on time domain attention provided by the embodiment of the present application comprises the following steps:

[0042] Step 1: data set preparation, including registered time series remote sensing images and manually annotated building masks, as shown in Figure 1 , wherein each remote sensing image corresponds to a building mask. The building mask can be manually annotated or obtained through GIS information.

[0043] Step 2: training the time series building segmentation model with the prepared data set. The input of the model is a time series segment, each segment consisting of a plurality of registered multi-temporal remote sensing images. The structure of the building segmentation model is as shown in Figure 2As shown, it mainly consists of three parts: a backbone network, a temporal Transformer, and a segmentation head. The backbone network extracts features from each image in the temporal segment; the temporal Transformer performs temporal self-attention on all images in the temporal segment, fusing features from all time phases; the segmentation head then processes each image in the temporal segment separately, outputting the final building semantic segmentation result (only two categories: building and background). The backbone network is ResNet18-V1c, the temporal Transformer has one attention module and one multi-head, and the segmentation head is the Spatial Pyramid Pooling (ASPP) module used in DeepLab v3. Furthermore, the first batch normalization layer in the backbone network and the first normalization layer in the temporal Transformer are replaced with instance normalization layers.

[0044] Step 3: Calculate the model's loss function. The loss function consists of two parts: (1) semantic segmentation loss; (2) change detection loss. The formula for the semantic segmentation loss is as follows: Where H and W represent the height and width of the image; C represents the number of semantic segmentation categories, which is 2; P and Y represent the model's prediction result (category probability) and the labeled ground truth, respectively. ij ω represents the class probability predicted by the model for the pixel at coordinates (i,j) in the image; ij This represents the weight of the pixel at coordinates (i,j). When this pixel changes within the time series, ω... ij Select 2, otherwise select 1. Simultaneously, calculate the change detection loss for every two images within the time series segment, and denote the prediction results for the two time phases as P0. 1 and P 2 Then the change detection loss Where θ ij θ represents the weight of the pixel at coordinates (i,j), when the pixel belongs to the change class. ij Take -10, otherwise take 1; ||·||1 represents the L1 norm, and the model iterates a total of 40,000 times.

[0045] Step 4: After training the time-series building segmentation model, use the model to predict the building probability map for all time phases within the time series, and then compress the prediction results for all time phases into a single probability map. in This represents the predicted position of (i,j) in the image at time t. This is an indicator function; its value is 1 when the condition within the parentheses is true, and 0 otherwise; ∈ is used to prevent division by zero errors and is set to 10. -8 α is the prediction probability threshold, set to 0.1; t = 1, 2, ... N t Represents the phase coordinates, where N t Indicates the total number of time phases.

[0046] Step 5: Convert the compressed building probability map S ij into building instance segmentation results to determine the shape and location of buildings. Let M 0 represent the region of all S ij > β h , that is, if S > β Similarly, let M 1 represent the region of all S ij > β l , where β h = 0.6, β l = 0.4. Based on the inverse number -S of the probability map S obtained in the foregoing, let M 0 be the seed, and the shape and location of all building instances are obtained by using the watershed algorithm in each region of M 1 = 1, respectively.

[0047] Step 6: After obtaining the shape and location of each building instance in step 5, it is necessary to determine the probability of the appearance of the building in each time phase. First, the appearance probability of each building instance in each time phase is calculated wherein represents the lth building instance, |·| represents the number of elements in the set, represents the appearance probability of the lth building instance in the tth time phase.

[0048] Step 7: Calculate the forward sliding average sequence of the appearance probability of the building in each time phase and the reverse sliding average sequence

[0049] Step 8: Determine whether the state of the building in the time sequence changes according to the two sliding average sequences calculated in step 7, and only consider the case where the change occurs only once, that is, the building is newly built or demolished at most, and is not demolished and rebuilt or built and demolished. If it is determined that the state of the building in the time sequence does not change, 0 is represented as the building does not exist, and 1 is represented as the building exists, and then the state of the lth building in the tth time phase can be determined by the formula If it is determined that the state of the building in the time sequence changes, then can be determined by the formula wherein is the time point of the building appearance, inf represents the minimum value, γ d = 0.5, γ m = 0.4.

[0050] The method is verified on the SpaceNet7 dataset, which contains 101 sequences of ortho-rectified satellite remote sensing images, each time sequence contains 18-26 images. The data set was taken from 2017 to 2020. The resolution of the data set is about 4m, and the size of each image is 1024x1024. Since only 60 time sequences in this data set provide labeled true values, only these 60 time sequences are used for experiments, and specific examples are shown in Figure 1 .

[0051] During the experiment, precision, recall, IoU (Intersection over Union), Boundary IoU (BIoU) and SCOT (SpaceNet Change and Object Tracking) are used as evaluation indexes of building extraction. In the pixel sample, define TP as the number of true examples, FP as the number of false positives, and FN as the number of false negatives, then precision = TP / (TP+FP), recall = TP / (TP+FN), IoU = TP / (TP+TN+FP). Where G represents the mask true value, P represents the mask predicted by the model, G d represents the pixel area at the boundary of the true value instance, P d represents the pixel area at the boundary of the predicted instance.

[0052] The SCOT score is an instance-level evaluation index, which considers both tracking and change detection scores, and is calculated by formula , where β = 2. The tracking item score Where TP s , FP s , FN s have the same meaning as TP, FP, FN, but change from pixel samples to instance samples. If the IoU between the instance predicted by the model and the instance in the labeled true value is greater than 0.25, it is considered as TP s . Similarly, Here TP new , FP new , FN new only consider newly appearing building instances.

[0053] The experiment compares the CFFM (Coarse-To-Fine Feature Mining) method and the MRCFA (Mining Relations among Cross-Frame Affinities) method, both of which are relatively new methods in video semantic segmentation. In addition, the experiment also compares the method of using only a depth model for single-image segmentation instead of time-series segmentation (steps 1-3).

[0054] The experimental results are shown in Table 1. It can be seen that among all the compared methods, the method proposed in the present application is better than the other compared methods in all comprehensive indicators, including the pixel indicators representing the accuracy of the building mask and the SCOT indicators representing the tracking of existing buildings and the detection of changes. The present application can effectively utilize time-series information in the case of large differences in apparent style between different temporal images, and achieve high-precision building extraction and change monitoring of time-series remote sensing images.

[0055] Table 1 Comparison of results of different methods

[0056] Method precision recall IoU mIoU SCOT BIoU single-image segmentation 66.94 41.69 34.57 65.00 36.63 31.64 CFFM 74.35 37.83 33.46 64.55 34.71 31.19 MRCFA 74.09 3611 32.06 63.81 33.91 30.32 the method herein 68.46 44.82 37.15 66.37 38.42 34.86

[0057] In specific implementation, the method proposed in the technical solution of the present application can be automatically run by a person skilled in the art using computer software technology, and the system device of the method, such as a computer readable storage medium storing the corresponding computer program of the technical solution of the present application and a computer device including the running of the corresponding computer program, should also be within the protection scope of the present application.

[0058] The embodiment of the present application also provides a time-series remote sensing image building change monitoring system based on time-domain attention, which comprises the following modules:

[0059] A data preparation module, a data set is prepared, including time-series remote sensing images that are well registered and artificial annotated building masks;

[0060] A model construction module, a time-series building segmentation model is trained using the prepared data set, the input of the model is a time-series segment, each segment is composed of a plurality of registered multi-temporal remote sensing images, the building segmentation model is divided into three parts, which are a backbone network, a time-domain Transformer and a segmentation head;

[0061] The backbone network extracts the features of each image in the time-series segment, respectively; the time-domain Transformer performs time-domain self-attention on all images in the time-series segment, and fuses the features of all time phases; the segmentation head processes each image in the time-series segment, respectively, and outputs the final building semantic segmentation result;

[0062] The loss function calculation module calculates the loss function of the model, which includes two parts: semantic segmentation loss and change detection loss.

[0063] The probability map calculation module, after training the time-series building segmentation model, predicts the building probability map of all time phases in the time series using the model, and compresses all the predicted results of the time phases into a probability map S ij .

[0064] The building shape and position determination module converts the compressed building probability map S ij into a building instance segmentation result, and determines the shape and position of the building.

[0065] The probability determination module, after obtaining the shape and position of each building instance, then determines the probability of the building appearing in each time phase.

[0066] First, the probability of each building instance appearing in each time phase is calculated wherein represents the prediction result of the position with horizontal and vertical coordinates (i, j) in the image of the tth time phase, represents the lth building instance, and |·| represents the number of elements in the set, represents the probability of the lth building instance appearing in the tth time phase.

[0067] The average sequence calculation module calculates the forward sliding average sequence and the reverse sliding average sequence of the probability of the building appearing in each time phase.

[0068] The change monitoring module determines whether the state of the building in the time series changes according to the two calculated sliding average sequences.

[0069] The specific implementation of each module corresponds to each step, and the present application will not be described.

[0070] It should be understood that the parts not described in detail in the specification are all prior art.

[0071] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or replace them with similar ways, without deviating from the spirit of the present application or exceeding the scope defined by the appended claims.

Claims

1. A time-domain attention-based building change monitoring method for time-series remote sensing images, characterized in that, Comprising the following steps: Step 1, data set preparation, including registered time series remote sensing images and manually annotated building masks; Step 2, training a time series building segmentation model with the prepared data set, the input of the model is a time series segment, each segment is composed of several registered multi-temporal remote sensing images, the building segmentation model is divided into three parts, respectively, backbone network, time domain Transformer and segmentation head; Wherein the backbone network extracts the features of each image in the time series segment; The time domain Transformer performs time domain self-attention on all images in the time series segment, and fuses the features of all time phases; The segmentation head processes each image in the time series segment respectively, and outputs the final building semantic segmentation result; Step 3, calculating the loss function of the model, the loss function includes two parts: semantic segmentation loss and change detection loss; Step 4, after training the time-series building segmentation model, use the model to predict the building probability map of all time phases in the time series, and compress all the prediction results into a probability map S ij ; Step 5, converting the compressed building probability map S ij into a building instance segmentation result, to determine the shape and location of the building; Step 6, after obtaining the shape and position of each building instance in step 5, the probability of the appearance of the building in each time phase is determined; First, the appearance probability of each building instance in each time phase is calculated where represents the prediction result of the position with horizontal and vertical coordinates (i,j) in the image of the tth time phase, represents the lth building instance, |·| represents the number of elements in the set, represents the appearance probability of the lth building instance in the tth time phase; Step 7, calculating the forward sliding average sequence and reverse sliding average sequence of the appearance probability of the building in each time phase; Step 8, determining whether the state of the building in the time series changes according to the two sliding average sequences calculated in step 7.

2. The time-domain attention-based time-series remote sensing image building change monitoring method according to claim 1, wherein: The backbone network is ResNet18-V1c, the number of attention modules in the time domain Transformer is 1, the number of multi-head is 1, and the segmentation head is the ASPP module used by DeepLab v3; In addition, the first batch normalization layer in the backbone network and the first layer normalization layer in the time domain Transformer are replaced by an instance normalization layer. 3.The time-series remote sensing image building change monitoring method based on time-domain attention of claim 1, wherein: The formula of semantic segmentation loss is where H and W represent the height and width of the image; C represents the number of semantic segmentation categories; P and Y represent the prediction result and the labeled true value of the model respectively, ij represents the category probability predicted by the model for the pixel with coordinates (i, j) in the image, and the prediction result is the category probability; ω ij represents the weight of the pixel with coordinates (i, j), when there is a change in the time sequence, ω ij takes 2, otherwise takes 1; at the same time, the change detection loss is calculated for each two images in the time sequence segment, and the prediction results of the two time phases are respectively P 1 and P 2 , then the change detection loss is where θ ij represents the weight of the pixel with coordinates (i, j), when the pixel belongs to the change category, θ ij takes -10, otherwise takes 1; ||·||1 represents the L1 norm. 4.The time-series remote sensing image building change monitoring method based on time-domain attention of claim 1, wherein: The calculation formula of the probability map is wherein represents the prediction result of the position with horizontal and vertical coordinates (i, j) in the image of the tth time phase; is an indicator function, whose value is 1 when the condition in the parentheses is true, and 0 otherwise; ∈ is used to prevent division by zero error, and α is a prediction probability threshold, t = 1, 2, … N t represents the time phase coordinates, wherein N t represents the total number of time phases.

5. The time-domain attention-based time-series remote sensing image building change monitoring method of claim 1, wherein: M 0 represents the region of all S ij > β h , i.e. if S ij > β h , else M 1 represents the region of all S ij > β l , where β h and β l are constants, based on the inverse number -S of the probability map S, with M 0 as seeds, the shape and location of all building instances are obtained by respectively segmenting each M 1 = 1 region with the watershed algorithm.

6. The time-domain attention-based time-series remote sensing image building change monitoring method of claim 1, wherein: The calculation formula of the forward moving average sequence in step 7 is The calculation formula of the reverse average sequence is 7. The time-domain attention-based time-series remote sensing image building change monitoring method of claim 1, wherein: The specific implementation mode of determining whether the state of the building in the time series changes in step 8 is as follows: If the state of the building in the time series does not change, 0 is assigned to the building not existing and 1 to the building existing, then the state of the 1th building in the tth time phase is determined by the formula If the state of the building in the time series changes, then the state of the 1th building in the tth time phase is determined by the formula wherein is the time point of the building appearing, inf denotes the minimum value and γ d and γ m are constants. 8.The time-domain attention-based time-series remote sensing image building change monitoring method of claim 1, wherein: Further comprising step 9, using precision, recall, IoU, Boundary IoU and SCOT as evaluation indexes to evaluate the detection effect. 9.A time-domain attention-based time-series remote sensing image building change monitoring system, characterized in that, Comprising the following modules: Data preparation module, data set preparation, including registered time series remote sensing images and manually annotated building masks; Model building module, training a time series building segmentation model with the prepared data set, the input of the model is a time series segment, each segment is composed of several registered multi-temporal remote sensing images, the building segmentation model is divided into three parts, respectively, backbone network, time domain Transformer and segmentation head; Wherein the backbone network extracts the features of each image in the time series segment; The time domain Transformer performs time domain self-attention on all images in the time series segment, and fuses the features of all time phases; The segmentation head processes each image in the time series segment respectively, and outputs the final building semantic segmentation result; Loss function calculation module, calculating the loss function of the model, the loss function includes two parts: semantic segmentation loss and change detection loss; a probability map calculation module, after the time-series building segmentation model is trained, predicting a building probability map of all time phases in the time series by using the model, and compressing the prediction results of all time phases into one probability map S ij ; The building shape and position determination module converts the compressed building probability map S ij into a building instance segmentation result, and determines the shape and position of the building. Probability determination module, after obtaining the shape and position of each building instance, the probability of the appearance of the building in each time phase is determined; First, the appearance probability of each building instance in each time phase is calculated where represents the prediction result of the position with horizontal and vertical coordinates (i,j) in the image of the tth time phase, represents the lth building instance, |·| represents the number of elements in the set, represents the appearance probability of the lth building instance in the tth time phase; a mean sequence calculation module, which calculates a forward sliding mean sequence and a reverse sliding mean sequence of the occurrence probability of the building in each time phase; a change monitoring module, which determines whether the state of the building in the time sequence changes according to the two calculated sliding mean sequences.

10. The time-domain attention-based time-series remote sensing image building change monitoring system of claim 9, wherein: The formula of semantic segmentation loss is where H and W represent the height and width of the image; C represents the number of semantic segmentation categories; P and Y represent the prediction result and the labeled true value of the model respectively, ij represents the category probability predicted by the model for the pixel with coordinates (i, j) in the image, and the prediction result is the category probability; ω ij represents the weight of the pixel with coordinates (i, j), when there is a change in the time sequence, ω ij takes 2, otherwise takes 1; at the same time, the change detection loss is calculated for each two images in the time sequence segment, and the prediction results of the two time phases are respectively P 1 and P 2 , then the change detection loss is where θ ij represents the weight of the pixel with coordinates (i, j), when the pixel belongs to the change category, θ ij takes -10, otherwise takes 1; ||·||1 represents the L1 norm.

Images