An outdoor change monitoring method based on gradual change and mutation

Abstract

The present invention belongs to the technical field of image processing, and discloses an outdoor change monitoring method based on gradual change and mutation. For outdoor change monitoring in data with spatial-temporal multi-dimensions, a temporal interactive image pair encoder based on a window coupling attention mechanism is used to extract dual-temporal mutation features; the change temporal evolution is simulated by interpolation in time series to capture the gradual change features in the image sequence; finally, the fusion of multi-scale features is completed by a simple encoder operation, improving the model's attention to the changed area so as to accurately monitor outdoor change targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an outdoor change monitoring method based on gradual change and sudden change. Background Art

[0002] Outdoor change monitoring addresses issues such as open environments, poor lighting, and potential safety hazards by identifying changes through time-series detection. As a key technology for monitoring regional conditions, outdoor change monitoring plays a significant role in numerous tasks, including security monitoring, outdoor environmental perception, and assisted driving. Routine urban surveillance often involves harsh weather conditions or poor lighting, creating an urgent need for fast, stable, and robust methods to process multi-dimensional spatial and temporal outdoor images.

[0003] The traditional outdoor change monitoring method based on manually extracted features does not have an ideal representation effect. However, the use of deep learning networks can capture abstract features through nonlinear operations, thus having stronger representation. For example, convolutional neural networks (CNNs) can extract local features from data in a sliding window manner, while Transformers rely on self-attention mechanisms to dynamically calculate global context. Although CNNs can focus on local information, they ignore global dependencies; in contrast, Transformers capture similar global features at each layer, resulting in redundant representation. Therefore, in this invention, we propose to construct a coupled global attention distribution on the feature map captured by CNN for sufficient local and global representation. At the same time, in this project, we propose to use the mutation features between the two temporal states and the evolution process on the temporal state, so that the method's attention is more focused on the features of the changed areas and the changes, thereby further improving the performance of outdoor change monitoring. Summary of the invention

[0004] The purpose of the present invention is to provide an outdoor change monitoring method based on gradual and sudden changes, which performs outdoor change monitoring in multiple spatial and temporal dimensions. The sudden change features in image pairs are interactively modeled by coupled attention, and the gradual change features in image sequences are extracted by using a large spatial kernel selection mechanism to simulate the temporal evolution process of the changes. Finally, the gradual and sudden change features are fused through a simple decoder to improve the monitoring accuracy and generalization performance.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A method for monitoring outdoor changes based on gradual and sudden changes, comprising the following steps:

[0007] S1. Process the image pairs T1 and T2 at different times in the same outdoor scene into a gradient image sequence S through the temporal interpolation method. T1 is the image at the first time in the image pair, and T2 is the image at the second time in the image pair;

[0008] S2. Use a pre-trained convolutional neural network model to extract 4 feature maps F1 of different scales from the image pairs T1 and T2 respectively l and l ∈ [1, 4], the feature map F1 l is the l-th feature map extracted from the image T1, and the feature map is the l-th feature map extracted from the image T2;

[0009] S3. Through the temporal interactive image pair encoder based on the window coupling attention mechanism, perform coupled modeling on the feature maps F1 of each scale l and respectively to capture mutation features. The operation of the temporal interactive image pair encoder based on the window coupling attention mechanism is as follows:

[0010] S31. Supplement the position encoding information for the feature maps F1 of each scale l and respectively;

[0011] S32. Divide the feature maps F1 with the position encoding information supplemented l and into non-overlapping and same-sized small blocks and to obtain the small block sets corresponding to each feature map. The small block is obtained after the division of the feature map F1 l , and the small block is obtained after the division of the feature map ;

[0012] S33. Concatenate the small blocks and in the small block sets along the channel dimension, and perform linear expansion after convolution operation to obtain the dual-temporal joint query representation

[0013] S34. Downsample the small blocks and in the small block sets at the rates of {2, 4, 8, 16} respectively to obtain multi-scale feature blocks of different spatial sizes. Linearly expand all the multi-scale feature blocks belonging to the same feature map and then perform feature splicing at each scale to obtain the key-value pair representation groups and

[0014] S35. Represent through the dual-temporal joint query in step S33 and the key-value pair representation group in step S34 and respectively calculate the coupled spatial attention to obtain the global attention output for each temporal state and

[0015] S36. Further encode the global attention output through a convolutional layer with shared parameters and and calculate the absolute error between the output features of the convolutional layer with shared parameters to obtain the mutation feature map

[0016] S4. Explore the changing gradual process in the gradual image sequence S through an image sequence encoder based on a large spatial kernel selection module. The operation of the image sequence encoder based on the large spatial kernel selection module is as follows:

[0017] S41. Reshape the four-dimensional gradual image sequence S into a three-dimensional gradual image sequence S′, and the channel dimension of the gradual image sequence S′ contains the changing temporal evolution;

[0018] S42. Perform 4-fold spatial downsampling on the gradual image sequence S′ through convolutional operations to obtain the temporal evolution features;

[0019] S43. Encode the temporal evolution features using three consecutive large spatial kernel selection modules, and add convolutional operations for 2-fold spatial downsampling of the features between every two large spatial kernel selection modules to obtain 3 different-scale gradual feature maps output by the three large spatial kernel selection modules s ∈ [1, 3];

[0020] S5. Concatenate the mutation feature maps and the gradual feature maps along the channel dimension to obtain 4-scale difference maps

[0021] S6. Obtain the fused difference map according to the 4-scale difference maps ;

[0022] S7. Uniform the size of the fused difference map to the size of the image pair through interpolation upsampling, and set the pixel values of the changing region and the non-changing region in the fused difference map through a convolutional layer and a Sigmoid function to achieve outdoor change monitoring.

[0023] The following also provides several optional methods, which are not additional limitations to the above overall solution, but merely further supplements or optimizations. On the premise of no technical or logical contradictions, each optional method can be combined with the above overall solution separately, or multiple optional methods can be combined with each other.

[0024] Preferably: in step S1, the image pairs T1 and T2 at different times in the same outdoor scene are processed into a gradient image sequence S by a temporal interpolation method, where t is the number of frames of the gradient image sequence S, and the calculation formula for each frame in the temporal dimension of the t frames is as follows:

[0025] Frame j = T1 + (T2 - T1) * (j - 1) / t - 1, j ∈ [1, t]

[0026] In the formula, Frame j is the j-th frame in the temporal dimension. When j = 1, this frame is T1; when j = t, this frame is T2.

[0027] Preferably: in step S2, the convolutional neural network model is the GhostNetV2 model or the Transformer model, and the sizes of the 4 different-scale feature maps output by the convolutional neural network model are respectively and where, W is the width, H is the height, and d1, d2, d3, and d4 are the channel depths.

[0028] Preferably: in step S31, the position encoding information is supplemented for the feature maps F1 l and The calculation formula is as follows:

[0029] F i l = F i l + DepthwiseConv(F i l ), i ∈ [1, 2]

[0030] where, DepthwiseConv(·) is the depthwise separable convolution operation, i is different time states, i = 1 corresponds to the first moment, i = 2 corresponds to the first moment, and F i l is the l-th feature map extracted from the image T i in the i-th time state.

[0031] Preferably: in step 34, the patches and Perform spatial downsampling at multiples of {2, 4, 8, 16} respectively to obtain multi-scale feature blocks of different spatial sizes, linearly expand all multi-scale feature blocks belonging to the same feature map, and then perform feature splicing at each scale to obtain a key-value pair representation group. and The calculation formula is as follows:

[0032]

[0033]

[0034] In the formula, the small block is obtained by dividing the feature map F i l MultiPool(·) is the spatial downsampling operation at multiples of {2, 4, 8, 16}, Linear(·) is the linear expansion operation, and cat(·) is the splicing operation. is the key corresponding to the l-th feature map F i l at the i-th time step, and V i l is the value corresponding to the l-th feature map F i l at the i-th time step.

[0035] Preferably, in step 35, the dual-temporal joint query representation in step S33 and the key-value pair representation group and in step S34 are used to calculate the coupled spatial attention respectively to obtain the global attention output and at each time step. The calculation formula is as follows:

[0036]

[0037] where Softmax(·) is the non-linear mapping function and T is the transpose operation. is the global attention output corresponding to the l-th feature map F i l at the i-th time step.

[0038] Preferably, in step 36, the global attention output and are further encoded by a convolutional layer with shared parameters, and the absolute error between the output features of the convolutional layer with shared parameters is calculated to obtain the mutation feature map The calculation formula is as follows:

[0039]

[0040] Among them, Abs(·) is the absolute value operation, and Conv share (·) is a convolutional layer with shared parameters.

[0041] Preferably: In step 43, the specific operation of the single large spatial kernel selection module is as follows:

[0042] Input feature map F seq After being extracted by the first depthwise separable convolution, the feature F seq-1 is obtained. Then, the feature F seq-1 is extracted by the second depthwise separable convolution to obtain the feature F seq-2 . The feature F seq-1 and F seq-2 are concatenated along the channel dimension, and then average pooling operation and max pooling operation are performed along the channel dimension. Then, the pooled features are concatenated along the channel. After concatenation, a convolutional layer and a Sigmoid function are used to extract the spatial kernel selection attention. After the spatial kernel selection attention is split along the channel direction, it is multiplied with the features F seq-1 and F seq-2 respectively. Then, the features after multiplication are superimposed spatially and sent into a convolutional layer with a kernel size of 1×1 to obtain the large spatial kernel selection features. The large spatial kernel selection features are then subjected to Hadamard product with the input feature map F seq to obtain the output features of the single large spatial kernel selection module.

[0043] Preferably: In step 6, obtaining the fused difference map according to the difference maps of 4 scales includes:

[0044] The low-resolution difference map is upsampled and concatenated with the difference map of the next higher resolution along the channel direction. After concatenation, it is sent into a decoder composed of residual blocks to complete feature fusion, completing one fusion operation of low and high resolutions. The fused features are used as the low-resolution difference map to perform another fusion operation with the difference map of the next higher resolution until all difference maps are fused, and the fused difference map is output by the last encoder.

[0045] Preferably: In step 7, the pixel values of the changed regions are set to 1, and the pixel values of the unchanged regions are set to 0.

[0046] An outdoor change monitoring method based on gradual change and sudden change provided by the present invention monitors outdoor changes in data with spatial-temporal multi-dimensions. It uses a temporal interactive image pair encoder based on a window coupling attention mechanism to extract dual-temporal sudden change features; simulates the change temporal evolution through interpolation in time series to capture the gradual change features in the image sequence; and finally completes the fusion of multi-scale features with a simple encoder operation, improving the model's attention to the changed regions to accurately monitor outdoor change targets. Description of the Drawings

[0047] Figure 1 This is the flowchart of the outdoor change monitoring based on gradual change and mutation of the present invention;

[0048] Figure 2 This is the flowchart of the window-coupled attention mechanism of the present invention;

[0049] Figure 3 This is the flowchart of the large-space kernel selection module of the present invention;

[0050] Figure 4 This is the detection result graph of the embodiment of the present invention. Detailed implementation manners

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

[0053] In view of the spatio-temporal multi-dimensional characteristics of data in multi-scale change target retrieval, the present application realizes accurate multi-scale change target retrieval by constructing a cross-temporal joint global attention mechanism, a joint pixel-by-pixel subtraction and parameterized learning double-branch, and a lightweight global spatial attention mechanism.

[0054] As Figures 1 - 3 shown, the outdoor change monitoring method based on gradual change and mutation in this embodiment includes the following steps:

[0055] S1. Process the image pairs T1 and T2 of the same outdoor scene at different times with a size of W×H×3 into a gradual change image sequence S with a size of W×H×3×t through a temporal interpolation method, where T1 is the image at the first time in the image pair and T2 is the image at the second time in the image pair.

[0056] Taking the image size of the image pairs T1 and T2 as 256×256×3 as an example for illustration, it can be adjusted according to actual needs in other embodiments. The image pairs T1 and T2 are processed into a gradual change image sequence S with a size of 256×256×3×8 through a temporal interpolation method, and the calculation formulas of each frame in the 8-frame temporal dimension are as follows:

[0057] Framej = T1 + (T2 - T1) * (j - 1) / 8 - 1, j ∈ [1, 8] (1)

[0058] In the formula, Frame j is the j-th frame in the time sequence dimension. When j = 1, this frame is T1; when j = 8, this frame is T2. The operation on the image refers to the operation on the pixel values of each pixel point on the image.

[0059] S2. Use the Siamese backbone network GhostNetV2 model pre-trained on the ImageNet dataset to extract 4 feature maps F1 of different scales from the image pairs T1 and T2 respectively l and l ∈ [1, 4], the feature map F1 l is the l-th feature map extracted from image T1, and the feature map is the l-th feature map extracted from image T2. The output sizes of the feature maps are and respectively. Among them, W is the width, H is the height, and d1, d2, d3, and d4 are the channel depths.

[0060] Taking the image size of the image pairs T1 and T2 as 256×256×3 as an example, the output sizes of each stage of the GhostNetV2 model are 128×128×32, 64×64×48, 32×32×80, and 16×16×192.

[0061] S3. Use the temporal interactive image pair encoder based on the window coupling attention mechanism to perform coupled modeling on F1 l and F2 l of each scale respectively to capture mutation features. The specific operation of the temporal interactive image pair encoder based on the window coupling attention mechanism is as follows:

[0062] S31. By adding the result of performing depthwise separable convolution operation on each temporal F1 l and on this basis, the supplementary of position encoding information is realized. The calculation formula is as follows:

[0063] F i l = F i l + DepthwiseConv(F i l ), i ∈ [1, 2] (2)

[0064] Among them, DepthwiseConv(·) is the depthwise separable convolution operation, i is different temporal moments, i = 1 corresponds to the first moment, i = 2 corresponds to the first moment, F il The image T in the i tense i The l-th feature map extracted.

[0065] S32. The dual-tense feature map F1 supplemented with positional encoding information l and are divided into non-overlapping patches of the same size w×h×d l (e.g., 16×16×d l ) to obtain a set of patches corresponding to each feature map. The patch and is obtained by dividing the feature map F1 , and the patch l is obtained by dividing the feature map .

[0066] S33. The patches in the set of patches and are concatenated along the channel dimension to generate a patch of w×h×2d l . Then, the channel dimension is reduced to d through a convolution operation, and the output is linearly unfolded to obtain a dual-tense joint query representation with a size of N×d l , where N = w×h, and the calculation formula is as follows:

[0067]

[0068] where Linear(·) is the linear unfolding operation, Conv(·) is the convolution operation, and Cat(·) is the concatenation function along the channel direction. It should be noted that when performing patch concatenation, the patches at the corresponding positions in the corresponding set of patches are concatenated one by one and merged into a set after concatenation. For example, for the set of patches containing the patch and the set of patches containing , the patches at the corresponding positions are concatenated one by one, and after subsequent convolution operations and linear unfolding operations, a dual-tense joint query representation is obtained. Therefore, in formula 3 refers to the concatenation of all the patches at the corresponding positions in the set of patches one by one.

[0069] S34. The patches in the set of patches and are each subjected to spatial downsampling at a ratio of {2, 4, 8, 16} to obtain multi-scale feature blocks of different spatial sizes. All the multi-scale feature blocks belonging to the same feature map are linearly unfolded and then subjected to multi-scale feature concatenation for each tense to obtain their respective key-value pair representation groups and The sizes of both K and V are N'×d​l , N' = w' × h', and the calculation formula is as follows:

[0070]

[0071]

[0072] In the formula, the small block is obtained by dividing the feature map F i l MultiPool(·) is the spatial downsampling operation with a magnification of {2, 4, 8, 16}, Linear(·) is the linear expansion operation, and cat(·) is the concatenation operation. is the key corresponding to the l-th feature map F in the i-th time step i l V i l is the value corresponding to the l-th feature map F in the i-th time step i l In this step, the spatial downsampling operation is performed on each small block in sequence. Therefore, after the spatial downsampling operation, a small block set obtains four new sets. After performing the linear expansion operation and the concatenation operation on these four new sets, a set of key-value pair representations corresponding to one time step is obtained.

[0073]

[0074] S35. Through the joint query representation in step S33 and the key-value pair representation group in step S34 and calculate the coupled spatial attention respectively to obtain the global attention output for each time step and both have the size of w × h × d l and the calculation formula is as follows:

[0075]

[0076] where Softmax(·) is the non-linear mapping function, and T is the transpose operation. is the global attention output corresponding to the l-th feature map F in the i-th time step i l In step S36, further encode the global attention output through a convolutional layer with shared parameters

[0077] and calculate the absolute error between the output features of the convolutional layer with shared parameters to obtain the mutation feature map and The calculation formula is as follows:

[0078] ​

[0079] Among them, Abs(·) is the absolute value operation, and Conv share (·) is a convolutional layer with shared parameters.

[0080] S4. Explore the gradual change process of the image sequence S through the image sequence encoder based on the large spatial kernel selection module. The operation of the image sequence encoder based on the large spatial kernel selection module is as follows:

[0081] S41. Reshape the four-dimensional image sequence S of size W×H×3×t into a three-dimensional gradual change image sequence S′ of W×H×3t, and its channel dimension contains the changing temporal evolution.

[0082] S42. Perform 4-fold spatial downsampling on the gradual change image sequence S′ through a convolutional operation with a convolutional kernel size of 9×9, a stride of 4×4, and a spatial padding size of 4×4 to obtain the temporal evolution feature.

[0083] S43. Use three consecutive large spatial kernel selection modules to encode the temporal evolution feature. A convolutional operation with a convolutional kernel size of 2×2 and a stride of 2×2 is added between every two large spatial kernel selection modules for 2-fold spatial downsampling of the feature to obtain the gradual change feature maps of the multi-scale image sequences with sizes respectively and where s ∈ [1, 3], and d2′, d3′, and d4′ are the channel depths. s ∈ [1, 3], d2′, d3′ and d4′ are the channel depths.

[0084] In one embodiment, in step S43, a single large spatial kernel selection module is as follows: the input feature map F seq successively passes through a depthwise separable convolution with a convolutional kernel size of 5×5 and a spatial padding size of 2×2 and a depthwise separable convolution with a convolutional kernel size of 7×7, a dilation rate of 3, and a spatial padding size of 9×9 to extract features. Then, the features F seq-1 and F seq-2 extracted by the two convolutions are concatenated along the channel dimension, and then average pooling and max pooling operations are performed along the channel dimension. Then, the pooled features are concatenated along the channel, and a convolutional operation with a convolutional kernel size of 7×7 and a spatial padding size of 3×3 and a Sigmoid function are used to extract the spatial kernel selection attention. After splitting along the channel direction, they are multiplied with F seq-1 and F seq-2 respectively. Then, the calculated features are stacked spatially and fed into a convolutional layer with a convolutional kernel size of 1×1 to obtain the large spatial kernel selection feature. The large spatial kernel selection feature is then subjected to a Hadamard product with the original input feature map F seq to enhance the representation, and the output feature of a single large spatial kernel selection module is obtained.

[0085] S5. By concatenating the features of the mutation feature maps and the gradual change feature maps along the channel dimension, four-scale difference maps with sizes of and are obtained.

[0086] S6. Based on the four-scale difference maps a fused difference map is obtained.

[0087] The low-resolution difference map is upsampled and concatenated with the higher-resolution difference map along the channel direction. After concatenation, it is fed into a decoder composed of residual blocks (formed by convolutional layers with a convolutional kernel size of 3×3) to complete feature fusion, completing one fusion operation between low and high resolutions. The fused features are used as the low-resolution difference map to perform another fusion operation with the higher-resolution difference map until the fusion of all difference maps is completed, and the fused difference map is output by the last encoder. The calculation formula for one fusion operation is as follows:

[0088]

[0089] Among them, is the output feature of one fusion operation, ReLU(·) is the non-linear mapping, BN(·) is the batch normalization, UP(·) is the upsampling operation, is the low-resolution difference map, is the higher-resolution difference map.

[0090] S7. Finally, the feature map after feature fusion is interpolated and upsampled to the size of the image in the image pair in step S1, and the pixel values of the changing target pixels in the fused difference map are changed to 1 and the pixel values of the non-changing regions are 0 through a convolutional layer with a convolutional kernel size of 3×3 and a Sigmoid function, realizing outdoor change monitoring.

[0091] As a preferred embodiment, taking the images collected at different times from 2006 to 2019 in a suburban area of Guangzhou as an example, an outdoor change monitoring method based on gradual change and mutation provided by this application is used for experiments. The images of the detection dataset include three bands of R, G, and B, with a resolution of 0.55m. The specific steps are as follows:

[0092] Step 1: Load the images collected at different times from 2006 to 2019 in a suburban area of Guangzhou, such as Figure 4As shown, the image in Tense 1 (denoted as X1) is the image before the change, and the image in Tense 2 (denoted as X2) is the image after the change. (a)-(e) are six groups of images. Select the three displayed bands (default R, G, B), cut the image pairs and their corresponding detection label maps into 256×256, and divide the data into three parts: training, validation, and testing.

[0093] Step 2: Pretrain a CNN model (specifically using the GhostNetV2 network) on the ImageNet classification dataset. The CNN model can also be replaced by a Transformer model.

[0094] Step 3: Load the training set and validation set data cut in Step 1, and send them into the pre-trained GhostNetV2 model to capture feature maps F1 and F2 with sizes of 128×128×32, 64×64×48, 32×32×80, and 16×16×192. F1 and F2 are multi-scale abstract feature maps of the corresponding sub-images in the image pairs.

[0095] Step 4: Execute a temporal interactive image pair encoder based on the window coupling attention mechanism on the feature maps F1 and F2 to capture multi-scale mutation features.

[0096] Step 5: Explore the gradual change process of the changes in the image sequence S through an image sequence encoder based on the large spatial kernel selection module to obtain multi-scale gradual change features.

[0097] Step 6: Concatenate the image pair difference features and the image sequence gradual change features of the same scale along the channel dimension, and use a simple residual block to complete multi-scale feature fusion.

[0098] Step 7: After the fused feature map undergoes 2x upsampling, convolution operation, and Sigmoid function, obtain a change detection map with a size of 256×256×1. The model training can be completed by minimizing the loss between the detection prediction map and the label map through the cross-entropy loss function and the stochastic gradient descent algorithm, which are well-known techniques for those skilled in the art and will not be elaborated here.

[0099] Step 8: Load the parameters trained by this method, obtain the prediction output on the test set, compare the change detection results with the test set labels, and calculate the precision, recall, F1 score, intersection over union, and overall accuracy. As Figure 4 shown, there are a total of six groups of image pairs (a)-(f), corresponding to the result comparison of the change detection maps of the proposed method (the method of this application) and the reference method (dual-time transformer). The detection evaluation results are shown in Table 1.

[0100] Table 1 Detection Evaluation Results

[0101]

[0102] Among them, the dual-temporal transformer is a method proposed in the prior art. For reference: H. Chen, Z. Qi and Z. Shi, "Remote Sensing Image Change Detection With Transformers," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-14, 2022, Art no. 5607514, doi: 10.1109 / TGRS.2021.3095166. The detection results of the method proposed in Table 1 comprehensively exceed the results of the dual-temporal transformer method in the prior art numerically. At the same time Figure 4 it has fewer missed detections (light gray pixel points) and false detections (dark gray pixel points), as well as higher accuracy (white pixel points), further verifying the effectiveness of the present invention in retrieving changing targets.

[0103] The method proposed in this application is for outdoor change monitoring in data with spatial-temporal multi-dimensions. It uses a temporal interactive image pair encoder based on a window-coupled attention mechanism to extract dual-temporal mutation features; simulates the change temporal evolution through temporal interpolation to capture the gradual change features in the image sequence; and finally completes the fusion of multi-scale features with a simple encoder operation, improving the model's attention to the changing area to accurately monitor outdoor changing targets.

[0104] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0105] The above-described embodiments merely represent several implementation manners of the present invention. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. An outdoor change monitoring method based on gradual change and mutation, characterized in that, The outdoor change monitoring method based on gradual change and mutation includes the following steps: S1. Process the image pairs T1 and T2 of the same outdoor scene at different times into a gradual change image sequence S through a temporal interpolation method, where T1 is the image at the first time in the image pair, and T2 is the image at the second time in the image pair; S2. Extract feature maps F1 of four different scales for the image pairs T1 and T2 respectively using a pre-trained convolutional neural network model l and Feature map F1 l is the l-th feature map extracted for image T1, and the feature map is the l-th feature map extracted for image T2; S3. Respectively perform coupled modeling on the feature maps F1 of each scale through a temporal interactive image pair encoder based on a window-coupled attention mechanism l and to capture mutation features. The specific operation of the temporal interactive image pair encoder based on the window-coupled attention mechanism is as follows: S31. Supplement the positional encoding information for the feature maps F1 at each scale l and ; The feature map F1 supplemented with position encoding information l and are divided into non - overlapping and same - sized small blocks and to obtain a set of small blocks corresponding to each feature map. The small block is obtained after the division of the feature map F1 l and the small block is obtained after the division of the feature map ; S33. Concatenate the chunks in the chunk set and along the channel dimension, and perform a linear expansion after a convolution operation to obtain a bi-temporal joint query representation S34. For the patches in the small patch set and perform spatial downsampling at {2, 4, 8, 16} times magnification respectively to obtain multi-scale feature patches of different spatial sizes, linearly expand all the multi-scale feature patches belonging to the same feature map, and then perform feature splicing at each scale to obtain a key-value pair representation group and S35. Represent through the dual-temporal joint query in step S33 with the key-value pair representation group in step S34 and calculate the coupled spatial attention respectively to obtain the global attention output for each temporal state and S36. Further encode the global attention output through the convolutional layer sharing parameters and Calculate the absolute error between the output features of the convolutional layer sharing parameters to obtain the mutation feature map S4. Explore the gradual change process of the changes in the gradual change image sequence S through an image sequence encoder based on a large spatial kernel selection module. The specific operations of the image sequence encoder based on the large spatial kernel selection module are as follows: S41. Reshape the four-dimensional gradual change image sequence S into a three-dimensional gradual change image sequence S′, and the channel dimension of the gradual change image sequence S′ contains the temporal evolution of the changes; S42. Perform spatial 4-fold downsampling on the gradual change image sequence S′ through a convolution operation to obtain the temporal evolution features; S43. Use three consecutive large spatial kernel selection modules to encode the temporal evolution features. Add a convolution operation between every two large spatial kernel selection modules for 2-fold spatial downsampling of the features, and obtain three gradually changing feature maps of different scales output by the three large spatial kernel selection modules. S5. Concatenate the mutation feature maps and the gradient feature maps of the same scale along the channel dimension to obtain the difference maps of 4 scales and the gradient feature maps along the channel dimension to obtain the difference maps of 4 scales S6. Obtain a fused difference map based on the difference maps of four scales Obtain a fused difference map; S7. Unify the size of the fused difference map to the size of the image pair through interpolation upsampling, and set the pixel values of the change region and the non-change region in the fused difference map through a convolutional layer and a Sigmoid function to achieve outdoor change monitoring.

2. The outdoor change monitoring method based on gradual change and mutation as described in claim 1, wherein: In step S1, when processing the image pairs T1 and T2 of the same outdoor scene at different times into a gradual change image sequence S through a temporal interpolation method, where t is the number of frames of the gradual change image sequence S, the calculation formula for each frame in the t-frame temporal dimension is as follows: Frame j = T1 + (T2 - T1) * (j - 1) / t - 1, j ∈ [1, t] where Frame j is the j-th frame in the time sequence dimension. When j = 1, this frame is T1; when j = t, this frame is T2.

3. The outdoor change monitoring method based on gradual change and mutation as claimed in claim 1, wherein: In step S2, the convolutional neural network model is a GhostNetV2 model or a Transformer model, and the sizes of the four feature maps with different scales output by the convolutional neural network model are respectively and where W is the width, H is the height, and d1, d2, d3, and d4 are the channel depths.

4. The outdoor change monitoring method based on gradual change and mutation as described in claim 1, wherein: In step S31, for the feature maps F1 of each scale l and supplementary position encoding information is performed, and the calculation formula is as follows: F i l = F i l + DepthwiseConv(F i l ), i ∈ [1, 2] Among them, DepthwiseConv(·) is a depthwise separable convolution operation, i represents different time tenses, i = 1 corresponds to the first moment, i = 2 corresponds to the second moment, and F i l is the l-th feature map extracted from the image T i in the i-th time tense.

5. The outdoor change monitoring method based on gradual change and mutation as claimed in claim 4, wherein: In step 34, the small blocks in the small block set and are each subjected to spatial downsampling at a magnification of {2, 4, 8, 16} to obtain multi-scale feature blocks of different spatial sizes. All the multi-scale feature blocks belonging to the same feature map are linearly expanded and then feature stitching is performed at each scale to obtain a key-value pair representation group and The calculation formula is as follows: In the formula, the small block is the feature map F i l obtained after division. MultiPool(·) is the spatial downsampling operation with a magnification of {2, 4, 8, 16}, Linear(·) is the linear expansion operation, and cat(·) is the concatenation operation. is the key corresponding to the l-th feature map F i l at the i-th time step, and V i l is the value corresponding to the l-th feature map F i l at the i-th time step.

6. The outdoor change monitoring method based on gradual change and mutation as described in claim 5, characterized in that: In step 35, the representation by the dual-temporal joint query in step S33 and the key-value pair representation group in step S34 and respectively calculate the coupled spatial attention to obtain the global attention output for each temporal state and The calculation formula is as follows: Among them, Softmax(·) is a non-linear mapping function, and T is a transpose operation. is the global attention output corresponding to the l-th feature map F i l at the i-th time step.

7. The outdoor change monitoring method based on gradual change and mutation as described in claim 2, characterized in that: In step 36, the convolutional layer that shares parameters further encodes the global attention output and calculate the absolute error between the output features of the convolutional layer that shares parameters to obtain the mutation feature map The calculation formula is as follows: Among them, Abs(·) is the absolute value operation, and Conv share (·) is a convolutional layer with shared parameters.

8. The outdoor change monitoring method based on gradual change and mutation as described in claim 1, wherein: In step 43, the specific operations of a single large spatial kernel selection module are as follows: Input feature map F seq Feature F is obtained by extraction through the first depthwise separable convolution seq-1 , and then feature F seq-1 is further extracted through the second depthwise separable convolution to obtain feature F seq-2 . Feature F seq-1 and F seq-2 are concatenated along the channel dimension, and then average pooling operation and max pooling operation are performed along the channel dimension. Then the pooled features are concatenated along the channel. After concatenation, a convolutional layer and a Sigmoid function are used to extract spatial kernel selection attention. After splitting the spatial kernel selection attention along the channel direction, it is multiplied with feature F seq-1 and F seq-2 respectively. Then the features after multiplication are spatially superimposed and fed into a convolutional layer with a kernel size of 1×1 to obtain large spatial kernel selection features. The large spatial kernel selection features are then subjected to Hadamard product with the input feature map F seq to obtain the output feature of a single large spatial kernel selection module.

9. The outdoor change monitoring method based on gradual change and mutation as described in claim 1, wherein: In step 6, the fused difference map is obtained based on the difference maps of 4 scales, including: ​ Upsample the low-resolution difference map and splice it with the higher-resolution difference map along the channel direction. After splicing, send it to a decoder composed of residual blocks to complete feature fusion, complete a single fusion operation of low and high resolutions, and use the fused features as the low-resolution difference map to perform a fusion operation with the higher-resolution difference map again until all difference maps are fused, and the fused difference map is output by the last encoder.

10. The outdoor change monitoring method based on gradual change and mutation as claimed in claim 1, wherein: In step 7, the pixel value of the change region is set to 1, and the pixel value of the non-change region is set to 0.

Images