A method for detecting temporal changes in remote sensing mining images based on dual-branch deep supervision

Through the remote sensing mining pattern timing change detection method based on dual-branch deep supervision, deep learning technology is used to improve the accuracy and efficiency of remote sensing mining pattern change detection in complex scenarios, and the traditional method is solved in large-scale or long-term time series detection. insufficient performance problem.

CN118230159BActive Publication Date: 2024-11-26CHONGQING HUADI RESOURCES ENVIRONMENT TECH CO LTD +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410334186.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-22
Publication Date
2024-11-26
Estimated Expiration
2044-03-22

AI Technical Summary

Technical Problem

The traditional remote sensing mining pattern change detection method has limited performance in complex geographical environments and large-scale data, and is difficult to adapt to different scenarios and lighting conditions, and is sensitive to noise, resulting in low detection accuracy and efficiency, especially in large areas or long-term There are challenges in the detection of sequence changes.

Method used

Using the remote sensing mining pattern timing change detection method based on dual-branch deep supervision, the semantic change detection model including the front-end timing difference detection network and the back-end timing difference detection network is constructed by obtaining high-resolution timing remote sensing data for preprocessing, and using depth Learning techniques capture image features and time differences from a bi-time phase perspective and generate differences maps for change detection.

Benefits of technology

It improves the accuracy and efficiency of timing change detection, can be applicable to large-scale or long-term time series change detection, and enhances the adaptability to complex scenarios and automation of detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118230159B_ABST
    Figure CN118230159B_ABST
Patent Text Reader

Abstract

The present invention provides a method for detecting time series changes of remote sensing mining spots based on dual-branch deep supervision, including: obtaining time series remote sensing data of a mining area, the time series remote sensing data including high-resolution remote sensing images at multiple time points and covering the entire range of the mining area; preprocessing the time series remote sensing data, the preprocessing including atmospheric correction, radiation calibration and image registration operations; constructing and training a semantic change detection model based on deep learning, the semantic change detection model including a front-end time series difference detection network and a back-end time series difference detection network, forming a semantic change detection model based on dual-branch time series change detection; using the trained semantic change detection model, generating a difference map according to the time series remote sensing data, and obtaining a time series change detection result. The present invention improves the accuracy and efficiency of time series change detection, and can be applied to change detection of a large range or a long time series.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing data processing, and in particular to a remote sensing mining pattern temporal change detection method based on dual-branch deep supervision. Background Art

[0002] Remote sensing mining patch change detection is the use of remote sensing technology to monitor and analyze surface features in mineral areas in a time series to detect changes in the surface. This process involves acquiring, processing and interpreting remote sensing data, aiming to provide key surface information for the mining industry to facilitate more effective resource management and decision making. High-resolution remote sensing images are acquired using satellites, aircraft or other sensors, which should cover the mining area and be collected at different times. Time series remote sensing data are key because they provide surface information at different points in time, making change detection possible.

[0003] Traditional change detection methods usually use threshold-based, ratio-based or statistical methods to detect pixel-level changes. These methods may be effective in simple scenes, but their performance may be limited in complex geographical environments and large-scale data. Traditional methods have poor adaptability to different scenes, lighting conditions and sensors, and often require manual adjustment of parameters or use of different thresholds, resulting in difficulty in unifying sensitivity. Traditional methods are sensitive to noise or unstable factors in remote sensing images, and may mistakenly identify noise as surface changes, affecting the accuracy of change detection. Traditional methods usually only consider pixel-level changes in images and ignore spatial contextual information, so they have poor detection effects on complex object boundaries or small-scale changes. The selection and adjustment of parameters usually need to rely on experience or manual intervention, so they are not automated enough and are difficult to adapt to change detection over large areas or long time series. Due to these problems, traditional methods may face some challenges when dealing with change detection of remote sensing data in large-scale and complex scenes.

[0004] Therefore, there is an urgent need for a remote sensing mining pattern temporal change detection method that can be applied to large-scale or long-time series change detection and improve the accuracy and efficiency of detection results. Summary of the invention

[0005] Based on this, it is necessary to provide a remote sensing mining pattern temporal change detection method based on dual-branch deep supervision to address the above technical problems.

[0006] A method for detecting temporal changes in remote sensing mining patterns based on dual-branch deep supervision comprises the following steps: acquiring temporal remote sensing data of a mining area, wherein the temporal remote sensing data comprises high-resolution remote sensing images at multiple time points and covering the entire range of the mining area; preprocessing the temporal remote sensing data, wherein the preprocessing comprises atmospheric correction, radiation calibration and image registration operations; constructing and training a semantic change detection model based on deep learning, wherein the semantic change detection model comprises a front-end temporal difference detection network and a back-end temporal difference detection network, forming a semantic change detection model based on dual-branch temporal change detection; using the trained semantic change detection model, generating a difference map according to the temporal remote sensing data, and obtaining a temporal change detection result.

[0007] In one embodiment, the front-end temporal difference detection network includes: a ResNet50 network, a front feature enhancement module and an encoder-decoder architecture; the ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; the feature enhancement module includes a multi-head self-attention unit, a multi-layer perceptron block and a batch normalization layer, which is used to capture global context information with long-range dependencies based on the self-attention mechanism, and calculate the temporal difference features of the obtained dual-phase feature map; the front encoder-decoder architecture is used to obtain the corresponding detection results based on the obtained temporal difference features; the ResNet50 network includes an input layer, an initial convolution layer, a pooling layer, and multiple residual blocks , global average pooling layer and fully connected layer; wherein the input layer is used to receive time series remote sensing data; the initial convolution layer and pooling layer are used to extract features from the time series remote sensing data to obtain feature maps, the initial convolution layer is a 7×7 convolution kernel with a step size of 2, and the pooling layer is a 3×3 maximum pooling with a step size of 2; the multiple residual blocks are stacked to form a deep network, each residual block includes a first convolution layer and a second convolution layer, the output of the first convolution layer is directly added to the output of the second convolution layer to form a residual connection; the global average pooling layer is used to perform global average pooling on the feature maps, reducing the spatial size of each feature map to 1×1; the fully connected layer is used to map the feature maps after global average pooling to category scores.

[0008] In one embodiment, the encoder-decoder architecture includes four convolutional layers and three deconvolutional layers; wherein, the number of convolutional kernels of the four convolutional layers is 32, 64, 128 and 256 respectively, the convolutional kernel size is 5×5, and the relu function is used. Three residual layers are connected after each convolutional layer, and each residual layer contains two consecutive convolutional layers. The feature map of the input image is obtained through the convolutional layer; the number of convolutional kernels of the three deconvolutional layers is 128, 64, and 32 respectively, the convolutional kernel size is 2×2, and the step size is 2. The feature map obtained by continuous encoding is then subjected to three consecutive residual operations. Each residual layer contains two consecutive convolutional layers, and the pixel information is restored layer by layer through the deconvolution network, and the final classification result is obtained through the pixel-level classifier; a copy layer is provided after the deconvolution layer for copying and bitwise adding the feature matrix of the encoder part.

[0009] In one embodiment, the formula of the front-end timing difference detection network is:

[0010]

[0011] Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way of expressing differences in features.

[0012] In one embodiment, the back-end time series difference detection network includes: a ResNet50 network, an encoder-decoder architecture and a feature enhancement module; the ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; the encoder-decoder architecture is used to generate decoding features of two time series; the feature enhancement module is used to enhance the decoding features based on the input dual-phase data, and calculate the change distance between the decoded dual-phase features to generate a probability map for change prediction.

[0013] In one embodiment, the formula of the back-end timing difference detection network is:

[0014]

[0015] Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way of expressing differences in features.

[0016] In one embodiment, it also includes: using a feature difference method to perform front-end dual-phase difference calculation, the formula is:

[0017]

[0018] Where, d i Represents the time series difference characteristics, and They represent the feature maps of scale i generated based on the front ResNet50 network at the first time and the second time respectively; the Euclidean distance map of the two-phase feature pairs is calculated by the back-end temporal difference detection network, and the formula is:

[0019]

[0020] In the formula, o i Represents the Euclidean distance map; the contrast loss is calculated by comparing the Euclidean distance map with the actual label truth value.

[0021] In one embodiment, the training loss function of the semantic change detection model is:

[0022] L=L f (f t (X 1 ,X 2 ),Y)+L b (f b (X 1 ,X 2 ),Y),

[0023] Using binary cross entropy loss function L bce and Dice loss function L dice As the loss function for training the two branches, the formulas are:

[0024] L bce = -Y·log f(X 1 ,X 2 )-(1-Y)·log(1-f(X 1 ,X 2 )),

[0025]

[0026] Where, L f represents the training loss function of the front-end timing difference detection network, L b represents the training loss function of the backend temporal difference detection network, X 1 and X 2 are the input time series remote sensing data, Y is the corresponding true label result of the change area, and f t (·,·) and f b (·,·) are the front-end timing difference detection network and the back-end timing difference detection network respectively.

[0027] In one of the embodiments, it also includes: integrating the first detection result of the front-end temporal difference detection network and the second detection result of the back-end temporal difference detection network to obtain a target difference detection result; generating target difference detection results for temporal remote sensing data at different time points through a trained semantic change detection model, and obtaining a temporal change detection result based on the target difference detection results of all mining areas.

[0028] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: by acquiring high-resolution remote sensing images of a mining area at multiple time points and covering the entire range of the mining area, time series remote sensing data is obtained, atmospheric correction, radiation calibration and image registration operations are performed on the time series remote sensing data, and a semantic change detection model based on deep learning is constructed and trained, wherein the semantic change detection module includes a front-end time series difference detection network and a back-end time series difference detection network, forming a semantic change detection model based on dual-branch time series change detection, thereby realizing the capture of effective features and time differences in dual-phase images from both the front-end and back-end perspectives, through the correlation between the front-end and back-end time series difference detection networks, sharing the backbone and feature enhancement attention modules, promoting the collaborative work of the two, and strengthening the long-range dependency modeling between the marked features in the dual-branch architecture; using the trained semantic change detection model, a difference map is generated based on the time series remote sensing data to obtain a time series change detection result, and the synergy of the front-end and back-end dual-branch networks can simultaneously capture the features from the dual-time series mining patch data, and emphasize the time series differences, thereby improving the accuracy and efficiency of time series change detection, and can be applicable to change detection over a large range or over a long time series. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a flow chart of a method for detecting time series changes of remote sensing mining spots based on dual-branch deep supervision in one embodiment;

[0030] Figure 2 This is a diagram showing the detection effects of multiple algorithms on changes in remote sensing mining spots in one embodiment. DETAILED DESCRIPTION

[0031] Before describing the specific embodiments of the present invention, the overall concept of the present invention is described as follows:

[0032] The present invention is mainly developed based on the remote sensing data processing process. Traditional remote sensing data processing methods have poor performance and low detection accuracy when detecting remote sensing data changes in large-scale and complex scenes.

[0033] Therefore, the present invention proposes a remote sensing mining pattern temporal change detection method based on dual-branch deep supervision, obtains high-resolution remote sensing images of multiple time points in the mining area and covering the entire range of the mining area, obtains time-series remote sensing data, performs atmospheric correction, radiation calibration and image registration operations on the time-series remote sensing data, constructs and trains a semantic change detection model based on deep learning, wherein the semantic change detection module includes a front-end temporal difference detection network and a back-end temporal difference detection network, forming a semantic change detection model based on dual-branch temporal change detection, thereby realizing the capture of dual-phase from both the front-end and back-end perspectives. The effective features and time differences in the image are detected through the correlation between the front-end and back-end time series difference networks, sharing the backbone and feature enhancement attention module to promote the collaborative work of the two, and strengthening the long-range dependency modeling between the marked features in the dual-branch architecture; the trained semantic change detection model is used to generate a difference map based on the time series remote sensing data to obtain the time series change detection results. Through the synergy of the front-end and back-end dual-branch networks, the features from the dual-time series mining patch data can be captured simultaneously, and the time series differences are emphasized, which improves the accuracy and efficiency of time series change detection and can be applied to change detection over a large range or over a long time series. .

[0034] After introducing the overall concept of the present invention, in order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail by specific implementation methods in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] In one embodiment, Figure 1 As shown, a remote sensing mining pattern temporal change detection method based on dual-branch deep supervision is provided, comprising the following steps:

[0036] Step S101, acquiring time series remote sensing data of a mining area, wherein the time series remote sensing data includes high-resolution remote sensing images at multiple time points and covering the entire range of the mining area.

[0037] Specifically, high-resolution remote sensing images covering the entire mining area at multiple time points are acquired through satellites or other sensors as time series remote sensing data of the mining area. The time series remote sensing data covers all mineral areas and different time points, so as to facilitate the detection of changes in mining patches based on time.

[0038] Step S102, preprocessing the time series remote sensing data, the preprocessing includes atmospheric correction, radiation calibration and image registration operations.

[0039] Specifically, the acquired time series remote sensing data are preprocessed, including atmospheric correction, radiation calibration and image registration, to ensure the consistency and quality of the time series remote sensing data, which helps to eliminate noise in the image, improve the quality of subsequent analysis, and make the time series remote sensing data at different time points more comparable.

[0040] Step S103, constructing a semantic change detection model based on deep learning, wherein the semantic change detection model includes a front-end temporal difference detection network and a back-end temporal difference detection network, forming a semantic change detection model based on dual-branch temporal change detection.

[0041] Specifically, a semantic change detection model based on deep learning is constructed, which includes a front-end temporal difference detection network and a back-end temporal difference detection network, forming a unified semantic change detection model based on dual-branch temporal change detection, thereby capturing effective features and temporal differences in dual-phase images from both the front-end and back-end perspectives.

[0042] Among them, the front-end temporal difference detection network includes: ResNet50 network, feature enhancement module and encoder-decoder architecture; ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; the feature enhancement module includes multi-head self-attention units, multi-layer perception blocks and batch normalization layers, which are used to capture global context information with long-range dependencies based on the self-attention mechanism, and calculate the temporal difference features of the obtained dual-phase feature maps; the encoder-decoder architecture is used to obtain the corresponding detection results based on the obtained temporal difference features; the ResNet50 network includes an input layer, an initial convolutional layer, a pooling layer, multiple residual blocks, Global average pooling layer and fully connected layer; wherein, the input layer is used to receive time series remote sensing data; the initial convolution layer and the pooling layer are used to extract features from the time series remote sensing data to obtain feature maps, the initial convolution layer is a 7×7 convolution kernel with a step size of 2, and the pooling layer is a 3×3 maximum pooling with a step size of 2; multiple residual blocks are stacked to form a deep network, each residual block includes a first convolution layer and a second convolution layer, and the output of the first convolution layer is directly added to the output of the second convolution layer to form a residual connection; the global average pooling layer is used to perform global average pooling on the feature map, reducing the spatial size of each feature map to 1×1; the fully connected layer is used to map the feature map after global average pooling to the category score.

[0043] Specifically, the front-end temporal difference detection network focuses on temporal difference feature extraction at the front end of the network, followed by encoding and decoding calculations. The architecture of the front-end temporal difference detection network uses the ResNet50 network as the backbone network to generate dual-phase feature maps at different scales, and uses the feature enhancement module to process each feature map. The obtained temporal difference features are input into the standard encoder-decoder architecture to obtain the corresponding detection results. The effect diagram of the detection results is shown in the figure below. Figure 2 shown.

[0044] The input layer of the ResNet50 network receives dual-phase time series remote sensing data, and extracts features of the dual-phase remote sensing images through the initial convolutional layer and pooling layer to obtain feature maps.

[0045] The core of the ResNet50 network is the residual block. Each residual block contains two convolutional layers, where the output of the first convolutional layer is directly added to the output of the second convolutional layer to form a residual connection. This structure allows the model to learn residuals, making it easier for gradients to propagate, solving the problem of gradient vanishing. In the residual block, when the dimensions of the input and output do not match, a 1×1 convolutional layer can be used for downsampling to ensure that they can be added. The ResNet50 network constructs a deep network by pairing multiple residual blocks. The entire network contains some residual blocks of different layers, including 3 residual blocks of 50 layers.

[0046] At the top of the ResNet50 network, global average pooling is performed on the feature maps to reduce the spatial size of each feature map to 1×1, and the last layer is a fully connected layer to map the extracted features to category scores.

[0047] The feature enhancement module uses the self-attention mechanism to enable each target feature to take into account the information of every other target feature in the entire image representation, capture global contextual information with long-range dependencies, thereby facilitating visual change detection, and calculate the temporal difference features of the obtained dual-phase feature map.

[0048] In the encoder-decoder architecture, the encoder gradually downsamples the input features to capture advanced abstract features and increase the map depth, allowing the network to capture global context and abstract features; the decoder maps these abstract features back to the original input dimension through upsampling, retaining complex details and improving the accuracy of change detection.

[0049] Among them, the encoder-decoder architecture includes four convolutional layers and three deconvolutional layers; the number of convolutional kernels of the four convolutional layers is 32, 64, 128 and 256 respectively, the convolutional kernel size is 5×5, and the relu function is used. Three residual layers are connected after each convolutional layer, and each residual layer contains two consecutive convolutional layers. The feature map of the input image is obtained through the convolutional layer; the number of convolutional kernels of the three deconvolutional layers is 128, 64, and 32 respectively, the convolutional kernel size is 2×2, and the step size is 2. The feature map obtained by continuous encoding is continuously subjected to three residual operations. Each residual layer contains two consecutive convolutional layers, and the pixel information is restored layer by layer through the deconvolution network, and the final classification result is obtained through the pixel-level classifier; a copy layer is set after the deconvolution layer to copy and bitwise add the feature matrix of the encoder part.

[0050] Specifically, in the encoder-decoder architecture, there are four convolutional layers, each of which is followed by three residual layers. Each residual layer contains two consecutive convolutional layers. The parameters of the first consecutive convolutional layer are the same as those of the convolutional layer. The second consecutive convolutional layer uses linear activation, and the other parameters are the same as those of the first consecutive convolutional layer. During encoding, the convolutional layer is used to obtain the feature map of the input image.

[0051] There are three deconvolution layers, and a copy layer is set after the deconvolution layer to copy and bitwise add the feature matrix of the encoder part. The feature map obtained during the connection encoding is subjected to three consecutive residual operations. Each residual layer contains two consecutive convolution layers. The parameters of the first consecutive convolution layer are the same as those of the convolution layer. The second consecutive convolution layer uses linear activation, and the other parameters are the same as those of the first consecutive convolution layer. During decoding, the deconvolution network is used to restore the pixel information layer by layer, and the final classification result is obtained by a pixel-level classifier in the last layer.

[0052] In this embodiment, the change feature generated by the front-end timing difference detection network is input into the head of the network to generate the prediction probability map of the specific branch, wherein the formula of the front-end timing difference detection network is:

[0053]

[0054] Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way of expressing differences in features.

[0055] Among them, the back-end time series difference detection network includes: ResNet50 network, encoder-decoder architecture and feature enhancement module; ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; encoder-decoder architecture is used to generate decoding features of two time series; feature enhancement module is used to enhance the decoding features according to the input dual-phase data, and calculate the change distance between the decoded dual-phase features to generate a probability map for change prediction.

[0056] Specifically, the back-end temporal difference detection network first performs independent encoding and decoding operations on the dual-phase images, and focuses on extracting the distance feature map between the dual-phase images. The back-end temporal difference detection network has the same ResNet50 network structure, encoder-decoder architecture, and feature enhancement module as the front-end temporal difference detection network. In the back-end temporal difference detection network, the dual-phase feature map is generated by the ResNet50 network and input into the encoder-decoder architecture to generate the decoding features of the two time series for subsequent feature difference calculations; the dual-phase features are input into the feature enhancement module to enhance the representation ability of the decoding features, and by calculating the change distance between the decoded dual-phase features, a probability map for change prediction is generated as the detection result.

[0057] Among them, the formula of the back-end timing difference detection network is:

[0058]

[0059] Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way to represent feature differences. By calculating the change distance between the decoded bi-temporal features, the back-end temporal difference detection network generates a probability map for change prediction.

[0060] It also includes: using the feature difference method to calculate the front-end dual-phase difference, the formula is:

[0061]

[0062] Where, d i Represents the time series difference characteristics, and They represent the feature maps of scale i generated based on the front ResNet50 network at the first time and the second time respectively; the Euclidean distance map of the two-phase feature pairs is calculated by the back-end temporal difference detection network, and the formula is:

[0063]

[0064] In the formula, oi Represents the Euclidean distance map; the contrast loss is calculated by comparing the Euclidean distance map with the actual label truth value.

[0065] Specifically, for the front-end dual-phase difference calculation, a feature difference method is adopted, which includes an element-by-element subtraction operation followed by an absolute value operation, to obtain the temporal difference features from the dual-phase features; a back-end temporal difference detection network is adopted to calculate the Euclidean distance map of the dual-phase feature pairs. In order to optimize the model, the model calculates the contrast loss by comparing the distance map with the actual label truth value.

[0066] Step S104, using the trained semantic change detection model, generating a difference map according to the time series remote sensing data to obtain a time series change detection result.

[0067] Specifically, the constructed semantic change detection model is trained, and the acquired time series remote sensing data is input into the trained semantic change detection model to generate a difference map. Based on the difference map, the time series change detection results are obtained, and the time series change detection results are analyzed and interpreted to identify surface changes in the mining area, thereby obtaining information on resource exploration, mineral management, and environmental monitoring.

[0068] Among them, the training loss function of the semantic change detection model is:

[0069] L=L f (f t (X 1 ,X 2 ),Y)+L b (f b (X 1 ,X 2 ),Y),

[0070] Using binary cross entropy loss function L bce and Dice loss function L dice As the loss function for training the two branches, the formulas are:

[0071] L bce = -Y·log f(X 1 ,X 2 )-(1-Y)·log(1-f(X 1 ,X 2 )),

[0072]

[0073] Where, L f represents the training loss function of the front-end timing difference detection network, L b represents the training loss function of the backend temporal difference detection network, X 1and X 2 are the input time series remote sensing data, Y is the corresponding true label result of the change area, and f t (·,·) and f b (·,·) are the front-end timing difference detection network and the back-end timing difference detection network respectively.

[0074] Specifically, in order to compare the change predictions from the two branches with the ground truth, the objective functions of the respective branch networks are calculated, and the overall goal is to optimize the entire model under computational constraints through a dual-branch loss function. The synergy of the two branches integrates the calculation of dual-phase features and difference features to achieve deep supervision, using the training loss function of the front-end branch and the training loss function of the back-end branch as the objective function of the entire model; the objective function uses the binary cross entropy loss function and the Dice loss function as the loss function for training the two branches. The optimization of the model relies on the constraints of the two branches, aiming to minimize the distance between the change prediction and the true label results of the change area.

[0075] Among them, step S104 also includes: integrating the first detection result of the front-end temporal difference detection network and the second detection result of the back-end temporal difference detection network to obtain the target difference detection result; generating target difference detection results for the temporal remote sensing data at different time points through the trained semantic change detection model, and obtaining the temporal change detection result based on the target difference detection results of all mining areas.

[0076] Specifically, the corresponding first detection result and the second detection result are obtained respectively through the front-end temporal difference detection network and the back-end temporal difference detection network, and the target difference detection result is obtained by comparison based on the first detection result and the second detection result. Through the trained semantic change detection model, the target difference detection results are generated for the temporal remote sensing data at different time points, and the temporal change detection results are obtained based on the target difference detection results of all mining areas. The dual-branch method is used to perform deep supervised analysis of remote sensing image changes, which improves the accuracy of the detection results.

[0077] In this embodiment, time series remote sensing data is obtained by acquiring high-resolution remote sensing images of a mining area at multiple time points and covering the entire range of the mining area, and the time series remote sensing data is preprocessed by atmospheric correction, radiation calibration and image registration operations, and a semantic change detection model based on deep learning is constructed and trained, wherein the semantic change detection module includes a front-end time series difference detection network and a back-end time series difference detection network, forming a semantic change detection model based on dual-branch time series change detection, thereby realizing the capture of effective features and time differences in dual-phase images from both the front-end and back-end perspectives, and through the correlation between the front-end and back-end time series difference detection networks, the backbone and feature enhancement attention modules are shared to promote the collaborative work of the two, and the long-range dependency modeling between the marked features is strengthened in the dual-branch architecture; the trained semantic change detection model is used to generate a difference map based on the time series remote sensing data to obtain a time series change detection result, and the synergy of the front-end and back-end dual-branch networks can simultaneously capture the features from the dual-time series mining patch data, and emphasize the time series differences, thereby improving the accuracy and efficiency of time series change detection, and can be applicable to change detection over a large range or over a long time series.

[0078] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0079] Obviously, those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer storage medium (ROM / RAM, magnetic disk, optical disk) and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0080] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A method for detecting temporal changes in remote sensing mining patterns based on dual-branch deep supervision, characterized in that: The following steps are involved: Acquire time-series remote sensing data of the mining area, wherein the time-series remote sensing data includes high-resolution remote sensing images at multiple time points and covering the entire range of the mining area; Preprocessing the time series remote sensing data, wherein the preprocessing includes atmospheric correction, radiation calibration and image registration operations; Constructing and training a semantic change detection model based on deep learning, wherein the semantic change detection model includes a front-end temporal difference detection network and a back-end temporal difference detection network, forming a semantic change detection model based on dual-branch temporal change detection; Among them, the characteristic difference method is used to calculate the front-end dual-phase difference, and the formula is: Where, d i Represents the time series difference characteristics, and They represent the feature maps of scale i generated based on the ResNet50 network at the first time and the second time respectively; The Euclidean distance map of the bi-phase feature pair is calculated through the back-end temporal difference detection network, and the formula is: In the formula, o i represents the Euclidean distance graph; Compute the contrast loss by comparing the Euclidean distance map with the actual label truth value; Using the trained semantic change detection model, generating a difference map according to the time series remote sensing data, and obtaining a time series change detection result; Among them, the training loss function of the semantic change detection model is: L=L f (f t (X 1 ,X 2 ),Y)+L b (f b (X 1 ,X 2 ),Y), Using binary cross entropy loss function L bce and Dice loss function L dice As the loss function for training the two branches, the formulas are: L bce =-Y·logf(X 1 ,X 2 )-(1-Y)·log(1-f(X 1 ,X 2 )), Where, L f represents the training loss function of the front-end timing difference detection network, L b represents the training loss function of the backend temporal difference detection network, X 1 and X 2 are the input time series remote sensing data, Y is the corresponding true label result of the change area, and f t (·,·) and f b (·,·) are the front-end timing difference detection network and the back-end timing difference detection network respectively, f(X 1 ,X 2 ) represents a front-end timing difference detection network or a back-end timing difference detection network.

2. According to the method for detecting time series changes of remote sensing mining patterns based on dual-branch deep supervision in claim 1, it is characterized in that: The front-end timing difference detection network includes: a ResNet50 network, a feature enhancement module and an encoder-decoder architecture; The ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; The feature enhancement module includes a multi-head self-attention unit, a multi-layer perceptron block and a batch normalization layer, which is used to capture global context information with long-range dependencies based on the self-attention mechanism, and calculate the temporal difference feature of the obtained dual-phase feature map; The encoder-decoder architecture is used to obtain corresponding detection results according to the obtained time sequence difference characteristics; The ResNet50 network includes an input layer, an initial convolutional layer, a pooling layer, multiple residual blocks, a global average pooling layer and a fully connected layer; Among them, the input layer is used to receive time series remote sensing data; the initial convolution layer and pooling layer are used to extract features from the time series remote sensing data to obtain a feature map, the initial convolution layer is a 7×7 convolution kernel with a step size of 2, and the pooling layer is a 3×3 maximum pooling with a step size of 2; the multiple residual blocks are stacked to form a deep network, each residual block includes a first convolution layer and a second convolution layer, and the output of the first convolution layer is directly added to the output of the second convolution layer to form a residual connection; the global average pooling layer is used to perform global average pooling on the feature map to reduce the spatial size of each feature map to 1×1; the fully connected layer is used to map the feature map after global average pooling to the category score.

3. The method for detecting time series changes of remote sensing mining patterns based on dual-branch deep supervision according to claim 2 is characterized in that: The encoder-decoder architecture includes four convolutional layers and three deconvolutional layers; The number of convolution kernels of the four convolution layers is 32, 64, 128 and 256 respectively, the size of the convolution kernel is 5×5, and the relu function is used. Three residual layers are connected after each convolution layer, and each residual layer contains two consecutive convolution layers. The feature map of the input image is obtained through the convolution layer. The number of convolution kernels of the three deconvolution layers is 128, 64, and 32 respectively, the size of the convolution kernel is 2×2, the step size is 2, and the feature map obtained by continuous encoding is subjected to three consecutive residual operations. Each residual layer contains two consecutive convolution layers. The pixel information is restored layer by layer through the deconvolution network, and the final classification result is obtained through the pixel-level classifier; A copy layer is arranged after the deconvolution layer, for copying and bitwise adding the feature matrix of the encoder part.

4. The method for detecting temporal changes in remote sensing mining patterns based on dual-branch deep supervision according to claim 3 is characterized in that: The formula of the front-end timing difference detection network is: Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way of expressing differences in features.

5. The method for detecting time series changes of remote sensing mining patterns based on dual-branch deep supervision according to claim 1 is characterized in that: The back-end timing difference detection network includes: a ResNet50 network, an encoder-decoder architecture, and a feature enhancement module; The ResNet50 network is used to generate dual-phase feature maps at different scales based on time series remote sensing data; The encoder-decoder architecture is used to generate two time-series decoding features; The feature enhancement module is used to enhance the decoded features according to the input bi-phase data, and calculate the change distance between the decoded bi-phase features to generate a probability map for change prediction.

6. The method for detecting temporal changes in remote sensing mining patterns based on dual-branch deep supervision according to claim 1 is characterized in that: The formula of the back-end timing difference detection network is: Among them, X 1 and X 2 are the input time series remote sensing data, e(·) and d(·) are the encoder and decoder respectively, A way of expressing differences in features.

7. The method for detecting temporal changes in remote sensing mining patterns based on dual-branch deep supervision according to claim 1 is characterized in that: Also includes: Integrate the first detection result of the front-end timing difference detection network and the second detection result of the back-end timing difference detection network to obtain a target difference detection result; Through the trained semantic change detection model, the target difference detection results are generated for the time series remote sensing data at different time points, and the time series change detection results are obtained based on the target difference detection results of all mining areas.