A gan instance-level local enhancement method for remote sensing change detection in open-pit mine

Abstract

The application provides a GAN instance-level local enhancement method for remote sensing change detection of an open-pit mine area, and comprises the following steps: S1. label generation, comprising double-time mine area label collection, construction and training of a label editing model, and local directional editing; S2. image generation, comprising double-time mine area image collection, construction and training of an image editing model, and local semantic editing; and S3. data set construction, wherein double-time data are randomly combined to construct a new mine area change detection data set. The method can realize instance-level local transformation, so that new real and diverse change detection samples are generated by using non-change mine area data, and the problem of lack of change detection data of the open-pit mine area based on a supervised deep learning model is solved to a great extent; and the method can expand the remote sensing change detection data set of the open-pit mine area, and thus the accuracy and robustness of the model are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data enhancement, in particular, to a GAN instance-level local enhancement method for remote sensing change detection of open-pit mining areas. BACKGROUND

[0002] Mineral resources are one of the most important natural resources. However, the exploitation of mineral resources, especially open-pit mining, has caused serious damage to the ecological environment (Chen, et al. 2018; Pratiwi et al. 2021). Reasonable and orderly development of mineral resources is crucial to the protection of the ecological environment and mineral resources. Therefore, it is necessary to detect changes in the surface mining area to effectively supervise the development process of the mining area. The method of change detection based on remote sensing images is widely used in open-pit mining change detection tasks (Yang et al. 2021; Nie et al. 2022; Han et al. 2021; Barenblitt et al. 2021), and high-resolution satellite data provides strong support for fine change detection (Zhi Yong et al. 2022). However, due to the complexity of high-resolution remote sensing image information, current high-resolution remote sensing image-based extraction of open-pit mining area change information is still mainly based on professional manual visual interpretation (Du et al. 2022).

[0003] In recent years, supervised deep learning technology has attracted more and more attention in open-pit mining change detection due to its strong image information extraction capability (Du et al. 2022; Tang et al. 2021; Camalan et al. 2022; Gallwey et al. 2020). However, this method usually requires a large amount of labeled data set. However, in the open-pit mining change detection task, due to the small number of mining area change regions, it is difficult to collect effective data. And self-annotation of change detection data set is also time-consuming and laborious, and even it may not be possible to obtain enough data even after searching the entire research area. For example, the open-pit mining change detection data set used by Du et al. (Du et al. 2022) has only 45 images. Secondly, the change area in the open-pit mining change detection task in units of years is smaller, and manual annotation is more difficult. However, if there is not enough training data, the change detection model based on supervised deep learning will lack enough generalization ability to adapt to complex remote sensing feature information and the influence of different imaging conditions.

[0004] Data augmentation provides a solution for supervised deep learning with limited data, and currently mainly focuses on methods based on image-level global transformation and image-level local transformation. Image-level global transformation has been widely used since early times, including basic methods such as flipping, rotation, color transformation, and adding noise (Kuny, Hammer, and Thiele 2022; Luo et al. 2021; Raghavan et al. 2022; Cui and Jiang 2022; Ding et al. 2016). Subsequently, some image-level local transformation methods were proposed, including Cutout (Devries and Taylor 2017), Cutmix (Yun et al. 2019), Mosica (Bochkovskiy, Wang, and Liao 2020), and others. However, these methods only rely on expanding existing data sets, which may not effectively solve the problem of small sample size or biased samples, thereby hindering the generalization and robustness of change detection models.

[0005] To solve this problem, some methods have recently emerged that can use other regional image data or semantic segmentation data to create new change detection data to expand training samples, mainly based on GAN methods. These methods include instance patch-based image synthesis methods (Yan, Yang, and Zhang 2022; Chen, Li, and Shi 2022), GAN-based image generation methods (Luo et al. 2021), and GAN-based local image editing methods (Rui et al. 2021). However, most of these methods (Yan, Yang, and Zhang 2022; Chen, Li, and Shi 2022; Rui et al. 2021) are designed for the appearance, disappearance, or attribute changes of the entire detection target, which belong to instance-level global transformation and cannot be used in mining areas where instance-level local changes may occur. The remaining methods, such as the method proposed by Luo et al. (Luo et al. 2021), belong to image-level global transformation, which has greater randomness in the generation results and requires manual labeling of data. There is currently no change detection data augmentation method that can achieve instance-level local transformation. SUMMARY

[0006] The present application provides a GAN instance-level local augmentation method for remote sensing change detection in open-pit mining areas to solve the problem of implementing instance-level local transformation methods and the lack of change detection data for supervised deep learning models in open-pit mining areas.

[0007] To solve the above technical problems, the technical solution of the present application is:

[0008] The present invention provides a GAN instance-level local augmentation method for remote sensing change detection in open-pit mines, comprising the following steps: S1. Label generation, including dual-time mine label collection, construction and training of a label editing model, and local directional editing; S2. Image generation, including dual-time mine image collection, construction and training of an image editing model, and local semantic editing; S3. Dataset construction: randomly combining dual-time data to construct a new mine change detection dataset.

[0009] Optionally, in the above-described GAN instance-level local augmentation method for remote sensing change detection in open-pit mines, in step S1, during the dual-time mine area label collection, the dual-time mine area images are first registered, then mine area feature labels are drawn based on these images, and finally, the images and labels are cropped into 256×256 pixels as a dataset, with the mine area feature labels being the original labels L used in the label generation stage. O .

[0010] Optionally, in the above-mentioned GAN instance-level local augmentation method for remote sensing change detection in open-pit mines, in step S1, during the construction and training of the label editing model, a GAN-based label editing model is first constructed; then, the mask, synthesized labels, input, and output of the label editing model are set during the training phase: the mask is one or more rectangular masks M randomly generated at any position on the label boundary. R The input is the masked M. R Original tag L after masking O That is, input = M R ×L O The output is the original label L. O Finally, the label editing model is trained so that it can edit M. R ×L O Automatically reverts to random but authentic non-directional mining area labels.

[0011] Optionally, in the above-described GAN instance-level local enhancement method for remote sensing change detection in open-pit mines, in step S1, during local orientation editing, an orientation label mask M is used. L To guide the label editing model in performing localized directional editing to generate data, the mask and input of the label editing model are set during the training phase: the mask is a rectangular mask M. R Only retain the original labels of the mining area features L O One side, namely the directional label mask M L The input is M after directional label masking. L Original tag L after masking O That is, input = M L ×L O; finally, directional editing of the label editing model is performed, so that the label editing model can synthesize real and diverse mine area synthetic labels L R ×L O for input, to synthesize real and diverse mine area synthetic labels L G .

[0012] Optionally, in the above GAN instance-level local enhancement method for open-pit mine area remote sensing change detection, in step S2, in the double-time mine area image collection, the mine area feature image is the original image I O .

[0013] Optionally, in the above GAN instance-level local enhancement method for open-pit mine area remote sensing change detection, in step S2, first, based on the DeepFill v2 model, combined with the DeepLab v3+ model, a GAN-based image editing model is constructed, and the DeepLab v3+ model is used to perform semantic segmentation on the generated result I G of the DeepFill v2 model to obtain a prediction result L P , in order to exclude the interference outside the editing area M I , the result of this area is replaced with the original label L O , and a new prediction result L PM is obtained:

[0014] L PM =(1-M I )×L O +M I ×L P

[0015] Then, according to the similarity between L PM and the synthetic label L G as the guide label, a semantic segmentation loss Loss seg is calculated, where L O is the original label, and M I is a semantic image mask:

[0016] Loss seg =|L G -L PM |

[0017] The generator loss value is adjusted by Loss seg to optimize the loss function and assist in training the DeepFill v2 model:

[0018] Loss=Loss D +(1-λ seg )×Loss G +λ seg ×Lossseg

[0019] In the formula, Loss D The total loss of the discriminator is represented by the Loss value. G Let λ represent the total loss of the original generator. seg Indicates Loss seg Optimization parameters for generator loss,

[0020] Then, set the mask, guidance labels, inputs, and outputs for the image editing model during the training phase: the mask is a rectangular mask M. R The guide label is the original label L. O The input is the masked M. R Original image I after masking O That is, input = M R ×I O The output is the original image I. O Finally, the image editing model is trained so that it can edit images based on the original label L. O Under the guidance of M R ×I O Automatically restores the original image of the actual mining area. O .

[0021] Optionally, in the above-described GAN instance-level local augmentation method for remote sensing change detection in open-pit mines, in step S2, during local semantic editing, the mask, guidance labels, and input of the image editing model are set during the training phase: the mask consists of three parts, the change region LC = |L O -L G |、M R Located in the mining area, feature label L O The inner part and the buffer zones of the former two, namely M I This allows for the simulation of changes in the mining area, construction traces within the mining area, and the edges of the mining area; the guiding label is the synthetic label L. G The input is the masked M. I Original image I after masking O That is, input = M I ×I O Finally, semantic editing of the image editing model is performed, enabling the model to synthesize labels L. G Under the guidance of M I ×I O As input, synthesize a realistic and semantically consistent synthetic image of the mining area I. G .

[0022] Optionally, in the GAN instance-level local enhancement method for remote sensing change detection of open-pit mine area described above, in step S3, the strategy of randomly combining double-time data includes two kinds: one is to randomly combine the original data at time T1 with the generated data at time T2 or to randomly combine the original data at time T2 with the generated data at time T1; the other is to randomly combine the generated data at time T1 with the generated data at time T2.

[0023] The beneficial effects of the present application are:

[0024] The method of the present application can realize instance-level local transformation, thereby generating new real and diverse change detection samples using no-change mine area data, and greatly solving the problem of lack of change detection data of open-pit mine area based on supervised deep learning model. Specifically, GAN instance-level local enhancement (GILA) realizes instance-level local transformation by sequentially editing the mine area label and image, and further constructs a new mine area change detection data set through random combination of double-time data. As an instance-level local transformation data enhancement method, GILA can expand the remote sensing change detection data set of open-pit mine area, and further improve the accuracy and robustness of the model. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the specific embodiments or prior art description will be briefly introduced as follows.

[0026] Figure 1 is a flowchart of the GAN instance-level local enhancement method for remote sensing change detection of open-pit mine area of the present application;

[0027] Figure 2 is a result display diagram of the GAN instance-level local enhancement method for remote sensing change detection of open-pit mine area of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application.

[0029] As Figure 1 shown, the GAN instance-level local enhancement method for remote sensing change detection of open-pit mine area of the present application includes the following steps:

[0030] S1. Label generation

[0031] a. Double-time mine area label collection

[0032] First, the dual-time (time T1 and time T2) images of the mining area were registered. Then, feature labels for the mining area were drawn based on these images. Finally, the images and labels were cropped into 256×256 pixels to form the dataset. The feature labels for the mining area are the original labels L used in the label generation stage. O The original label L, including time T1 O The original label L for time T2 O .

[0033] b. Construction and training of the tag editing model

[0034] First, based on the PICNet model (Zheng et al., 2019), a GAN-based label editing model is constructed. Then, the mask, synthesized labels, input, and output of the GAN-based label editing model are set during the training phase: the mask consists of one or more rectangular masks M randomly generated at arbitrary positions along the label boundaries. R The input is the masked M. R Original tag L after masking O That is, input = M R ×L O The output is the original label L. O Finally, the label editing model is trained so that it can edit M. R ×L O Automatically reverts to random but authentic non-directional mining area labels.

[0035] c. Localized directional editing

[0036] After training, the label editing model can already edit the original label L. O After local editing, directional tag mask M is also needed. L This guides the label editing model to perform localized directional editing to generate data, where the directional label mask M L Used to control the editing area. During localized directional editing, it is necessary to set the mask and input for this label editing model during the training phase: the mask is a rectangular mask M. R Only retain the original labels of the mining area features L O One side, namely M L This enables directional control of the expansion or contraction of mining areas; the input is the M-shaped directional label mask. L Original tag L after masking O That is, input = M L ×L O Finally, the targeted editing of the tag editing model is performed, enabling the tag editing model to use M... R ×L O As input, synthesize realistic and diverse mining area synthetic tags L G, including synthetic label L of time T1 G and synthetic label L of time T2 G , so as to simulate the process of destruction or repair of the ground surface in the mining of the mining area.

[0037] S2. Image generation

[0038] a. Double-time mining area image collection

[0039] In the double-time mining area label collection step of the previous label generation, the image and label have been cropped into 256x256 size patches as a dataset. The mining area feature image is the original image I used in the image generation stage O , including the original image I of time T1 O and the original image I of time T2 O .

[0040] b. Construction and training of image editing model

[0041] First, based on the DeepFill v2 model (Yu et al., 2019), combined with the DeepLab v3+ model (Chen et al., 2018), an image editing model based on GAN is constructed. The DeepLab v3+ model is used to perform semantic segmentation on the generated result I G of the DeepFill v2 model, to obtain the prediction result L P . In order to exclude the interference outside the editing area M I , the result of this area is replaced with the original label L O , to obtain a new prediction result L PM :

[0042] L PM = (1-M I ) x L O + M I x L P

[0043] Then, according to the similarity between L PM and the guide label L G (the guide label is the synthetic label L G ), the semantic segmentation loss Loss seg is calculated, where L O is the original label, and M I is the semantic image mask:

[0044] Loss seg = |L G -L PM |

[0045] Through Loss segAdjusting the generator loss value and optimizing the loss function helps train the DeepFill v2 model:

[0046] Loss = Loss D +(1-λ seg )×Loss G +λ seg ×Loss seg

[0047] In the formula, Loss D The total loss of the discriminator is represented by the Loss value. G Let λ represent the total loss of the original generator. seg Indicates Loss seg Optimization parameters for generator loss.

[0048] Then, the mask, guidance labels, inputs, and outputs of the GAN-based image editing model are set during the training phase: the mask is a rectangular mask M. R The guide label is the original label L. O (Original labels L at time T1 and time T2) O ); The input is the masked M R Original image I after masking O That is, input = M R ×I O The output is the original image I. O Finally, the image editing model is trained so that it can edit images based on the original label L. O Under the guidance of M R ×I O Automatically restores the original image of the actual mining area. O .

[0049] c. Local semantic editing

[0050] After training, the image editing model can already edit the original image I. O Local editing is required, followed by the use of a semantic image mask M. I And the previously generated synthetic tag L G This guides the model to perform local semantic editing, where the semantic image mask M I Used to control the editing area, and the composition tag L G Used to guide the editing process. During local semantic editing, it is necessary to set the mask, guidance labels, and input for the image editing model during the training phase: the mask consists of three parts, the change region LC = |L... O -L G |、M R Located in the mining area, feature label L O The inner part and the buffer zones of the former two, namely MI , thus achieving simulation of mining area changes, construction traces within the mining area, and the mining area edge; the guidance label is a synthetic label L G ; the input is the original image I I masked by the mask M O , that is, input = M I * I O . Finally, semantic editing of the image editing model is performed, so that the model can synthesize a real and semantically correct mining area synthetic image I G with M I * I O as input under the guidance of the synthetic label L G , including a synthetic image I G at time T1 and a synthetic image I G at time T2, thus simulating the damage or repair process to the ground surface during mining.

[0051] S3. Dataset construction

[0052] The double-time data are randomly combined to construct a new mining area change detection dataset. During the execution process, two different combination strategies are considered: the first strategy is to randomly combine the original data at time T1 (T2) with the generated data at time T2 (T1), and the second strategy is to randomly combine the generated data at time T1 and time T2. Finally, the constructed dataset includes double-time mining area images and a change label L, and the GAN-based image and label editing results and dataset construction results are shown in Figure 2 .

[0053] The following takes “the 3-band high-resolution remote sensing fused images of Liaoning Province in 2018 and 2019 provided by the Liaoning Province Mining Geological Environment Change Remote Sensing Image Dynamic Monitoring Project” as research data to illustrate the GAN instance-level local enhancement method for open-pit mining area remote sensing change detection of the present application.

[0054] Data introduction: The present study uses the 3-band high-resolution remote sensing fused images of Liaoning Province in 2018 and 2019 provided by the Liaoning Province Mining Geological Environment Change Remote Sensing Image Dynamic Monitoring Project. These images are multi-source fused images, including satellite data such as Gaofen 1, Gaofen 2, Gaofen 6, Beijing 2, and Resource 3, with a spatial resolution of 0.5 to 2 meters and a coverage time of every year from June to December. After preprocessing such as registration, the change map patches of the open-pit mining area are drawn by experts using artificial visual interpretation according to relevant standards [50, 51]. The images are resampled to 1 m to unify the image resolution, and are cropped into 256-sized non-overlapping images, finally obtaining 845 groups of open-pit mining area change detection data. The label is a 0-1 binary label, that is, only whether a change occurs is considered. The change detection dataset is randomly divided into a training set, a validation set, and a test set according to a ratio of 6:2:2.

[0055] An instance-level local augmentation method based on Generative Adversarial Network (GAN) for remote sensing change detection in open-pit mines mainly consists of three steps:

[0056] S1. Tag Generation

[0057] a. Dual-time mining area tag collection

[0058] First, the dual-time mining area images were registered. Then, mining area feature labels were drawn based on these images. Finally, the images and labels were cropped to a size of 256×256 patches to form the dataset. The mining area feature labels are the original labels L used in the label generation stage. O .

[0059] b. Construction and training of the tag editing model

[0060] First, based on the PICNet model (Zheng et al., 2019), a GAN-based label editing model is constructed. Then, the mask, synthesized labels, input, and output of the GAN-based label editing model are set during the training phase: the mask consists of one or more rectangular masks M randomly generated at arbitrary positions along the label boundaries. R The input is the masked M. R Original tag L after masking O That is, input = M R ×L O The output is the original label L. O Finally, the label editing model is trained so that it can edit M. R ×L O Automatically reverts to random but authentic non-directional mining area labels.

[0061] c. Localized directional editing

[0062] After training, the label editing model can already edit the original label L. O After local editing, directional tag mask M is also needed. L This guides the label editing model to perform localized directional editing to generate data, where the directional label mask M L Used to control the editing area. During localized directional editing, it is necessary to set the mask and input for this label editing model during the training phase: the mask is used to control the M... R Only retain the mining area feature label L O One side, namely M L This enables directional control of the expansion or contraction of mining areas; the input is the M-shaped directional label mask. L Original tag L after masking O That is, input = M LX L O Finally, the directional editing of the label editing model is performed, so that the label editing model can synthesize real and diverse mining area labels L R X L O as input, to synthesize real and diverse mining area labels L G including the synthesized label L G at time T1 and the synthesized label L G at time T2, so as to simulate the process of damage or repair to the ground surface in the mining of the mining area.

[0063] (2) Image generation

[0064] a. Double-time mining area image collection

[0065] In the double-time mining area label collection step of the previous label generation, the image and the label are cropped to a 256x256 size patch as a data set. The mining area feature image is the original image I O including the original image I O at time T1 and the original image I O at time T2, which is used in the image generation stage.

[0066] b. Construction and training of image editing model

[0067] First, based on the DeepFill v2 model (Yu et al., 2019), combined with the DeepLab v3+ model (Chen et al., 2018), an image editing model based on GAN is constructed. The DeepLab v3+ model is used to perform semantic segmentation on the generated result I G of the DeepFill v2 model, to obtain a prediction result L P , in order to exclude the interference outside the editing area M I , the result of this area is replaced with the original label L O , to obtain a new prediction result L PM :

[0068] L PM = (1 - M I ) x L O + M I x L P

[0069] Then, according to the similarity between L PM and the guide label L G , the semantic segmentation loss Loss seg is calculated, where L O is the original label and M I is the semantic image mask:

[0070] Loss seg= |L G -L PM |

[0071] By Loss seg Adjusting the generator loss value optimizes the loss function and assists DeepFill v2 in training:

[0072] Loss = Loss D + (1 - λ seg ) x Loss G + λ seg x Loss seg

[0073] In the formula, Loss D represents the total loss of the discriminator, Loss G represents the total loss of the original generator, λ seg represents the optimization parameter of Loss seg to the generator loss.

[0074] Secondly, the mask, guide label, input and output of the image editing model based on GAN need to be set in the training stage: the mask is a rectangular mask M R ; the guide label is the original label L O ; the input is the original image I O shielded by the mask M R , that is, input = M R x I O ; the output is the original image I O . Finally, the image editing model is trained so that the image editing model can automatically restore M R x I O to the real original image I O under the guidance of the original label L O .

[0075] c. Local semantic editing

[0076] After the image editing model is trained, it can already realize local editing of the original image I O , and then the semantic image mask M I and the previously generated label L G are used to guide the model to perform local semantic editing, wherein the semantic image mask M I is used to control the editing area, and the synthesized label L G is used to guide the editing process. In local semantic editing, the mask, guide label and input of the model need to be set in the training stage: the mask includes three parts, the change area LC O = |L G |, M RLocated in the mining area, feature label L O The inner part and the buffer zones of the former two, namely M I This allows for the simulation of changes in the mining area, construction traces within the mining area, and the edges of the mining area; the guiding label is the synthetic label L. G The input is the masked M. I Original image I after masking O That is, input = M I ×I O Finally, semantic editing of the image editing model is performed, enabling the model to synthesize labels L. G Under the guidance of M I ×I O As input, synthesize a realistic and semantically consistent synthetic image of the mining area I. G The composite image I includes time T1. G Composite image I at time T2 G This simulates the process of surface damage or repair during mining operations.

[0077] S3. Dataset Construction

[0078] A new dataset for detecting changes in the mining area was constructed by randomly combining data from two different time periods. During the process, two different combination strategies were considered: the first strategy randomly combined the original data from time T1 (T2) with the generated data from time T2 (T1), and the second strategy randomly combined the generated data from time T1 and time T2. The final dataset includes dual-time mining area images and change labels L.

[0079] The quality of the synthesized images was evaluated using Fréchet perceptual distance (FID) (Hensel et al. 2017) and kernel perceptual distance (KID) (Binkowski et al. 2018). Lower values ​​for both indicate better image realism and diversity. The results are shown in Table 1. Low FID and KID values ​​indicate high image quality, demonstrating the realism and diversity of the synthesized images.

[0080] Table 1. Image Quality Assessment

[0081]

[0082] For data augmentation effect evaluation, five advanced supervised deep learning change detection models are used as the baseline to verify the accuracy of the precision evaluation, which are FC-Siam-conc (Daudt, Saux, and Boulch 2018), SNUNet-CD (Fang et al. 2021), BIT-CD (Chen, Qi, and Shi 2021), A2Net (Li et al. 2023) and DMINet (Feng et al. 2023). In addition, the mean of the above five methods (Mean) is used as the final evaluation result. The evaluation indicators include overall accuracy (OA) and F1 score (F1). The data augmentation experiment results are shown in Table 2. After the GAN instance-level local augmentation (GILA) enhancement, the average OA of the model is improved from 97.57% to 98.12%, and the average F1 score is improved from 77.27% to 83.67%, which shows that GILA improves the effect of change detection. In addition, if only GILA synthetic data is used for training, the average OA reaches 97.64%, and the average F1 score is 78.65%, which can achieve the effect of training using real data.

[0083] In summary, GILA can use synthetic data to simulate real data, expand the open-pit mine remote sensing change detection dataset, and further improve the accuracy and robustness of the model.

[0084] Table 2. Data augmentation effect evaluation

[0085]

[0086] The above embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or improvements to the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features, within the technical scope disclosed by the present application. Such modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A GAN instance-level local enhancement method for remote sensing change detection in open-pit mines, characterized in that, Includes the following steps: S1. Label generation, including dual-time mining area label collection, construction and training of the label editing model, and local directional editing; in the construction and training of the label editing model, firstly, a GAN-based label editing model is constructed; then, the mask, synthesized labels, input, and output of the label editing model are set during the training phase: the mask is one or more rectangular masks M randomly generated at arbitrary positions on the label boundary. R The input is the masked M. R Original tag L after masking O That is, input = M R ×L O The output is the original label L. O Finally, the label editing model is trained so that it can edit M. R ×L O Automatically reconstructed into random but authentic non-directional mining area labels; S2. Image Generation; including dual-time mining area image collection, construction and training of image editing models, and local semantic editing; firstly, based on the DeepFill v2 model, combined with the DeepLab v3+ model, a GAN-based image editing model is constructed. The DeepLab v3+ model is used to process the generated results of the DeepFill v2 model. G Perform semantic segmentation to obtain the prediction result L. P To exclude the editing region M I External interference will cause the results in that region to be replaced with the original label L. O A new prediction result L was obtained. PM : L PM =(1―M I )×L O +M I ×L P Afterwards, according to L PM Combined with the synthetic label L as a guide label G Similarity calculation for semantic segmentation loss. seg , where L O For the original tag, M I For semantic image masking: Loss seg =|L G ―L PM | By Loss seg Adjusting the generator loss value and optimizing the loss function helps train the DeepFill v2 model: Loss=Loss D +(1-l seg )×Loss G +λ seg ×Loss seg In the formula, Loss D The total loss of the discriminator is represented by the Loss value. G Let λ represent the total loss of the original generator. seg Indicates Loss seg Optimization parameters for generator loss, Then, the mask, guidance labels, inputs, and outputs of the image editing model are set during the training phase: the mask is a rectangular mask M. R The guide label is the original label L. O The input is the masked M. R Original image I after masking O That is, input = M R ×I O The output is the original image I. O Finally, the image editing model is trained so that it can edit images in the original label L. O Under the guidance of M R ×I O Automatically restores the original image of the actual mining area. O ; S3. Dataset Construction: Randomly combine the two time-series data to construct a new mining area change detection dataset.

2. The GAN instance-level local enhancement method for remote sensing change detection in open-pit mines according to claim 1, characterized in that, In step S1, during the dual-time mining area label collection, the dual-time mining area images are first registered, then mining area feature labels are drawn based on these images, and finally, the images and labels are cropped into 256×256 pixels to form a dataset. The mining area feature labels are the original labels L used in the label generation stage. O .

3. The GAN instance-level local enhancement method for remote sensing change detection in open-pit mines according to claim 1, characterized in that, In step S1, during the local orientation editing, an orientation tag mask M is used. L To guide the label editing model in performing localized directional editing to generate data, the mask and input of the label editing model are set during the training phase: the mask is a rectangular mask M. R Only retain the original labels of the mining area features L O One side, namely the directional label mask M L The input is the directional label mask M. L Original tag L after masking O That is, input = M L ×L O Finally, the targeted editing of the label editing model is performed, enabling the label editing model to use M... R ×L O As input, synthesize realistic and diverse mining area synthetic tags L G .

4. The GAN instance-level local enhancement method for remote sensing change detection in open-pit mines according to claim 1, characterized in that, In step S2, during the dual-time mining area image collection, the mining area feature image is the original image I used in the image generation stage. O .

5. The GAN instance-level local enhancement method for remote sensing change detection in open-pit mines according to claim 1, characterized in that, In step S2, during the local semantic editing, the mask, guidance labels, and input of the image editing model are set during the training phase: the mask consists of three parts, the change region LC = |L O -L G |、M R Located in the mining area, feature label L O The inner part and the buffer zones of the former two, namely M I This allows for the simulation of changes in the mining area, construction traces within the mining area, and the edges of the mining area; the guiding label is the synthetic label L. G The input is the masked M. I Original image I after masking O That is, input = M I ×I O Finally, semantic editing of the image editing model is performed, enabling the model to synthesize labels L. G Under the guidance of M I ×I O As input, synthesize a realistic and semantically consistent synthetic image of the mining area I. G .

6. The GAN instance-level local enhancement method for remote sensing change detection in open-pit mines according to claim 1, characterized in that, In step S3, there are two strategies for randomly combining the dual-time data: one is to randomly combine the original data of time T1 with the generated data of time T2 or to randomly combine the original data of time T2 with the generated data of time T1; the other is to randomly combine the generated data of time T1 and time T2.

Images