An unsupervised domain adaptive method for transferring interpretation of different oil spill events

Abstract

The present invention provides an unsupervised domain adaptive migration interpretation method for different oil spill events, which belongs to the cross-technical field of marine remote sensing and artificial intelligence. The method first uses historical oil spill data to learn the target information of different oil spill events; secondly, the oil spill event data is accurately divided into strong samples and weak samples based on satellite image knowledge, marine environmental information and oil-seawater power ratio; finally, the Top- K Marginal probability improves the integrity of strong sample pseudo labels, uses adversarial training to improve the oil spill extraction capability of weak samples, and improves the target information of different oil spill events. The present invention can solve the differences in oil spill characteristics and the impact of only a small amount of data for each oil spill event, improve the oil spill extraction capability, and accurately realize the extraction of oil spill information.

Description

Technical Field

[0001] The invention belongs to the cross-technical field of marine remote sensing and artificial intelligence, and relates to an unsupervised domain adaptive migration interpretation method for different oil spill events. Background Art

[0002] Oil spill is an emergency incident at sea caused by the leakage of oil products during the exploration, exploitation and transportation of offshore oil. With the growing demand for oil, offshore oil spills occur frequently. The oil film flows with the waves and causes great harm to the natural environment, ecological environment, and social economy after spreading. Therefore, it is urgent to enhance the level of oil spill detection and improve the oil spill monitoring network. Synthetic aperture radar (SAR) satellites have the advantages of wide coverage, strong timeliness, and no interference from clouds and fog. They are of great significance for monitoring oil spill areas and protecting the marine ecological environment.

[0003] Due to the influence of factors such as the marine environment, SAR sensors, oil film thickness and type, there are obvious differences in the imaging features of oil spills in different SAR images. In addition, the number of SAR images obtained for each oil spill incident is limited, and it is impossible to effectively train the deep learning network. For example, the Chinese invention patent "Method for obtaining offshore oil spill area by analyzing SAR images using self-attention" (CN115546199A) only mentions the use of vertical polarization as the original image for oil spill detection, but does not analyze the differences in the imaging features of oil spills on different SAR images; on the other hand, the Chinese invention patent "Method for offshore oil spill detection based on multi-scale conditional adversarial network" (CN113269092A) uses a generative adversarial network to reduce the impact of small sample data, but ignores the significant differences between oil spill features under factors such as different satellites and different oil spill incidents, and does not consider the use of historical oil spill data to guide the generation of sample data in new oil spill incidents.

[0004] In view of the above problems, the present invention proposes an unsupervised domain adaptive migration interpretation method for different oil spill incidents, which is used for oil spill information extraction of different oil spill incidents. Summary of the invention

[0005] The present invention mainly solves the impact of the differences in oil spill features and oil spill events with a small amount of data on the performance of traditional semantic segmentation models. In view of the fact that the imaging features of the oil spill target and the seawater background in the oil spill event are different on different SAR images, an unsupervised domain adaptive migration interpretation method for different oil spill events is proposed. This method can handle different oil spill events in a timely and effective manner, and improve the timeliness and accuracy of oil spill extraction. Based on the historical oil spill data that is downloaded and publicly collected for free from satellite resource websites, the unsupervised domain adaptive method based on adversarial training in the output space is migrated to different oil spill events to extract oil spill features, so that the distribution of historical oil spill data can be extended to different oil spill events. A knowledge decision module is constructed based on satellite image knowledge, marine environmental information, and oil-seawater power ratio to accurately divide strong samples and weak samples in oil spill events, so as to prepare data for learning the unique information of different oil spill events. In addition, to ensure the integrity of oil spill features, the present invention proposes a method of in-domain iterative pseudo-labeling, that is, using pseudo-labels based on Top-K marginal probabilities to update strong samples, improve the oil spill integrity of strong samples, and use adversarial training to enable weak samples to further learn oil spill features, improve oil spill extraction capabilities, and accurately realize oil spill information extraction.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] An unsupervised domain adaptive migration interpretation method for different oil spill events, the migration interpretation method for different oil spill events first uses historical oil spill data to quickly and efficiently learn the target information of different oil spill events, and accurately divides the oil spill event data into strong samples and weak samples based on satellite image knowledge, marine environmental information and oil-seawater power ratio, and then uses Top-K marginal probability to improve the integrity of strong sample pseudo labels, uses adversarial training to improve the oil spill extraction ability of weak samples, and improves the target information of different oil spill events. Specifically, it includes the following steps:

[0008] In the first step, historical oil spill data is obtained from satellite resource websites and public data sets, and after preprocessing operations, it is cropped to obtain a historical oil spill data set, and labels are made based on labelme software, which is set as the source domain data, and the domain label of the source domain data is artificially set to 0. In addition, single oil spill event data different from historical oil spill data is obtained from satellite resource websites, and after preprocessing operations, it is cropped to obtain a single oil spill event data set, which is set as the target domain data, and the domain label of the target domain data is artificially set to 1. The preprocessing operations include thermal noise removal, radiation calibration, coherent scatter filtering, and terrain correction.

[0009] The second step, the inter-spill adaptation phase, minimizes the inter-domain differences between the source and target domains through adversarial training, which can be divided into two main parts: uncertainty measurement, see step 2.1. Segmentation model in inter-domain adaptation and the discriminator Adversarial training based on source domain data and target domain data can make the segmentation model It has the ability to generalize to different oil spill incidents. The details are as follows:

[0010] Step 2.1: In SAR images, some oil spill areas have a high degree of similarity with background noise, feature distribution is uniform, and inter-class differences are small, resulting in clear oil spill areas and seawater background in the semantic segmentation output results, as well as decision boundaries where oil spill pixels and seawater background pixels are similar. First, the source domain data and target domain data are input into the segmentation model respectively. In the above, the probability prediction map of the source domain data and the target domain data is obtained, and then the probability prediction map of the source domain data and the target domain data is converted into a boundary map of the source domain data and the target domain data containing only similar oil spill pixels and seawater background pixels based on the uncertainty measurement. The above oil spill pixels are set to 0, and the seawater background pixels are set to 1.

[0011] (1)

[0012] (2)

[0013] (3)

[0014] (4)

[0015] in, Represents the source domain data, Represents the target domain data, represents the segmentation model, represents the uncertainty measurement module, represents the source domain boundary map, represents the target domain boundary graph, is the probability of oil spill pixels in the probability prediction map, is the probability of the seawater background pixel in the probability prediction image.

[0016] Step 2.2: Segmentation model in inter-domain adaptation and the discriminator Adversarial training:

[0017] For the segmentation model Since the source domain data has labels, its probability prediction map and labels are used for segmentation loss; since the target domain data has no labels, the target domain boundary map is input into the discriminator The discrimination result of the target domain is obtained, and the domain label of the source domain data is subjected to adversarial loss.

[0018] (5)

[0019] (6)

[0020] (7)

[0021] in, represents the objective function of the segmentation model, represents the segmentation loss, Represents resistance to loss, is the loss weight that balances the two losses, represents the discriminator, represents the source domain label, Indicates the height of the image. Indicates the width of the image. Indicates the category of the image.

[0022] For the discriminator Optimize the source domain data and the target domain data to input the boundary map into the discriminator The discrimination results of the source domain and the target domain are obtained, and then the adversarial loss is performed with their respective domain labels, as shown in formula (8):

[0023] L inter D ( X s ,X t ) = − ∑ h,w [ log ( D inter ( U s ) ) +log ( 1 − D inter ( U t ) ) ] (8)

[0024] in, Representation Discriminator The objective function of .

[0025] The third step is to build a knowledge decision module to evaluate the image quality of the target domain data and divide it into strong samples and weak samples. First, we use the satellite image knowledge such as the resolution and polarization mode of the SAR image to select high-resolution and vertically polarized SAR images, and then determine whether the wind speed information at the location of the oil spill is between 2 and 7 m / s. Finally, we sort the oil-seawater power ratio by size, and divide the first 75% with smaller values ​​into strong samples in the target domain, and the rest into weak samples in the target domain.

[0026] The calculation method of the oil-seawater power ratio is as follows:

[0027] (9)

[0028] in, represents the average backscatter power of the corresponding area, and represent the backscattering coefficients of oil spill and seawater respectively.

[0029] Step 4: Use the segmentation model obtained in step 2 The loss weight generates the initial pseudo-label for the strong sample in the target domain. In order to reduce the error accumulation and negative transfer risk caused by the initial pseudo-label, an in-domain iterative pseudo-label method is designed, which mainly has two parts: the segmentation model in the in-domain adaptation and the discriminator For adversarial training, see step 4.1. For iterative update of pseudo labels of strong samples in the target domain, see step 4.2. Considering strong samples in the target domain as the source domain and weak samples in the target domain as the target domain, the domain label of strong samples in the target domain is 0, and the domain label of weak samples in the target domain is 1.

[0030] Step 4.1: In-domain adaptation of the segmentation model and the discriminator For the weak samples in the target domain, the weak samples in the target domain are input into the segmentation model. The probability prediction map is obtained from the image, and then the target domain weak sample boundary map is obtained through uncertainty measurement, and then input into the discriminator The discrimination result of weak samples in the target domain is obtained, and the adversarial loss is performed with the domain label of strong samples in the target domain to optimize the segmentation model. , as shown in formula (10):

[0031] (10)

[0032] in, represents the weak sample of the target domain, represents the discriminator, represents the weak sample boundary map of the target domain, Represents resistance to loss.

[0033] Input strong samples from the target domain into the segmentation model The probability prediction map is obtained from the image, and then the strong sample boundary map of the target domain is obtained through uncertainty measurement, and then input into the discriminator The discrimination result of the target domain strong sample is obtained in . The discrimination result of the target domain weak sample is subjected to adversarial loss with the domain label of the target domain weak sample. The adversarial loss made by combining the discrimination result of the target domain strong sample with the domain label of the target domain strong sample is used to optimize the discriminator. , as shown in formula (11):

[0034] L intra D ( X ts ,X tw ) = − ∑ h,w [ log ( D intra ( U ts ) ) +log ( 1 − D intra ( U tw ) ) ] (11)

[0035] in, represents a strong sample in the target domain, represents the strong sample boundary map of the target domain, Representation Discriminator The objective function of .

[0036] Step 4.2, iterative update of pseudo labels of strong samples in the target domain. For strong samples in the target domain, in addition to using adversarial loss to optimize the discriminator In addition, the segmentation model is also used Iteratively update the initial pseudo-label of the strong sample in the target domain to make segmentation loss with the probability prediction map of the strong sample in the target domain to optimize the segmentation model Specifically:

[0037] In the iterative process, first, a blank tensor is created to store the pixels of the oil spill category; then, the segmentation model is transformed into The loss weights are updated to the segmentation model ; In the first iteration, the initial pseudo-label of the strong sample in the target domain and the probability prediction map of the strong sample in the target domain are used for segmentation loss, and the marginal probability of the oil spill category in the initial pseudo-label of the strong sample in the target domain is sorted from large to small, and the Top-K is set to 1, and the rest are set to 0, and stored in the tensor for updating the initial pseudo-label of the strong sample in the target domain in the subsequent iteration. In the second iteration, the segmentation model Generate the initial pseudo-label of the strong sample in the target domain of the second round, which is covered by the pixel point with a value of 1 in the tensor and updated as the pseudo-label of the strong sample in the target domain of the second round. After the segmentation loss is completed with the probability prediction map of the strong sample in the target domain, the tensor is still updated based on the Top-K marginal probability, and so on, iterates round by round until convergence is reached.

[0038] Final segmentation model The optimization is shown in the following formula:

[0039] (12)

[0040] (13)

[0041] (14)

[0042] (15)

[0043] (16)

[0044] (17)

[0045] in, represents a pseudo label, represents the initial pseudo label, represents the initial pseudo-label of each round in the iterative process, Represents a tensor storing oil spill pixels, Representation of segmentation model in inter-domain adaptation The optimal weight of represents the segmentation model that generates the initial pseudo-labels, Represents the minimum probability value of oil spill pixels based on Top-K marginal probability, Represents the probability set of oil spill pixels in the initial pseudo-label sorted from large to small.

[0046] The beneficial effects of the present invention are:

[0047] The present invention can solve the differences in oil spill characteristics and the impact of only a small amount of data for each oil spill event. The unsupervised domain adaptive migration interpretation method for different oil spill events adopts adaptation between oil spill events, only decision modules and domain iterative pseudo labels, and migrates the historical oil spill data collected for free to a single oil spill event with a small amount of data to extract oil spill information, so that the distribution of historical oil spill data is gradually extended to different oil spill events. In order to better learn the oil spill characteristics of different oil spill events, the data of oil spill events are more reasonably divided based on satellite image knowledge, marine environmental information and oil-seawater power ratio, and the risk of negative migration is initially reduced. The Top-K marginal probability is then used to improve the oil spill integrity of the strong sample in the target domain, so that it gradually approaches the real label, and adversarial training is used to enhance the ability of the weak sample in the target domain to extract oil spill features, so that it can further learn oil spill features. In the specific verification process, the collected experimental data covers oil spill events at different times, different locations, different marine environments, different SAR sensors, different oil film thicknesses and types, which verifies the effectiveness of the invention in different oil spill events. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A method for transferring interpretation of different oil spill incidents without supervised domain adaptation;

[0049] Figure 2 Iterative update graph of strong sample pseudo labels in the target domain; Figure 2 (a) in the figure represents an oil spill image with strong samples in the target domain. Figure 2 (b) in the figure is the label map. Figure 2 (c) in the figure is the initial pseudo label of the strong sample in the target domain. Figure 2 (d) in the figure is the pseudo label of the strong sample in the target domain after iterating i rounds. Figure 2 (e) in the figure is the pseudo label of the strong sample in the target domain after iterating j rounds. Figure 2 (f) in the figure is the pseudo label of the last round of strong samples in the target domain, where i<j.

[0050] Figure 3 The oil spill extraction result of SAR image is shown with Sentinel-1 as source domain data and oil spill incident A as target domain data; Figure 3 (a) in the figure represents the oil spill image of oil spill event A. Figure 3 (b) in the figure is the label map. Figure 3(c) in the figure is the segmentation result image.

[0051] Figure 4 The oil spill extraction result of SAR image is shown with Sentinel-1 as source domain data and oil spill event B as target domain data; Figure 4 (a) in the figure represents the oil spill image of oil spill event B. Figure 4 (b) in the figure is the label map. Figure 4 (c) in the figure is the segmentation result image.

[0052] Figure 5 The oil spill extraction result of SAR image is shown with Sentinel-1 as source domain data and oil spill event C as target domain data; Figure 5 (a) in the figure represents the oil spill image of oil spill event C. Figure 5 (b) in the figure is the label map. Figure 5 (c) in the figure is the segmentation result image.

[0053] Figure 6 The oil spill extraction result of SAR image is shown with Sentinel-1 as source domain data and oil spill event D as target domain data; Figure 6 (a) in the figure represents the oil spill image of oil spill event D. Figure 6 (b) in the figure is the label map. Figure 6 (c) in the figure is the segmentation result image.

[0054] Figure 7 The oil spill extraction result of SAR image with ERS-1 / 2 as source domain data and oil spill event A as target domain data is shown in the figure. Figure 7 (a) in the figure represents the oil spill image of oil spill event A. Figure 7 (b) in the figure is the label map. Figure 7 (c) in the figure is the segmentation result image.

[0055] Figure 8 The oil spill extraction result of SAR image with ERS-1 / 2 as source domain data and oil spill event B as target domain data is shown in the figure. Figure 8 (a) in the figure represents the oil spill image of oil spill event B. Figure 8 (b) in the figure is the label map. Figure 8 (c) in the figure is the segmentation result image.

[0056] Fig. 9 The oil spill extraction result of SAR image with ERS-1 / 2 as source domain data and oil spill event C as target domain data is shown in the figure. Fig. 9 (a) in the figure represents the oil spill image of oil spill event C. Fig. 9 (b) in the figure is the label map. Fig. 9 (c) in the figure is the segmentation result image.

[0057] Fig.10 The oil spill extraction result of SAR image with ERS-1 / 2 as source domain data and oil spill event D as target domain data is shown in the figure. Fig.10 (a) in the figure represents the oil spill image of oil spill event D. Fig.10 (b) in the figure is the label map. Fig.10 (c) in the figure is the segmentation result image. DETAILED DESCRIPTION

[0058] In order to make the method problems solved by the present invention, the method solutions adopted and the method effects achieved clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It is understood that the specific examples described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the contents.

[0059] like Figure 1 As shown, an unsupervised domain adaptive migration interpretation method for different oil spill incidents provided by an example of the present invention includes:

[0060] Compiled in python3.6, pytorch1.7.1 and cuda12.2 under Windows10 system, run on NVIDIA GeForce RTX 3090 GPU, input data size of 512×512 SAR image, output 512×512 semantic segmentation result map, where the segmentation model , and Use SGD optimizer to optimize, discriminator and The Adam optimizer is used for optimization, the decay rate of EMA is set to 0.99, and the loss weight and It is set to 0.001 and K in Top-K is set to 0.8.

[0061] In the first step, historical oil spill data is obtained from satellite resource websites and public data sets, and after preprocessing operations, it is cropped to obtain a historical oil spill data set, and labels are made based on labelme software, which is set as the source domain data, and the domain label of the source domain data is artificially set to 0. In addition, single oil spill event data different from historical oil spill data is obtained from satellite resource websites, and after preprocessing operations, it is cropped to obtain a single oil spill event data set, which is set as the target domain data, and the domain label of the target domain data is artificially set to 1. The preprocessing operations include thermal noise removal, radiation calibration, coherent scatter filtering, and terrain correction.

[0062] The source domain data includes two datasets, 130 scenes of Sentinel-1 data and 14 scenes of ERS-1 / 2 data. The target domain data includes four datasets, each of which represents a real oil spill incident, which are divided into oil spill incident A, oil spill incident B, oil spill incident C and oil spill incident D. The data used are GF-3, COSMO-SkyMed or Radarsat-2 data. Each time, only one source domain data and one target domain data are selected for experiment. For example, when the Sentinel-1 dataset is selected as the source domain data, the oil spill incident A dataset is first selected as the target domain data for experiment. Figure 3 As shown in (a) in Figure 2, we select the oil spill incident B dataset as the target domain data for the experiment. Figure 4 As shown in (a) in , the oil spill incident C dataset is selected as the target domain data for the experiment, as shown in Figure 5 As shown in (a) in the figure, the oil spill incident D dataset is finally selected as the target domain data for the experiment, as shown in Figure 6 When the ERS-1 / 2 dataset is selected as the source domain data, the oil spill event A dataset, oil spill event B dataset, oil spill event C dataset or oil spill event D dataset are selected as the target domain data for experiments, respectively. Figure 7 (a) Figure 8 (a) Fig. 9 (a) and Fig.10 As shown in (a) in .

[0063] Step 1.1, thermal noise removal: The thermal noise removal operation is used to reduce or eliminate the impact of thermal noise.

[0064] Step 1.2, Radiometric calibration: Radiometric calibration is used to correct for sensor errors and determine the precise radiometric value at the sensor entrance, taking into account cloud cover.

[0065] Step 1.3, coherent scattering filtering: Since coherent speckle noise is a common phenomenon in SAR images, the most commonly used Refined Lee filter is used to remove the coherent speckle noise, where the filter window size is set to 7×7.

[0066] Step 1.4, terrain correction: Use the range Doppler method to perform terrain correction to ensure that the real geographic coordinates correspond to the remote sensing image.

[0067] Step 1.5, create data sets and labels: After the above preprocessing, the SAR images of the source domain data and the target domain data are cropped to 512×512 pixels as the input of the model. In addition, the labelme software is used to create labels for the source domain data.

[0068] The second step, the inter-spill adaptation phase, minimizes the inter-domain differences between the source and target domains through adversarial training, which can be divided into two main parts: uncertainty measurement, see step 2.1. Segmentation model in inter-domain adaptation and the discriminator Adversarial training based on source domain data and target domain data can make the segmentation model It has the ability to generalize to different oil spill incidents. The details are as follows:

[0069] Step 2.1: First, input the source domain data and target domain data into the segmentation model respectively. In the above, the probability prediction map of the source domain data and the target domain data is obtained, and then the probability prediction map of the source domain data and the target domain data is converted into a boundary map of the source domain data and the target domain data containing only similar oil spill pixels and seawater background pixels based on the uncertainty measurement. The oil spill pixels are set to 0, and the seawater background pixels are set to 1. The more similar the oil spill is to the seawater background, and The closer, the The bigger, the The greater the uncertainty, the or The corresponding pixel in is set to 1. When the difference between the oil spill and the seawater background is large, Can accurately identify oil spills and seawater background, resulting in and The greater the difference, the The smaller, or The corresponding pixel in is set to 0.

[0070] (1)

[0071] (2)

[0072] (3)

[0073] (4)

[0074] in, Represents the source domain data, Represents the target domain data, represents the segmentation model, represents the uncertainty measurement module, represents the source domain boundary map, represents the target domain boundary graph, is the probability of oil spill pixels in the probability prediction map, is the probability of the seawater background pixel in the probability prediction image.

[0075] Step 2.2: Segmentation model in inter-domain adaptation and the discriminator Adversarial training:

[0076] For the segmentation model Since the source domain data has labels, its probability prediction map and labels are used for segmentation loss to learn the feature distribution of the source domain data. As the training progresses, its loss continues to decrease until convergence. Since the target domain data has no labels, the target domain boundary map is input into the discriminator. The target domain discrimination result is obtained from the source domain data, and the domain label of the source domain data is subjected to adversarial loss, so that the target domain data gradually learns the characteristic distribution of the source domain data. In the early stage of training, the loss is large, and gradually decreases as the training progresses. Ideally, the loss will tend to a balanced state and no longer fluctuate violently. Finally, save the segmentation model The optimal weight of is used to generate a probability prediction map for the strong sample in the target domain, which is used as the initial pseudo label of the strong sample in the target domain, such as Figure 2 As shown in (c) in .

[0077] (5)

[0078] (6)

[0079] (7)

[0080] in, represents the objective function of the segmentation model, represents the segmentation loss, Represents resistance to loss, is the loss weight that balances the two losses, represents the discriminator, represents the source domain label, Indicates the height of the image. Indicates the width of the image. Indicates the category of the image.

[0081] For the discriminator Optimize the source domain data and the target domain data to input the boundary map into the discriminator The discrimination results of the source domain and the target domain are obtained, and then adversarial losses are performed with their respective domain labels, and finally the discriminator Confusion occurs, and it becomes increasingly difficult to distinguish between source domain data and target domain data. The total loss is low at the beginning of training, and gradually increases as training progresses, and finally reaches a medium level when the training stabilizes, as shown in formula (8):

[0082] L inter D ( X s ,X t ) = − ∑ h,w [ log ( D inter ( U s ) ) +log ( 1 − D inter ( U t ) ) ] (8)

[0083] in, represents the objective function of the discriminator.

[0084] The third step is to build a knowledge decision module to evaluate the image quality of the target domain data and divide it into strong samples in the target domain and weak samples in the target domain. First, the satellite image knowledge such as the resolution and polarization mode of the SAR image is used. For example, the higher the resolution, the better the image quality. The vertically polarized ocean image is brighter when the incident angle is large, while the intensity difference between the four polarization modes is similar when the incident angle is small. In addition, the wind speed information of the oil spill location is judged. At a wind speed of 2~7m / s, the sea surface is slightly rough, and the oil film is easier to observe on the SAR image. When the wind speed is 0~2 m / s, thin oil films are not visible on the SAR image. When the wind speed is 7~12 m / s, only thicker oil films can be observed. Due to the forced diffusion of wind, thinner oil films are easy to break and cannot be observed on the SAR image. When the wind speed exceeds 12 m / s, even thick oil films are difficult to observe on the SAR image. Finally, the oil-seawater power ratio is used to quantify the difference between the oil spill and seawater. The smaller the oil-seawater power ratio, the better. The smaller the value, the greater the contrast and difference. The larger the value, the smaller the contrast between the oil film and the seawater, and the greater the similarity. The oil-seawater power ratio is sorted from small to large, and the first 75% is classified as the target domain strong sample, such as Figure 2 The leftmost column in is shown, and the rest are weak samples in the target domain. The calculation method of the oil-seawater power ratio is as follows:

[0085] (9)

[0086] in, represents the average backscatter power of the corresponding area, and represent the backscattering coefficients of oil spill and seawater respectively.

[0087] Step 4: Use the segmentation model obtained in step 2 The weights generate initial pseudo labels for strong samples in the target domain, such as Figure 2 As shown in (c) in the figure, in order to reduce the error accumulation and negative transfer risk caused by the initial pseudo-label, an in-domain iterative pseudo-label method is designed, which mainly has two parts: the segmentation model in the in-domain adaptation and the discriminator For adversarial training, see step 4.1. For iterative update of pseudo labels of strong samples in the target domain, see step 4.2. Considering strong samples in the target domain as the source domain and weak samples in the target domain as the target domain, the domain label of strong samples in the target domain is 0, and the domain label of weak samples in the target domain is 1.

[0088] Step 4.1: In-domain adaptation of the segmentation model and the discriminator For the weak samples in the target domain, the weak samples in the target domain are input into the segmentation model. The probability prediction map is obtained from the image, and then the target domain weak sample boundary map is obtained through uncertainty measurement, and then input into the discriminator The discrimination result of weak samples in the target domain is obtained, and the adversarial loss is performed with the domain label of strong samples in the target domain to optimize the segmentation model. , in the early stage of training, its loss is large, and gradually decreases as the training progresses. Ideally, its loss will tend to a balanced state and no longer fluctuate violently, as shown in formula (10):

[0089] (10)

[0090] in, represents the weak sample of the target domain, represents the discriminator, Represents the weak sample boundary map of the target domain, Represents resistance to loss.

[0091] Strong samples from the target domain are input to the segmentation model The probability prediction map is obtained from the image, and then the strong sample boundary map of the target domain is obtained through uncertainty measurement, and then input into the discriminator The discrimination result of the target domain strong sample is obtained in . The discrimination result of the target domain weak sample is subjected to adversarial loss with the domain label of the target domain weak sample, and the adversarial loss made by combining the discrimination result of the target domain strong sample with the domain label of the target domain strong sample is used to optimize the discriminator. , the total loss is low at the beginning of training, gradually increases as training progresses, and finally reaches a medium level when the training stabilizes, as shown in formula (11):

[0092] L intra D ( X ts ,X tw ) = − ∑ h,w [ log ( D intra ( U ts ) ) +log ( 1 − D intra ( U tw ) ) ] (11)

[0093] in, represents a strong sample in the target domain, Represents the strong sample boundary map of the target domain, Representation Discriminator The objective function of .

[0094] Step 4.2, iterative update of the pseudo-labels of strong samples in the target domain. The iterative update process is as follows: Figure 2 As shown, for strong samples in the target domain, in addition to using adversarial loss to optimize the discriminator In addition, the segmentation model is also used Iteratively update the initial pseudo-label of the strong sample in the target domain to make segmentation loss with the probability prediction map of the strong sample in the target domain to optimize the segmentation model , as the training progresses, its loss continues to decrease until convergence. Specifically, in the iterative process, a blank tensor is first created to store the pixels of the oil spill category, and then the segmentation model is transformed into The weights of the segmentation model are updated , in the first iteration, Figure 2 (c) in the figure, that is, the initial pseudo-label of the strong sample in the target domain and the probability prediction map of the strong sample in the target domain are used for segmentation loss, and the marginal probabilities of the oil spill category in the initial pseudo-label of the strong sample in the target domain are sorted from large to small, and the Top-K are set to 1, and the rest are set to 0, and stored in the tensor for updating the initial pseudo-label of the strong sample in the target domain in the subsequent iteration process. In the second iteration, the segmentation model Generate the initial pseudo-label of the strong sample in the target domain of the second round, which is covered by the pixel points with a value of 1 in the tensor and updated as the pseudo-label of the strong sample in the target domain of the second round, such as Figure 2 As shown in (d) in the figure, after the segmentation loss is done with the probability prediction graph of the strong sample in the target domain, the tensor is still updated based on the Top-K marginal probability, and so on, iterating round by round to gradually improve the pseudo label quality of the strong sample in the target domain, as shown in Figure 2 As shown in (e) in , until the convergence state is reached, at this time, the pseudo label of the strong sample in the target domain is optimal, as shown in Figure 2 Finally, save the segmentation model The optimal weight of is used to generate a probability prediction map for the target domain data, which is used as the segmentation result of the target domain data. Figure 3 (c) Figure 4 (c) Figure 5 (c) Figure 6 (c) Figure 7 (c) Figure 8 (c) Fig. 9 (c) Fig.10 As shown in (c) in .

[0095] (12)

[0096] (13)

[0097] (14)

[0098] (15)

[0099] (16)

[0100] (17)

[0101] in, represents a pseudo label, represents the initial pseudo-label, represents the initial pseudo-label of each round in the iterative process, Represents a tensor storing oil spill pixels, Representation of segmentation model in inter-domain adaptation The optimal weight of represents the segmentation model that generates the initial pseudo-labels, Represents the minimum probability value of oil spill pixels based on Top-K marginal probability, Represents the probability set of oil spill pixels in the initial pseudo-label sorted from large to small.

[0102] Finally, it should be noted that the above examples are only used to illustrate the method scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above examples, ordinary method personnel in the field should understand that modifying the method schemes recorded in the above examples, or equivalently replacing some or all of the method features therein, does not cause the essence of the corresponding method scheme to deviate from the scope of the method schemes of the examples of the present invention.

Claims

1. An unsupervised domain adaptive migration interpretation method for different oil spill events, characterized in that: The migration interpretation method of different oil spill events first uses historical oil spill data to learn the target information of different oil spill events, and accurately divides the oil spill event data into strong samples and weak samples based on satellite image knowledge, marine environmental information and oil-seawater power ratio, and then uses Top-K marginal probability to improve the integrity of strong sample pseudo labels, and uses adversarial training to improve the oil spill extraction ability of weak samples, and improves the target information of different oil spill events. Specifically, it includes the following steps: The first step is to obtain historical oil spill data, perform preprocessing operations on it, and then crop it to obtain a historical oil spill data set, and create labels, set it as source domain data, and artificially set the domain label of the source domain data to 0; at the same time, obtain single oil spill event data different from the historical oil spill data, perform preprocessing operations on it, and then crop it to obtain a single oil spill event data set, set it as target domain data, and artificially set the domain label of the target domain data to 1; the source domain data and the target domain data are SAR image data; The second step is to minimize the inter-domain difference between the source domain and the target domain through adversarial training in the inter-oil spill adaptation stage, which is divided into two parts: uncertainty measurement and segmentation model G in inter-domain adaptation. inter and the discriminator D inter The adversarial training is as follows: Step 2.1: First, input the source domain data and target domain data into the segmentation model G inter In the method, the probability prediction map of the source domain data and the target domain data is obtained, and then the probability prediction map of the source domain data and the target domain data is converted into a boundary map of the source domain data and the target domain data containing only similar oil spill pixels and seawater background pixels based on the uncertainty measurement; the oil spill pixels are set to 0, and the seawater background pixels are set to 1; U s =U inter (G inter (X s ))(1) U t =U inter (G inter (X t ))(2) Uinter(Ginter(Xs))=1-|p0-p1|(3) Uinter(Ginter(Xt))=1-|p0-p1|(4) Among them, X s represents the source domain data, X t represents the target domain data, G inter represents the segmentation model, U inter represents the uncertainty measurement module, U s represents the source domain boundary graph, U t represents the target domain boundary map, p0 is the probability of oil spill pixels in the probability prediction map, and p1 is the probability of seawater background pixels in the probability prediction map; Step 2.2: Segmentation model G in inter-domain adaptation inter and the discriminator D inter Adversarial training: For the segmentation model G inter The source domain data has labels, and its probability prediction map and labels are used for segmentation loss; the target domain data has no labels, and the target domain boundary map is input into the discriminator D inter The discrimination result of the target domain is obtained, and the domain label of the source domain data is subjected to adversarial loss; in, represents the objective function of the segmentation model, represents the segmentation loss, represents the adversarial loss, α adv is the loss weight that balances the two losses, D inter represents the discriminator, Y s represents the source domain label, h represents the height of the image, w represents the width of the image, and c represents the category of the image; For the discriminator D inter The boundary map of the source domain data and the target domain data is input into the discriminator D inter The discrimination results of the source domain and the target domain are obtained, and then adversarial losses are performed with their respective domain labels, as shown in formula (8): in, Represents the discriminator D inter The objective function of The third step is to build a knowledge decision-making module to evaluate the image quality of the target domain data and divide it into strong samples and weak samples. First, select high-resolution and vertically polarized SAR images, and then determine whether the wind speed information at the location of the oil spill is between 2 and 7 m / s. Combined with the oil-seawater power ratio, it is divided into strong samples in the target domain, and the rest are weak samples in the target domain. Specifically, combined with the size sorting of the oil-seawater power ratio, the first 75% with smaller values ​​are divided into strong samples in the target domain, and the rest are weak samples in the target domain. The fourth step is to use the segmentation model G obtained in the second step inter The loss weights generate initial pseudo labels for strong samples in the target domain and design an in-domain iterative pseudo labeling method, which mainly consists of two parts: the in-domain adaptation segmentation model G intra and the discriminator D intra The adversarial training and iterative update of pseudo labels of strong samples in the target domain are performed; the strong samples in the target domain are regarded as the source domain, and the weak samples in the target domain are regarded as the target domain. Then, the domain label of the strong samples in the target domain is 0, and the domain label of the weak samples in the target domain is 1; the details are as follows: Step 4.1: In-domain adaptation of the segmentation model G intra and the discriminator D intra For the weak samples in the target domain, the weak samples in the target domain are input into the segmentation model G intra The probability prediction map is obtained from the image, and then the target domain weak sample boundary map is obtained through uncertainty measurement, and then input into the discriminator D intra The discrimination result of weak samples in the target domain is obtained, and the adversarial loss is performed with the domain label of strong samples in the target domain to optimize the segmentation model G. intra , as shown in formula (10): Among them, X tw represents the weak sample in the target domain, D intra represents the discriminator, U tw represents the weak sample boundary map of the target domain, It means fighting against loss; Strong samples from the target domain are input to the segmentation model G intra The probability prediction map is obtained from the image, and then the strong sample boundary map of the target domain is obtained through uncertainty measurement, and then input into the discriminator D intra The discrimination result of the strong sample in the target domain is obtained; the discrimination result of the weak sample in the target domain is subjected to adversarial loss with the domain label of the weak sample in the target domain, and the adversarial loss made by combining the discrimination result of the strong sample in the target domain with the domain label of the strong sample in the target domain is used to optimize the discriminator D intra , as shown in formula (11): Among them, X ts represents a strong sample in the target domain, U ts represents the strong sample boundary map of the target domain, Represents the discriminator D intra The objective function of Step 4.2, iterative update of pseudo labels of strong samples in the target domain. For strong samples in the target domain, in addition to using adversarial loss to optimize the discriminator D intra In addition, the segmentation model G is also used teacher Iteratively update the initial pseudo-label of the strong sample in the target domain to make segmentation loss with the probability prediction map of the strong sample in the target domain to optimize the segmentation model G intra , the final segmentation loss of the target domain strong sample and the iterative update of the target domain strong sample pseudo label are shown in the following formula: p K =min({p i1 ,p i2 ,…,p ik })(17) in, represents a pseudo label, represents the initial pseudo-label, represents the initial pseudo-label of each round in the iteration process, T represents the tensor storing the oil spill pixel points, Represents the segmentation model G in inter-domain adaptation inter The optimal weight of G teacher represents the segmentation model that generates the initial pseudo-labels, p K represents the minimum probability value of oil spill pixels based on Top-K marginal probability, {p i1 ,p i2 ,…,p ik } represents the probability set of oil spill pixels in the initial pseudo-label sorted from large to small.

2. According to claim 1, the unsupervised domain adaptive migration interpretation method for different oil spill incidents is characterized in that: The pre-processing operation in the first step comprises the following steps: Step 1.1, thermal noise removal; Step 1.2, radiation calibration; Step 1.3, coherent scatter filtering; Step 1.4, terrain correction.

3. The unsupervised domain adaptive migration interpretation method for different oil spill incidents according to claim 1 is characterized in that: In the third step, the oil-seawater power ratio is calculated as follows: Where, mean(·) represents the average backscatter power of the corresponding area, and represent the backscattering coefficients of oil spill and seawater respectively.

4. The unsupervised domain adaptive migration interpretation method for different oil spill incidents according to claim 1 is characterized in that: The specific step 4.2 is as follows: In the iterative process, first create a blank tensor to store the pixels of the oil spill category, and then use EMA to transform the segmentation model G intra The weights of the segmentation model G are updated teacher ; In the first iteration, the initial pseudo-label of the strong sample in the target domain and the probability prediction map of the strong sample in the target domain are used for segmentation loss, and the marginal probability of the oil spill category in the initial pseudo-label of the strong sample in the target domain is sorted from large to small, and the Top-K is set to 1, and the rest are set to 0, and stored in the tensor for updating the initial pseudo-label of the strong sample in the target domain in the subsequent iteration process; In the second iteration, the segmentation model G teacher Generate the initial pseudo-label of the strong sample in the target domain of the second round, which is covered by the pixel point with a value of 1 in the tensor and updated as the pseudo-label of the strong sample in the target domain of the second round. After the segmentation loss is completed with the probability prediction map of the strong sample in the target domain, the tensor is still updated based on the Top-K marginal probability, and so on, iterates round by round until convergence is reached.

Images