Accurate classification and semantic segmentation method for multiple types of oil spill analogues
By employing a multi-type oil spill analogue accurate classification and semantic segmentation model, utilizing a dual-network structure and a label probability controller to adaptively adjust sample probabilities, and combining marine environmental information, the model solves the problems of misclassification and incomplete segmentation caused by class imbalance in marine oil spill monitoring, and achieves high-precision identification and segmentation of oil spill areas.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN UNIV OF TECH
- Filing Date
- 2024-07-03
- Publication Date
- 2026-07-17
AI Technical Summary
Existing technologies for marine oil spill monitoring suffer from biased estimations leading to misclassification due to class imbalance and incomplete oil spill segmentation results, especially in complex scenarios where it is difficult to effectively learn global and local information.
A multi-type oil spill analogue accurate classification and semantic segmentation model is adopted. It utilizes a dual-network structure, a label probability controller and a dual-generator collaborative network, combined with marine environmental information and prior knowledge. The generator is optimized through cross-complementary operations and discriminator feedback, and the sample probability is adaptively adjusted to alleviate the class imbalance problem, thereby extracting global and local feature information of oil spills.
It improves the accuracy and reliability of oil spill classification, effectively alleviates model bias caused by imbalance of similar samples, and enhances the accuracy of oil spill area identification and segmentation integrity.
Smart Images
Figure CN118864846B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the interdisciplinary field of marine remote sensing and artificial intelligence, and relates to a method for accurate classification and semantic segmentation of multiple types of oil spill analogues. Background Technology
[0002] Marine oil spills are sudden maritime incidents caused by the leakage of oil products during oil exploration, extraction, and transportation. The increasing global demand for oil, the development of offshore oil platforms, and the continuous growth of the maritime transport industry have led to frequent marine oil spills. After an oil spill, it flows with the waves on the ocean surface, spreading and posing a significant threat to the health of the marine ecosystem, seriously endangering the marine ecological environment and coastal socio-economic development. Compared to traditional methods such as ship-based and fixed-point monitoring, satellite remote sensing is widely used in marine oil spill monitoring due to its advantages of wide monitoring range and high efficiency. Marine oil spill satellite remote sensing monitoring is mainly divided into two categories: optical satellite remote sensing and Synthetic Aperture Radar (SAR) satellite remote sensing. SAR, with its advantages of all-weather, all-time coverage and high resolution, has become an important means of oil spill monitoring. Different polarization methods in SAR images affect oil spill detection; VV polarization, due to its strong backscattering ability, can effectively detect oil spills. However, due to the capillary waves and short gravity waves that suppress sea surface roughness caused by oil spills, the oil-slicked sea surface appears as a dark area in SAR images. Yet, SAR images also contain many similar-looking dark areas, such as upwellings, biofilms, leeward areas, and low-wind-speed areas. Therefore, quickly and effectively identifying the true oil spill area is of significant scientific importance.
[0003] Furthermore, the complex imaging characteristics of dark areas in wide-area SAR imagery make it unclear how to distinguish between oil spill analogues and actual oil spills. Class imbalance among oil spill analogue data can lead to biased estimations and misclassifications of smaller categories. Additionally, in oil spill segmentation tasks, oil spill segmentation networks struggle to effectively guarantee both global and local information in complex scenarios.
[0004] To address the aforementioned problems, this invention proposes a precise classification and semantic segmentation model for multiple types of oil spill analogues. The model comprises two networks: a precise classification network for oil spill analogues and a semantic segmentation network for oil spills. In the precise classification network, the categories of oil spill analogues are determined based on marine environmental information. A one-stage discriminator is used for refined multi-classification of oil spill analogues, reducing false positives. Based on this, a label probability controller is proposed, which controls the generator through prior and posterior probabilities, automatically adjusting the probability of generating false samples and balancing the number of samples to reduce biased estimation. In the semantic segmentation network, a two-stage dual-generator oil spill segmentation and cross-complementary operations are proposed, effectively ensuring both global and local information about the oil spill, ultimately achieving precise classification of analogues and segmentation of the oil spill area within a single framework. Summary of the Invention
[0005] This invention primarily addresses the issues of biased estimation and incomplete oil spill segmentation results caused by class imbalance among oil spill analogues in intelligent oil spill interpretation methods. It effectively mitigates the impact of sample imbalance and the difficulty in effectively learning global and local information about oil spills on model results. Based on long-term time-series satellite imagery such as Sentinel-1, GF-3, and ERS-1 / 2, a multi-type oil spill analogue accurate classification and semantic segmentation model is proposed. Random noise and multi-class label information are input into the generator to obtain pseudo-samples with class information. A label probability controller is designed to adaptively control the sample probability generated by the generator, mitigating the class imbalance problem among oil spill analogues. Furthermore, prior knowledge from experts and the marine environment is embedded into the accurate classification network for oil spill analogues to improve the accuracy and reliability of classification results. A dual-generator collaborative oil spill semantic segmentation network is proposed to extract global and local features of the oil spill. The generator is optimized by feedback from the discriminator, and iterative updates mitigate the problems of missed detection in small oil spill areas and incomplete oil spill edge regions.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] A method for accurate classification and semantic segmentation of multiple types of oil spill analogues is proposed. First, based on various oil spill analogues and oil spill monitoring data from SAR images in complex scenarios, a dual-network structure is established, including an oil spill analogue accurate classification network and an oil spill semantic segmentation network. The oil spill analogue accurate classification network includes a label probability controller, a one-stage generator, and a one-stage discriminator; the oil spill semantic segmentation network includes a two-stage dual generator and a two-stage discriminator, with cross-complement operations. Second, random noise and multi-class label information are input into the one-stage generator, and a label probability controller is designed to adaptively control the sample probability generated by the one-stage generator. Furthermore, prior knowledge such as marine environmental information is embedded in the one-stage discriminator to improve the accuracy and reliability of the classification results. Finally, a method is proposed whereby the two-stage dual generator collaboratively extracts global and local oil spill features, and the two-stage discriminator provides feedback to optimize the two-stage dual generator, iteratively updating the process. Cross-complement operations are then used to generate the oil spill segmentation results. Specifically, the method includes the following steps:
[0008] The first step involves collecting remote sensing satellite data on oil spills and similar objects from Sentinel-1, GF-3, and ERS-1 / 2, and cropping SAR images to a size of 256x256 containing oil spills and various similar targets. A multi-classification training strategy and a label probability controller for oil spill similar objects are designed. The multi-classification training strategy achieves refined multi-classification of dark areas in the SAR images, while the label probability controller alleviates the class imbalance problem among oil spill similar objects, improving the accuracy and robustness of the first-stage discriminator in classifying oil spill similar objects. The specific details are as follows:
[0009] 1.1) Establish a one-stage discriminator D c and the first-stage generator G c A precise classification network for oil spill analogues.
[0010] The first-stage generator G c It consists of convolutional layers, Leaky ReLU function, Batch Normalization (BN) layer, Tanh function, and upsampling layer. The BN layer prevents changes in data distribution during training, avoiding gradient vanishing or exploding, and speeds up training. The upsampling layer increases the width and length of the input features. The Tanh function connects the convolutional layers as the end. Pseudo-samples with random noise z and oil spill classification information c are input into the one-stage generator G. c Then, output the result G. c (z) Input to the first-stage discriminator D c The first-stage discriminator D is described in the text. c It consists of multiple convolutional blocks, each containing a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function, and a one-stage discriminator D. c Output x iThe formula is shown in (1):
[0011] x i =D c (G c (z),X c (1)
[0012] Where, x i This represents the one-stage discriminator D. c The output feature vector, where i represents the number of feature vector elements, X c Represents remote sensing imagery with category c information, D c (·) represents a one-stage discriminator that can learn feature information from oil spill analogue images and oil spill images. G c (z) represents the one-stage generator G. c The output result.
[0013] One-stage discriminator D c The last convolutional block is followed by a convolutional block using Sigmoid and Logsoftmax activation functions as a one-stage discriminator D. c The output of the Sigmoid function can effectively distinguish between real images and pseudo images generated by the generator, and the Logsoftmax function is used for classification, transforming the single classification problem into a multi-class classification problem. The activation function formulas of Sigmoid and Logsoftmax are shown in equations (2) and (3) respectively:
[0014]
[0015] Where exp(·) is the exponential operation, log(·) is the logarithmic operation, and x i This represents the one-stage discriminator D. c The output is a feature vector, where i represents the number of feature vector elements. The Logsoftmax function outputs classification probability values, enabling multi-classification of oil spills and similar substances.
[0016] 1.2) Due to the imbalance among oil spill analogue categories, low-wind-speed areas account for a larger proportion, while leeward areas and biofilms are relatively less. This category imbalance can lead to biased estimation problems in the model. A label probability controller is designed to adaptively control the sample probabilities generated by the generator to mitigate the category imbalance problem among oil spill analogues.
[0017] The tag probability controller consists of prior probability and posterior probability. Prior probability P c (i) is a one-stage generator G c The probability of each type of oil spill analogue is generated from the training data, and the posterior probability P is... e (i) is a one-stage generator G cIn each iteration of the oil spill analogue precise classification network, based on the one-stage discriminant D... c The probability of generating the output error rate is determined by designing a sample balancing calculation function, and P is then solved from the balancing calculation function. c (i) The probability calculation function is as shown in equations (4) and (5):
[0018]
[0019] Where L represents the total number of categories, C n G represents the total number of samples used in model training. n G represents a one-stage generator c Total number of samples generated, P c (i) represents the control of the first-stage generator G. c The probability of each class is determined by the training samples. C(i) represents the number of training samples for each class.
[0020] To reduce the error rate of oil spill analogue classification in the precise classification network, the error rate of each class is calculated in each iteration of the precise classification network, and the error rate is used to guide the first-stage generator G. c Adaptive processing is employed. Categories of oil spill analogues with higher error rates are assigned higher probability values, and the generator produces a large number of pseudo-sample data for that category. This, in turn, helps the oil spill analogue classification network improve the accuracy of its classification. e (i) is the probability calculation function, as shown in equation (6):
[0021]
[0022] Among them, P e (i) indicates that the first-stage generator G is guided. c Generate the error rate for each class probability, where Acc(i) represents the accuracy during training for each class. This represents the sum of error rates for all categories. A probability balance parameter is added to balance the ratio between these values.
[0023] The probability value for each category is generated by a one-stage generator G. c The generated prior probability P c (i) and posterior probability P e (i) Determined jointly, as shown in equation (7):
[0024] P(i)=βP c (i)+(1-β)P e (i)(7)
[0025] Where β is the probability balance parameter, and P(i) represents the final control of the first-stage generator G. cThe probability value generated for each category.
[0026] The second step involves designing a deep and shallow dual-generator oil spill semantic segmentation network to preserve both global and local information about the oil spill and accurately extract the oil spill area. Specifically:
[0027] In the oil spill semantic segmentation network, a deep two-stage generator G is used. a Shallow two-stage generator G b Two-stage discriminator D s The deep two-stage generator G consists of intersection and complement operations. a It is a U-shaped fully convolutional symmetric structure with both downsampling and upsampling symmetric structures, used to extract local feature information of oil spills. Shallow two-stage generator G b It consists of multiple convolutional layers and convolutional blocks, where each convolutional block contains a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function. Its output layer consists of a convolutional layer and a Tanh activation function, used to extract global feature information of the oil spill. Two-stage discriminator D s It consists of multiple convolutional blocks connected together, each containing the same convolutional layer, BN layer, and LeakyReLU function. The most special one is that the last convolutional block contains only convolutional layers.
[0028] The two-stage generator G a and G b Using the original oil spill images as input, the oil spill segmentation result is output after interpolation. The oil spill segmentation result, the original oil spill image, and the segmentation labels are then combined. Figure 1 The input is fed into the two-stage discriminator D. s Discriminant learning is performed to learn the differences in data distribution among various oil spill outcomes, and a discrimination score is output. The discrimination score is used as feedback to optimize the dual two-stage generator G. a and G b This is to generate more accurate oil spill segmentation results. At this point, the two-stage generator G... a and G b It can improve its ability to generate oil spill segmentation images and successfully deceive the two-stage discriminator D. s Finally, the two-stage discriminator D... s After training with the oil spill semantic segmentation network, the data is discarded, and the dual two-stage generator G is used. a and G b reserve.
[0029] The bi-stage generator G trained as described above a and G b This is used to generate oil spill segmentation results containing global and local feature information. Further, through cross-complementation operations, misjudged areas caused by SAR image noise are removed, and detailed information of the oil spill boundaries is completed to obtain the final segmentation results, as shown in equations (8) and (9):
[0030]
[0031] y = Comp(G a (X oil ),G b (X oil ))(9)
[0032] Among them, X oil For a one-stage discriminator D c Output oil spill image data set, For oil spill classification, y represents the generated oil spill segmentation result, and Comp(·) represents the cross-complement operation.
[0033] The third step is to design the loss function of the precise classification network for oil spill analogues, so that the stable gradient generated by the precise classification network for oil spill analogues neither vanishes nor explodes.
[0034] The objective loss function of the precise classification network for oil spill analogues consists of two parts:
[0035] The first part is L N It is a loss function used to determine whether data is true; the second part, L. C The function is a cost function for the classification accuracy of network data for precise classification of oil spill analogues, L. N Functions and L C The functions are shown in equations (10) and (11):
[0036] L N =E[logP(N=real|X real )]+E[logP(N=fake|X fake )](10)
[0037] L C =E[logP(C=c|X real )]+E[logP(C=c|X fake )](11)
[0038] Where C represents the category, X fake =G c (C,z) represents random noise z passing through a one-stage generator G. c Generated pseudo-samples. X real This represents real data. For the one-stage discriminator D... c Determine the probability distribution P(N|X) of whether the original data is real data and the probability distribution P(C|X) of the classification label, respectively.
[0039] During the training of the network for accurate classification of oil spill analogues, the one-stage generator G...c The optimization direction is to minimize L N -L C This means that the generated data can be treated as realistically as possible, and the data can be effectively categorized. Generator G c The optimization function is defined as shown in equation (12):
[0040]
[0041] Where min is the minimization operation function.
[0042] One-stage discriminator D c The optimization direction is to maximize L. N +L C The corresponding physical meaning is that we hope the first-stage discriminator D c It can distinguish between real data and generated fake data as much as possible, and can effectively classify the data. One-stage discriminator D c The optimization function is defined as shown in equation (13):
[0043]
[0044] Where min is the minimization operation function.
[0045] The fourth step is to design the loss function for the oil spill semantic segmentation network.
[0046] During the training of the network for accurate classification of oil spill analogues, a one-stage discriminator D was used. c Classification is performed. However, in the multi-classification and segmentation stages of oil spill analogues, the convolutional parts of the bi-discriminators in the precise classification network for oil spill analogues and the semantic segmentation network for oil spills have the same network design, using a pre-trained one-stage discriminator D. c Weights are shared with the two-stage discriminator D s To improve training speed and accuracy, the following measures are taken:
[0047] The loss function of the oil spill semantic segmentation network is trained using WGAN-GP gradient penalty loss to ensure the stability of the network. The function is defined as shown in equation (14):
[0048]
[0049] Among them, G a,b G represents a or G b λ1 is the L1 norm constraint as a balance parameter, I is the oil spill image, and S is the input oil spill segmentation label image. To generate G a and G b The results of the oil spill separation. It is the L1 norm, and its function is to penalize the distance.
[0050] By minimizing the training of the two-stage dual generator G a and G b The goal is to generate a system capable of deceiving the two-stage discriminator D. s The real segmented image, two-stage discriminator D s The function is defined as shown in equation (15):
[0051]
[0052] Where λ2 is the gradient penalty weight balancing parameter. To counteract the loss and improve the discriminator's discriminative ability, the training of the two-stage discriminator D is minimized. s The generated segmented label image is distinguished from the oil spill segmentation image, and the discriminant score is output to guide the two-stage dual generator G. a and G b .
[0053] Based on the functions shown in equations (14) and (15), the target loss function of the oil spill semantic segmentation network is defined as shown in equation (16):
[0054]
[0055] Where min is the minimization operation function and max is the maximization operation function.
[0056] The oil spill semantic segmentation network discards the two-stage discriminator D after training. s Retain the two-stage generator G a and G b Using a two-stage generator G a and G b Generate global and local oil spill segmentation maps and perform cross-complementation operations to achieve accurate segmentation.
[0057] The beneficial effects of this invention are as follows:
[0058] (1) To address the challenges of traditional deep learning algorithms in SAR image oil spill identification tasks, such as biased estimation leading to misclassification due to class imbalance among oil spill analogues and incomplete oil spill segmentation results caused by a large amount of speckle noise, this invention analyzes long-term satellite images such as Sentinel-1, GF-3, and ERS-1 / 2, inputting random noise and multi-class label information into the generator. A label probability controller is specifically designed to adaptively control the sample probability generated by the generator, mitigating the model collapse problem caused by class imbalance among oil spill analogues. Prior knowledge such as marine environmental information is embedded into the network to improve the accuracy and reliability of the classification results. Simultaneously, a dual-generator collaborative network is proposed to extract global and local oil spill features, combining discriminator feedback to optimize the generator, iteratively updating to effectively alleviate the problems of missed detection in small oil spill areas and incomplete oil spill edge regions, enhancing the model's performance and robustness.
[0059] (2) The method proposed in this invention has high accuracy and high reliability, and can meet the requirements of accurate classification of real oil spills and multiple similar substances, alleviate the problem of model collapse caused by unbalanced samples, and accurately identify irregular oil spills. Attached Figure Description
[0060] Figure 1 A method for accurate classification and semantic segmentation of multiple types of oil spill analogues;
[0061] Figure 2 Schematic diagram of remote sensing image data of oil spill and some similar substances: (a) leeward slope, (b) biofilm, (c) upwelling, (d) low wind speed area, (e) oil spill;
[0062] Figure 3 Schematic diagram of remote sensing image data and labels for oil spill, (a) is oil spill image data, (b) is the true value of the label;
[0063] Figure 4 The classification results of the precise classification and semantic segmentation method for multiple types of oil spill analogues are as follows: (a) oil spill, (b) biofilm, (c) leeward slope, (d) upwelling, and (e) low wind speed area.
[0064] Figure 5 The segmentation results of the precise classification and semantic segmentation method for multiple types of oil spill analogues: (a) is the original oil spill image, (b) is the ground truth label, and (c) is the segmentation result. Detailed Implementation
[0065] To make the problem solved by the present invention, the method adopted, and the effect achieved by the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the accompanying drawings, not all of them.
[0066] like Figure 1 As shown, this invention relates to a method for accurate classification and semantic segmentation of multiple types of oil spill analogues, specifically including the following steps:
[0067] The code was compiled using Python 3.8.10, PyTorch 1.7.1, and CUDA 10.1 on a Windows 10 system and run on an RTX 2080Ti GPU.
[0068] The first step was to collect SAR remote sensing satellite data on oil spills and similar substances from Sentinel-1, GF-3, and ERS-1 / 2 between 2014 and 2022. The data on oil spills and similar substances were then subjected to radiometric calibration, image enhancement, and terrain correction. Finally, datasets and labels were created through preprocessing.
[0069] 1.1) Radiometric calibration: Converting digital image data into actual physical quantities such as reflectivity, radiance, or temperature to eliminate radiometric bias in SAR images caused by sensors.
[0070] 1.2) Image enhancement: The Lee filtering method with a window size of 7×7 is used to reduce speckle noise in SAR images.
[0071] 1.3) Terrain Correction: SAR remote sensing satellite data on oil spills and similar objects have certain geometric distortions, so registration and correction are required.
[0072] 1.4) Dataset Creation: Multiple remote sensing images containing oil spills and similar substances were selected and cropped to 256×256 pixels. A total of 200 images were collected from the Sentinel-1 dataset, with oil spill similar substances accounting for 68%. The training and testing set ratio was set to 7:3. Specifically, four oil spill images were used for training on the Sentinel-1 and ERS-1 / 2 datasets, and 20 images were used for testing. Four oil spill images from the GF-3 satellite imagery were used as training data, and five oil spill images were used to test the accurate classification and semantic segmentation methods for multiple types of oil spill similar substances. Training and testing data information is shown in Table 1, and schematic diagrams of oil spills and some similar substances are shown below. Figure 2 As shown.
[0073] Table 1. Dataset information used for model training and testing.
[0074]
[0075] Finally, LabelMe software was used to create labels through visual interpretation. The oil spill remote sensing imagery data and labels are shown below. Figure 3 As shown.
[0076] The second step is to establish a one-stage discriminator D. c and the first-stage generator G c A precise classification network for oil spill analogues.
[0077] 2.1) The first-stage generator G c The generator G consists of a convolutional layer, a LeakyReLU function, a BN layer, a Tanh function, and an upsampling layer. It takes pseudo-samples with random noise z and oil spill classification information c as input to a one-stage generator. c Then, output the result G. c (z) Input to the first-stage discriminator D c The first-stage discriminator D is described in the text. c It consists of 4 convolutional blocks, each containing a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function, with a one-stage discriminator D. c Output x i The formula is shown in (1):
[0078] x i =D c (G c (z),X c (1)
[0079] Where, x i This represents the one-stage discriminator D. c The output feature vector, where i represents the number of feature vector elements, X c Represents remote sensing imagery with category c information, D c (·) represents a one-stage discriminator, G c (z) represents the one-stage generator G. c The output result.
[0080] One-stage discriminator D c The last convolutional block is followed by a convolutional block using Sigmoid and Logsoftmax activation functions as a one-stage discriminator D. c The output of the Sigmoid and Logsoftmax activation functions is shown in equations (2) and (3), respectively:
[0081]
[0082] Where exp(·) is the exponential operation, log(·) is the logarithmic operation, and x i This represents the one-stage discriminator D. c The output feature vector, where i represents the number of feature vector elements, allows the trained model to automatically determine the category from unknown images. Simultaneously, the Logsoftmax function outputs classification probability values, enabling multi-class classification of five categories: oil spill, upwelling, oil film, leeward area, and low-wind-speed area.
[0083] 2.2) The tag probability controller consists of prior probability and posterior probability. Prior probability P c (x i ) is a one-stage generator G c The probability of each type of oil spill analogue is generated from the training data, and the posterior probability P is... e (x i ) is a one-stage generator G c In each iteration of the oil spill analogue precise classification network, based on the one-stage discriminant D... c The probability of generating the output error rate is determined by designing a sample balancing calculation function, and P is then solved from the balancing calculation function. c (i) The probability calculation function is as shown in equations (4) and (5):
[0084]
[0085] Where L represents the total number of categories, C n G represents the total number of samples used in model training. n G represents a one-stage generator c Total number of samples generated, P c (x i ) indicates that the first-stage generator G is controlled. c The probability of being in one of the five classes determined by the training data, C(x) i ) indicates the amount of training data for the 5 categories.
[0086] The generator will produce a large number of pseudo-sample data for oil spill analogue categories with high error rates. This is to improve the accuracy of the oil spill analogue classification network. e (i) is the probability calculation function, as shown in equation (6):
[0087]
[0088] Among them, P e (x i ) indicates the guidance of the one-stage generator G c The error rate for generating the probability of each class, Acc(x) i () represents the accuracy during training for each category. This represents the sum of error rates for all categories. A probability balance parameter is added to balance the ratio between these values.
[0089] The probability value for each category is generated by a one-stage generator G. c The generated prior probability P c (i) and posterior probability P e (i) Determined jointly, as shown in equation (7):
[0090] P(x i )=βP c (x i )+(1-β)P e (x i (7)
[0091] Where β is the probability balance parameter, P(x) i ) represents the final control one-stage generator G c The generated probability values for the 5 categories, the second step as a whole, are as follows: Figure 1 The upper part shows the classification results. Figure 4 As shown.
[0092] The third step is to design a dual-generator oil spill segmentation network for identifying deep and shallow layers of the oil spill. (See...) Figure 1 The lower half is shown below:
[0093] In the oil spill semantic segmentation network, a deep two-stage generator G is used. a Shallow two-stage generator G b Two-stage discriminator D s The deep two-stage generator G consists of intersection and complement operations. a It features a dual-symmetric U-shaped structure with four downsampling and four upsampling operations, followed by a series connection of one convolutional layer and three convolutional blocks. Shallow two-stage generator G b It consists of one convolutional layer and three convolutional blocks linked in series. Each convolutional block contains three layers: one convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function. Its output layer consists of one convolutional layer and a Tanh activation function. Two-stage discriminator D s The structure consists of four convolutional blocks connected together, each consisting of the same convolutional layer, BN layer, and LeakyReLU function. The only special feature is that the last one only has a convolutional layer.
[0094] The two-stage generator G a and G b Using the original oil spill images as input, the oil spill segmentation result is output after interpolation. The oil spill segmentation result, the original oil spill image, and the segmentation labels are then combined. Figure 1 The input is fed into the two-stage discriminator D. s Then, the two-stage discriminator D... sPerform discriminative learning and output a discriminative score. The discriminative score feedback optimizes the bi-two-stage generator G. a and G b This is to generate accurate oil spill segmentation results. At this point, the two-stage generator G... a and G b It can improve its ability to generate oil spill segmentation results and successfully deceive the two-stage discriminator D. s Finally, the two-stage discriminator D... s After training with the oil spill semantic segmentation network, the data is discarded; the two-stage generator G... a and G b reserve.
[0095] The bi-stage generator G trained as described above a and G b This is used to generate oil spill segmentation results containing global and local feature information. Further, through cross-complementation operations, misclassified areas caused by SAR image noise are removed, and detailed information of the oil spill boundaries is completed to obtain the final segmentation results, as shown in equations (8) and (9):
[0096]
[0097] y = Comp(G a (X oil ),G b (X oil ))(9)
[0098] Where X oil For a one-stage discriminator D c Output oil spill image data set, For oil spill classification, y represents the generated segmentation result, and Comp(·) represents the cross-complement operation.
[0099] The fourth step is to design the loss function of the precise classification network for oil spill analogues. The target loss function of the precise classification network for oil spill analogues consists of two parts:
[0100] The first part is L N It is a loss function used to determine whether data is true. Part Two L C It is a cost function oriented towards the classification accuracy of the model data, L N Functions and L C The functions are shown in equations (10) and (11):
[0101] L N =E[logP(N=real|X real )]+E[logP(N=fake|X fake )](10)
[0102] LC =E[logP(C=c|X real )]+E[logP(C=c|X fake )](11)
[0103] Where C represents the category, X fake =G c (C,z) represents random noise z passing through a one-stage generator G. c Generated pseudo-samples. X real This represents actual data.
[0104] For the one-stage discriminant D c Determine the probability distribution P(N|X) of whether the original data is real data and the probability distribution P(C|X) of the classification label, respectively.
[0105] During the training of the network for accurate classification of oil spill analogues, the one-stage generator G... c The optimization direction is to minimize L N -L C One-stage discriminator D c The optimization direction is to maximize L. N +L C One-stage generator G c With the one-stage discriminator D c The optimization function is defined as shown in equations (12) and (13):
[0106]
[0107] Here, min represents the minimization operation.
[0108] The fifth step is to design the loss function for the oil spill semantic segmentation network. The oil spill semantic segmentation network is trained using WGAN-GP gradient penalty loss to ensure its stability. The function is defined as shown in equation (14):
[0109]
[0110] Among them, G a,b G represents a or G b λ1 is the L1 norm constraint as a balance parameter, I is the oil spill image, and S is the input oil spill segmentation label image. To generate G a and G b The results of the oil spill separation It is an L1 norm.
[0111] By minimizing the training of the two-stage dual generator G a and G bThe goal is to generate a system capable of deceiving the two-stage discriminator D. s The real segmented image, two-stage discriminator D s The function is defined as shown in equation (15):
[0112]
[0113] Where: λ2 is the gradient penalty weight balancing parameter. To counteract the loss and improve the discriminator's discriminative ability, the training of the two-stage discriminator D is minimized. s The generated segmented label image is distinguished from the oil spill segmentation image, and the discriminant score is output to guide the two-stage dual generator G. a and G b .
[0114] Based on the functions shown in equations (14) and (15), the target loss function of the oil spill semantic segmentation network is defined as shown in equation (16):
[0115]
[0116] Where min is the minimization operation function and max is the maximization operation function.
[0117] During the oil spill segmentation phase, the two-stage discriminator D is discarded after training. s Retain the two-stage generator G a and G b Using a two-stage generator G a and G b Global and local oil spill segmentation maps are generated, and cross-complement operations are performed to achieve accurate segmentation. The segmentation results of the accurate classification and semantic segmentation method for multiple types of oil spill analogues are shown below. Figure 5 As shown.
[0118] Finally, it should be noted that the above embodiments are only used to illustrate the method solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications to the method solutions described in the foregoing embodiments, or equivalent substitutions for some or all of the method features, do not cause the essence of the corresponding method solutions to deviate from the scope of the method solutions of the embodiments of the present invention.
Claims
1. A method for accurate classification and semantic segmentation of multiple types of oil spill analogues, characterized in that, The proposed method first establishes a dual-network structure based on various oil spill analogues and oil spill monitoring data from SAR images in complex scenarios. This structure includes an oil spill analogue precise classification network and an oil spill semantic segmentation network. The precise classification network comprises a label probability controller, a one-stage generator, and a one-stage discriminator, while the semantic segmentation network comprises a two-stage dual generator, a two-stage discriminator, and a complement operation. Second, random noise and multi-class label information are input into the one-stage generator, and a label probability controller is designed to adaptively control the sample probability generated by the one-stage generator. Finally, a two-stage dual generator is proposed to collaboratively extract global and local oil spill features, and the two-stage discriminator provides feedback to optimize the two-stage dual generator. This process is iteratively updated, and the complement operation is used to generate the oil spill segmentation result. Specifically, it includes the following steps: The first step involves collecting remote sensing satellite data on oil spills and similar objects from Sentinel-1, GF-3, and ERS-1 / 2, and cropping SAR images containing oil spills and various similar targets. A multi-classification training strategy and a label probability controller for oil spill similar objects are designed. This multi-classification training strategy is used to achieve refined multi-classification of dark areas in the SAR images, as detailed below: 1.1) Establish a one-stage discriminator and one-stage generator A precise classification network for oil spill analogues; The first-stage generator The first-stage discriminator consists of convolutional layers, LeakyReLU function, BN layer, Tanh function, and upsampling layer. It consists of multiple convolutional blocks, each containing a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function; it will include random noise. Oil Spill Classification Information The pseudo-samples are input into the first-stage generator. Then, output the result. Input to the first-stage discriminator In the middle, the first-stage discriminator Output The formula is shown in (1): (1) in, Indicates a one-stage discriminator The output feature vector, i This is represented by the number of elements in the feature vector. Indicates having categories Remote sensing images of information, As a one-stage discriminator, it can learn feature information from oil spill analogue images and oil spill images. Represents a one-stage generator The output result; One-stage discriminator D c The last convolutional block is followed by a convolutional block using an activation function as a one-stage discriminator. The output enables multi-classification of oil spills and similar substances; 1.2) Design a label probability controller to adaptively control the sample probability generated by the generator; specifically: The tag probability controller is divided into prior probability and posterior probability; prior probability It is a one-stage generator The probability of each type of oil spill analogue is generated from the training data, and the posterior probability is... It is a one-stage generator In each iteration of the oil spill analogue precise classification network, a one-stage discriminant is used. The probability of generating the output error rate; design a sample balance calculation function, and solve the error rate from the balance calculation function. The probability calculation functions are shown in equations (4) and (5): (4) (5) in, Represents the total number of categories. This represents the total number of samples used in model training. Represents a one-stage generator Total number of samples generated Indicates control of the first-stage generator The probability of each class determined by the training samples; This indicates the number of training samples for each category; In each iteration of the oil spill analogue accurate classification network, the error rate for each class is calculated, and the error rate is used to guide the first-stage generator. Adaptation is performed; oil spill analogues with higher error rates will be assigned higher probability values, and the generator will generate a large number of pseudo-sample data for that category; The probability calculation function is shown in equation (6): (6) in, Indicates guidance for the first-stage generator Generate the error rate for the probability of each category. This represents the accuracy during training for each category; This represents the sum of error rates for all categories; a probability balance parameter is added to balance the ratio between these values. The probability value for each category is generated by a one-stage generator. The generated prior probabilities and posterior probability As determined by common factors, as shown in equation (7): (7) in, For probability balance parameters, Indicates the final control one-stage generator The probability value generated for each category; The second step involves designing a deep and shallow dual-generator network for oil spill semantic segmentation to preserve both global and local information about the oil spill and accurately extract the oil spill area; specifically as follows: In the oil spill semantic segmentation network, a deep two-stage generator is used. G a Shallow two-stage generator G b Two-stage discriminator D s The deep two-stage generator consists of intersection and complement operations. G a It is a U-shaped fully convolutional symmetric structure, with both downsampling and upsampling symmetric structures, used to extract local feature information of oil spills; the shallow two-stage generator G b It consists of multiple convolutional layers and convolutional blocks, where each convolutional block contains a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU function. Its output layer consists of a convolutional layer and a Tanh activation function, used to extract global feature information of the oil spill; the two-stage discriminator D s The convolutional blocks are connected by multiple identical convolutional layers, BN layers, and LeakyReLU functions. The special case is that the last convolutional block contains only convolutional layers. The two-stage generator G a and G b The original oil spill images are used as inputs, and the oil spill segmentation result is output after interpolation. The oil spill segmentation result, the original oil spill image, and the segmentation label image are then input into a two-stage discriminator. D s Discriminant learning is performed to learn the differences in data distribution among various oil spill outcomes, and a discrimination score is output; the discrimination score is used to optimize the two-stage generator. G a and G b This generates more accurate oil spill segmentation results; finally, a two-stage discriminator... D s After training with the oil spill semantic segmentation network, the data is discarded, and a dual two-stage generator is used. G a and G b reserve; The bi-stage generator trained as described above G a and G b This is used to generate oil spill segmentation results containing global and local feature information; and further, through cross-complementation operations, misjudged areas caused by SAR image noise are removed, and detailed information of the oil spill boundary is completed to obtain the final segmentation results, as shown in equations (8) and (9): (8) (9) in, For a one-stage discriminator Output oil spill image data set, For oil spill classification operations, For the generated oil spill segmentation results, This is a complement operation; The third step is to design the loss function of the precise classification network for oil spill analogues so that the stable gradient generated by the precise classification network for oil spill analogues neither vanishes nor explodes. The objective loss function of the precise classification network for oil spill analogues consists of two parts: Part One L N The function is a loss function used to determine whether data is true; Part Two. The function is a cost function for the classification accuracy of network data for accurately classifying oil spill analogues; During the training of the network for accurate classification of oil spill analogues, the first-stage generator... G c The optimization direction is to minimize L N - L C generator G c The optimization function is defined as shown in equation (12): (12) Where min is the minimization operation function; One-stage discriminator D c The optimization direction is to maximize L N + L C One-stage discriminator D c The optimization function is defined as shown in equation (13): (13) Where min is the minimization operation function; The fourth step is to design the loss function for the oil spill semantic segmentation network; A one-stage discriminator was used during the training of the network for accurate classification of oil spill analogues. D c Classification is performed; in the oil spill analogue multi-classification and oil spill segmentation stages, the convolutional parts of the bi-discriminators in the oil spill analogue precise classification network and the oil spill semantic segmentation network have the same network design, using a pre-trained one-stage discriminator. D c Weights shared with the two-phase discriminator D s Improve training speed and accuracy, specifically as follows: The loss function of the oil spill semantic segmentation network is trained using WGAN-GP gradient penalty loss to ensure the stability of the oil spill semantic segmentation network. By minimizing the training of the two-stage dual generator G a and G b The goal is to generate a system capable of deceiving a two-phase discriminator. D s The true segmented image; Based on the loss function of the oil spill semantic segmentation network, and the two-stage discriminator D s If the function is , then the target loss function of the oil spill semantic segmentation network is defined as shown in equation (16): (16) Where min is the minimization operation function and max is the maximization operation function; The oil spill semantic segmentation network discards the two-stage discriminator after training. D s Retain the two-stage generator G a and G b Use a two-stage generator G a and G b Generate global and local oil spill segmentation maps and perform cross-complementation operations to achieve accurate segmentation.
2. The method for accurate classification and semantic segmentation of multiple types of oil spill analogues according to claim 1, characterized in that, In step 1.1), using and Convolutional blocks of activation functions as a one-stage discriminator Output: The function is used to distinguish between real images and fake images generated by the generator, employing... Classify functions and The activation function formulas are shown in equations (2) and (3) respectively: (2) (3) in, For index operations, For logarithmic operations, Indicates a one-stage discriminator The output feature vector, i This is represented by the number of elements in the feature vector; The function outputs classification probability values to achieve multi-classification of oil spills and similar substances.
3. The method for accurate classification and semantic segmentation of multiple types of oil spill analogues according to claim 1, characterized in that, In the third step, functions and The functions are shown in equations (10) and (11): (10) (11) in, C Indicates category, Represents random noise z Through a one-stage generator G c Generated pseudo-samples; Represents real data; for a one-stage discriminator D c Determine the probability distribution of whether the original data is real data. P (N|X) Probability distribution of category labels P(C|X) .
4. The method for accurate classification and semantic segmentation of multiple types of oil spill analogues according to claim 1, characterized in that, In the third step: The loss function of the aforementioned oil spill semantic segmentation network is defined as shown in equation (14): (14) in, G a,b express G a or G b λ1 is an L1 norm constraint that serves as a balance parameter. I Image of the oil spill. S To input an oil spill segmentation label image, To generate G a and G b Oil spill separation results; It is the L1 norm, and its function is to penalize distance; The two-stage discriminator D s The function is defined as shown in equation (15): (15) Where λ2 is the gradient penalty weight balancing parameter. To counteract the loss and improve the discriminator's discriminative ability, the training of a two-stage discriminator is minimized. D s The generated segmented label image is distinguished from the oil spill segmentation image, and the discriminant score is output to guide the two-stage dual generator. G a and G b .