A sea surface oil spill semantic segmentation method and device based on improved Mask2Former and a star-ground cooperative simulation detection system

Abstract

The application discloses a sea surface oil leakage semantic segmentation method and device based on an improved Mask2Former and a star-ground cooperative simulation detection system, belongs to the technical field of remote sensing image semantic segmentation and intelligent detection, and solves the problems that the double demands of real-time performance and accuracy cannot be met in rapid response to marine pollution accidents and the response efficiency of the overall system is low.The method comprises the following steps: a marine scene context aggregation module captures oil film diffusion rules through directional bar pooling and inhibits background interference through noise perception gating; a multi-scale feature enhancement module considers feature expression under different diffusion through a scale self-adaptive residual mechanism; a category balance query attention module reweights a query vector by using a category frequency logarithmic smooth weight; a boundary refining module implements residual refining based on a boundary attention weight; and an adaptive boundary up-sampling module maintains the safety of bilinear interpolation and sharpens a boundary of a probability graph.The application is suitable for automatic monitoring and emergency early warning of sea surface oil pollution.

Description

Technical Field

[0001] This application relates to the field of semantic segmentation and intelligent detection technology of remote sensing images, and in particular to automatic detection and fine segmentation technology of marine oil pollution for multi-source remote sensing data. Background Technology

[0002] Oil spills at sea are severe, sudden marine pollution incidents that cause immense damage to marine ecosystems, fishery resources, and coastal economies. Rapid and accurate detection and quantitative assessment of the spill's extent are paramount for pollution control and emergency response. With the development of remote sensing satellite and unmanned aerial vehicle (UAV) technologies, using sensors such as optical, multispectral, and synthetic aperture radar (SAR) to acquire sea surface images, combined with deep learning semantic segmentation methods for automatic detection, has become a crucial technical means for marine pollution monitoring. However, oil spill scenarios at sea exhibit significant complexity: on the one hand, oil slicks constitute a very small proportion of remote sensing images, and their category distribution is highly uneven; on the other hand, the marine environment is subject to complex background interference such as wave reflections, cloud shadows, and ocean current disturbances, while the oil film boundaries are often blurred and discontinuous, posing a significant challenge to accurate segmentation. Furthermore, oil slicks vary significantly in spatial scale, ranging from thin oil films to large-area diffusion morphologies, further increasing the difficulty of model building.

[0003] Existing semantic segmentation methods are mostly designed based on general visual scenes and lack specific optimizations for the characteristics of marine environments. Mask2Former, as an advanced semantic segmentation framework, combines the advantages of Transformer and mask prediction and has achieved excellent performance in general semantic segmentation tasks. However, when it is directly applied to the segmentation of oil spills on the sea surface, it still faces core technical challenges such as extreme class imbalance, severe background noise interference, blurred oil spill boundaries, and multi-scale target changes.

[0004] In existing technologies, improvements to oil spill segmentation at sea mostly focus on optimizing single modules, lacking systematic solutions that address multiple challenges through collaborative design. Furthermore, few consider the deployment constraints of deep learning models on actual hardware platforms and system-level collaborative processing mechanisms. This makes it difficult to translate algorithmic improvements into directly usable oil spill detection systems, failing to meet the dual demands of real-time performance and accuracy in rapid response to marine pollution incidents. In addition, existing solutions typically download all remote sensing images to the ground for processing, failing to fully utilize the edge computing capabilities of spaceborne platforms for initial data filtering. This results in a large amount of invalid data occupying limited communication bandwidth, hindering the overall system's response efficiency. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems of failing to meet the dual requirements of real-time performance and accuracy in rapid response to marine pollution incidents, as well as the low system response efficiency. It provides a semantic segmentation method, device, and satellite-ground collaborative simulation detection system for marine oil spills based on an improved Mask2Former.

[0006] The technical solution adopted by this invention to solve the above problems is: an improved semantic segmentation method for sea surface oil spills based on Mask2Former, the method comprising:

[0007] Step S1: Acquire remote sensing images of the sea surface and extract multi-scale features from them to obtain multi-scale feature maps of different levels, C2, C3, and C4, and a high-level semantic feature map, C5.

[0008] Step S2: Input the C5 high-level semantic feature map into the multi-scale deformable attention pixel decoder, and generate a mask feature map after upsampling and fusion; then enhance the mask feature map with marine context through the marine scene context aggregation module to suppress sea surface background noise interference;

[0009] Step S3: The C2, C3, and C4 multi-scale feature maps are subjected to scale-adaptive residual enhancement through the multi-scale feature enhancement module to obtain enhanced multi-scale feature maps, thereby improving the multi-scale feature representation capability.

[0010] Step S4: Input the enhanced mask feature map output from step S2 and the enhanced multi-scale feature map output from step S3 into the transformer decoder; In the self-attention layer of the transformer decoder, the query vector is reweighted by the category-aware class balance query attention module to solve the problem of extreme class imbalance in complex sea surface scenes.

[0011] Step S5: Predict the final mask output from the converter decoder, and enhance the boundary through the boundary refinement module to improve the accuracy of oil stain edge segmentation;

[0012] Step S6: Upsample the mask prediction output by the boundary refinement module using the adaptive boundary upsampling module to output the final semantic segmentation result of the marine oil spill.

[0013] Furthermore, the ocean scene context aggregation module described in step S2 includes three sub-modules:

[0014] Directional bar pooling submodule: Performs adaptive bar pooling on the mask feature map along the horizontal and vertical directions respectively, to obtain horizontal bar pooling features and vertical bar pooling features, specifically represented as follows:

[0015] ;

[0016] in, and The horizontal and vertical mask feature maps are represented respectively, B represents the batch size, C represents the number of feature channels, and H and W represent the height and width of the mask feature map, respectively.

[0017] The strip pooling results are convolved and expanded to restore the original spatial dimensions. A directional attention weight map is then generated through a fusion layer to weight the mask features output by the pixel decoder, thereby capturing the directional features of the oil film spreading along the coastal current.

[0018] , This is a directional attention weight map;

[0019] Multi-scale context pyramid submodule: Uses depthwise separable dilated convolution branches with different dilation rates and global average pooling branches to extract multi-scale context features in parallel, and outputs a multi-scale context feature map after channel concatenation and convolution fusion.

[0020] Noise-aware gating submodule: Extracts channel statistical features through global average pooling, and generates channel attention gating through a fully connected layer. Simultaneously, spatial attention gating is generated through spatial convolution. The calculation formula for the gating mechanism is as follows:

[0021] ;

[0022] in, This is the feature map after noise-sensing gated modulation. Represents multi-scale contextual feature maps; uses joint two-way gating to modulate features and suppress background noise interference such as sea surface wave reflection;

[0023] The outputs of each submodule are fused with the original mask feature map to obtain the enhanced mask feature map.

[0024] Furthermore, the multi-scale feature enhancement module in step S3 includes:

[0025] Scale Attention Submodule: Performs global average pooling on feature maps at each level to extract global semantic descriptors.

[0026] ;

[0027] in, Indicates the captured number A global feature vector that aggregates global contextual information at various scales;

[0028] After concatenating the global feature vectors of all levels, The scale-weighted prediction network consists of two fully connected layers, a ReLU activation function, and a Softmax function. It predicts an importance weight scalar for each layer, achieving adaptive weight allocation for features at different scales. The calculation formula is as follows:

[0029]

[0030] in, This is the multi-scale weight matrix output by the scale-weighted prediction network. These represent the global feature vectors of the captured aggregated global context information at the 1st, 2nd, and 3rd scales, respectively;

[0031] Residual Scale Enhancement Submodule: Performs depthwise separable convolution and pointwise convolution sequentially on each layer's feature map to enhance the features. The enhanced features are multiplied by a learnable residual scaling factor and the corresponding scale weight, then superimposed onto the original features to form the residual enhanced output. The calculation formula is as follows:

[0032] ;

[0033] in, For the enhanced multi-scale feature map, and The first The height and width of the feature map at each scale. Indicates the first The original characteristics of dimensionality Indicates enhanced branching features, This represents the learnable residual scaling factor. Indicates the first The adaptive weights are applied to each scale; the learnable residual scaling factor is initialized to zero to ensure that the module output is completely consistent with the input during the initial training phase.

[0034] Furthermore, the category-balanced query attention module described in step S4 includes:

[0035] The query category estimation submodule: Processes the query feature vectors from the transformer decoder through a classification network, outputting the probability distribution for each query vector corresponding to different categories of the sea surface oil slick.

[0036] ;

[0037] in, This represents a predefined number of queries. Representing a classification network, Represented as the number of semantic categories;

[0038] Category-aware reweighting submodule: Calculates a category importance score for each query vector based on log-smoothed weights of category frequencies and a learnable weight modulation factor.

[0039]

[0040] in, Each query in each batch is estimated as the first... The probability of each category This represents the effective class weights, after processing the class weight vector. Introducing a learnable modulation-like factor That is, by The calculated value is then applied as a scaling factor to the self-attention score matrix to enhance the attention response of rare class query vectors. Represented as the first Effective category weights for each category;

[0041] The dynamic balancing factor submodule estimates the degree of class imbalance based on the information entropy of the probability distribution of query classes within the current batch. It then dynamically outputs a balancing factor through a fully connected network, interpolating and fusing the reweighted attention with the original attention to make the class balance intensity adaptively change with the class distribution within the batch.

[0042] Furthermore, the boundary refining module described in step S5 includes:

[0043] Lightweight edge detection submodule: Calculates the horizontal and vertical gradients of the mean values ​​of each channel of the mask feature map using a fixed Sobel operator.

[0044] ;

[0045] in, The gradient is in the horizontal direction. The gradient is in the vertical direction. Represented as the average mask feature obtained by the pixel decoder, and These represent the horizontal and vertical gradient operators, respectively; after taking the gradient magnitude and normalizing, an edge intensity map with the same resolution as the mask prediction is generated.

[0046] The boundary attention weight submodule processes the edge intensity map through convolution, batch normalization, ReLU-convolution, and a sigmoid network to output a spatial boundary attention weight map, highlighting the oil film edge region and suppressing interference from non-boundary regions. Specifically, it is represented as follows:

[0047] ;

[0048] in, For spatial boundary attention weights, Represented as a boundary probability diagram, It is a lightweight weight generation network. express function;

[0049] The residual refinement submodule: After batching the final mask prediction tensor, it calculates the residual adjustment amount and limits the residual range through a convolutional network. This is then multiplied by the learnable residual scaling factor, multiplied by the boundary attention weights, and finally superimposed onto the original mask prediction. Specifically, it is represented as follows:

[0050] ;

[0051] in, This is a refined mask prediction after boundary attention weight modulation. This represents the original mask prediction. This represents the residual correction term, which is subject to a learnable residual scaling factor. Restrictions.

[0052] Furthermore, the adaptive boundary upsampling module in step S6 includes:

[0053] The original logit tensor of the mask is always upsampled using standard bilinear interpolation to avoid introducing the risk of performance degradation;

[0054] After upsampling is completed and a probability map is obtained through activation function, the Sobel operator is used to detect edge positions on the probability map. A contrast enhancement term is applied to the probability value according to the edge intensity, so that the probability prediction at the boundary is shifted to the extremes of 0 or 1 to sharpen the segmentation boundary.

[0055] Secondly, the present invention provides a space-ground collaborative simulation detection system, the system comprising:

[0056] The spaceborne edge processing subsystem is constructed using equivalent simulation and validated on a ground computing platform to simulate the on-orbit processing process deployed on a low-Earth orbit satellite platform. It includes: a remote sensing image acquisition unit for acquiring sea surface remote sensing images; a spaceborne inference unit that simulates a space-level programmable logic device mounted on a low-Earth orbit satellite platform, running a quantized and compressed lightweight segmentation model in a ground environment to perform fast detection inference on remote sensing images and output an oil spill detection confidence map and a coarse segmentation mask; and an adaptive transmission control unit that executes differentiated data transmission strategies based on the confidence map, determines the data content to be transmitted, and generates the corresponding downlink data set, including the confidence map, coarse segmentation mask, and selectively retained original remote sensing image data.

[0057] The ground receiving and refining subsystem includes: a remote sensing data receiving unit, used to receive the downlink data set generated by the adaptive transmission control unit; and a full model inference unit, which runs an improved Mask2Former-based semantic segmentation method for marine oil spills as described above, performs fine segmentation on the received remote sensing images, and outputs the segmentation results.

[0058] The cloud-based analysis subsystem is used to aggregate historical segmentation results and push lightweight model update instructions to the simulation module corresponding to the onboard inference unit.

[0059] Furthermore, the lightweight segmentation model in the spaceborne inference unit is obtained by compressing and optimizing the model of the improved Mask2Former-based semantic segmentation method for marine oil spills described above, including:

[0060] Using the complete segmentation model as the teacher model, the lightweight student model is trained through distillation learning, so that the student model maintains high segmentation accuracy while reducing model complexity.

[0061] Integer quantization is performed on the student model to convert floating-point operations into low-ratio fixed-point operations, thereby reducing the computational complexity and storage requirements of the model and adapting to the computing resource constraints of edge processors.

[0062] Based on the contribution of each module to the segmentation performance, some computational branches in the complete model are pruned, while key sub-modules that play a significant role in object detection and boundary recognition are retained, in order to reduce the number of model parameters and computational load, so as to meet the deployment requirements of edge computing environment.

[0063] Furthermore, the differentiated data transmission strategy executed by the adaptive transmission control unit includes:

[0064] Low confidence scenario: When the maximum confidence of the confidence map is lower than the preset threshold. If it is determined that there is no oil pollution target in the current remote sensing image, only auxiliary information or low-resolution thumbnail is transmitted to minimize the amount of data transmitted.

[0065] Medium confidence scenario: When the maximum confidence level is at the low confidence threshold With high confidence threshold When a suspected oil spill target is identified in the current image, a coarse segmentation mask and compressed remote sensing image data are transmitted for further analysis and confirmation by the ground end.

[0066] High confidence scenario: When the maximum confidence level is higher than the preset threshold When there are complex target boundary structures, the system determines that there are high-confidence oil pollution targets in the current image and transmits high-resolution remote sensing image data to support the ground end to run the fully improved Mask2Former model to perform high-precision semantic segmentation processing.

[0067] Thirdly, the present invention provides a semantic segmentation device for marine oil spills based on an improved Mask2Former, the device comprising:

[0068] The data acquisition unit is used to acquire remote sensing image data of the sea surface;

[0069] The preprocessing unit is used to normalize, randomly crop, and augment remote sensing images;

[0070] A semantic segmentation unit is used to implement an improved Mask2Former-based semantic segmentation method for marine oil spills, as described above.

[0071] The result output unit is used to display the oil spill area segmentation and labeling, area statistics and early warning information in real time; when the detection results meet the preset alarm conditions, an alarm is triggered.

[0072] Storage unit, used to store semantic segmentation model parameters and program instructions;

[0073] The processing unit is connected to the data acquisition unit, preprocessing unit, semantic segmentation unit, and result output unit, and is used to coordinate the collaborative operation of each unit.

[0074] The beneficial effects of this invention are:

[0075] 1. Effectively solves the problem of extreme class imbalance in offshore oil spill scenarios. Through the class-aware reweighting mechanism and dynamic balance factor of the class-balanced query attention module, the query response of rare classes is enhanced in a targeted manner, effectively improving the problem of small targets and rare classes being suppressed by the dominant class, and improving the segmentation accuracy of rare classes;

[0076] 2. Significantly improves the accuracy of oil film boundary segmentation. By constructing a collaborative optimization mechanism between a boundary refinement module and an adaptive boundary upsampling module, the former constrains the residual refinement to the boundary region through boundary attention weights, while the latter applies boundary sharpening post-processing to the probability map. Together, they solve the problems of blurred oil film boundaries and edge distortion introduced by upsampling, making the oil film edges clearer and more continuous, thus improving the boundary quality of the segmentation results.

[0077] 3. Significantly enhances the ability to suppress background noise from the sea surface. The marine scene context aggregation module uses directional strip pooling and noise-aware gating, combined with channel attention and spatial attention, to jointly modulate features, effectively suppressing background noise interference such as wave reflections and cloud shadows, and reducing the false positive rate;

[0078] 4. Enhance the feature representation capability of multi-scale oil film targets. The multi-scale feature enhancement module, through scale-adaptive weight allocation and residual enhancement mechanism, enables the model to simultaneously take into account multi-scale morphologies such as thin oil films, small-scale oil traces, and large-area oil pollution diffusion, thereby improving the overall segmentation performance and generalization ability;

[0079] 5. Significantly improves system real-time performance and resource utilization efficiency. Based on traditional semantic segmentation methods, this invention introduces a space-ground collaborative processing architecture for low-Earth orbit satellite applications. It constructs a spaceborne edge processing flow through equivalent simulation, using a lightweight segmentation model optimized by model compression and quantization to simulate rapid on-orbit detection, achieving preliminary screening and target localization of remote sensing data. Combined with a confidence-driven data selection mechanism, only candidate regions or key data are transmitted, reducing invalid data usage. Simultaneously, a fully improved Mask2Former model is deployed on the ground for fine-grained segmentation, while the cloud handles model updates and historical data management. This hierarchical processing mechanism effectively reduces data transmission pressure and computational resource consumption while ensuring segmentation accuracy, improving the overall system response efficiency and engineering applicability.

[0080] 6. Excellent engineering deployability and cross-platform adaptability. This invention adopts a modular and lightweight design strategy, allowing each functional module to be flexibly deployed on different computing platforms, including ground-based GPU servers, embedded edge computing devices, and equivalent spaceborne computing environments. Through a unified data processing flow and interface design, it achieves efficient processing and system integration of multi-source remote sensing data, meeting the application requirements of marine oil spill monitoring under real-time, stability, and resource-constrained conditions, and has good engineering promotion value.

[0081] This invention achieves high-precision identification and boundary extraction of oil spill targets on the sea surface through deep learning model optimization and a satellite-ground collaborative progressive inference mechanism. Furthermore, this invention relates to a hardware-software co-design method, which can be deployed on ground computing devices or edge computing platforms, and verifies the onboard processing flow for low-Earth orbit satellite applications through equivalent simulation, making it suitable for automated monitoring and emergency early warning of oil spills on the sea surface. Attached Figure Description

[0082] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0083] Figure 1 This is a schematic diagram of the overall structure of the space-ground collaborative simulation and testing system.

[0084] Figure 2 This is a schematic diagram of the overall structure of the semantic segmentation method for marine oil spills based on the improved Mask2Former.

[0085] Figure 3 This is a structural diagram of the ocean scene context aggregation module.

[0086] Figure 4 This is a schematic diagram of the multi-scale feature enhancement module.

[0087] Figure 5 This is a structural diagram of the category-balanced query attention module.

[0088] Figure 6 This is a structural diagram of the boundary refining module.

[0089] Figure 7 This is a schematic diagram of the model lightweighting process in the spaceborne inference unit.

[0090] Figure 8 This is a schematic diagram of the differentiated data transmission process executed by the adaptive transmission control unit.

[0091] Figure 9 This is a schematic diagram of a semantic segmentation device for marine oil spills based on an improved Mask2Former. Detailed Implementation

[0092] Specific Implementation Method 1: This implementation method provides a semantic segmentation method for marine oil spills based on an improved Mask2Former, such as... Figure 2 As shown, the specific steps include:

[0093] Step S1: Acquire remote sensing images of the sea surface and extract multi-scale features from them to obtain multi-scale feature maps of different levels, C2, C3, and C4, and a high-level semantic feature map, C5.

[0094] Step S2: Input the C5 high-level semantic feature map into the multi-scale deformable attention pixel decoder, and generate a mask feature map after upsampling and fusion; then enhance the mask feature map with marine context through the marine scene context aggregation module to suppress sea surface background noise interference.

[0095] like Figure 3 As shown, the ocean scene context aggregation module is a feature enhancement module designed for ocean surface scenes, comprising three sub-modules, as follows:

[0096] (1) Directional bar pooling submodule: Adaptive bar pooling is performed on the mask feature map along the horizontal and vertical directions respectively, specifically as follows:

[0097] ;

[0098] in and These represent the mask feature maps in the horizontal and vertical directions, respectively. The pooling results are then convolved and expanded back to their original spatial dimensions. A directional attention weight map is generated through a fusion layer to weight the input features, capturing the directional characteristics of the oil film spreading along the coastal current. Specifically, this is represented as follows:

[0099] .

[0100] (2) Multi-scale context pyramid submodule: Multi-scale context features are extracted in parallel using depth-separable dilated convolution branches with different dilation rates and global average pooling branches. After channel splicing and convolution fusion, the multi-scale context feature map is output.

[0101] (3) Noise perception gating submodule: Extract channel statistical features through global average pooling, and generate channel attention gating through a fully connected layer. Simultaneously, spatial attention gating is generated through spatial convolution. The calculation formula for the gating mechanism is as follows:

[0102] ;

[0103] in This represents the context feature map generated by the multi-scale context pyramid submodule. Two-way gating is used to modulate the features, suppressing background noise interference such as sea wave reflections.

[0104] Step S3: The multi-scale feature map is enhanced by scale-adaptive residual enhancement through the multi-scale feature enhancement module to obtain the enhanced multi-scale feature map, thereby improving the multi-scale feature representation capability.

[0105] like Figure 4 As shown, the multi-scale feature enhancement module adopts a lightweight design and includes two sub-modules, as detailed below:

[0106] (1) Scale Attention Submodule: Global average pooling is performed on the feature maps of each level to extract the global semantic descriptor, specifically represented as:

[0107] ;

[0108] in Indicates the captured number Aggregate global context information at each scale. After concatenating global feature vectors from all levels... The scale-weighted prediction network predicts an importance weight scalar for each layer, achieving adaptive weight allocation for features at different scales. The calculation formula is as follows:

[0109] .

[0110] in, This is the multi-scale weight matrix output by the scale-weighted prediction network. These represent the global feature vectors of the captured aggregated global context information at the 1st, 2nd, and 3rd scales, respectively.

[0111] (2) Residual Scale Enhancement Submodule: For each level of feature map, depthwise separable convolution and pointwise convolution are performed sequentially to complete feature enhancement; the enhanced features are multiplied by the learnable residual scaling factor and the corresponding scale weight, and then superimposed on the original features to form the residual enhanced output. The calculation formula is as follows:

[0112] ;

[0113] in Indicates the first The original characteristics of dimensionality Indicates enhanced branching features, This represents the learnable residual scaling factor. Indicates the first Adaptive weights at each scale. The learnable residual scaling factor is initialized to zero to ensure that the module output is completely consistent with the input during the initial training phase.

[0114] The enhanced mask feature map output from step S2 and the enhanced multi-scale feature map output from step S3 are input together into the transformer decoder. In the self-attention layer of the decoder, the query vector is reweighted by the category-aware class balance query attention module to solve the problem of extreme class imbalance in complex sea surface scenes.

[0115] like Figure 5 As shown, the category-balanced query attention module includes three sub-modules, as follows:

[0116] (1) Query Category Estimation Submodule: The query feature vectors in the transformer decoder are processed by a classification network to output the probability distribution of each query vector corresponding to different categories of sea surface oil film segmentation, specifically represented as follows:

[0117] ;

[0118] in This represents a predefined number of 100 queries. Represents a classification network.

[0119] (2) Category-aware reweighting submodule: Based on the log-smoothed weights of category frequencies and the learnable weight modulation factor, a category importance score is calculated for each query vector. The calculation formula is as follows:

[0120] ;

[0121] in This represents the effective class weights, after processing the class weight vector. Introducing a learnable modulation-like factor That is, by The result is calculated and then applied as a scaling factor to the self-attention score matrix to enhance the attention response of rare class query vectors.

[0122] (3) Dynamic balance factor submodule: Estimate the degree of class imbalance based on the information entropy of the probability distribution of query classes in the current batch, and dynamically output the balance factor through a fully connected network. Interpolate and fuse the reweighted attention and the original attention to make the class balance intensity adaptively change with the class distribution in the batch.

[0123] Step S5: The final mask prediction output by the converter decoder is enhanced by the boundary refinement module to improve the accuracy of oil stain edge segmentation.

[0124] like Figure 6 As shown, the boundary refining module includes three sub-modules, as follows:

[0125] (1) Lightweight edge detection submodule: The horizontal and vertical gradients of the mean values ​​of each channel of the mask feature map are calculated using a fixed Sobel operator. The calculation formula is as follows:

[0126] ;

[0127] in Represented as the average mask feature obtained by the pixel decoder, and These represent the horizontal and vertical gradient operators, respectively. After taking the gradient magnitude and normalizing, an edge intensity map with the same resolution as the mask prediction is generated.

[0128] (2) Boundary attention weight submodule: The edge intensity map is processed through a convolution-batch normalization-ReLU-convolution-Sigmoid network to output a spatial boundary attention weight map, highlighting the oil film edge region and suppressing interference from non-boundary regions, specifically as follows:

[0129] ;

[0130] in Represented as a boundary probability diagram, It is a lightweight weight generation network. express function.

[0131] (3) Residual Refinement Submodule: After batching the final mask prediction tensor, the residual adjustment amount is calculated by a convolutional network and the residual range is limited. It is then multiplied by the learnable residual scaling factor and then multiplied by the boundary attention weights. Finally, it is superimposed on the original mask prediction, specifically expressed as:

[0132] ;

[0133] in This represents the original mask prediction. This indicates that the residual correction term is subject to a learnable residual scaling factor. Restrictions.

[0134] Step S6: Upsample the refined mask prediction using the adaptive boundary upsampling module to output the final semantic segmentation result of the marine oil spill.

[0135] The adaptive boundary upsampling module specifically includes two steps:

[0136] (1) Standard bilinear upsampling steps: The original logit tensor of the mask is always upsampled using standard bilinear interpolation to avoid introducing the risk of performance degradation;

[0137] (2) Probability map boundary post-processing step: After the upsampling is completed and the probability map is obtained by the activation function, the Sobel operator is used to detect the edge position on the probability map, and the contrast enhancement term is applied to the probability value according to the edge intensity, so that the probability prediction at the boundary is shifted to the two extremes of 0 or 1 to sharpen the segmentation boundary; the scaling factor of the enhancement step is a learnable parameter, initialized to zero, to ensure that the module does not introduce the risk of performance degradation.

[0138] The learnable gain parameters in each improved module described in steps S2 to S6 are initialized with zero values ​​to ensure that each improved module degenerates into an identity mapping in the initial stage of training, so as not to destroy the baseline segmentation performance of the original Mask2Former framework, and gradually activate the enhancement effect of each module as training progresses.

[0139] At the algorithm level, this implementation method systematically solves core problems in marine oil spill segmentation, such as extreme class imbalance, background noise interference, blurred oil spill boundaries, and multi-scale target changes, through the design and integration of five collaborative improvement modules.

[0140] Specific Implementation Method Two: This implementation method is a space-ground collaborative simulation detection system based on an improved Mask2Former-based semantic segmentation method for marine oil spills, as described in Implementation Method One. It is used to verify the collaborative mechanism between spaceborne edge processing and ground refining for low-Earth orbit satellite applications in an equivalent simulation environment. For example... Figure 1 As shown, the system includes:

[0141] (1) Onboard edge processing subsystem, which is constructed in an equivalent simulation manner and verified on a ground computing platform to simulate the on-orbit processing process deployed on a low-orbit satellite platform, including: a remote sensing image acquisition unit for acquiring remote sensing images of the sea surface; an onboard inference unit for simulating a space-level programmable logic device mounted on a low-orbit satellite platform, running a quantized and compressed lightweight segmentation model in the ground environment, performing fast detection inference on the remote sensing images, and outputting an oil pollution detection confidence map and a coarse segmentation mask; and an adaptive transmission control unit for executing a differentiated data transmission strategy according to the confidence map, determining the data content to be transmitted, and generating a corresponding downlink data set, including the confidence map, the coarse segmentation mask, and selectively retained original remote sensing image data.

[0142] (2) Ground receiving and refining subsystem, including: remote sensing data receiving unit, used to receive the downlink data set generated by the adaptive transmission control unit; full model inference unit, running the sea surface oil spill semantic segmentation method described in Implementation 1, performing fine segmentation on the received remote sensing image, and outputting fine segmentation results;

[0143] (3) Cloud analysis subsystem, used to aggregate historical segmentation results, perform statistical analysis, and push lightweight model update instructions to the simulation module corresponding to the on-board inference unit.

[0144] At the system level, this implementation proposes a space-ground collaborative simulation detection system. It constructs a spaceborne edge inference process through equivalent simulation and achieves hierarchical processing by combining a confidence-driven differentiated data transmission strategy. It forms a complete system closed loop through ground refinement and cloud-based continuous optimization.

[0145] Specifically, such as Figure 7 As shown, the lightweight segmentation model in the spaceborne inference unit is obtained by performing model compression and optimization on the segmentation method described in Implementation Method 1, including:

[0146] (1) Knowledge distillation: The complete segmentation model is used as the teacher model, and the lightweight student model is trained by distillation learning, so that the student model maintains high segmentation accuracy while reducing model complexity;

[0147] (2) Integer quantization: The student model is subjected to integer quantization to convert floating-point operations into low-ratio fixed-point operations, so as to reduce the computational complexity and storage requirements of the model and thus adapt to the computing resource constraints of the edge processor.

[0148] (3) Structure pruning: Based on the contribution of each module to the segmentation performance, some computational branches in the complete model are pruned, and key sub-modules that have a significant effect on target detection and boundary recognition are retained to reduce the number of model parameters and computational load, so as to meet the deployment requirements in the edge computing environment.

[0149] Specifically, such as Figure 8 As shown, the differentiated data transmission strategy executed by the adaptive transmission control unit is based on a preset low confidence level. With high confidence (satisfy The processing results are divided into three categories for routing, including:

[0150] (1) Low confidence scenario: When the maximum confidence of the confidence map is lower than the preset threshold If it is determined that there is no oil pollution target in the current remote sensing image, only necessary auxiliary information or low-resolution thumbnails are transmitted to minimize the amount of data transmitted and reduce the consumption of communication resources.

[0151] (2) Medium confidence scenario: when the maximum confidence level is at the low confidence threshold With high confidence threshold When a suspected oil spill target is identified in the current image, a coarse segmentation mask and compressed remote sensing image data are transmitted for further analysis and confirmation by the ground end.

[0152] (3) High confidence scenario: When the maximum confidence level is higher than the preset threshold When there is a complex target boundary structure, it is determined that there is a high-confidence oil pollution target in the current image, and high-resolution remote sensing image data is transmitted to support the ground end to run the complete improved Mask2Former model to perform high-precision semantic segmentation processing;

[0153] Among them, the low confidence threshold With high confidence threshold All are preset parameters that can be adjusted according to the application scenario to meet the requirements. The differentiated data transmission strategy dynamically adjusts the transmission content through three-tiered routing, effectively reducing the average data transmission volume and improving the efficiency of communication link utilization while ensuring the complete transmission of key target information.

[0154] Specific Implementation Method Three: This implementation method is based on the semantic segmentation device for marine oil spills based on an improved Mask2Former, as described in Implementation Method One. Figure 9 As shown, it includes:

[0155] (1) Data acquisition unit: using optical cameras, synthetic aperture radar or multispectral sensors mounted on satellites, UAVs or shipborne platforms to acquire remote sensing image data of the sea surface;

[0156] (2) Preprocessing unit: used to normalize, randomly crop and augment the remote sensing image;

[0157] (3) Semantic segmentation unit: used to deploy the improved Mask2Former semantic segmentation model on the processor and perform semantic segmentation processing on the preprocessed remote sensing image. The model integrates the marine scene context aggregation module, multi-scale feature enhancement module, category balanced query attention module, boundary refinement module and adaptive boundary upsampling module to implement the sea surface oil spill semantic segmentation method described in Implementation 1.

[0158] (4) Result output unit: It consists of a display and an audible and visual alarm, which is used to display the oil pollution area segmentation marking, area statistics and early warning information in real time; when the detection result meets the preset alarm conditions (such as the oil pollution confidence level exceeding the set threshold or the detection area exceeding the preset range), the audible and visual alarm is triggered to remind the staff to intervene and handle the situation.

[0159] (5) Storage unit: used to store semantic segmentation model parameters and program instructions;

[0160] (6) Processing unit: connected to the data acquisition unit, preprocessing unit, semantic segmentation unit and result output unit, used to coordinate the collaborative operation of each unit, wherein the processing unit is selected as a graphics processor (GPU).

[0161] The device can be deployed on ground computing equipment or edge computing platforms, and supports application verification for spaceborne environments in an equivalent manner.

[0162] At the engineering deployment level, this implementation provides a hardware device solution that can be flexibly deployed on ground computing devices, embedded edge computing platforms, and equivalent spaceborne computing environments to meet the actual engineering needs of marine pollution monitoring.

[0163] Example 1: Semantic Segmentation Method for Marine Oil Spills Based on Improved Mask2Former

[0164] This embodiment provides a semantic segmentation method for marine oil spills based on an improved Mask2Former. This method can be deployed on ground computing devices or edge computing platforms and can support application verification in spaceborne environments through equivalent means. Based on the Mask2Former semantic segmentation framework, this method introduces five improved modules: a marine scene context aggregation module, a multi-scale feature enhancement module, a category-balanced query attention module, a boundary refinement module, and an adaptive boundary upsampling module. The specific implementation steps are as follows:

[0165] Data preparation and preprocessing: The data acquisition unit acquires remote sensing image data of the sea surface, including synthetic aperture radar (SAR) satellites, UAVs, or airborne sensors; the preprocessing unit performs normalization, random cropping, scale transformation, and data augmentation operations on the acquired images, and divides them into training sets, validation sets, and test sets for model training and performance evaluation.

[0166] Multi-scale feature extraction: Multi-scale feature extraction is performed on the preprocessed sea surface remote sensing image, and four levels of multi-scale feature maps (C2, C3, C4, and C5) are output (the number of channels are 256, 512, 1024, and 2048, respectively). Among them, C5 is a high-level semantic feature map, which is used for subsequent mask feature generation.

[0167] Mask Feature and Context Enhancement: The C5 high-level semantic feature map is input into a multi-scale deformable attention pixel decoder. After upsampling and feature fusion, a mask feature map is generated and then input into the marine scene context aggregation module for enhancement. Specifically, the directional bar pooling submodule captures the directional features of oil slick spreading along the current direction in both horizontal and vertical directions, generating a directional attention weight map and weighting the mask feature map accordingly. The multi-scale context pyramid submodule extracts and fuses context features from different receptive fields in parallel to adapt to the multi-scale morphological changes of the oil slick on the sea surface. The noise-aware gating submodule modulates the features using both channel attention and spatial attention to suppress background noise interference such as wave reflection. The outputs of each submodule are fused with the original mask feature map via residual connections to obtain the enhanced mask feature map.

[0168] Multi-scale feature enhancement: The multi-scale feature maps of the C2, C3, and C4 levels are input into the multi-scale feature enhancement module for scale-adaptive residual enhancement. The scale attention submodule aggregates global features from each level and predicts an importance weight scalar for each level; the residual scale enhancement submodule performs lightweight feature enhancement on the features of each level by multiplying the corresponding scale weight by a learnable residual scaling factor initialized to zero and then superimposing it onto the original features to obtain the enhanced multi-scale feature map.

[0169] Transformer Decoding and Class Balancing Optimization: The enhanced mask feature map and multi-scale feature map are jointly input into the Mask2Former transformer decoder, and a class-balanced query attention module is integrated into each self-attention layer. The query class estimation submodule outputs the probability distribution of each class from the query feature vector in the decoder; the class-aware reweighting submodule calculates the importance score for each query based on the preset log-smoothed class weights and learnable modulation factors and applies it to the self-attention matrix to enhance the query response of rare classes; the dynamic balancing factor submodule dynamically adjusts the balancing strength based on the information entropy of the current batch class distribution, and merges the reweighted attention with the original attention interpolation to avoid overemphasizing rare classes and damaging the performance of dominant classes.

[0170] Boundary Refinement Processing: The final mask prediction output from the converter decoder is input to the boundary refinement module and subjected to residual boundary enhancement. A lightweight edge detection submodule extracts horizontal and vertical gradient magnitudes from the mask feature map using a fixed Sobel operator, normalizes them, and generates an edge intensity map. The boundary attention weight submodule converts the edge intensity map into a spatial boundary attention weight map, highlighting the oil film edge region. The residual refinement submodule calculates the residual adjustment amount through a convolutional network, modulates it with an initialized, learnable residual scaling factor and boundary attention weights, and then superimposes it onto the original mask prediction. Boundary refinement only affects the boundary region and does not affect the prediction of the internal region, outputting the refined mask prediction.

[0171] Adaptive boundary upsampling: Upsampling is performed on the refined mask prediction. First, standard bilinear interpolation is used to upsample and restore the original image resolution. Then, gradient detection operators are used to locate edge positions on the probability map. A contrast enhancement term is applied to the probability value according to the edge intensity, which shifts the probability prediction at the boundary to both extremes to sharpen the segmentation boundary. The enhancement magnitude is controlled by a learnable parameter to obtain high-precision segmentation results.

[0172] Output results: The final segmentation results are output to the display terminal or early warning system through the output unit for marking oil pollution areas, area statistics and early warning information release.

[0173] Model Training: Training was performed on the LADOS dataset on a single NVIDIA RTX 4080 GPU (16 GB VRAM) with Automatic Mixed Precision (AMP) enabled. The batch size was set to 8, and the model was trained for 160k iterations using the AdamW optimizer with a base learning rate of 0.0001, a backbone learning rate of 0.1, and weight decay of 0.05. During training, input images were randomly resized and cropped to a spatial resolution of 512×512 pixels.

[0174] Example 2: Space-Ground Collaborative Simulation and Detection System

[0175] This embodiment provides an equivalent implementation of a space-ground collaborative simulation and testing system to verify the collaborative processing mechanism of spaceborne edge inference and ground refinement for low-Earth orbit satellite application scenarios. The system uses a ground computing platform as its carrier and constructs the spaceborne processing flow through equivalent simulation. It includes three parts: a spaceborne edge processing subsystem, a ground receiving and refining subsystem, and a cloud analysis subsystem. The specific implementation steps are as follows.

[0176] Spaceborne Edge Processing Subsystem: This subsystem simulates the spaceborne processing environment using a ground-based computing platform, constructing a lightweight inference process. First, the fully improved Mask2Former model trained in Example 1 is compressed in three steps: knowledge distillation, integer quantization, and structural pruning. This results in a lightweight segmentation model with significantly reduced parameter and computational costs, used to simulate rapid detection and inference on the spaceborne end in a ground-based equivalent verification environment. Then, the spaceborne inference unit performs rapid forward inference on the input remote sensing image, outputting an oil spill detection confidence map and a coarse segmentation mask. Finally, the adaptive transmission control unit, based on a preset low-confidence threshold... With high confidence threshold Three differentiated data transmission strategies are implemented to generate a set of downlink data to be transmitted. Complete data transmission is performed only on candidate regions or key frames, effectively reducing the occupation of invalid data.

[0177] Ground Reception and Refinement Subsystem: Receives the downlink data set generated by the spaceborne edge processing subsystem. For frames containing original remote sensing images, it performs high-precision semantic segmentation using the complete improved Mask2Former model described in Example 1 (including five improved modules: OCA, MSFE, CBQA, BRM, and ABU), outputting refined segmentation results. For frames containing only coarse segmentation masks and compressed images, it performs auxiliary analysis by combining mask information. Low-confidence frames containing only auxiliary information are directly determined as targetless frames, eliminating the need for refined segmentation, thereby achieving on-demand allocation and efficient utilization of ground-based computing resources.

[0178] The cloud-based analysis subsystem aggregates historical segmentation results from various orbits, performs cross-orbit statistical analysis and pollution diffusion trend modeling; it periodically performs incremental fine-tuning of the lightweight segmentation model based on newly labeled data, updates model parameters, and pushes optimized model weights to the simulation module corresponding to the spaceborne edge processing subsystem, thereby achieving continuous iteration and performance improvement of the system.

[0179] Example 3: Hardware Implementation

[0180] This embodiment provides a semantic segmentation device for marine oil spills based on an improved Mask2Former, which can be integrated into ground computing devices or embedded edge computing platforms.

[0181] The data acquisition unit uses an UAV equipped with an optical camera or an airborne synthetic aperture radar sensor to acquire remote sensing images of the sea surface, supporting multiple resolutions and imaging modes. The preprocessing unit is responsible for image normalization, random cropping, and data augmentation operations to ensure that the input data meets the format requirements of the segmentation model. The semantic segmentation unit deploys the complete improved Mask2Former segmentation model described above on a GPU to perform high-precision semantic segmentation inference and output segmentation results for various types of oil pollution areas. The result output unit outputs detection results and early warning information, triggering a combined audio-visual alarm when the confidence level meets preset alarm conditions. The storage unit stores model weights, parameters, and instructions. The processing unit uses a GPU to work in conjunction with other functional units to coordinate the overall workflow. The device can be deployed on a ground server or edge computing device and supports equivalent verification for spaceborne environments.

[0182] Performance testing:

[0183] To verify the effectiveness of this invention, performance tests were conducted using the LADOS dataset. The performance of the original Mask2Former and the improved method of this invention were compared. The test metrics included mIoU (mean intersection-over-union ratio), IoU for each class, and ACC. The test results are as follows:

[0184] 1. Overall performance: The original Mask2Former has an mIoU of 53.72%, while the improved method of this invention has an mIoU of 74.48%, an improvement of 20.76%;

[0185] 2. Performance of Rare Categories: The original Mask2Former's IoU and ACC for oil platforms were 25.32% and 35.31%, respectively, and for ships, they were 37.17% and 57.76%, respectively. The improved method of this invention achieves IoU and ACC of 63.94% (an improvement of 38.62%) and 67.97% (an improvement of 32.66%) for oil platforms, and 82.80% (an improvement of 45.63%) and 90.94% (an improvement of 33.18%) for ships, respectively.

[0186] Test results show that the improved method proposed in this invention can effectively solve key technical problems such as class imbalance, boundary ambiguity, and background noise interference in semantic segmentation of marine oil spills. It demonstrates significant improvements in overall segmentation accuracy and model robustness, especially in the identification and segmentation of rare categories such as oil platforms and ships, meeting the application requirements of high-precision marine monitoring. Furthermore, through the hierarchical processing mechanism of the space-ground collaborative simulation detection system, this invention achieves efficient hierarchical processing of remote sensing data while ensuring fine segmentation accuracy: a lightweight model undertakes rapid preliminary detection and data screening, reducing redundant transmission burden; the ground-based full model completes high-precision refined segmentation; and continuous iterative optimization in the cloud forms a closed-loop system. The coordinated operation of these three levels effectively improves overall processing efficiency under limited computing resources, balancing system real-time performance and detection accuracy, and possesses good engineering feasibility and application value.

[0187] In summary, this application provides a semantic segmentation method, device, and satellite-ground collaborative simulation detection system for marine oil spills based on an improved Mask2Former. Through algorithm optimization and hardware system co-design, it achieves high-precision detection and efficient processing of marine oil spills. Addressing core technical challenges in marine oil spill scenarios, such as extreme class imbalance (extremely small proportion of oil spill targets with significant class frequency differences), blurred oil film boundaries (low spectral difference between thin oil films and seawater, diffuse boundaries), and severe background noise interference from the sea surface (wave reflections, cloud shadows, etc.), this invention systematically introduces five collaborative improvements to the Mask2Former framework. The modules are as follows: Ocean Scene Context Aggregation (OCA) captures oil film diffusion patterns through directional strip pooling and suppresses background interference with noise-aware gating; Multi-Scale Feature Enhancement (MSFE) uses a scale-adaptive residual mechanism to balance feature representation under different diffusion morphologies; Category Balanced Query Attention (CBQA) reweights the query vector using class frequency log-smooth weights to enhance rare class awareness; Boundary Refinement (BRM) performs spatially selective residual refinement based on boundary attention weights; and Adaptive Boundary Upsampling (ABU) applies boundary sharpening post-processing to the probability map while maintaining the safety of bilinear interpolation. The learnable gain parameters of all five modules are initialized to zero to ensure that the initial training degenerates into an identity mapping, without compromising the baseline performance of the original Mask2Former. Building upon this foundation, this invention further proposes a space-ground collaborative simulation detection system: A spaceborne edge processing workflow is constructed on a ground platform using an equivalent simulation approach. A lightweight segmentation model, refined through knowledge distillation and quantization compression, is used to simulate rapid detection and inference on the spaceborne end. A confidence-driven differentiated data transmission strategy is combined to achieve hierarchical processing. The ground end runs a complete five-module model for fine semantic segmentation, while the cloud handles historical data aggregation and continuous model optimization. This invention also provides a supporting hardware semantic segmentation device, comprising six units: data acquisition, preprocessing, semantic segmentation, result output, storage, and processing. This device can be flexibly deployed on ground-based GPU servers, embedded edge computing platforms, and equivalent spaceborne computing environments, effectively improving the real-time performance, segmentation accuracy, and engineering deployability of offshore oil spill detection.

Claims

1. A semantic segmentation method for marine oil spills based on an improved Mask2Former, characterized in that, The method includes: Step S1: Acquire remote sensing images of the sea surface and extract multi-scale features from them to obtain multi-scale feature maps of different levels, C2, C3, and C4, and a high-level semantic feature map, C5. Step S2: Input the C5 high-level semantic feature map into the multi-scale deformable attention pixel decoder, and generate a mask feature map after upsampling and fusion; then enhance the mask feature map with marine context through the marine scene context aggregation module to suppress sea surface background noise interference; Step S3: The C2, C3, and C4 multi-scale feature maps are subjected to scale-adaptive residual enhancement through the multi-scale feature enhancement module to obtain enhanced multi-scale feature maps, thereby improving the multi-scale feature representation capability. Step S4: Input the enhanced mask feature map output from step S2 and the enhanced multi-scale feature map output from step S3 into the transformer decoder; In the self-attention layer of the transformer decoder, the query vector is reweighted by the category-aware class balance query attention module to solve the problem of extreme class imbalance in complex sea surface scenes. Step S5: For the final mask prediction output by the converter decoder, perform boundary enhancement through the boundary refinement module to improve the accuracy of oil stain edge segmentation; Step S6: Upsample the mask prediction output by the boundary refinement module using the adaptive boundary upsampling module to output the final semantic segmentation result of the marine oil spill.

2. The semantic segmentation method for marine oil spills based on the improved Mask2Former as described in claim 1, characterized in that: The ocean scene context aggregation module in step S2 includes three sub-modules: Directional bar pooling submodule: Performs adaptive bar pooling on the mask feature map along the horizontal and vertical directions respectively, to obtain horizontal bar pooling features and vertical bar pooling features, specifically represented as follows: ； in, and B represents the mask feature map in the horizontal and vertical directions, respectively; C represents the number of feature channels; H and W represent the height and width of the mask feature map, respectively. The strip pooling results are convolved and expanded to restore the original spatial dimensions. A directional attention weight map is then generated through a fusion layer to weight the mask features output by the pixel decoder, thereby capturing the directional features of the oil film spreading along the coastal current. , This is a directional attention weight map; Multi-scale context pyramid submodule: Uses depthwise separable dilated convolution branches with different dilation rates and global average pooling branches to extract multi-scale context features in parallel, and outputs a multi-scale context feature map after channel concatenation and convolution fusion. Noise-aware gating submodule: Extracts channel statistical features through global average pooling, and generates channel attention gating through a fully connected layer. Simultaneously, spatial attention gating is generated through spatial convolution. The calculation formula for the gating mechanism is as follows: ； in, This is the feature map after noise-aware gating modulation. Represents multi-scale contextual feature maps; uses joint two-way gating to modulate features and suppress background noise interference such as sea surface wave reflection; The outputs of each submodule are fused with the original mask feature map to obtain the enhanced mask feature map.

3. The semantic segmentation method for marine oil spills based on the improved Mask2Former as described in claim 1, characterized in that: The multi-scale feature enhancement module mentioned in step S3 includes: Scale Attention Submodule: Performs global average pooling on feature maps at each level to extract global semantic descriptors. ； in, Indicates the captured number A global feature vector that aggregates global contextual information at various scales; After concatenating the global feature vectors of all levels, The scale-weighted prediction network consists of two fully connected layers, a ReLU activation function, and a Softmax function. It predicts an importance weight scalar for each layer, achieving adaptive weight allocation for features at different scales. The calculation formula is as follows: in, This is the multi-scale weight matrix output by the scale-weighted prediction network. These represent the global feature vectors of the captured aggregated global context information at the 1st, 2nd, and 3rd scales, respectively; Residual Scale Enhancement Submodule: Performs depthwise separable convolution and pointwise convolution sequentially on each layer's feature map to enhance the features. The enhanced features are multiplied by a learnable residual scaling factor and the corresponding scale weight, then superimposed onto the original features to form the residual enhanced output. The calculation formula is as follows: ； in, For the enhanced multi-scale feature map, and The first The height and width of the feature map at each scale. Indicates the first The original characteristics of dimensional Indicates enhanced branching features, This represents the learnable residual scaling factor. Indicates the first The adaptive weights are applied to each scale; the learnable residual scaling factor is initialized to zero to ensure that the module output is completely consistent with the input during the initial training phase.

4. The semantic segmentation method for marine oil spills based on the improved Mask2Former as described in claim 1, characterized in that: The category-balanced query attention module mentioned in step S4 includes: The query category estimation submodule: Processes the query feature vectors from the transformer decoder through a classification network, outputting the probability distribution for each query vector corresponding to different categories of the sea surface oil slick. ； in, This represents a predefined number of queries. Representing a classification network, Represented as the number of semantic categories; Category-aware reweighting submodule: Calculates a category importance score for each query vector based on log-smoothed weights of category frequencies and a learnable weight modulation factor. in, Each query in each batch is estimated as the first... The probability of each category This represents the effective class weights, after processing the class weight vector. Introducing a learnable modulation-like factor That is, by The calculated value is then applied as a scaling factor to the self-attention score matrix to enhance the attention response of rare class query vectors. Represented as the first Effective category weights for each category; The dynamic balancing factor submodule estimates the degree of class imbalance based on the information entropy of the probability distribution of query classes within the current batch. It then dynamically outputs a balancing factor through a fully connected network, interpolating and fusing the reweighted attention with the original attention to make the class balance intensity adaptively change with the class distribution within the batch.

5. The semantic segmentation method for marine oil spills based on the improved Mask2Former as described in claim 1, characterized in that: The boundary refining module mentioned in step S5 includes: Lightweight edge detection submodule: Calculates the horizontal and vertical gradients of the mean values ​​of each channel of the mask feature map using a fixed Sobel operator. ； in, The gradient is in the horizontal direction. The gradient is in the vertical direction. Represented as the average mask feature obtained by the pixel decoder, and These represent the horizontal and vertical gradient operators, respectively; after taking the gradient magnitude and normalizing, an edge intensity map with the same resolution as the mask prediction is generated. The boundary attention weight submodule processes the edge intensity map through convolution, batch normalization, ReLU-convolution, and a sigmoid network to output a spatial boundary attention weight map, highlighting the oil film edge region and suppressing interference from non-boundary regions. Specifically, it is represented as follows: ； in, For spatial boundary attention weights, Represented as a boundary probability diagram, It is a lightweight weight generation network. express function; The residual refinement submodule: After batching the final mask prediction tensor, it calculates the residual adjustment amount and limits the residual range through a convolutional network. This is then multiplied by the learnable residual scaling factor, multiplied by the boundary attention weights, and finally superimposed onto the original mask prediction. Specifically, it is represented as follows: ； in, This is a refined mask prediction after boundary attention weight modulation. This represents the original mask prediction. This represents a predefined number of queries. This represents the residual correction term, which is subject to a learnable residual scaling factor. Restrictions.

6. The semantic segmentation method for marine oil spills based on the improved Mask2Former as described in claim 1, characterized in that: The adaptive boundary upsampling module mentioned in step S6 includes: The original logit tensor of the mask is always upsampled using standard bilinear interpolation to avoid introducing the risk of performance degradation; After upsampling is completed and a probability map is obtained through activation function, the Sobel operator is used to detect edge positions on the probability map. A contrast enhancement term is applied to the probability value according to the edge intensity, so that the probability prediction at the boundary is shifted to the extremes of 0 or 1 to sharpen the segmentation boundary.

7. A satellite-ground collaborative simulation and detection system, characterized in that: The system includes: The spaceborne edge processing subsystem is constructed using equivalent simulation and validated on a ground computing platform to simulate the on-orbit processing process deployed on a low-Earth orbit satellite platform. It includes: a remote sensing image acquisition unit for acquiring sea surface remote sensing images; a spaceborne inference unit that simulates a space-level programmable logic device mounted on a low-Earth orbit satellite platform, running a quantized and compressed lightweight segmentation model in a ground environment to perform fast detection inference on remote sensing images and output an oil spill detection confidence map and a coarse segmentation mask; and an adaptive transmission control unit that executes differentiated data transmission strategies based on the confidence map, determines the data content to be transmitted, and generates the corresponding downlink data set, including the confidence map, coarse segmentation mask, and selectively retained original remote sensing image data. The ground receiving and refining subsystem includes: a remote sensing data receiving unit for receiving a set of downlink data generated by an adaptive transmission control unit; and a full model inference unit for running the improved Mask2Former semantic segmentation method for marine oil spills according to any one of claims 1-6, performing fine segmentation on the received remote sensing images, and outputting the segmentation results. The cloud-based analysis subsystem is used to aggregate historical segmentation results and push lightweight model update instructions to the simulation module corresponding to the onboard inference unit.

8. The satellite-ground collaborative simulation and detection system according to claim 7, characterized in that: The lightweight segmentation model in the spaceborne inference unit is obtained by model compression and optimization of the improved Mask2Former semantic segmentation method for marine oil spills according to any one of claims 1-6, including: Using the complete segmentation model as the teacher model, the lightweight student model is trained through distillation learning, so that the student model maintains high segmentation accuracy while reducing model complexity. Integer quantization is performed on the student model to convert floating-point operations into low-ratio fixed-point operations, thereby reducing the computational complexity and storage requirements of the model and adapting to the computing resource constraints of edge processors. Based on the contribution of each module to the segmentation performance, some computational branches in the complete model are pruned, while key sub-modules that play a significant role in object detection and boundary recognition are retained, in order to reduce the number of model parameters and computational load, so as to meet the deployment requirements of edge computing environment.

9. The satellite-ground collaborative simulation and detection system according to claim 7, characterized in that: The differentiated data transmission strategy executed by the adaptive transmission control unit includes: Low confidence scenario: When the maximum confidence of the confidence map is lower than the preset threshold. If it is determined that there is no oil pollution target in the current remote sensing image, only auxiliary information or low-resolution thumbnail is transmitted to minimize the amount of data transmitted. Medium confidence scenario: When the maximum confidence level is at the low confidence threshold With high confidence threshold When a suspected oil spill target is identified in the current image, a coarse segmentation mask and compressed remote sensing image data are transmitted for further analysis and confirmation at the ground end. High confidence scenario: When the maximum confidence level is higher than the preset threshold When there are complex target boundary structures, the system determines that there are high-confidence oil pollution targets in the current image and transmits high-resolution remote sensing image data to support the ground end to run the fully improved Mask2Former model to perform high-precision semantic segmentation processing.

10. A semantic segmentation device for marine oil spills based on an improved Mask2Former, characterized in that: The device includes: The data acquisition unit is used to acquire remote sensing image data of the sea surface; The preprocessing unit is used to normalize, randomly crop, and augment remote sensing images; A semantic segmentation unit is used to implement the improved Mask2Former-based semantic segmentation method for marine oil spills as described in any one of claims 1-6; The result output unit is used to display the oil spill area segmentation and labeling, area statistics and early warning information in real time; when the detection result meets the preset alarm conditions, an alarm is triggered. Storage unit, used to store semantic segmentation model parameters and program instructions; The processing unit is connected to the data acquisition unit, preprocessing unit, semantic segmentation unit, and result output unit, and is used to coordinate the collaborative operation of each unit.

Images