A method, system, storage medium, and product for panoramic segmentation of SAR ocean images based on multi-granularity feature prototype guidance.

Abstract

This invention discloses a panoramic segmentation method, system, storage medium, and product for SAR marine images based on multi-granularity feature prototype guidance. Belonging to the field of computer vision and remote sensing image processing technology, it addresses the shortcomings of existing query-based panoramic segmentation methods, which generally employ random initialization query strategies and lack effective constraints on the physical characteristics and statistical distribution features of SAR images. This invention acquires the SAR marine image to be processed; constructs improved teacher and student models to process the SAR marine image to obtain student multi-granularity Gaussian feature prototypes; inputs these student multi-granularity Gaussian feature prototypes into an improved transformer decoder for decoding, resulting in panoramic segmentation of the SAR marine image. This invention is used for the unified interpretation of target instances and environmental background in complex marine scenes.

Description

Technical Field

[0001] A panoramic segmentation method, system, storage medium, and product for SAR marine images based on multi-granularity feature prototype guidance are presented. This method is used for the unified interpretation of target instances and environmental backgrounds in complex marine scenes and belongs to the field of computer vision and remote sensing image processing technology. Background Technology

[0002] Scene understanding technology based on Synthetic Aperture Radar (SAR) marine remote sensing images is one of the important research directions in the field of marine monitoring and remote sensing applications, and it has significant application value in maritime traffic supervision, fisheries management, marine environmental monitoring, and maritime safety assurance. Compared with tasks that focus only on the detection or semantic segmentation of a single target, panoramic segmentation technology can uniformly model the target instance region and the environmental background region in a marine scene, providing more comprehensive and detailed analytical results for complex marine scenes, and is gradually becoming an important development direction in the intelligent interpretation of SAR marine remote sensing images.

[0003] However, due to the unique nature of SAR imaging mechanisms, SAR marine images typically suffer from problems such as strong speckle noise, complex scattering characteristics, and unstable target-background contrast. This is especially true in nearshore areas, where strong scatterers from coastlines, land areas, and man-made structures severely interfere with the identification and segmentation of marine targets. Against this backdrop, achieving stable segmentation of target instances and background regions simultaneously in complex marine scenes remains a significant technical challenge.

[0004] Most existing intelligent interpretation methods for SAR marine images model target instance segmentation and background region segmentation as independent tasks. For example, one type of method focuses on the detection or segmentation of ship instances, while another focuses on the semantic segmentation of sea and land areas. This task decoupling approach simplifies model design to some extent, but it ignores the inherent relationship between target instances and their surrounding environment, making it difficult to achieve unified modeling of the marine scene. When the results of the above methods are simply superimposed or fused, problems such as inconsistent target boundaries, category conflicts, or semantic incoherence often arise, affecting the reliability of the overall interpretation results.

[0005] In recent years, panoramic segmentation models based on attention mechanisms and query representations have made some progress in the field of computer vision and have been gradually introduced into remote sensing image processing tasks. These methods typically employ randomly initialized query vectors and interact with image features through a multi-stage decoding process to achieve joint prediction of target instances and background regions. However, these methods suffer from the following technical problems in SAR marine image scenarios: First, existing query-based panoramic segmentation methods generally employ random initialization query strategies, lacking effective constraints on the physical characteristics and statistical distribution features of SAR images. For example, in SAR marine images, speckle noise and non-Gaussian scattering characteristics lead to strong uncertainty in feature distribution. Random queries are difficult to effectively focus on discriminative target areas in the early stages of decoding and are easily affected by background clutter, thereby reducing the model's learning efficiency and prediction stability. Second, existing methods typically employ a uniform feature representation during feature modeling, failing to distinguish the semantic differences between target boundary regions, target subject regions, and surrounding context regions. For example, in complex marine scenes, target boundaries play a crucial role in instance differentiation, target subject regions determine semantic consistency, and surrounding context regions are equally indispensable for suppressing background interference. If the features of these different functional regions are modeled uniformly, conflicting optimization objectives can easily be introduced during training, leading to deficiencies in segmentation results in terms of boundary accuracy, instance consistency, or background suppression capabilities.

[0006] Third, existing methods lack effective guidance mechanisms for the evolution of query representations during multi-scale or multi-stage decoding. For example, most methods rely solely on attention computation to facilitate information interaction between features at different scales, failing to fully utilize the structural and contextual information inherent in these features. In SAR marine scenarios, without reasonable prior guidance during decoding, query representations are prone to instability and fluctuations during multi-stage evolution, thus affecting the robustness and consistency of panoramic segmentation results in complex nearshore environments and dense target scenarios. Summary of the Invention

[0007] The purpose of this invention is to provide a panoramic segmentation method, system, storage medium, and product for SAR marine images based on multi-granularity feature prototype guidance. This addresses the problems of existing query-based panoramic segmentation methods, which generally employ random initialization query strategies, lacking effective constraints on the physical characteristics and statistical distribution features of SAR images; typically using a uniform feature representation method during feature modeling, failing to distinguish the semantic differences between target boundary regions, target subject regions, and surrounding context regions; and lacking an effective guidance mechanism for the evolution of query representations during multi-scale or multi-stage decoding.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance includes the following steps: Step 1: Acquire the SAR ocean image to be processed; Step 2: Construct and train the improved teacher and student models. After training, process the SAR ocean image to be processed to obtain the student multi-granular Gaussian feature prototype. The improved teacher and student models are: a teacher multi-granular Gaussian feature prototype layer that uses a label extraction mask to constrain multi-granular features after the feature extraction network of the known teacher model, and a student multi-granular Gaussian feature prototype layer that is introduced after the feature extraction network of the known student model. Step 3: Input the student's multi-granularity Gaussian feature prototype into the improved transformer decoder to obtain the panoramic segmentation of the SAR ocean image; The improved transformer decoder introduces a Sigmoid modulation function after the input layer to modulate the student multi-granular Gaussian feature prototype. The output modulated by the Sigmoid modulation function is multiplied by the student multi-granular Gaussian feature prototype channel by channel and element by element, and then added by the randomly obtained initial query representation channel by channel and element by element to obtain the final query representation for subsequent cascaded prototype-guided decoding.

[0009] Furthermore, the specific implementation steps of the teacher multi-granularity Gaussian feature prototype layer in step 2 are as follows: Step 2.1: Input the SAR ocean image into the feature extraction network for multi-scale feature extraction, and construct the input features based on the multi-scale features. The feature extraction network is ResNet50. Step 2.2: Based on the preset feature decomposition strategy, the input features are divided into multiple granularities to obtain a set of multiple granular features used to represent different spatial regions, different semantic levels, or different levels of detail. The feature decomposition strategy includes region expansion and distance transformation decomposition strategies. The set of multiple granular features includes boundary granular features, structural granular features, and context granular features. Step 2.3: Apply a masking formula to each granularity feature in the multi-granularity feature set to obtain the corresponding mask. Then multiply each mask with the input feature channel by channel and element by element to obtain the corresponding mask constraint feature. The mask formula is: in, A mask representing boundary-level features, used to characterize the boundary regions of target instances. A mask representing structural granular features, used to characterize the main structural regions of a target instance. A mask representing contextual granular features, used to characterize the contextual region surrounding the target instance. This indicates a morphological etching operation. This indicates a morphological dilation operation. This represents the true mask for multi-granularity feature labels, with target instances including ship targets. Represents a structural element; Step 2.4: Based on the mask constraint features corresponding to each granularity feature, perform convolution operations to perform feature aggregation processing to obtain the statistical representation of the features at the corresponding granularity. Step 2.5: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The spatial location index of the mask constraint features corresponding to each granularity The value at that location, The constant term for numerical stability in the improved teacher model. This represents the function used to ensure that the variance parameter is non-negative. The improved teacher model generates the first... The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. The improved teacher model generates the first... The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features Spatial location index values, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved teacher model; Step 2.6: Based on the mean vector and variance vector obtained in Step 2.5, the prototype of the teacher's Gaussian features is obtained, with the following formula: in, The improved teacher model is shown in the first... Teacher Gaussian feature prototypes generated at each granularity.

[0010] Furthermore, the specific implementation steps of the student multi-granularity Gaussian feature prototype layer in step 2 are as follows: Step 2-1: Input the SAR ocean image into the feature extraction network for multi-scale feature extraction, and construct the input features based on the multi-scale features. The feature extraction network is ResNet50. Step 2-2: Based on the preset feature decomposition strategy, the input features are divided into multiple granularities to obtain a set of multiple granular features used to represent different spatial regions, different semantic levels, or different levels of detail. The feature decomposition strategy includes region expansion and distance transformation decomposition strategies. The set of multiple granular features includes boundary granular features, structural granular features, and context granular features. Steps 2-3: Perform convolution operations on each granularity feature to perform feature aggregation processing, so as to obtain the statistical representation of the features at the corresponding granularity. Steps 2-4: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The total number of spatial locations aggregated from each granular feature reference feature. The constant term for numerical stability in the improved student model. This represents the function used to ensure that the variance parameter is non-negative. This indicates the generation of the improved student model. The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. This indicates the generation of the improved student model. The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features The value of the feature of spatial location index, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved student model; Step 2-5: Based on the mean vector and variance vector obtained in Step 2-4, the prototype of the student Gaussian feature is obtained, with the following formula: in, The improved student model is represented in the first... Student Gaussian feature prototypes generated at each granularity.

[0011] Furthermore, in step 2, during the training phase, the improved teacher model and student model measure the distribution difference between the teacher's Gaussian feature prototype and the student's Gaussian feature prototype, and apply a distillation constraint to the student's Gaussian feature prototype based on the distribution difference. This guides the improved student model to generate a student multi-granularity Gaussian feature prototype consistent with the improved teacher model. The distribution difference measure is expressed in the form of relative entropy, and the calculation formula is as follows: If the training iteration requirements are met, the student multi-granularity Gaussian feature prototype optimized by distillation is output, that is, the improved student model is guided to generate a multi-granularity Gaussian feature prototype consistent with the improved teacher model. Otherwise, the improved teacher model updates its parameters based on the exponential moving average (EMA), the improved student model performs backpropagation to update the student Gaussian feature prototype, and is trained again.

[0012] Furthermore, the specific decoding steps of the improved transformer decoder in step 3 are as follows: Step 3.1: Use the student multi-granularity Gaussian feature prototype obtained from the improved student model as prior information; Step 3.2: Modulate the prior information using the Sigmoid modulation function, multiply it channel-by-channel and element-by-element by the prior information, and then add it channel-by-channel and element-by-element to the randomly obtained initial query representation to obtain the final query representation. The formula is as follows: in, This represents the initial query. This represents a multilayer perceptron. This indicates the weights used to generate normalized modulation weights. This represents the final query result; Step 3.3: The final query representation input is decoded through a cascaded mask constraint feature interaction module, feature autocorrelation modeling module, and feature nonlinear mapping module using residual and normalization connection methods to obtain the panoramic segmentation of the SAR ocean image.

[0013] A panoramic segmentation system for SAR marine images based on multi-granularity feature prototype guidance includes a memory, a processor, and a computer program stored in the memory. The system is characterized in that the processor executes the computer program to implement the steps of the panoramic segmentation method for SAR marine images based on multi-granularity feature prototype guidance.

[0014] A computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance.

[0015] A computer program product, comprising a computer program, characterized in that: when the computer program is executed by a processor, it implements the steps of the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance.

[0016] Compared with the prior art, the advantages of the present invention are as follows: I. This invention fundamentally improves the query initialization method in the panoramic segmentation and decoding process by introducing a multi-granularity feature prototype guidance mechanism. Unlike existing technologies that rely on random initialization of queries for decoding prediction, this invention utilizes Gaussian feature prototypes obtained from multi-granularity feature distribution modeling as prior information to initialize or reconstruct the initial query representation. This ensures that the query has clear physical and statistical constraints before entering the decoding stage. This approach effectively reduces the uncertainty and instability caused by random initialization, guides the decoding process from semantically meaningful feature regions, improves query learning efficiency and the consistency of prediction results, and is particularly suitable for synthetic aperture radar marine scenarios with strong noise interference and complex scattering characteristics. Second, this invention employs Gaussian feature prototypes to parametrically model multi-granularity feature distributions and introduces a feature prototype distillation optimization mechanism based on an improved teacher model and an improved student model during the model training phase. This allows the improved student model to obtain stable constraints from the teacher model at the distribution level during the learning process. This design not only simultaneously constrains the centrality and uncertainty characteristics of the feature distribution but also maintains the consistency of feature representation even under conditions of high sample noise or missing local features. This effectively reduces random fluctuations during training and improves the robustness and fault tolerance of the model in low signal-to-noise ratio, weak scattering, and complex nearshore environments. Third, this invention introduces a feature prototype-based cascaded guidance mechanism into the panoramic segmentation decoding process. This mechanism progressively optimizes the query representation and feature fusion process across multiple decoding stages or scale levels, enabling the decoding process to first focus on locally discriminative regions and then gradually integrate broader contextual information. This cascaded guidance approach avoids the problem of numerous invalid responses generated in complex remote sensing backgrounds by existing unguided or single-stage decoding methods. Without significantly increasing model complexity, it improves the computational effectiveness and resource utilization efficiency of the decoding stage, making it more suitable for deployment and long-term stable operation in practical engineering scenarios. Fourth, while introducing a multi-granularity Gaussian feature prototype guidance mechanism, this invention does not make large-scale modifications to the overall structure of existing panoramic segmentation and decoding networks. The construction and guidance modulation process of the feature prototype can be implemented as an independent module, exhibiting good modularity and versatility. Therefore, the method of this invention can be adapted to different types of panoramic segmentation network structures without significantly increasing the scale of model parameters and computational complexity. It is easy to integrate and deploy in existing remote sensing image interpretation systems, while also helping to reduce the computational resource consumption during model training and inference. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a schematic diagram of the multi-granularity Gaussian feature prototype construction module of the present invention; Figure 3 This is a schematic diagram of the query update module based on Gaussian feature prototype guidance of the present invention; Figure 4 This is a schematic diagram of the characteristic prototype distillation optimization mechanism of the present invention; Figure 5 This is a performance comparison table in the embodiments of the present invention; Figure 6 This is a schematic diagram of the structure of a computer device according to the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, a panoramic segmentation method for SAR ocean images based on multi-granularity feature prototypes includes the following steps: Step 1: Acquire the SAR ocean image to be processed; Step 2: Construct and train the improved teacher and student models. After training, process the SAR ocean image to be processed to obtain the student multi-granular Gaussian feature prototype. The improved teacher and student models are: a teacher multi-granular Gaussian feature prototype layer that uses a label extraction mask to constrain multi-granular features after the feature extraction network of the known teacher model, and a student multi-granular Gaussian feature prototype layer that is introduced after the feature extraction network of the known student model. The specific implementation steps of the teacher multi-granularity Gaussian feature prototype layer are as follows: Step 2.1: Input the SAR ocean image into a feature extraction network for multi-scale feature extraction, and construct input features based on the multi-scale features. The feature extraction network is ResNet50. The multi-scale features are used to simultaneously preserve local detail information and global semantic information of the target instance, providing a foundation for subsequent multi-granularity feature partitioning. Step 2.2: Based on a preset feature decomposition strategy, perform multi-granularity partitioning on the input features to obtain a set of multi-granular features representing different spatial regions, different semantic levels, or different levels of detail. Specifically, as shown... Figure 4 As shown, based on the differences in the semantic roles of different spatial regions in the panoramic segmentation task, the multi-scale features are divided into boundary-granular features for representing the boundary regions of the target instance, structural-granular features for representing the main structure of the target instance, and context-granular features for representing the surrounding environment information of the target instance. The feature decomposition strategies include region expansion and distance transform decomposition strategies, which are used for manual segmentation.

[0021] Step 2.3: Apply a masking formula to each granularity feature in the multi-granularity feature set to obtain the corresponding mask. Then multiply each mask with the input feature channel by channel and element by element to obtain the corresponding mask constraint feature. The mask formula is: in, A mask representing boundary-level features, used to characterize the boundary regions of target instances. A mask representing structural granular features, used to characterize the main structural regions of a target instance. A mask representing contextual granular features, used to characterize the contextual region surrounding the target instance. This indicates a morphological etching operation. This indicates a morphological dilation operation. This represents the true mask for multi-granularity feature labels, with target instances including ship targets. Represents a structural element; Step 2.4: Based on the mask constraint features corresponding to each granularity feature, perform convolution operations to perform feature aggregation processing to obtain the statistical representation of the features at the corresponding granularity. Step 2.5: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The spatial location index of the mask constraint features corresponding to each granularity The value at that location, The constant term for numerical stability in the improved teacher model. This represents the function used to ensure that the variance parameter is non-negative. The improved teacher model generates the first... The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. The improved teacher model generates the first... The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features Spatial location index values, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved teacher model; Step 2.6: Based on the mean vector and variance vector obtained in Step 2.5, the prototype of the teacher's Gaussian features is obtained, with the following formula: in, The improved teacher model is shown in the first... Teacher Gaussian feature prototypes generated at each granularity.

[0022] The specific implementation steps of the student multi-granularity Gaussian feature prototype layer are as follows: Step 2-1: Input the SAR ocean image into the feature extraction network for multi-scale feature extraction, and construct the input features based on the multi-scale features. The feature extraction network is ResNet50. Step 2-2: Based on the preset feature decomposition strategy, the input features are divided into multiple granularities to obtain a set of multiple granular features used to represent different spatial regions, different semantic levels, or different levels of detail. The feature decomposition strategy includes region expansion and distance transformation decomposition strategies. The set of multiple granular features includes boundary granular features, structural granular features, and context granular features. Steps 2-3: Perform convolution operations on each granularity feature to perform feature aggregation processing, so as to obtain the statistical representation of the features at the corresponding granularity. Steps 2-4: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The total number of spatial locations aggregated from each granular feature reference feature. The constant term for numerical stability in the improved student model. This represents the function used to ensure that the variance parameter is non-negative. This indicates the generation of the improved student model. The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. This indicates the generation of the improved student model. The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features The value of the feature of spatial location index, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved student model; Step 2-5: Based on the mean vector and variance vector obtained in Step 2-4, the prototype of the student Gaussian feature is obtained, with the following formula: in, The improved student model is represented in the first... Student Gaussian feature prototypes generated at each granularity.

[0023] During the training phase, the improved teacher and student models measure the distribution difference between the teacher's and student's Gaussian feature prototypes. Based on this distribution difference, a distillation constraint is applied to the student's Gaussian feature prototype, guiding the improved student model to generate student multi-granularity Gaussian feature prototypes consistent with the improved teacher model. The distribution difference measure is expressed in the form of relative entropy, and the calculation formula is as follows: By using the above distributional difference measure, the consistency of the improved student model and the improved teacher model at the level of mean and variance parameters can be constrained simultaneously, thereby avoiding the instability problem caused by aligning only a single statistic.

[0024] If the training iteration requirements are met, the student multi-granularity Gaussian feature prototype optimized by distillation is output. This guides the improved student model to generate a multi-granularity Gaussian feature prototype consistent with the improved teacher model (the distribution difference measurement result applies distillation constraints to the multi-granularity Gaussian feature prototype output by the improved student model, and updates the parameters of the improved student model based on these constraints, so that the Gaussian feature prototypes generated by the student model at each granularity level gradually approach the Gaussian feature prototype corresponding to the improved teacher model in terms of distribution). Otherwise, the improved teacher model smooths the parameters of the improved student model over time based on the exponential moving average (EMA), thereby obtaining a stable update of the improved teacher model parameters. The improved student model then performs backpropagation to update the student Gaussian feature prototype and is trained again.

[0025] The multi-granularity Gaussian feature prototype is used to compactly represent features of different granularities at the statistical level, while simultaneously characterizing the centrality and uncertainty of the feature distribution.

[0026] Based on the input features, multi-granularity features of the SAR ocean image are extracted, and a corresponding multi-granularity feature prototype representation is constructed based on the multi-granularity features. The multi-granularity feature information is used to characterize feature representations in different spatial regions, semantic levels, or levels of detail within the image. The multi-granularity feature prototype representation is obtained through Gaussian prototype modeling, which parameterizes the distribution characteristics of each granularity feature to form a feature prototype representation with uncertainty representation capabilities. During the model training phase, a feature prototype distillation mechanism based on teacher and student models is introduced to constrain and optimize the multi-granularity feature prototype representation. The distilled and optimized multi-granularity feature prototype representation is injected into the panoramic segmentation and decoding process in a cascaded manner to guide the query representation, feature representation, or feature fusion process in the segmentation prediction process step by step, thereby continuously optimizing the query expression during multi-scale or multi-stage decoding.

[0027] In this embodiment, the improved student model is used to perform a panoptic segmentation prediction task guided by multi-granularity feature prototypes, while the improved teacher model provides a stable feature prototype distribution as distillation guidance for the student model. The parameters of the improved teacher model are adjusted by smoothly updating the parameters of the improved student model to ensure that the improved teacher model has more stable feature representation capabilities during training. In this embodiment, the smooth update method can be implemented using an exponential moving average.

[0028] Step 3: Input the student's multi-granularity Gaussian feature prototype into the improved transformer decoder to obtain the panoramic segmentation of the SAR ocean image; The improved transformer decoder introduces a Sigmoid modulation function after the input layer to modulate the student multi-granular Gaussian feature prototype. The output modulated by the Sigmoid modulation function is multiplied by the student multi-granular Gaussian feature prototype channel by channel and element by element, and then added by the randomly obtained initial query representation channel by channel and element by element to obtain the final query representation for subsequent cascaded prototype-guided decoding.

[0029] like Figure 3 As shown, the specific decoding steps of the improved transformer decoder are as follows: Step 3.1: Use the student multi-granularity Gaussian feature prototype obtained from the improved student model as prior information; Step 3.2: Modulate the prior information using the Sigmoid modulation function, multiply it channel-by-channel and element-by-element by the prior information, and then add it channel-by-channel and element-by-element to the randomly obtained initial query representation to obtain the final query representation. The formula is as follows: in, This represents the initial query. This represents a multilayer perceptron. This indicates the weights used to generate normalized modulation weights. This represents the final query result; Step 3.3: The final query representation input is decoded through a cascaded mask constraint feature interaction module, feature autocorrelation modeling module, and feature nonlinear mapping module using residual and normalization connection methods to obtain the panoramic segmentation of the SAR ocean image.

[0030] In an exemplary embodiment, to verify the feasibility and effectiveness of the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance described in this application, a SAR ocean image panoramic segmentation dataset for experimental verification was constructed during the specific implementation process, and the method of this application was experimentally tested on this dataset. The experimental dataset is constructed based on publicly available SAR ocean remote sensing image data. In this embodiment, it can be expanded and modified based on SAR ocean images in the SSDD (SAR ShipDetection Dataset) dataset. By supplementing pixel-level or region-level panoramic segmentation annotation information, the original target detection annotation data is transformed into an annotation format suitable for panoramic segmentation tasks, thereby forming an experimental dataset for verifying the method of this application. In this embodiment, the constructed experimental dataset is divided into a data subset for model training and a data subset for model testing. For example, it can be divided according to a ratio of approximately 8:2 between training data and test data to verify the training effect and generalization performance of the improved teacher model and student model. This invention does not impose a specific ratio limitation.

[0031] The method described in this invention was experimentally verified on the constructed SAR ocean image panoramic segmentation dataset. The relevant experimental results are as follows: Figure 5As shown. In this embodiment, to evaluate the overall performance of the panoramic segmentation results, commonly used evaluation metrics in the field of panoramic segmentation can be used to quantitatively analyze the model performance. For example, the Panoptic Quality (PQ) metric can be used to evaluate the overall performance of the model in the joint task of instance segmentation and semantic segmentation. The PQ metric comprehensively reflects the matching accuracy and recognition reliability of the segmented region. In a specific implementation, the PQ metric can be further decomposed into the Segmentation Quality (SQ) metric and the Recognition Quality (RQ) metric. SQ measures the segmentation accuracy of the model within the target region, while RQ measures the model's ability to recognize and match target instances. Through these evaluation metrics, the performance of this invention in panoramic segmentation tasks can be comprehensively analyzed from different perspectives. Experimental results show that the panoramic segmentation method based on multi-granularity feature prototype guidance exhibits good stability in terms of overall segmentation quality and can achieve a relatively balanced segmentation effect in the joint modeling of the target instance region and the background region. By introducing multi-granularity feature information such as boundary features, structural features, and contextual features, different spatial semantic regions in SAR marine images are distinguished and modeled. This helps improve the model's comprehensive perception ability of target boundaries, instance subjects, and environmental background, thereby improving the performance of panoramic segmentation results in complex scenes. Furthermore, the Gaussian feature prototype-based modeling approach, through parameterized description of multi-granularity feature distributions, enables the model to represent uncertainty at the feature level, theoretically helping to mitigate the impact of speckle noise and scattering instability in SAR images on segmentation results. Building upon this, a feature prototype distillation mechanism is introduced, allowing the student model to be guided and constrained by the stable feature distribution of the teacher model during training, thus reducing random fluctuations in the feature learning process and improving the consistency and reliability of segmentation results. Simultaneously, in the panoramic segmentation decoding stage, multi-granularity Gaussian feature prototypes are introduced into the decoding process in a cascaded manner, progressively guiding the evolution of the query representation. This allows the decoding process to gradually transition from local semantics to global semantic modeling, thereby suppressing background interference and enhancing the structural integrity of instance regions. The panoramic segmentation method proposed in this invention, based on multi-granularity feature modeling and prototype-guided decoding, can adapt to situations such as large target scale variations, strong background interference, and complex nearshore environments in complex SAR marine scenes. It provides a feasible technical solution for stable panoramic segmentation of target instance regions and background regions in SAR marine images.

[0032] Based on the same inventive concept, embodiments of this application also provide the aforementioned system, device, or apparatus, including deploying the method in a server or terminal device, the internal structure of which can be illustrated as follows: Figure 6 As shown.

[0033] The system, device, or apparatus includes a processor, memory, input / output (I / O) interfaces, and a communication interface, wherein the processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities to execute a SAR marine image panoramic segmentation related processing flow guided by multi-granularity feature prototypes.

[0034] The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and data resources for panoramic segmentation. The internal memory provides the operating environment for the operation of the operating system and computer programs. The data resources in the system are used to store SAR ocean images, multi-granularity feature information, Gaussian feature prototype parameters, and panoramic segmentation results, among other data.

[0035] The input / output interface is used to realize information interaction between the processor and external devices, and the communication interface is used to establish communication connections with external terminal devices through a network. When the computer program is executed by the processor, it is used to implement the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance described in the above embodiments.

[0036] Those skilled in the art will understand that Figure 6 The structures shown are merely schematic diagrams of parts related to the technical solutions of this invention and do not constitute a limitation on the computer device to which this application applies. In specific implementations, the computer device may include more or fewer components than shown, or some components may be combined, or different component connection methods and layout forms may be adopted.

[0037] In one exemplary embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored therein, which, when executed by a processor, is used to implement the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance described in the above method embodiments.

[0038] In one exemplary embodiment, a computer program product is also provided, the computer program product including a computer program, which, when executed by a processor, is used to implement the SAR ocean image panoramic segmentation method based on multi-granularity feature prototype guidance described in the above method embodiments.

[0039] It should be noted that the data involved in the embodiments of this application (including but not limited to SAR marine image data, multi-granularity feature data, Gaussian feature prototype parameters, panoramic segmentation result data, etc.) are all legally authorized or compliantly obtained data, and their collection, storage, use and processing processes shall comply with the requirements of relevant laws, regulations and industry standards.

[0040] Those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by computer program instructions and related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processing flow described in the above method embodiments. References to memory, databases, or other storage media in the embodiments of this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory, magnetically variable memory, ferroelectric memory, phase-change memory, and graphene memory, etc.; volatile memory can include random access memory or cache memory, wherein random access memory can be static random access memory or dynamic random access memory, etc.

Claims

1. A panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance, characterized in that, Includes the following steps: Step 1: Acquire the SAR ocean image to be processed; Step 2: Construct and train the improved teacher and student models. After training, process the SAR ocean image to be processed to obtain the student multi-granular Gaussian feature prototype. The improved teacher and student models are: a teacher multi-granular Gaussian feature prototype layer that uses a label extraction mask to constrain multi-granular features after the feature extraction network of the known teacher model, and a student multi-granular Gaussian feature prototype layer that is introduced after the feature extraction network of the known student model. Step 3: Input the student's multi-granularity Gaussian feature prototype into the improved transformer decoder to obtain the panoramic segmentation of the SAR ocean image; The improved transformer decoder introduces a Sigmoid modulation function after the input layer to modulate the student multi-granular Gaussian feature prototype. The output modulated by the Sigmoid modulation function is multiplied by the student multi-granular Gaussian feature prototype channel by channel and element by element, and then added by the randomly obtained initial query representation channel by channel and element by element to obtain the final query representation for subsequent cascaded prototype-guided decoding.

2. The panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance according to claim 1, characterized in that, The specific implementation steps of the teacher multi-granularity Gaussian feature prototype layer in step 2 are as follows: Step 2.1: Input the SAR ocean image into the feature extraction network for multi-scale feature extraction, and construct the input features based on the multi-scale features. The feature extraction network is ResNet50. Step 2.2: Based on the preset feature decomposition strategy, the input features are divided into multiple granularities to obtain a set of multiple granular features used to represent different spatial regions, different semantic levels, or different levels of detail. The feature decomposition strategy includes region expansion and distance transformation decomposition strategies. The set of multiple granular features includes boundary granular features, structural granular features, and context granular features. Step 2.3: Apply a masking formula to each granularity feature in the multi-granularity feature set to obtain the corresponding mask. Then multiply each mask with the input feature channel by channel and element by element to obtain the corresponding mask constraint feature. The mask formula is: in, A mask representing boundary-level features, used to characterize the boundary regions of target instances. A mask representing structural granular features, used to characterize the main structural regions of a target instance. A mask representing contextual granular features, used to characterize the contextual region surrounding the target instance. This indicates a morphological etching operation. This indicates a morphological dilation operation. This represents the true mask for multi-granularity feature labels, with target instances including ship targets. Represents a structural element; Step 2.4: Based on the mask constraint features corresponding to each granularity feature, perform convolution operations to perform feature aggregation processing to obtain the statistical representation of the features at the corresponding granularity. Step 2.5: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The spatial location index of the mask constraint features corresponding to each granularity The value at that location, The constant term for numerical stability in the improved teacher model. This represents the function used to ensure that the variance parameter is non-negative. The improved teacher model generates the first... The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. The improved teacher model generates the first... The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features Spatial location index values, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved teacher model; Step 2.6: Based on the mean vector and variance vector obtained in Step 2.5, the prototype of the teacher's Gaussian features is obtained, with the following formula: in, The improved teacher model is shown in the first... Teacher Gaussian feature prototypes generated at each granularity.

3. The panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance according to claim 2, characterized in that, The specific implementation steps of the student multi-granularity Gaussian feature prototype layer in step 2 are as follows: Step 2-1: Input the SAR ocean image into the feature extraction network for multi-scale feature extraction, and construct the input features based on the multi-scale features. The feature extraction network is ResNet50. Step 2-2: Based on the preset feature decomposition strategy, the input features are divided into multiple granularities to obtain a set of multiple granular features used to represent different spatial regions, different semantic levels, or different levels of detail. The feature decomposition strategy includes region expansion and distance transformation decomposition strategies. The set of multiple granular features includes boundary granular features, structural granular features, and context granular features. Steps 2-3: Perform convolution operations on each granularity feature to perform feature aggregation processing, so as to obtain the statistical representation of the features at the corresponding granularity. Steps 2-4: Represent each granularity feature in a unified Gaussian distribution form. Specifically, the statistical representation of each feature is processed by average pooling, and after the statistical aggregation is processed, it is mapped to the mean vector through a fully connected layer. The variance is calculated based on the statistical representation of the features and the variance formula of the mean vector, and the average pooling operation is performed to construct the variance constraint function to map the variance into a variance vector. The formulas for the mean vector and variance vector of the Gaussian feature prototypes at each granularity are as follows: in, This represents the spatial location index in the input features. Indicates the first The total number of spatial locations aggregated from each granular feature reference feature. The constant term for numerical stability in the improved student model. This represents the function used to ensure that the variance parameter is non-negative. This indicates the generation of the improved student model. The mean parameter of each granularity feature distribution is used to characterize the centrality of the feature distribution. This indicates the generation of the improved student model. The variance parameter of each granularity feature distribution is used to characterize the uncertainty of the feature distribution. Representing input features The value of the feature of spatial location index, mean vector The central characteristic and variance vector characterizing this granularity feature The uncertainty characteristics that characterize this granularity feature This represents the improved student model; Step 2-5: Based on the mean vector and variance vector obtained in Step 2-4, the prototype of the student Gaussian feature is obtained, with the following formula: in, The improved student model is represented in the first... Student Gaussian feature prototypes generated at each granularity.

4. The panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance according to claim 3, characterized in that, In step 2, during the training phase, the improved teacher and student models measure the distribution difference between the teacher's Gaussian feature prototype and the student's Gaussian feature prototype. Based on this distribution difference, a distillation constraint is applied to the student's Gaussian feature prototype to guide the improved student model in generating student multi-granularity Gaussian feature prototypes consistent with the improved teacher model. The distribution difference measure is expressed in the form of relative entropy, and the calculation formula is as follows: If the training iteration requirements are met, the student multi-granularity Gaussian feature prototype optimized by distillation is output, that is, the improved student model is guided to generate a multi-granularity Gaussian feature prototype consistent with the improved teacher model. Otherwise, the improved teacher model updates its parameters based on the exponential moving average (EMA), the improved student model performs backpropagation to update the student Gaussian feature prototype, and is trained again.

5. The panoramic segmentation method for SAR ocean images based on multi-granularity feature prototype guidance according to claim 4, characterized in that, The specific decoding steps of the improved transformer decoder in step 3 are as follows: Step 3.1: Use the student multi-granularity Gaussian feature prototype obtained from the improved student model as prior information; Step 3.2: Modulate the prior information using the Sigmoid modulation function, multiply it channel-by-channel and element-by-element by the prior information, and then add it channel-by-channel and element-by-element to the randomly obtained initial query representation to obtain the final query representation. The formula is as follows: in, This represents the initial query. This represents a multilayer perceptron. This indicates the weights used to generate normalized modulation weights. This represents the final query result; Step 3.3: The final query representation input is decoded through a cascaded mask constraint feature interaction module, feature autocorrelation modeling module, and feature nonlinear mapping module using residual and normalization connection methods to obtain the panoramic segmentation of the SAR ocean image.

6. A panoramic segmentation system for SAR ocean images based on multi-granularity feature prototype guidance, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.

8. A computer program product, comprising a computer program, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.

Images