A high-dimensional coupled segmentation method based on SAM for fine-grained target semantics

Abstract

This invention discloses a high-dimensional coupled segmentation method based on the fine semantics of the target (SAM). First, the SAM model is improved by introducing a point-level influence decomposition (PID) module after the upsampling operation of the mask decoder in the existing SAM model's segmentation backbone. Then, a reinforcement learning-based point cue optimizer is introduced before the existing SAM model's cue encoder. The segmentation backbone is trained first, and then the current round's cue point embedding, point-level contribution mask, and prediction mask are obtained from the trained backbone. Reinforcement learning states and reward signals are constructed, and the reinforcement learning-based point cue optimizer is trained using the proximal policy optimization (PPO) algorithm, achieving high-dimensional closed-loop coupling between the segmentation backbone and cue generation. This method offers advantages such as significantly improved segmentation accuracy, strong interpretability, good cross-domain generalization and engineering adaptability, and compatibility with multiple versions of the SAM model.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image segmentation technology, and particularly relates to a high-dimensional coupled segmentation method based on the fine semantics of targets using SAM. Background Technology

[0002] Fine-grained semantic segmentation of targets refers to pixel-level, refined, and structured semantic classification and contour segmentation of targets in non-natural domain images such as remote sensing images and synthetic aperture radar (SAR) images, outputting a pixel-by-pixel annotated fine-grained semantic mask. This has significant application value in fields such as marine monitoring, remote sensing reconnaissance, and maritime safety. However, non-natural domain images such as remote sensing images and SAR images generally suffer from weak boundary contrast, low signal-to-noise ratio, and complex texture structures, making it difficult for traditional segmentation methods to achieve high-precision target extraction.

[0003] In recent years, general segmentation models, such as the Segment Anything Model (SAM), have been trained on large-scale natural image datasets and can complete target segmentation tasks under zero-shot conditions. However, because their training data is mainly distributed in natural scenes, they have obvious limitations in fine semantic segmentation of targets in non-natural domain images: firstly, weak boundaries and background noise can easily cause foreground and background confusion, resulting in insufficient segmentation accuracy; secondly, the cue information mainly acts on the high-level semantic space and lacks effective constraints on the target edge pixels.

[0004] Currently, SAM has been improved in two ways to address the aforementioned problems. Firstly, in the segmentation backbone, multi-scale modeling, local feature enhancement, or lightweight adaptation modules are introduced into the intermediate representation layer through structural adaptation and feature enhancement to improve the model's expressive ability in low-contrast and complex texture scenes. Secondly, in the prompt generation process, improved prompting mechanisms are introduced, such as multi-point prompts, prompt self-correction, or reinforcement learning-based prompt generation, to reduce manual interaction and improve segmentation stability. However, existing methods generally separate the feature modeling of the segmentation backbone from the prompt generation process, often employing weakly coupled strategies with staged or low-dimensional feedback. For example, in reinforcement learning-based prompt optimization methods, the prompting strategy typically only provides feedback based on the final segmentation result (binary mask), lacking a characterization of the specific contribution of point prompts in the segmentation process, making it difficult to achieve fine and stable prompt optimization in complex structures and weak edge scenes. Therefore, a high-dimensional coupled segmentation mechanism that effectively coordinates the segmentation backbone and prompt generation and accurately characterizes the contribution of point prompts is urgently needed to improve the accuracy and stability of non-natural domain target segmentation. Summary of the Invention

[0005] The present invention aims to solve the aforementioned technical problems existing in the prior art by providing a high-dimensional coupled segmentation method based on the fine semantics of targets using SAM.

[0006] The technical solution of this invention is a high-dimensional coupled segmentation method based on the fine semantics of targets using SAM, which is carried out according to the following steps: Step 1. Construct an improved SAM segmentation model, including a segmentation backbone and a cue encoder. The segmentation backbone consists of an image encoder and a mask decoder. After the upsampling operation of the mask decoder, a point-level influence decomposition module (PID) is introduced. The point-level influence decomposition module (PID) consists of a point information feature modulation unit, a point-level mask generation unit, and a linear mask reconstruction unit, and is subject to regularization constraints. A reinforcement learning-based point cue optimizer is introduced before the cue encoder. The reinforcement learning-based point cue optimizer consists of a state representation module and a policy network. Step 2. Input training set images and train the segmentation backbone; Step 3. Input the training set images and train the reinforcement learning-based point cue optimizer using the output of the segmentation backbone that has been trained; Step 4. Use the improved SAM segmentation model that has been trained to perform fine semantic segmentation of the target in the image to be segmented.

[0007] Step 2 involves inputting training set images and obtaining image embeddings through an image encoder. F Embedding images F and the cue point embedding vector generated by the cue encoder P i As input to the mask decoder, the backbone feature map is obtained after upsampling. F base The i Provide indexes for in-sample points; backbone feature map F base As input to the point-level influence decomposition module PID; The point information feature modulation unit embeds each prompt point into a vector. P i Input is processed by a lightweight MLP to generate a modulation vector. g i Then, the modulation vector is multiplied channel by channel to... g i With the main feature map F base By fusing the data, we obtain the point feature maps corresponding to each point prompt. f i ; The point-level mask generation unit generates each point feature map f i Decoding to point-level contribution masks using a shared lightweight convolutional head. M i ; The linear mask reconstruction unit uses a linear combination of point-level contribution masks. M i Reconstruct the prediction mask M final ; The regularization constraint is introduced by incorporating regularization loss. ， The L local For point-level local consistency loss, constrain point-level contribution mask M i Within its neighborhood, it conforms to point-type semantics; the L overlap To compensate for overlapping losses, penalize contribution masks at different point levels. M i Spatial overlap; the λ local , λ overlap These are the hyperparameters that balance the two losses.

[0008] The modulation vector ; in It is a Layer 2 fully connected network. W 1 、W 2 For learnable weights, b 1 、b 2 For bias terms; The point-level contribution mask ; The prediction mask Through the following Obtained using the pixel-by-pixel Sigmoid function; ; Where N is the number of cue points in each sample, and the weights are... w i With bias b For learnable parameters, w fg 、w edge 、w bg It is a globally learnable scalar. w fg For the foreground, w edge For edge points, w bg As background point, P L For dot type tags; The ; in, B Indicates the batch quantity. m For batch indexing, N r Indicates a prompt with dots i Location pos i Centered on, with radius r The set of neighborhood points, q The location of the neighboring points, sign PLi The symbolic function is determined by the point type label; max(0, ) represents the Hinge function. M i (m) ( q ) indicates the first m In the nth sample i Point hint in the neighborhood q Point-level contribution mask at the location, M i (m) ( pos i ) indicates the first m In the nth sample i The prompt is in its own location. pos i Point-level contribution mask; The ; in, This is a normalization factor used to eliminate batch size differences. B and the number of point hints in each sample N The impact on the magnitude of the loss, the j Not equal to i The in-sample point hint index, the M i (m) and M j (m) They represent the first m In the nth sample i The point-level contribution mask and the first point hint j Point-level contribution mask for each point.

[0009] Step 3 uses the training set images as input to the image encoder in the segmentation backbone that has been trained, and outputs the image embedding. F img Image embedding F img Embedded with the current round cue point generated by the cue encoder E tAs input to the mask decoder in the segmentation backbone after training, the output is a point-level contribution mask. and prediction mask The t The current interaction round index; the state representation module is based on image embedding. F img Previous prediction mask and embedding the current round prompt point E t Constituting a reinforcement learning state S t The policy network is based on reinforcement learning states. S t Output the next round of point hints; employ a near-end strategy optimization algorithm based on point-level contribution masks. and prediction mask The calculated reward serves as a monitoring signal, which is used to update network parameters until the termination condition is met.

[0010] The rewards include a global reward and a hypothesis testing reward, with the global reward using... Dice The coefficient quantifies the degree of overlap between the predicted mask and the ground truth mask obtained in the current interaction round; the hypothesis test reward is the difference between the global reward when the point hint is active and the global reward when the point hint is inactive, which is used to characterize the marginal contribution of a single point hint to the segmentation result.

[0011] global rewards Calculate using the following formula: ; in, This indicates the set of prompts at the current point. P(t) The predicted mask obtained under the given conditions, where Y is the true mask. It is the numerical stability constant; Hypothesis testing reward Calculate using the following formula: ; in, N t Indicates the first t The number of hints that participate in decoding during the round-robin interaction. M k For PID to the first k The point-level contribution mask generated by the point prompt. w k For learnable type-related weights, b 3 This is a bias term.

[0012] The policy network adopts an Actor-Critic architecture and is trained according to the following steps: Step 3.1 Input the training set images and obtain the image embeddings using the trained segmentation backbone image encoder. F img ; Step 3.2 Input initial point prompt; Step 3.3 Embed the cue points generated by the cue encoder. E t With image embedding F img The mask decoder of the segmented backbone, which has been trained, is fused, and the point-level contribution mask is obtained through PID. and prediction mask ; Step 3.4 The state representation module embeds the image. F img Previous prediction mask and current prompt embedding Et Concatenating to form reinforcement learning states S t ; Step 3.5 Policy Network Based on Reinforcement Learning State S t Output the next point hint; Step 3.6 uses a point-based contribution mask. and prediction mask The calculated hypothesis test reward serves as a supervision signal. The policy gradient variance is reduced and the policy network parameters are optimized by using generalized advantage estimation (GAE). The generalized dominance estimation function is as follows: ; Where, r t For the policy network in reinforcement learning state s t The hypothesis testing reward obtained after performing the action, where γ is the discount factor. This represents the state-value function parameterized by the Critic network. δ t For timing difference error, λ It is the attenuation coefficient of GAE, used to balance bias and variance; Repeat steps 3.3 to 3.6 until the termination condition is met.

[0013] This invention first improves the SAM model by introducing a point-level influence decomposition (PID) module after the upsampling operation of the mask decoder in the existing SAM model's segmentation backbone. It then introduces a reinforcement learning-based point cue optimizer before the existing SAM model's cue encoder. By first training the segmentation backbone, and then using the trained backbone to obtain the current round's cue point embedding, point-level contribution mask, and prediction mask, a reinforcement learning state and reward signal are constructed. The reinforcement learning-based point cue optimizer is then trained using the Proximal Policy Optimization (PPO) algorithm, achieving high-dimensional closed-loop coupling between the segmentation backbone and cue generation. The specific technical effects of this invention are as follows: 1. Significantly improved segmentation accuracy: By explicitly modeling the spatial contribution of cue points through the PID module and combining it with reinforcement learning point-level reward feedback, a high-dimensional closed-loop collaboration between the segmentation backbone and cue generation is achieved, effectively solving the problem of weak boundary confusion; Experimental data show that the Dice, IoU and other indicators are superior to the existing SAM improvement model, and the target segmentation edges are more accurate. 2. High interpretability: The point-level contribution mask intuitively presents the segmentation effect of a single prompt point, and the double regularization constraint enhances the contribution distinguishability, making it easy to trace the generation logic of the segmentation result and providing a clear direction for model optimization; 3. Good cross-domain generalization and engineering adaptability: The high-dimensional coupling mechanism reduces the dependence on natural domain training data and adapts to non-natural domain images such as remote sensing and SAR; it is compatible with multiple versions of SAM models, does not require reconstruction of the segmentation backbone, and has strong engineering adaptability. 4. High stability: The hypothesis testing reward function enables accurate credit allocation, avoids blindness in the optimization process, and improves the stability and consistency of segmentation results in complex scenarios. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the structure of the improved SAM segmentation model according to an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the PID module of the point-level influence decomposition module in an embodiment of the present invention.

[0016] Figure 3 This is a comparison diagram of the segmentation effects of the present invention and existing methods.

[0017] Figure 4 This is a line graph showing the ablation experiments of the present invention and existing methods. Detailed Implementation

[0018] The present invention provides a high-dimensional coupled segmentation method for fine-grained semantics of targets based on SAM, which is carried out according to the following steps: Step 1. Construct an improved SAM segmentation model, such as Figure 1The diagram shows a segmentation backbone and a cue encoder. The segmentation backbone consists of an image encoder and a mask decoder. A point-level influence decomposition (PID) module is introduced after the upsampling operation of the mask decoder. A reinforcement learning-based point cue optimizer is introduced before the cue encoder. This reinforcement learning-based point cue optimizer consists of a state representation module and a policy network. The policy network can simultaneously perceive image semantics, segmentation progress, and interaction history. Its action space selects the next coordinate point on the image plane as the cue. The point-level influence decomposition (PID) module is as follows: Figure 2 As shown: It consists of a point information feature modulation unit, a point-level mask generation unit, and a linear mask reconstruction unit, and is equipped with regularization constraints; Step 2. Train the segmentation backbone; Specifically, the input training set images are processed by an image encoder to obtain image embeddings. F Embedding images F and the cue point embedding vector generated by the cue encoder P i As input to the mask decoder, it is processed by a bidirectional Transformer and upsampled to obtain the backbone feature map. F base The i Provide indexes for in-sample points; backbone feature map F base As input to the point-level influence decomposition module PID; The point information feature modulation unit embeds each prompt point into a vector. P i Input is processed by a lightweight MLP to generate a modulation vector. g i Then, the modulation vector is multiplied channel by channel to... g i With the main feature map F base By fusing the data, we obtain the point feature maps corresponding to each point prompt. fi Thus, in the shared backbone feature map F base Based on this, each point can obtain a feature view with unique semantic bias through its own modulation vector; The modulation vector ; in It is a Layer 2 fully connected network. W 1 、W 2 For learnable weights, b 1 、b 2 For bias terms; The point-level mask generation unit generates each point feature map f i Decoding to point-level contribution masks using a shared lightweight convolutional head. M i ; The point-level contribution mask Convolution head It includes 3×3 convolution, layer normalization, ReLU activation and 1×1 convolution, and outputs the original contribution logits graph corresponding to each point. The shared convolution head design ensures that all points follow the same decoding rules, making the contribution masks of different points comparable. The linear mask reconstruction unit uses a linear combination of point-level contribution masks. M i Reconstruct the prediction mask M final ; The prediction mask M final Through the following Obtained using the pixel-by-pixel Sigmoid function; ; Where N is the number of cue points in each sample, and the weights are... w i With bias b For learnable parameters, w fg 、w edge 、w bg It is a globally learnable scalar. w fg For the foreground, w edge For edge points, w bg As background point, P L For point-type labels; this structure transforms the global segmentation task into a linear combination of a set of point-level contributions, each... M i By explicitly representing the spatial impact of corresponding points, the segmentation process is visualized and interpretable, providing reward variables for the subsequent point hint optimizer. To ensure point-level contribution mask for PID output M i Possessing the characteristics of being interpretable, distinguishable, and locally concentrated, this invention introduces two important regular expressions, namely... ， The L local For point-level local consistency loss, constrain point-level contribution mask Mi conforms to the point - type semantics within its neighborhood; the L overlap is the overlap loss, which penalizes the spatial overlap of different point - level contribution masks M i ; the λ local , λ overlap are hyperparameters for balancing the two losses respectively; the ; where, B represents the number of batches, m is the batch index, Nr represents the position by point prompt i pos i as the center and with a radius of r is the set of neighborhood points, at the q are the positions of the neighborhood points, sign PLi is the sign function determined by the point - type label, and max(0, ) represents the Hinge function, which only produces a penalty when the local relationship does not satisfy the constraint, M i (m) ( q ) represents the point - level contribution mask of the m th point in the neighborhood points i for the q th point in the M i (m) ( pos i ) represents the point - level contribution mask of the m th point at its own position i for the pos i th point; right - hand side the ; where, is the normalization factor, used to eliminate the influence of the number of batches B and the number of point prompts in each sample N on the loss magnitude, the j is the in - sample point - prompt index not equal to i , for each sample m, by traversing all point pairs satisfying i < j through ∑ i<j to avoid double - counting, the M i (m) and M j (m) respectively represent the mIn the nth sample i The point-level contribution mask and the first point hint j Point-level contribution mask for each point hint; calculate the product matrix. L1 The norm is a quantified value of the degree of overlap; minimizing it... L overlap This can encourage different points to minimize the overlap of their contribution masks, thereby improving the distinguishability and complementarity of point contributions; Step 3. Train the reinforcement learning-based point cue optimizer by using the training set images as input to the image encoder in the segmentation backbone, and outputting the image embedding. F img Image embedding F img Embedded with the current round cue point generated by the cue encoder E t As input to the mask decoder in the segmentation backbone after training, the output is a point-level contribution mask. and prediction mask The t The current interaction round index; the state representation module is based on image embedding. F img Previous prediction mask and embedding the current round prompt point E t Constituting a reinforcement learning state S t The policy network is based on reinforcement learning states. S t Output the next round of point hints; employ a near-end strategy optimization algorithm based on point-level contribution masks. and prediction mask The calculated reward serves as a monitoring signal, which is used to update network parameters until the termination condition is met. Specifically, the policy network adopts an Actor-Critic architecture and is trained using the PPO algorithm, with the following steps: Step 3.1 Input the training set images and obtain the image embeddings using the trained segmentation backbone image encoder. F img ; Step 3.2 Input initial point prompt; Step 3.3 Embed the cue points generated by the cue encoder. E t With image embedding F img The mask decoder of the segmented backbone, which has been trained, is fused, and the point-level contribution mask is obtained through PID. and prediction mask ; Step 3.4 The state representation module embeds the image. Fimg Previous prediction mask and current prompt embedding Et Concatenating to form reinforcement learning states S t ; Step 3.5 Policy Network Based on Reinforcement Learning State S t Output the next point hint; Step 3.6 uses a point-based contribution mask. and prediction mask The calculated hypothesis test reward serves as a supervision signal. The policy gradient variance is reduced and the policy network parameters are optimized by using generalized advantage estimation (GAE). The rewards include a global reward and a hypothesis testing reward. The global reward uses the Dice coefficient to quantify the overlap between the predicted mask and the ground truth mask obtained in the current interaction round. The hypothesis testing reward is the difference between the global reward when the point hint is active and the global reward when the point hint is inactive, and is used to characterize the marginal contribution of a single point hint to the segmentation result. global rewards Calculate using the following formula: ; in, This indicates the set of prompts at the current point. P(t) The predicted mask obtained under the given conditions, where Y is the true mask. It is the numerical stability constant; Hypothesis testing reward Calculate using the following formula: ; in, N t Indicates the first t The number of hints that participate in decoding during the round-robin interaction. M k For PID to the first k The point-level mask generated by the point prompt. w k For learnable type-related weights, b 3 For bias terms; The generalized dominance estimation function is as follows: ; Where, r t For the policy network in reinforcement learning state s t The hypothesis testing reward obtained after performing the action, where γ is the discount factor. This represents the state-value function parameterized by the Critic network. δ t For timing difference error, λ It is the attenuation coefficient of GAE, used to balance bias and variance; Repeat steps 3.3 to 3.6 until the termination condition is met; Step 4. Use the improved SAM segmentation model that has been trained to perform fine semantic segmentation of the target in the image to be segmented.

[0019] This invention constructs a pixel-level semantic segmentation dataset FSS-Seg based on FuSARShip1.0, and performs fine annotation on the original dataset. The final dataset contains 11,857 ship images in 7 classes, which are divided into 9,470 training images and 2,387 test images according to the standard.

[0020] All models are implemented using an NVIDIA GeForce RTX 3090 24GB graphics card and PyTorch 1.7. The models use the AdamW optimizer with an initial learning rate of 1e-3 and are trained for 100 epochs with a batch size of 8. Input images are uniformly scaled to 224×224, and the random seed is fixed at 1234. After the segmentation backbone is trained, the policy network learning rate is set to 2e-4, the discount factor γ=0.98, and the GAE parameter λ=0.97. Each interaction epoch consists of a maximum of 8 steps, with an initial point count of 1.

[0021] The experiment was conducted on the FSS-Seg semantic annotation dataset proposed in this invention, covering the three generations of basic SAM models and their representative improvement methods, and incorporating special modification methods for SAR. Quantitative analysis was performed using Dice, IoU, Precision, and HD95 indicators. The results of the large-scale comparative experiment are shown in Table 1.

[0022] Table 1 ; As can be seen from Table 1: In the SAM1 group experiments, the proposed mechanism (SAM1+PIDD+RL) was compared with the original SAM1, SAM-Adapter, CAM-SAM, Grounded SAM and PPO. The proposed mechanism significantly improved the core accuracy metrics compared with the original SAM (ViT-L) and its improved methods. Compared with PPO, which is mainly based on reinforcement learning, Dice and IoU achieved relative gains of 6.68% and 11.19%, respectively.

[0023] In the SAM2 group experiments, the proposed mechanism (SAM1+PIDD+RL) was compared with the original SAM2, SAM2-Adapter, Det-SAM2, SAM-OCTA2, and CA-SAM2. The proposed mechanism improved Dice and IoU by 63.75% and 23.41% respectively compared with SAM2(ViT-L). Even with multiple strong adaptation and heuristic suggestion methods coexisting, the proposed mechanism still achieved the best performance in three metrics.

[0024] In the SAM3 group, the mechanism proposed in this invention (SAM1+PIDD+RL) still improves the overall accuracy, but the gain is smaller than that of SAM1 and SAM2. Nevertheless, the results still verify that the mechanism has cross-generational transferability and can maintain stable improvement on stronger models.

[0025] This invention simultaneously selected CWSAM, pointSAM, and PolSAM for comparative evaluation. The CWSAM method achieved a high accuracy of 90.03%, but had significant boundary errors, with an HD95 of 23.6307 and an IoU index of 70.23%. In contrast, this invention achieved a better balance between overlap quality and boundary accuracy.

[0026] The results show that the method of the present invention is significantly better than the comparative method in all indicators, and the segmentation accuracy and edge smoothness are greatly improved.

[0027] To better demonstrate the segmentation effect of this invention, the qualitative results of this invention, along with four other models including Grounded SAM and CASAM, on FSS-Seg are compared. The results are as follows: Figure 3 As shown.

[0028] from Figure 3 As can be seen, in complex situations such as strong scattering points, streaks, trailing interference, and slender target structures, some methods are prone to background false detections or target contour breaks in images of bulk carriers, container ships, fishing boats, general cargo ships, cruise ships, and other vessels. In contrast, this invention can better maintain the continuity and integrity of the hull area and effectively suppress clutter interference.

[0029] To evaluate the independent contribution of each component in the model of this invention, an ablation experiment was designed and conducted. The experimental results are shown in Table 2 and... Figure 4 As shown.

[0030] Table 2 ; Table 2 illustrates the significant performance improvements achieved by introducing the PID module (Point Influence Decomposition), the PPO optimization strategy (Reinforcement Learning), and their synergistic (coupled) effects on the baseline model. The PPO optimization strategy improves Dice, IoU, and Precision metrics while decreasing HD95, concentrating predicted points more within error regions and balancing false positives with false negatives. The PID module primarily improves Dice and IoU, focusing on finer boundary handling, but its effect on reducing false positives is limited. Without coupling, inconsistencies in signals can easily lead to boundary overflow; coupling effectively resolves conflicts, optimizing all three metrics and minimizing HD95, achieving simultaneous optimization of the region and boundary.

[0031] Figure 4 This is a line graph of the ablation experiment, containing three sub-graphs, each corresponding to a performance comparison under different test scenarios or ablation conditions. From Figure 4 As can be clearly seen, compared to random and heuristic strategies, the PPO optimization strategy of this invention exhibits more stable performance curves and better overall performance when handling positive and negative points, verifying its effectiveness in sample selection and optimization. The consistency results of the three subgraphs further demonstrate that the advantages of the PPO optimization strategy of this invention are universal and not limited by specific scenarios or conditions.

Claims

1. A high-dimensional coupled segmentation method based on SAM (Significant Objective Aspect-Oriented Semantics), characterized in that... Follow these steps: Step 1. Construct an improved SAM segmentation model, including a segmentation backbone and a cue encoder. The segmentation backbone consists of an image encoder and a mask decoder. After the upsampling operation of the mask decoder, a point-level influence decomposition module (PID) is introduced. The point-level influence decomposition module (PID) consists of a point information feature modulation unit, a point-level mask generation unit, and a linear mask reconstruction unit, and is subject to regularization constraints. A reinforcement learning-based point cue optimizer is introduced before the cue encoder. The reinforcement learning-based point cue optimizer consists of a state representation module and a policy network. Step 2. Input training set images and train the segmentation backbone; Step 3. Input the training set images and train the reinforcement learning-based point cue optimizer using the output of the segmentation backbone that has been trained; Step 4. Use the improved SAM segmentation model that has been trained to perform fine semantic segmentation of the target in the image to be segmented; Step 2 involves inputting training set images and obtaining image embeddings through an image encoder. F Embedding images F and the cue point embedding vector generated by the cue encoder P i As input to the mask decoder, the backbone feature map is obtained after upsampling. F base The i Provide indexes for in-sample points; backbone feature map F base As input to the point-level influence decomposition module PID; The point information feature modulation unit embeds each prompt point into a vector. P i Input is processed by a lightweight MLP to generate a modulation vector. g i Then, the modulation vector is multiplied channel by channel to... g i With the main feature map F base By fusing the data, we obtain the point feature maps corresponding to each point prompt. fi ; The point-level mask generation unit generates each point feature map f i Decoding to point-level contribution masks using a shared lightweight convolutional head. M i ; The linear mask reconstruction unit uses a linear combination of point-level contribution masks. M i Reconstruct the prediction mask M final ; The regularization constraint is introduced by incorporating regularization loss. ， The L local For point-level local consistency loss, constrain point-level contribution mask M i Within its neighborhood, it conforms to point-type semantics; the L overlap To compensate for overlapping losses, penalize contribution masks at different point levels. M i Spatial overlap; The λ local , λ overlap These are the hyperparameters that balance the two losses; Step 3 uses the training set images as input to the image encoder in the segmentation backbone that has been trained, and outputs the image embedding. F img Image embedding F img Embedded with the current round cue point generated by the cue encoder Et As input to the mask decoder in the segmentation backbone after training, the output is a point-level contribution mask. and prediction mask The t The current interaction round index; the state representation module is based on image embedding. F img Previous prediction mask and embedding the current round prompt point E t Constituting a reinforcement learning state St The policy network is based on reinforcement learning states. St Output the next round of point hints; A near-end strategy optimization algorithm is adopted to optimize based on point-level contribution masks. and prediction mask The calculated reward serves as a monitoring signal, which is used to update network parameters until the termination condition is met. The rewards include a global reward and a hypothesis testing reward, with the global reward using... Dice The coefficient quantifies the degree of overlap between the predicted mask and the ground truth mask obtained in the current interaction round; the hypothesis test reward is the difference between the global reward when the point hint is active and the global reward when the point hint is inactive, which is used to characterize the marginal contribution of a single point hint to the segmentation result.

2. The high-dimensional coupled segmentation method based on SAM for fine-grained target semantics according to claim 1, characterized in that: The modulation vector ; in It is a Layer 2 fully connected network. W 1 、W 2 For learnable weights, b 1 、b 2 For bias terms; The point-level contribution mask ; The prediction mask Through the following Obtained using the pixel-by-pixel Sigmoid function; ； Where N is the number of cue points in each sample, and the weights are... w i With bias b For learnable parameters, w fg 、w edge 、w bg It is a globally learnable scalar. w fg For the foreground, w edge For edge points, w bg As background point, P L For dot type tags; The ; in, B Indicates the batch quantity. m For batch indexing, Nr Indicates a prompt with dots i Location pos i Centered on, with radius r The set of neighborhood points, q The location of the neighboring points, sign PLi The symbolic function is determined by the point type label; max(0, ) represents the Hinge function. M i (m) ( q ) indicates the first m In the nth sample i Point hint in the neighborhood q Point-level contribution mask at the location, M i (m) ( pos i ) indicates the first m In the nth sample i The prompt is in its own location. pos i Point-level contribution mask; The ; in, This is a normalization factor used to eliminate batch size differences. B and the number of point hints in each sample N The impact on the magnitude of the loss, the j Not equal to i The in-sample point hint index, the M i (m) and M j (m) They represent the first m In the nth sample i The point-level contribution mask and the first point hint j Point-level contribution mask for each point.

3. The high-dimensional coupled segmentation method based on SAM for fine-grained target semantics according to claim 2, characterized in that: global rewards Calculate using the following formula: ； in, This indicates the set of prompts at the current point. P(t) The predicted mask obtained under the given conditions, where Y is the true mask. It is the numerical stability constant; Hypothesis testing reward Calculate using the following formula: ； in, Nt Indicates the first t The number of hints that participate in decoding during the round-robin interaction. M k For PID to the first k The point-level contribution mask generated by the point prompt. wk For learnable type-related weights, b 3 This is a bias term.

4. The high-dimensional coupled segmentation method based on SAM for fine-grained target semantics as described in claim 3 The method, characterized by The policy network adopts an Actor-Critic architecture and is trained according to the following steps: Step 3.1 Input the training set images and obtain the image embeddings using the trained segmentation backbone image encoder. F img ; Step 3.2 Input initial point prompt; Step 3.3 Embed the cue points generated by the cue encoder. E t With image embedding F img The mask decoder of the segmented backbone, which has been trained, is fused, and the point-level contribution mask is obtained through PID. and prediction mask ; Step 3.4 The state representation module embeds the image. F img Previous prediction mask and current prompt embedding E t Concatenating to form reinforcement learning states S t ; Step 3.5 Policy Network Based on Reinforcement Learning State St Output the next point hint; Step 3.6 uses a point-based contribution mask. and prediction mask The calculated hypothesis test reward serves as a supervision signal. The policy gradient variance is reduced and the policy network parameters are optimized by using generalized advantage estimation (GAE). The generalized dominance estimation function is as follows: ； Where, r t For the policy network in reinforcement learning state s t The hypothesis testing reward obtained after performing the action, where γ is the discount factor. This represents the state-value function parameterized by the Critic network. δ t For timing difference error, λ It is the attenuation coefficient of GAE, used to balance bias and variance; Repeat steps 3.3 to 3.6 until the termination condition is met.

Images