Multi-expert collaborative image segmentation method and system based on postponement mechanism

Abstract

The disclosure provides a multi-expert collaborative image segmentation method and system based on a postponement mechanism, relates to the technical field of computer vision and image analysis, and comprises the following steps: acquiring an image to be segmented; inputting the preprocessed image to be segmented into a SAM image encoder for multi-scale feature extraction, and outputting an image semantic feature map; inputting the image semantic feature map into a SAM decoder to generate four types of outputs, namely, a mask, a display postponement area map, a postponement prediction, and an IOU score; inputting the postponement prediction into an implicit routing path for normalization, and outputting a routing mask; and performing pixel-by-pixel selective fusion on the outputs of an expert prediction model and a main model prediction model according to the routing mask, constructing a joint loss for optimization, and finally outputting a segmentation result. The disclosure realizes interactive collaborative decision-making of the model and the experts at the pixel level, and improves the stability of the model in the key area.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision and image analysis, in particular to a multi-expert collaborative image segmentation method and system based on a deferring mechanism. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute the prior art.

[0003] With the increasing scale and complexity of image data in complex environments, image segmentation tasks have higher requirements for the accuracy and generalization ability of the system. In ambiguous conditions, images are often affected by multiple factors, such as illumination changes, scale inconsistencies, boundary ambiguities, and background clutter, which are particularly evident in target detection tasks, making targets vulnerable to camouflage, occlusion, and deformation. Traditional semantic segmentation models often make mistakes and have unstable predictions when facing these uncertain areas. Existing methods usually rely on a single model to perform full-image prediction, lack effective strategies for handling uncertain areas, and are difficult to meet the robust reasoning needs in high-complexity scenarios.

[0004] In recent years, the idea of learning to defer decisions has been proposed, which is based on the model's ability to autonomously decide whether to complete the prediction independently or to defer the decision to a trusted external expert if uncertain, thereby improving the robustness and safety of the overall system in high-risk samples. This mechanism has shown significant results in classification tasks.

[0005] However, the existing deferring mechanism has not been effectively extended to dense prediction tasks such as image segmentation, especially in medical images and remote sensing images where a large number of pixels are in an uncertain state. The model needs to have more refined pixel-level judgment and responsibility division capabilities. In addition, with the emergence of pre-trained base models such as Segment Anything Model (SAM), their strong segmentation capabilities provide a good foundation for building a flexible and controllable deferring decision system. However, the existing native SAM processing methods still have the following shortcomings:

[0006] 1) When facing uncertain areas, the segmentation of boundary ambiguous areas in complex imaging is uncertain, and the image segmentation result is not accurate;

[0007] 2) Lack of flexible extension interface for multi-expert collaboration, the collaborative image segmentation process for uncertain areas selected for expert assistance prediction is not simple, and pixel-level collaborative image segmentation cannot be quickly realized;

[0008] 3) Lack of explainable responsibility division capability, resulting in poor transparency of optimized decisions and overall segmentation performance, and unable to accurately implement problem feedback. SUMMARY

[0009] The present disclosure proposes a multi-expert collaborative image segmentation method and system based on a postponement mechanism to solve the above problems. While segmenting based on a SAM base model, a learnable postponement mechanism is introduced to realize interactive dynamic collaborative decision-making of the model and the experts at the pixel level to adapt to high-uncertainty areas in complex and fuzzy images, solve segmentation uncertainty in fuzzy boundary areas, and improve processing capacity and interpretability for fuzzy targets.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions:

[0011] The multi-expert collaborative image segmentation method based on the postponement mechanism comprises:

[0012] An image to be segmented is obtained, and the image to be segmented is preprocessed;

[0013] The preprocessed image to be segmented is input into a SAM image encoder for multi-scale feature extraction, and an image semantic feature map is output. The image semantic feature map is input into a SAM decoder to generate four types of outputs, including a mask, a display postponement area map, postponement prediction, and an IOU score;

[0014] The postponement prediction is input into an implicit routing path for normalization, and a routing mask is output. The outputs of the expert prediction model and the main model prediction model are selectively fused pixel by pixel according to the routing mask, a joint loss is constructed for optimization, and finally a segmentation result is output.

[0015] According to some embodiments, the present disclosure adopts the following technical solutions:

[0016] The multi-expert collaborative image segmentation system based on the postponement mechanism comprises:

[0017] A data acquisition module is configured to obtain an image to be segmented and preprocess the image to be segmented;

[0018] A segmentation module is configured to input the preprocessed image to be segmented into a SAM image encoder for multi-scale feature extraction, and output an image semantic feature map. The image semantic feature map is input into a SAM decoder to generate four types of outputs, including a mask, a display postponement area map, postponement prediction, and an IOU score;

[0019] The postponement prediction is input into an implicit routing path for normalization, and a routing mask is output. The outputs of the expert prediction model and the main model prediction model are selectively fused pixel by pixel according to the routing mask, a joint loss is constructed for optimization, and finally a segmentation result is output.

[0020] According to some embodiments, the present disclosure adopts the following technical solutions:

[0021] A computer program product comprising a computer program which, when executed by a processor, implements the multi-expert collaborative image segmentation method based on the postponement mechanism.

[0022] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0023] A non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the multi-expert collaborative image segmentation method based on the postponement mechanism.

[0024] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0025] An electronic device comprising a processor, a memory, and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the multi-expert collaborative image segmentation method based on the postponement mechanism.

[0026] Compared with the prior art, the present disclosure has the beneficial effects that:

[0027] The multi-expert collaborative image segmentation method based on the postponement mechanism of the present disclosure performs collaborative decision on the preprocessed image to be segmented to output an image segmentation result. On the basis of a pre-trained segmentation model, the collaborative decision model introduces a multi-expert collaborative postponement mechanism, an explicit postponement channel, and a compound loss guidance strategy, which significantly improves the segmentation performance, inference transparency, and reliability of system deployment in complex environments such as image blurring.

[0028] The multi-expert collaborative image segmentation method based on the postponement mechanism of the present disclosure makes the decision parameters such as postponement threshold and expert calling ratio visible and adjustable through an explicit routing path: through a postponement probability map and fixed / adaptive threshold, low-confidence pixels are explicitly labeled and handed over to experts for processing, reducing the risk of missegmentation; the adjustability of the threshold enables the system to flexibly trade off "accuracy vs cost" under different scenarios and computing power conditions; the visualization of the postponement area provides a basis for the review process, enhancing the safety and compliance of deployment.

[0029] The multi-expert collaborative image segmentation method based on the postponement mechanism of the present disclosure implements pixel-level responsibility mask generation based on Softmax probability and argmax through an implicit routing path, realizing: automatic allocation of "who is responsible" at the pixel level, with the main model and each expert performing their respective functions, reducing redundant calculations; end-to-end differentiable weight learning enables dynamic adjustment of responsibility allocation during training and consistent convergence with the main task; by monitoring the routing proportion of each expert in real time, subsequent optimization and expansion are facilitated.

[0030] The multi-expert collaborative image segmentation method based on the postponement mechanism of the present disclosure, the joint loss includes the responsibility unified supervision loss, the postponement guidance loss and the load balancing regularization term, the responsibility unified supervision loss guides the consistent allocation of responsibility, the postponement guidance loss uses the expert_cover pseudo label to supervise the pass predictor, and the load balancing regularization term controls the channel load deviation. Avoiding "accurate division but wrong allocation", the responsibility unified supervision loss optimizes the segmentation result and the responsibility routing synchronously, ensuring that the pixels are correctly segmented and correctly assigned; accurately locking uncertain areas, the postponement guidance loss concentrates the postponement probability on low confidence pixels, improving the consistency and interpretability of explicit / implicit paths; stable collaboration and scalable deployment, the load balancing regularization term prevents expert channels from being overloaded or idle for a long time, improving the robustness and scalability of the system under different computing power and scenarios; realizing the efficiency improvement of end-to-end training, the three losses are cooperatively constrained, reducing the instability caused by manual parameter tuning, and making the model reach the balance point of "high precision + low postponement rate" faster. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which form a part of this disclosure, are intended to provide further understanding of the present disclosure and are incorporated herein for illustrative purposes. The schematic embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0032] Figure 1 The overall architecture of the segmentation of the embodiments of the present disclosure is shown in the figure;

[0033] Figure 2 The pixel-level task division process of the implicit routing path of the embodiments of the present disclosure is shown in the figure;

[0034] Figure 3 The expert model flowchart of the embodiments of the present disclosure is shown in the figure;

[0035] Figure 4 The three types of loss joint construction composite supervision mechanism of the embodiments of the present disclosure is shown in the figure. DETAILED DESCRIPTION

[0036] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present disclosure belongs.

[0038] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0039] Embodiment 1

[0040] In an embodiment of the present disclosure, a multi-expert collaborative image segmentation method based on a postponement mechanism is provided, the method comprising:

[0041] Step 1: Obtain an image to be segmented and pre-process it;

[0042] Step 2: input the pre-processed image to be segmented into a SAM image encoder for multi-scale feature extraction, and output an image semantic feature map; input the image semantic feature map into a SAM decoder to generate four types of outputs, including a mask, a display postponement area map, a postponement prediction, and an IOU score;

[0043] input the postponement prediction into an implicit routing path for normalization, and output a routing mask; according to the routing mask, perform pixel-by-pixel selective fusion on the outputs of the expert prediction model and the main model prediction model, construct a joint loss for optimization, and finally output a segmentation result.

[0044] As an embodiment, the multi-expert collaborative image segmentation method based on the postponement mechanism of the present disclosure realizes the responsibility division and dynamic cooperation of the model and the expert at the pixel level, and significantly improves the segmentation accuracy of uncertain areas. By expanding the structure of the SAM decoder, an explicit routing path and an implicit routing path are added to support fine-grained responsibility division. The specific implementation process is as follows:

[0045] Step 1: Obtain an image to be segmented and pre-process it;

[0046] Specifically, the present embodiment uses the Medical Segmentation Decathlon dataset and the TrashCan dataset as the training and verification dataset. The former is a public dataset in the Medical Segmentation Decathlon challenge, and the fifth task, prostate segmentation, is selected, which contains T2 and ADC MR images of 48 patients, and is officially divided into 32 training sets and 16 test sets. The latter contains 7212 real sea surface remote sensing images and provides pixel-level segmentation annotations for various water surface garbage objects such as plastic bottles, fishing gear, and pop cans.

[0047] First, the image is pre-processed, including:

[0048] 1) Color correction and contrast enhancement: Z-Score normalization is applied to eliminate the intensity difference between sequences for RI data (prostate); then, Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to enhance the soft tissue boundary.

[0049] For sea surface remote sensing images: Hydro-Retinex is used for water color correction (compensate for blue-green deviation caused by water absorption / scattering), and linear white balance is combined to improve the dynamic range of brightness.

[0050] 2) Min-Max normalization: Min-Max normalization is used to map the pixel values of the image to the [0, 1] interval. Specifically, it includes:

[0051]

[0052] wherein, to avoid the denominator being zero.

[0053] Step 2: input the preprocessed image to be segmented into the SAM image encoder for multi-scale feature extraction, and output the image semantic feature map; input the image semantic feature map into the SAM decoder to generate four types of outputs: mask, display deferred area map, deferred prediction, and IOU score;

[0054] Specifically, the modified SAM decoder outputs contains explicit routing path and implicit routing path, the explicit routing path includes: (1) defer_map branch: outputs the explicit deferred probability of each pixel; (2) threshold judgment module: generates the deferred area mask according to the set threshold (fixed during training, adjustable during inference); (3) deferred area visualization and review interface: highlights high deferred probability pixels and provides expert / artificial review;

[0055] (4) deferred guidance loss supervision unit: uses expert_cover pseudo label to supervise defer_map, ensuring that the deferred area is consistent with the uncertain pixels.

[0056] The implicit routing path includes: (1) pass_preds branch: outputs (N+1) channel pixel-level responsibility probability; (2) Softmax normalization and Argmax selection module: generates routing mask to determine which expert or main model is responsible for each pixel; (3) expert pixel distribution and fusion module: according to the routing mask, the corresponding pixels are sent to the expert branch and fused with the main model result pixel by pixel; (4) load balancing regularization unit: statistics the routing proportion of each channel and applies variance penalty to prevent extreme deviation of expert load.

[0057] Further, the specific decision segmentation process of the collaborative decision model comprises:

[0058] The preprocessed image to be segmented enters the SAM image encoder, and the output is obtained multi-scale features, and the multi-scale features are input into the SAM mask decoder, and the output is obtained four explicit outputs of mask, deferred prediction, display deferred area graph and IoU score;

[0059] Specifically, the preprocessed image tensor first passes through the frozen ViT-H encoder to obtain an image embedding (image_embeddings, Bx256xH / 16xW / 16) of 1 / 16 scale and a corresponding position encoding; then enter the modified four-way mask decoder, as shown in Table 1, four explicit outputs are generated at a time, and the deferred prediction is the responsibility weight of Softmax activation, and each channel corresponds to the main model / specialist.

[0060] Table 1 Four-way branch output by mask decoder

[0061]

[0062] Among them, the modified four-way mask decoder is obtained by modifying the mask decoder of the Segment Anything main model and expanding the mask decoder to output four-way branches.

[0063] Next, the mask and deferred prediction are input into the implicit routing path, and the specific process is as follows:

[0064] Softmax and routing mask generation: do Softmax on pass_preds ∈ ℝ^{B×(N+1)×H×W} in the channel dimension to get the responsibility probability, and then perform argmax on each pixel to get the routing mask route_mask ∈{0,…,N}^{B×H×W}, wherein 0…N-1 corresponds to the specialist, and the channel N corresponds to the main model; At the same time, one-hot it to R ∈ {0,1}^{B×(N+1)×H×W} in order to calculate and fuse.

[0065] Specialist / main model prediction:

[0066] Main model prediction: directly use main_mask as the output of the main model in channel 0.

[0067] Specialist prediction: for each specialist k ∈ (0…N-1), forward inference on the same input image or its routed pixel area to get M_k ∈ ℝ^{B×1×H×W}. In actual implementation, the specialist can be forward in the whole image, and then mask the non-channel pixels with R_k to avoid slicing overhead.

[0068] Pixel-wise selective fusion: one-hot routing mask is used to do pixel-level weighted sum of all channel outputs:

[0069]

[0070] where ⊙ is pixel-wise multiplication.

[0071] Joint loss optimization Y and true value GT into the unified supervision loss; pass_preds and expert_cover into the deferred guidance loss; the global channel proportion of pass_preds into the load balancing regularization. Back propagation only updates the four-way decoder and expert branch parameters to complete end-to-end optimization. The final output segmentation result is Y_fusion, while defer_map and route_mask are retained as explainable deferred area and responsibility distribution visualization.

[0072] Specifically, three types of expert behaviors are configured through the synthesis expert module in the expert prediction model:

[0073] Table 2 Expert settings

[0074]

[0075] The image tensor (B, C, H, W) of the routing mask is sent into the expert prediction model and the main model prediction model. According to the configuration of three types of expert capabilities in Table 2, the image and its pixel-level true value are first read, and then a pseudo-prediction mask is synthesized on the responsible pixels, and finally the Expert-k mask is output, which represents the segmentation result of k experts on their assigned areas, with a dimension of (B, 1, H, W).

[0076] Specifically, the deferred prediction (pass_preds) is first normalized by Softmax, and then argmax is performed on the channel to obtain the routing mask (B, N+1, H, W), which is used to select the responsible person for each pixel. N represents the main model, and the remaining k=0…N-1 represents k experts. At the same time, the expert responsibility area is generated:

[0077]

[0078] The deferred area map outputs the probability of each pixel being deferred and compared with the threshold τ to generate a deferred binary prediction map (B, 1, H, W). During the training phase, τ=0.5 is fixed, and during the inference phase, it can be dynamically adjusted according to the task sensitivity (sea surface garbage 0.30, prostate edge 0.45, etc.). Pixels higher than the threshold are explicitly marked as expert responsible areas.

[0079] As an embodiment, forward inference obtains output { } and These 5 tensors are split into 3 loss branches according to the arrow direction, each branch only acts on the decoder parameters and expert parameters in the backpropagation phase, and the encoder is always frozen. Figure 4

[0080] Specifically, the three loss branches constitute a joint loss, which includes a unified responsibility supervision loss, a deferred guidance loss, and a load balancing regularization term. The unified responsibility supervision loss guides the consistent allocation of responsibilities, the deferred guidance loss supervises the pass predictor using the expert_cover pseudo label, and the load balancing regularization term controls the channel load deviation.

[0081] Wherein, (1) the unified responsibility supervision loss: softmax_loss_segmentation guides the consistent allocation of responsibilities:

[0082]

[0083] Wherein, is the unified responsibility supervision loss. is the expectation over all pixels in the batch size B, height H, and width W. is the Softmax probability of the i-th channel from the pass_preds branch at this pixel. is the probability of the corresponding main model channel. is the pixel assignment label of the i-th expert in the expert_cover label (0 / 1). y is the true value class (or binary foreground label) of the pixel. is the Iverson bracket, which takes 1 if the condition is true, otherwise 0. is the pixel probability output by the main branch (main_mask). is the binary cross-entropy term between the main branch segmentation result and the true value.

[0084] (2) deferred guidance loss: defer_guidance_loss supervises the pass predictor using the expert_cover pseudo label, as follows:

[0085]

[0086] Wherein, is the total value of the deferred guidance loss; is the logits output by the deferred prediction branch (the value before Sigmoid in defer_map); is the Sigmoid function, which maps logits to [0,1] probability; ​defer_area_ratio is the ratio of deferred area (pixels) in the whole image, which is used to monitor the model's performance in the deferred area. binary cross-entropy softmax probability of the i-th expert branch in the responsibility weight branch (pass_preds), i = 1...E. the sum of all expert branch probabilities (excluding the main model branch), which can be regarded as the "implicit probability taken over by experts". mean squared error, to keep the explicit deferred probability consistent with the implicit expert probability. weight coefficient, controlling the contribution of the second and third terms. average value (expectation) of the whole image / batch, used as sparse regularization to suppress excessive deferral.

[0087] (3) Load balancing penalty term, add_load_balancing_penalty controls the channel load deviation. As follows:

[0088]

[0089] As an embodiment, freeze the SAM ViT-H image encoder (630 million parameters) and Prompt encoder (4 million parameters) during training, only update the decoder parameters, improve training efficiency and stability.

[0090] Mixed precision training. By combining single-precision (FP32) and half-precision (FP16) data types to accelerate deep learning model training.

[0091] Gradient accumulation optimization strategy. Set batch size = 2, accumulation step = 8, equivalent batch size = 16, which can effectively solve the problem of training on small memory devices. The optimization configuration is shown in Table 3.

[0092] Table 3 Optimization configuration

[0093]

[0094] Early stopping condition

[0095] Standard early stopping strategy based on Dice coefficient: if the Dice coefficient of the model on the validation set does not improve significantly in consecutive training rounds, trigger early stopping to save the model with the current optimal Dice.

[0096] Auxiliary early stopping strategy based on deferred ratio: monitor the deferred area ratio (defer_area_ratio) at the same time, if the deferred area cannot be further compressed for consecutive rounds, it means that the model has reached a stable cooperative division state, at this time, trigger early stopping to avoid overfitting.

[0097] This dual mechanism ensures that the model maintains segmentation accuracy while compressing areas requiring expert assistance as much as possible, improving the independence and reasoning ability of the main model.

[0098] Decision-making explainable enhancement

[0099] Enhance decision transparency through multi-dimensional visualization output, making the model reasoning process traceable, verifiable, and explainable. The specific implementation includes three types of core visualization tools:

[0100] Taking sea surface remote sensing images as an example, the final segmentation map uses semi-transparent overlay technology to visually present the segmentation results, with plastic bottles marked in red (RGB 255, 0, 0), fishing nets marked in blue (RGB 0, 120, 255), and metal cans marked in yellow (RGB 255, 204, 0). The difference between the 50% transparency of the boundary area and the 30% transparency of the internal area significantly improves the target recognition.

[0101] The deferred area map identifies the model's uncertain areas through a heat map (Jet color spectrum), establishing a three-level decision-making guidance mechanism: dark blue areas ([0, 0.3)) represent high-confidence results that are directly adopted, yellow-green transition zones ([0.3, 0.6)) suggest that the algorithm can handle autonomously, and orange-red areas ([0.6, 1.0]) require expert review.

[0102] The expert responsibility distribution map uses pseudo-color synthesis and data panel linkage to reveal the decision-making mechanism: the main model working area is rendered in cyan (0, 255, 255, 70%), and different expert areas are marked in different colors.

[0103] Embodiment 2

[0104] In an embodiment of the present disclosure, a multi-expert collaborative image segmentation system based on a deferred mechanism is provided, comprising:

[0105] A data acquisition module is configured to acquire an image to be segmented and pre-process the image to be segmented.

[0106] A segmentation module is configured to input the pre-processed image to be segmented into a SAM image encoder to extract multi-scale features, output an image semantic feature map; input the image semantic feature map into a SAM decoder to generate four types of outputs, including a mask, a deferred area map, a deferred prediction, and an IOU score.

[0107] The deferred prediction is input into an implicit routing path for normalization, and a routing mask is output; the outputs of the expert prediction model and the main model prediction model are selectively fused pixel by pixel according to the routing mask, a joint loss is constructed for optimization, and finally a segmentation result is output.

[0108] Embodiment 3

[0109] In an embodiment of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method for multi-expert collaborative image segmentation based on postponing mechanism.

[0110] Embodiment 4

[0111] In an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, for storing computer instructions which, when executed by a processor, implement the method for multi-expert collaborative image segmentation based on postponing mechanism.

[0112] Embodiment 5

[0113] In an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the method for multi-expert collaborative image segmentation based on postponing mechanism.

[0114] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The means for implementing the functions specified in one or more flows and / or blocks.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable data processing device to produce a computer-implemented process, so that the instructions executed by the computer or other programmable data processing device provide a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The steps for implementing the functions specified in one or more flows and / or blocks.

[0116] The specific embodiments of the present disclosure are described above with reference to the accompanying drawings, but are not intended to limit the protection scope of the present disclosure, and those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present disclosure without creative labor are still within the protection scope of the present disclosure.

Claims

1. A multi-expert collaborative image segmentation method based on a delay mechanism, characterized in that, include: Obtain the image to be segmented and preprocess it; The preprocessed image to be segmented is input into the SAM image encoder for multi-scale feature extraction, and the output is the image semantic feature map; the image semantic feature map is input into the SAM decoder to generate four types of outputs: mask, display retardation region map, retardation prediction and IOU score; The delayed prediction input is normalized in the implicit route path, and the output is the route mask; Based on the routing mask, the outputs of the expert prediction model and the main model prediction model are selectively fused pixel by pixel, a joint loss is constructed for optimization, and the final output is the segmentation result. The implicit routing path includes a responsibility weight prediction branch, which outputs pixel-level responsibility probabilities for N+1 channels; a Softmax normalization and Argmax selection module, which generates a routing mask to determine which expert or the main model is responsible for each pixel; and an expert pixel distribution and fusion module, which sends the corresponding pixels to the expert branch according to the routing mask and fuses them pixel by pixel with the main model results. The load balancing regularization unit is used to calculate the routing ratio of each channel and apply variance penalty to prevent extreme load skew by experts. The joint loss includes the unified responsibility supervision loss, the delayed boot loss, and the load balancing regularization term. The unified responsibility supervision loss guides the consistent allocation of responsibilities, the delayed boot loss uses the expert_cover pseudo-label to supervise the pass predictor, and the load balancing regularization term controls the channel load skew.

2. The multi-expert collaborative image segmentation method based on a delay mechanism as described in claim 1, characterized in that, The preprocessing process includes first performing color correction and contrast enhancement on the image, and then using Min-Max normalization to map the pixel values ​​of the image to the [0,1] interval.

3. The multi-expert collaborative image segmentation method based on a delay mechanism as described in claim 1, characterized in that, The preprocessed image to be segmented is input to the SAM image encoder. First, it passes through the frozen ViT-H encoder in the SAM image encoder to obtain multi-scale features. Then it enters the SAM decoder, which generates four outputs in one forward pass, including a mask, a display delayed region map, delayed prediction, and IOU score. The delayed prediction is the responsibility weight of the Softmax activation.

4. The multi-expert collaborative image segmentation method based on a delay mechanism as described in claim 1, characterized in that, The mask and delayed prediction are input into the implicit routing path. The delayed prediction is first normalized by Softmax, and then argmax is applied to the channel to obtain the routing mask (B, N+1, H, W). For each pixel, a person in charge is selected, and an expert responsibility area is generated at the same time.

5. The multi-expert collaborative image segmentation method based on a delay mechanism as described in claim 1, characterized in that, The delay prediction map is used to display the delay probability of each pixel, and then compared with the threshold τ to generate a delay binary prediction map.

6. A multi-expert collaborative image segmentation system based on a delay mechanism, characterized in that, Specifically, the multi-expert collaborative image segmentation method based on a delay mechanism as described in any one of claims 1-5 includes: The data acquisition module is used to acquire the image to be segmented and to preprocess the image to be segmented. The segmentation module inputs the preprocessed image to be segmented into the SAM image encoder for multi-scale feature extraction and outputs an image semantic feature map. The image semantic feature map is then input into the SAM decoder to generate four types of outputs: mask, display retardation region map, retardation prediction, and IOU score. The delayed prediction input is normalized into the implicit routing path, and the output is the routing mask. Based on the routing mask, the outputs of the expert prediction model and the main model prediction model are selectively fused pixel by pixel to construct a joint loss for optimization, and the final output is the segmentation result.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-expert collaborative image segmentation method based on the deferral mechanism as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the multi-expert collaborative image segmentation method based on a delay mechanism as described in any one of claims 1-5.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the multi-expert collaborative image segmentation method based on a delay mechanism as described in any one of claims 1-5.

Images