A medical image automatic segmentation method and system based on semantic prior guidance dynamic adaptation

Abstract

The application relates to the technical field of medical image analysis, and discloses a medical image automatic segmentation method and system based on semantic prior guidance dynamic adaptation, which comprises the following steps: converting a medical image to be processed into different scale images, and respectively extracting image embedding and prior features; automatically generating prompt information and condition information according to the prior features, and inputting relevant feature adaptation to the image embedding by using the condition information; encoding the prompt information, and generating a medical image segmentation mask according to the adapted image embedding and the prompt embedding; and in the training stage, jointly optimizing the semantic prior network, the prior condition adapter and the mask decoder by using a main segmentation loss and an auxiliary prior loss. The application can reduce the dependence on artificial prompts in the reasoning stage, and improve the automation degree, segmentation precision and cross-dataset applicability of medical image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, and in particular to an automatic medical image segmentation method and system based on semantic prior-guided dynamic adaptation. Background Technology

[0002] Medical image segmentation is used to identify and extract diagnostically significant target regions from medical images. In gastrointestinal endoscopy, accurate segmentation of polyp lesions aids in lesion localization, morphological analysis, and computer-aided diagnosis.

[0003] SAM (Search Engine Image Segmentation) possesses strong cue-based image segmentation capabilities, but it typically requires operators to input point or bounding box prompts, making it difficult to adapt to automated batch processing scenarios for medical images. Existing adaptation methods often use the same adaptation parameters for different input images, limiting their adaptability to variations in lesion size, morphology, boundary conditions, and image quality. Therefore, a medical image segmentation scheme is needed that can automatically generate cue information and adjust image features based on the semantic meaning of lesions in the current medical image. Summary of the Invention

[0004] The main objective of this application is to provide a method and system for automatic medical image segmentation based on semantic prior guidance and dynamic adaptation, which aims to reduce the dependence of medical image segmentation on manual prompts and improve the adaptability of the segmentation model to different lesion appearances and different image distributions.

[0005] To achieve the above objectives, this invention provides an automatic medical image segmentation method based on semantic prior-guided dynamic adaptation, the method comprising the following steps:

[0006] Acquire the medical image to be processed, and convert the medical image to be processed into a first-scale image and a second-scale image;

[0007] Image encoding is performed on the first-scale image to obtain image embedding, and semantic prior extraction is performed on the second-scale image to obtain prior features;

[0008] Based on the prior features, prompt information and conditional information are generated. The conditional information is used to adapt the image embedding to the features corresponding to the medical image to be processed, thereby obtaining the adapted image embedding.

[0009] The prompt information is encoded to obtain a prompt embedding, and a medical image segmentation mask is generated based on the adapted image embedding and the prompt embedding.

[0010] To achieve the above objectives, the present invention also provides an automatic medical image segmentation system based on semantic prior-guided dynamic adaptation, comprising:

[0011] The image processing module is used to acquire the medical image to be processed and convert the medical image to be processed into a first-scale image and a second-scale image;

[0012] The feature extraction module is used to encode the first-scale image to obtain the image embedding, and to extract the semantic prior of the second-scale image to obtain the prior features.

[0013] The prior guidance module is used to generate prompt information and condition information based on the prior features, and use the condition information to perform feature adaptation of the image embedding to the medical image to be processed, so as to obtain the adapted image embedding.

[0014] The segmentation output module is used to encode the prompt information to obtain the prompt embedding, and generate a medical image segmentation mask based on the adapted image embedding and the prompt embedding.

[0015] One or more technical solutions proposed in this application have at least the following technical effects:

[0016] This invention extracts lesion-related semantics through a low-resolution semantic prior processing path and automatically generates sparse and dense prompts based on the lesion-related semantics, enabling medical image segmentation to be completed during the inference stage without receiving manual prompt coordinates.

[0017] The present invention also generates dynamic modulation weights based on the semantic condition vector corresponding to the current medical image, so that different medical images can obtain feature adjustment methods corresponding to the semantics of their lesions, thereby improving the model's adaptability to changes in lesion morphology, scale and boundary state.

[0018] This invention freezes the SAM image encoder and trains the semantic prior network, prior condition adapter, and SAM mask decoder. It controls the number of trainable parameters while preserving the basic image representation capability, and improves the localization quality of automatic prompts through auxiliary prior loss. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments are briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a schematic diagram of the overall process of the automatic medical image segmentation method of the present invention.

[0022] Figure 2 This is a schematic diagram of the image preprocessing process of the present invention.

[0023] Figure 3 This is a schematic diagram of the PriorSAM medical image segmentation network of the present invention.

[0024] Figure 4 This is a schematic diagram of a gastrointestinal endoscopic image to be processed in a specific example of the present invention.

[0025] Figure 5 This is a schematic diagram of the actual segmentation mask in a specific example of the present invention.

[0026] Figure 6 This is a schematic diagram of the predicted segmentation mask and boundary comparison in a specific example of the present invention.

[0027] Figure 7 This is a schematic diagram of the module structure of the automatic medical image segmentation system of the present invention.

[0028] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are used to explain the technical solutions of this application and are not intended to limit this application. For those skilled in the art, equivalent substitutions or adaptive adjustments made based on the technical content disclosed in this application can all be used as executable implementation methods of this application.

[0030] The main solution of this application embodiment is as follows: by constructing the PriorSAM medical image segmentation network, the medical image to be processed is converted into images of different scales, and image embeddings and prior features are extracted respectively; prompt information and conditional information are automatically generated according to the prior features, and the image embedding is adapted to the input relevant features using the conditional information; the prompt information is encoded, and a medical image segmentation mask is generated according to the adapted image embedding and prompt embedding; during the training phase, the main segmentation loss and the auxiliary prior loss are used to jointly optimize the semantic prior network, the prior condition adapter, and the mask decoder.

[0031] It should be noted that the executing entity in this embodiment can be a computing device with image processing and neural network computing capabilities, or a server or workstation deployed with an automatic medical image segmentation program. This embodiment uses the segmentation of polyp lesions in gastrointestinal endoscopy images as an example for illustration, and the method can also be applied to other medical images with pixel-level segmentation requirements.

[0032] Based on this, embodiments of this application provide an automatic medical image segmentation method based on semantic prior guidance and dynamic adaptation. For example... Figure 1 As shown, it includes steps S100 to S400.

[0033] Step S100: Obtain the medical image to be processed and convert the medical image to be processed into a first-scale image and a second-scale image.

[0034] It should be noted that step S100 unifies the data format of the medical images to be processed and provides inputs suitable for the computational scale of both the SAM image encoding path and the semantic prior path. For example... Figure 2 As shown, step S100 includes: adjusting the medical image to be processed to a preset base resolution and performing normalization processing to obtain a base image; interpolating the base image to a first preset resolution to obtain a first scale image; and interpolating the base image to a second preset resolution lower than the first preset resolution to obtain a second scale image.

[0035] In this embodiment of the invention, the medical image to be processed can be a color image acquired by a gastrointestinal endoscopy device. During the model training phase, pixel-level ground truth segmentation masks corresponding to the training medical images are also obtained. Pixels in the polyp lesion region are set as foreground pixels, and pixels in other regions are set as background pixels. A one-to-one correspondence can be established between the training medical images and the ground truth segmentation masks according to file names, and a consistent image format and storage rules are used to ensure that no mismatch between images and annotations occurs when reading training data.

[0036] Specifically, the original medical image and its corresponding ground truth segmentation mask are first adjusted to 512×512. Bilinear interpolation can be used to resize the medical image, while pixel-class-preserving interpolation can be used to resize the ground truth segmentation mask. Medical image normalization transforms the original pixel values ​​to a range that the neural network can stably process, while the ground truth segmentation mask preserves the class relationship between foreground and background pixels.

[0037] During the training phase, data augmentation can be performed simultaneously on the medical image and the ground truth segmentation mask. Simultaneous execution means that both use the same cropping region, flip direction, and rotation parameters to ensure that the lesion location still corresponds after augmentation. In practical applications, data augmentation methods that can be used include random cropping, horizontal flipping, vertical flipping, rotation, brightness and contrast adjustment, and Gaussian noise addition. Brightness and contrast adjustment and noise addition are applied to the medical image, while geometric transformations are applied to both the medical image and the ground truth segmentation mask. During the validation and testing phases, only size adjustment and normalization can be performed to ensure consistent evaluation conditions for different models.

[0038] In one executable implementation, a 512×512 base image is interpolated to 1024×1024 to obtain a first-scale image; the base image is then interpolated to 256×256 to obtain a second-scale image. The first-scale image is used to extract image embeddings with more spatial details, while the second-scale image is used to extract lesion semantics with lower computational cost. The above dimensions are specific implementation parameters; those skilled in the art can set other preset resolutions based on the input specifications of the SAM image encoder used and the video memory capacity of the computing device.

[0039] Step S200: Image encoding is performed on the first scale image to obtain image embedding, and semantic prior extraction is performed on the second scale image to obtain prior features.

[0040] It should be noted that, as Figure 3 As shown, step S200 utilizes a Frozen SAM Image Encoder, SemanticPrior Network, Prior-Conditioned Adapter, SAM Prompt Encoder, and SAM Mask Decoder to construct the PriorSAM medical image segmentation network. It employs two complementary feature processing paths: a high-resolution image encoding path to maintain the image representation capability of the SAM base model, and a low-resolution semantic prior path to extract the lesion semantics required for automatic prompt generation and image feature adaptation. Step S200 includes: freezing the SAM image encoder with input parameters from the first-scale image and extracting image embeddings; performing pooling processing on the second-scale image and encoding the pooling results using a convolutional sequence to obtain prior features.

[0041] In this embodiment of the invention, the SAM image encoder can be a SAM ViT-B image encoder. For a first-scale image with dimensions of 1024×1024×3, the SAM image encoder outputs an image embedding with a spatial size of 64×64 and 256 channels. Parameter freezing means that the parameters of the SAM image encoder are not updated during model backpropagation, while the forward computation of the SAM image encoder is still performed during the training and inference phases.

[0042] By freezing the SAM image encoder, the existing image representation capabilities of the base model can be preserved, and the number of parameters that need to be updated during medical image training can be reduced. Each spatial location in the image embedding corresponds to a local region in the first-scale image, and each channel is used to represent the texture, contour, and contextual features of that local region. The subsequent prior condition adapter enhances the feature responses related to the current lesion based on the image embedding.

[0043] The feature extraction process performed by the semantic prior network on the second-scale image is as follows:

[0044]

[0045] in, This represents the second-scale image input to the semantic prior network; Indicates pooling operation; Represents a convolutional sequence; This represents prior features. Pooling operations and convolutional processing with downsampling capabilities work together to reduce the feature space size, enabling the semantic prior network to aggregate a large range of lesion information with lower computational cost.

[0046] In one executable implementation, the semantic prior network adopts the Original SPN structure. First, a start layer consisting of convolutional layers, group normalization layers, and ReLU activation functions is used to process the second-scale image of 256×256×3, resulting in 256×256×32 features. Then, a first residual coding layer is used to convert the features to 128×128×64, and a second residual coding layer is used to convert the features to 64×64×128. Finally, dilated convolutional layers, group normalization layers, and ReLU activation functions are used to process the features, resulting in 64×64×128 prior features.

[0047] Group normalization calculates the mean and variance of features separately for each preset channel group. Its operation does not depend on a large number of images in a single batch, making it suitable for scenarios with small batch sizes in medical image training. Dilated convolution expands the convolution sampling range while maintaining a 64×64 spatial size, enabling prior features to simultaneously represent information inside the lesion, information at the lesion boundary, and information about the background around the lesion.

[0048] Prior features are fed to the heatmap prediction branch, the dense cueing branch, and the conditional information branch, respectively. The heatmap prediction branch is used to generate the spatial response of lesions; the dense cueing branch is used to form a continuous semantic representation of regions; and the conditional information branch is used to summarize the channel response distribution of the current medical image and provide input conditions for image embedding adaptation.

[0049] Table 1 shows one parameter configuration for each stage of the semantic segmentation network.

[0050] Table 1 Semantic Segmentation Network Stage Configuration

[0051] Basic Input Size adjustment and normalization Original medical images 512×512 3 SAM input branch Interpolation processing 512×512 1024×1024 3 SAM image encoding Frozen SAM ViT-B encoder 1024×1024 64×64 256 Prior input branch Interpolation processing 512×512 256×256 3 Prior starting layer Convolution, group normalization, and ReLU 256×256 256×256 32 First coding layer Residual coding 256×256 128×128 64 Second coding layer Residual coding 128×128 64×64 128 Prior processing layer Dilated convolution, group normalization, and ReLU 64×64 64×64 128 Heatmap Prediction Branch 1×1 convolution 64×64 64×64 1 Dense hint branches Convolutional projection, channel alignment, and upsampling 64×64 256×256 1 Box prompt branch Threshold comparison, boundary coordinate extraction, and coordinate adjustment during training. 64×64 1×4 frame coordinates Point hint branch Maximum response extraction and background position selection 64×64 2×2 Positive and negative coordinates Conditional statistics branch Global pooling and channel-dimensional splicing 64×64 1×1 256 Conditional projection layer Linear projection 256 128 Semantic condition vector Adapted projection Linear projection 64×64×256 64×64×128 Bottleneck characteristics Lightweight Hypernetwork Two-level linear projection and nonlinear transformation 128 128 Dynamic modulation weights Adapted for projection Linear projection and residual fusion 64×64×128 64×64×256 Adaptive Image Embedding Hint Code Sparse and dense cue coding Prompt message Hint Embedded Sparse and dense forms Mask Decoding SAM mask decoder Image embedding and cue embedding low resolution mask 1 Output processing Interpolation, Sigmoid and binarization low resolution mask 512×512 1

[0052] Step S300: Generate prompt information and condition information based on the prior features, and use the condition information to perform feature adaptation of the image embedding to the medical image to be processed, thereby obtaining the adapted image embedding.

[0053] It should be noted that step S300 ensures that both automatic prompt generation and image embedding adaptation are based on the semantic basis of prior features corresponding to the current medical image. Step S300 includes spatial heatmap generation, sparse prompt generation, dense prompt generation, conditional information generation, dynamic modulation weight generation, and image embedding adaptation.

[0054] In this embodiment of the invention, the 64×64×128 prior features are first processed by a heatmap prediction layer using a 1×1 convolution to compress the number of channels to 1, resulting in a 64×64×1 heatmap prediction result. Sigmoid normalization is then applied to the heatmap prediction result to ensure that the response values ​​at each spatial location are within a preset range, thus obtaining a spatial heatmap. Locations with higher response values ​​in a spatial heatmap are more likely to be lesion areas.

[0055] The response values ​​in the spatial heatmap are compared with a preset threshold, and pixels that reach the preset threshold are selected as foreground candidate regions. Specifically, the minimum and maximum coordinates of the foreground candidate regions in the horizontal and vertical directions are calculated, and a bounding rectangle is formed using these coordinates to obtain a bounding box. To improve the coverage of the lesion edge by the bounding box, the sides of the bounding rectangle can be extended outward by a preset number of pixels without exceeding the image range. The bounding box can be represented as 1×4 coordinate data including the coordinates of the upper left and lower right corners.

[0056] Positive point hints can select the pixel location with the highest response value in the spatial heatmap, which typically corresponds to the area the model considers most likely to be inside the lesion. Negative point hints can select a background location with a high response value in the background band near the hint boundary, or a location with a low response value in the spatial heatmap. Point hints can be represented as 2×2 coordinate data and configured with category data to distinguish between positive and negative points.

[0057] During the training phase, the coordinates of each side of the bounding box cues are randomly changed within a preset coordinate range, ensuring that the cues used during training contain varying degrees of positional error. This process enables the mask decoder to learn to comprehensively utilize point cues, dense cues, and image embeddings to complete lesion segmentation. During the inference phase, bounding box cues generated from the spatial heatmap are used directly, eliminating the need for operator input of bounding box coordinates.

[0058] When no foreground candidate region is generated from the spatial heatmap, the rectangle corresponding to the entire image area is used as the bounding box cue, the image center position is used as the positive cue, and a negative cue is selected from preset corner positions. This process ensures that the cue encoder has processable input data and avoids interrupting the segmentation process due to the lack of foreground candidate regions.

[0059] Dense cues are obtained from prior features through convolutional projection, channel alignment, upsampling interpolation, and normalized activation, expressed as:

[0060]

[0061] in, This represents a priori feature of 64×64×128; This represents a dense cue mapping function, which performs convolutional projection and channel alignment on prior features; This represents an interpolation operation that upsamples the dense semantic representation to 256×256. This represents the normalized activation function; This indicates a dense notification.

[0062] Box hints and dot hints together form sparse hints, used to represent the approximate extent of lesions and their foreground and background locations. Dense hints are used to represent the continuous spatial distribution of lesion responses. Sparse and dense hints describe the positional relationships of lesions from different data formats and together constitute the hint information.

[0063] Furthermore, global average pooling and global max pooling are performed on the prior features, respectively. Global average pooling calculates the average response of each channel across all spatial locations, representing the overall activation level of each channel; global max pooling extracts the maximum response of each channel across all spatial locations, representing the response intensity of salient regions. The results of the two pooling operations are concatenated along the channel dimension to obtain the statistical feature vector:

[0064]

[0065] Performing a linear projection on the statistical feature vector yields the semantic condition vector:

[0066]

[0067] in, Represents prior features; This indicates a global average pooling operation; This represents the global max pooling operation; This indicates a channel-dimensional splicing operation; Represents a statistical eigenvector; Represents a linear projection function; Represents a semantic condition vector.

[0068] In one executable implementation, the prior features have 128 channels, and both the global average pooling and global max pooling results are 128-dimensional. These two are concatenated to obtain a 256-dimensional statistical feature vector, which is then linearly projected to obtain a 128-dimensional semantic condition vector. The semantic condition vector is calculated from the prior features of the current medical image; therefore, different medical images will have different semantic condition vectors.

[0069] The semantic conditional vector is input into a lightweight hypernetwork. First, a linear transformation is performed using the first learnable projection matrix, followed by ReLU activation. Then, a linear transformation is performed using the second learnable projection matrix, and finally, the dynamically modulated weights are obtained using the Softplus activation function.

[0070]

[0071] in, Represents a semantic condition vector; This represents the first learnable projection matrix; This represents the second learnable projection matrix; Represents the linear rectification activation function; This represents the activation function used to obtain a positive output value; This represents the dynamic modulation weight.

[0072] Dynamic modulation weights are used to change the channel responses in the image embedding adaptation branch. Since the semantic condition vector changes with the medical image to be processed, the lightweight hypernetwork generates different dynamic modulation weights for different medical images to be processed, so that the adjustment of the image embedding corresponds to the current lesion semantics.

[0073] Specifically, a 64×64×256 image is embedded and compressed into a 64×64×128 bottleneck feature using a downward projection matrix. Activation processing is then performed on the bottleneck feature, and the activation result is multiplied element-wise with 128-dimensional dynamic modulation weights. The dynamic modulation weights can be replicated along the spatial dimension, so that the weights of the same channel act on each spatial location of that channel.

[0074] The modulated bottleneck features are restored to 64×64×256 using the upprojection matrix, and residual fusion is performed with the original image embedding to obtain the adapted image embedding:

[0075]

[0076] in, This represents the image embedding output by the SAM image encoder; Represent the downward projection matrix; This represents the activation function; in one implementation, the GELU activation function is used. This represents element-wise multiplication; Indicates dynamic modulation weights; Represents the projection matrix; Indicates the preset scaling factor; This indicates that the image is being adapted for embedding.

[0077] Residual fusion allows the adapted image embedding to retain the basic representation of the original image embedding while incorporating lesion-related representations controlled by the current semantic conditions. A preset scaling factor controls the degree to which the adapted branch output affects the original image embedding and can be set based on the model's training stability.

[0078] Step S400: Encode the prompt information to obtain the prompt embedding, and generate a medical image segmentation mask based on the adapted image embedding and the prompt embedding.

[0079] It should be noted that step S400 converts the prompt information generated by the semantic prior network into a feature representation that can be processed by the SAM mask decoder, and completes the lesion region prediction. Step S400 includes prompt encoding, mask decoding, output processing, and training optimization.

[0080] In this embodiment of the invention, sparse cues composed of box cues and dot cues are input into the SAM cue encoder to obtain sparse cue embeddings; dense cues of 256×256 are input into the SAM cue encoder to obtain dense cue embeddings. Sparse cue embeddings are used to represent discrete positional constraints, while dense cue embeddings are used to represent continuous spatial semantic distributions.

[0081] The adapted image embedding, sparse cue embedding, dense cue embedding, and SAM location encoding are input into the SAM mask decoder. The mask decoder defines the region to be segmented based on the cue embedding and extracts the lesion contours and internal features associated with the region from the adapted image embedding, outputting a single-channel low-resolution segmentation mask. In an optional embodiment, the mask decoder also outputs the predicted intersection-over-union ratio (IoU) to help evaluate the quality of the predicted segmentation mask.

[0082] The low-resolution segmentation mask is interpolated to be adjusted to 512×512. Then, the Sigmoid function is used to convert the output values ​​of each pixel to a preset range, and the normalized result is compared with a preset binarization threshold to obtain a medical image segmentation mask with the same size as the base image. When it is necessary to restore the original image size, the medical image segmentation mask can be further resized according to the width and height of the original medical image.

[0083] During model training, the spatial heatmap is supervised by the downsampled ground truth segmentation mask, while the predicted segmentation mask is supervised by the original-size ground truth segmentation mask. The total loss is:

[0084]

[0085] in, Indicates the total loss; Indicates the primary segmentation loss; Indicates auxiliary prior loss; This represents the preset balance coefficient, used to adjust the proportion of auxiliary prior loss in the total loss.

[0086] The primary segmentation loss is:

[0087]

[0088] in, This represents the binary cross-entropy loss, used to constrain the foreground or background classification results for each pixel; This represents the Dice loss, used to constrain the overall overlap between the predicted lesion area and the actual lesion area; Indicates the predicted segmentation mask; This represents the actual segmentation mask.

[0089] The auxiliary prior loss is:

[0090]

[0091] in, Indicates auxiliary prior loss; This represents the binary cross-entropy operation; This represents a spatial heatmap predicted by a semantic prior network. This represents the true mask obtained by downsampling the true segmentation mask to the size of the spatial heatmap.

[0092] The auxiliary prior loss enables the heatmap prediction layer to learn the approximate location and coverage of lesions. As the spatial heatmap localization capability improves, the bounding box and point cues generated from the spatial heatmap can more accurately describe the location of lesions, and the prior features can also provide more discriminative input for the generation of conditional information.

[0093] During training, the parameters of the SAM image encoder are frozen, and the parameters of the semantic prior network, prior condition adapter, and SAM mask decoder are updated according to the total loss. The SAM cue encoder can keep its parameters frozen according to the specific training configuration. Optionally, StructureLoss and auxiliary constraints based on predicted intersection-union ratio are employed, and the training method can be selected according to the dataset size and training objectives.

[0094] In one executable implementation, PriorSAM is implemented using PyTorch, with AdamW as the optimizer and an initial learning rate set to 1×10⁻⁶. -4The initial learning rate is set to 5×10⁻⁵, the number of warm-up training epochs is set to 5, the number of images per batch is set to 4, the gradient accumulation count is set to 4, the weight decay coefficient is set to 1×10⁻⁴, the gradient clipping threshold is set to 1.0, and the number of training epochs is set to 100. These two sets of parameters represent implementation configurations under different computing resource conditions.

[0095] After the model is trained, the medical image to be segmented is input into the trained PriorSAM medical image segmentation network. The model sequentially performs basic size adjustment and normalization, dual-scale image transformation, image embedding extraction, prior feature extraction, cue information generation, conditional information generation, image embedding adaptation, cue encoding, mask decoding, and output processing to obtain the medical image segmentation mask for the polyp lesion region.

[0096] The following provides a complete and specific example. Gastrointestinal endoscopy images and their corresponding ground truth segmentation masks from the Kvasir-SEG dataset are selected as training samples, and both the images and ground truth segmentation masks are uniformly adjusted to 512×512. The first processing path interpolates the medical images to 1024×1024 and uses a frozen SAM ViT-B image encoder to obtain a 64×64×256 image embedding; the second processing path interpolates the medical images to 256×256 and uses a semantic prior network to obtain 64×64×128 prior features.

[0097] like Figure 4 As shown, the example medical image contains a polyp lesion with an approximately elliptical outline, surrounded by a dark background, and the examination instruments in the image pass through the vicinity of the lesion. The semantic prior network generates a spatial heatmap based on the lesion's color, texture, and regional continuity, and forms bounding box cues covering the lesion, positive dot cues located inside the lesion, and negative dot cues located in the background region based on the spatial heatmap.

[0098] The dense cueing generated from prior features describes the continuous response of the lesion region, while the semantic conditional vector describes the average and maximum responses of each feature channel in the current image. The prior conditional adapter generates dynamic modulation weights based on the semantic conditional vector and performs channel-dimensional compression, activation, element-wise modulation, channel-dimensional restoration, and residual fusion on the image embedding. The resulting adapted image embedding retains the basic visual information of the endoscopic image and enhances the feature responses corresponding to the current polyp lesion.

[0099] like Figure 5 As shown, the true segmentation mask uses white areas to represent polyp lesions and black areas to represent the background. After embedding the automatically generated prompts and adapted image into the input SAM mask decoder, the result is as follows: Figure 6The predicted segmentation mask is shown. The predicted region maintains the same main outline as the actual lesion region, with minor boundary differences at the local edges of the lesion, indicating that the model can complete the localization and segmentation of the lesion region even with instrument occlusion and local highlighted areas.

[0100] In the in-dataset evaluation, 5-fold cross-validation was used for training and testing on the Kvasir-SEG dataset. Evaluation metrics included Dice similarity coefficient, intersection-union ratio, and 95% Hausdorff distance. Experimental results are shown in Table 2.

[0101] Table 2. Segmentation results of the Kvasir-SEG dataset

[0102] U-Net 0.8567 0.6930 16.47 TransUNet 0.8738 0.7825 14.35 HResFormer 0.8523 0.7494 12.27 VM-UNet 0.8744 0.7505 14.21 SAM 0.7521 0.7360 22.68 PriorSAM 0.8869 0.8305 12.30

[0103] As shown in Table 2, PriorSAM's Dice similarity coefficient is 0.8869 and its crossover-union ratio is 0.8305, both higher than other comparison methods in the table. PriorSAM's 95% Hausdorff distance is 12.30, which is better than U-Net, TransUNet, VM-UNet, and SAM, and close to the results of HResFormer, indicating that this embodiment can improve the overlap of lesion regions and the quality of boundary localization.

[0104] To evaluate the model's applicability to different image distributions, after training the model on the Kvasir-SEG dataset, it was directly tested on the CVC-ClinicDB and ETIS-LaribPolypDB datasets. No parameter fine-tuning was performed during the testing process. The results are shown in Table 3.

[0105] Table 3 Cross-dataset segmentation results

[0106] U-Net 0.7136 0.6201 77.99 0.4437 0.3714 110.96 TransUNet 0.7564 0.6726 70.31 0.3557 0.2654 199.78 HResFormer 0.7487 0.6381 78.60 0.4766 0.3812 153.49 VM-UNet 0.6365 0.5463 76.81 0.3243 0.2805 187.44 SAM 0.6246 0.5372 66.66 0.6066 0.5214 198.72 PriorSAM 0.8173 0.7401 57.78 0.6508 0.5667 111.45

[0107] Table 3 shows that PriorSAM has a Dice similarity coefficient of 0.8173, an intersection-over-union ratio of 0.7401, and a 95% Hausdorff distance of 57.78 on the CVC-ClinicDB dataset; and a Dice similarity coefficient of 0.6508, an intersection-over-union ratio of 0.5667, and a 95% Hausdorff distance of 111.45 on the ETIS-LaribPolypDB dataset. These results indicate that semantic prior extraction and input-related feature adaptation help the model maintain its segmentation ability under different imaging qualities, lighting conditions, and lesion appearance distributions.

[0108] To verify the role of each component, ablation experiments were conducted under the same dataset and training conditions. Baseline represents the baseline model using a static adapter without a semantic prior network, prior condition adapter, or randomly adjusted bounding box coordinates during training; Model A represents the model with a semantic prior network added to Baseline; Model B represents the model with the prior condition adapter replacing the static adapter; PriorSAM represents the complete model further incorporating randomly adjusted bounding box coordinates during training. The experimental results are shown in Table 4.

[0109] Table 4 Ablation Experiment Results

[0110] Baseline 0.7618 0.7312 24.33 Model A √ 0.7944 0.7430 17.75 Model B √ √ 0.8667 0.7552 14.09 PriorSAM √ √ √ 0.8869 0.8305 12.30

[0111] As shown in Table 4, after setting the semantic prior network, the model can extract the spatial semantics of lesions and automatically generate prompt information; after setting the prior condition adapter, the model can adjust the image embedding according to the semantic conditions corresponding to different medical images; after adding random adjustment of bounding box coordinates during the training phase, the model has a better ability to adapt to the positional error of automatically generated bounding box prompts.

[0112] In the evaluation of automatic suggestion quality, the intersection-union ratio (IU / UK), recall, accuracy of positive suggestions, and accuracy of negative suggestions can also be calculated. For images with small lesions, specular highlights, and blurred boundaries, suggestion quality and segmentation quality can be statistically analyzed separately to assess the performance of the automatic suggestion generation and mask decoding processes under challenging image conditions.

[0113] This application also provides an automatic medical image segmentation system based on semantic prior guidance and dynamic adaptation. Please refer to [link / reference]. Figure 7 The automatic medical image segmentation system based on semantic prior guidance and dynamic adaptation includes:

[0114] The image processing module is used to acquire the medical image to be processed and convert the medical image to be processed into a first-scale image and a second-scale image;

[0115] The feature extraction module is used to encode the first-scale image to obtain the image embedding, and to extract the semantic prior of the second-scale image to obtain the prior features.

[0116] The prior guidance module is used to generate prompt information and condition information based on the prior features, and use the condition information to perform feature adaptation of the image embedding to the medical image to be processed, so as to obtain the adapted image embedding.

[0117] The segmentation output module is used to encode the prompt information to obtain the prompt embedding, and generate a medical image segmentation mask based on the adapted image embedding and the prompt embedding.

[0118] Compared with the prior art, the beneficial effects of the medical image automatic segmentation system based on semantic prior guidance and dynamic adaptation provided in this application are the same as the beneficial effects of the medical image automatic segmentation method based on semantic prior guidance and dynamic adaptation provided in the above embodiments. Moreover, the other technical features of the medical image automatic segmentation system based on semantic prior guidance and dynamic adaptation are the same as the features disclosed in the above embodiments, and will not be repeated here.

[0119] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0120] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct or indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for automatic medical image segmentation based on semantic prior-guided dynamic adaptation, characterized in that, The method includes the following steps: Acquire the medical image to be processed, and convert the medical image to be processed into a first-scale image and a second-scale image; Image encoding is performed on the first-scale image to obtain image embedding, and semantic prior extraction is performed on the second-scale image to obtain prior features; Based on the prior features, prompt information and conditional information are generated. The conditional information is used to adapt the image embedding to the features corresponding to the medical image to be processed, thereby obtaining the adapted image embedding. The prompt information is encoded to obtain a prompt embedding, and a medical image segmentation mask is generated based on the adapted image embedding and the prompt embedding.

2. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 1, characterized in that, The steps of acquiring a medical image to be processed and converting the medical image to be processed into a first-scale image and a second-scale image include: Acquire the medical image to be processed, adjust the medical image to a preset base resolution and perform normalization processing to obtain the base image; The base image is interpolated to a first preset resolution to obtain the first scale image; The base image is interpolated to a second preset resolution lower than the first preset resolution to obtain the second scale image.

3. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 1, characterized in that, The steps of encoding the first-scale image to obtain an image embedding and extracting semantic priors from the second-scale image to obtain prior features include: The SAM image encoder with frozen input parameters of the first-scale image is used to extract the image embedding. Pooling is performed on the second-scale image, and the pooling result is encoded using a convolutional sequence to obtain prior features. The expression for the prior features is as follows: in, This represents the second-scale image. This indicates a pooling operation. Represents a convolutional sequence. This represents prior features.

4. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 3, characterized in that, The steps of performing pooling on the second-scale image and encoding the pooling result using a convolutional sequence to obtain prior features include: The second-scale image is subjected to initial convolution processing, wherein the initial convolution processing includes convolution operation, group normalization operation and linear rectified activation operation; Perform residual encoding with downsampling function on the initial convolution processing result; The residual coding result is subjected to dilated convolution, and then group normalization and linear rectified activation are performed on the dilated convolution result to obtain prior features.

5. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 1, characterized in that, The steps of generating prompt information and conditional information based on the prior features, and using the conditional information to perform feature adaptation of the image embedding to the medical image to be processed to obtain the adapted image embedding include: The prior features are processed by channel mapping and normalization to obtain a spatial heatmap, and the pixels in the spatial heatmap whose response values ​​reach a preset threshold are used to form a foreground candidate region. A bounding box prompt is generated based on the boundary coordinates of the foreground candidate region, and a point prompt is generated based on the response value of the spatial heatmap, wherein the point prompt includes positive point prompts and negative point prompts; The box hints and the point hints are combined to form a sparse hint, the prior features are mapped to a dense hint, and the sparse hints and the dense hints are combined to form a hint message.

6. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 5, characterized in that, The steps of combining the box hints and the point hints into a sparse hint, mapping the prior features into a dense hint, and combining the sparse hints and the dense hints into a hint message include: The prior features are subjected to convolutional projection and channel alignment to obtain a dense semantic representation; The dense cue is obtained by upsampling interpolation and normalizing activation of the dense semantic representation. The expression of the dense cue is: in, Represents prior features, This represents a dense cue mapping function that includes convolutional projection and channel alignment. This indicates an upsampling interpolation operation. Represents the normalized activation function. This indicates a dense notification; The sparse and dense hints are combined into a hint message.

7. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 5, characterized in that, The step of generating prompt information and conditional information based on the prior features, and using the conditional information to perform feature adaptation of the image embedding to the medical image to be processed, to obtain the adapted image embedding, further includes: Global average pooling and global max pooling are performed on the prior features respectively, and the pooling results are concatenated along the channel dimension to obtain the statistical feature vector; By performing a linear projection on the statistical feature vector, a semantic condition vector serving as conditional information is obtained, expressed as: The semantic condition vector is subjected to two-level linear projection and nonlinear transformation using a lightweight hypernetwork to obtain dynamic modulation weights, expressed as follows: in, Represents prior features, This indicates a global average pooling operation. This represents the global max pooling operation. This indicates a channel dimension splicing operation. Represents a statistical eigenvector. Represents a linear projection function. Represents a semantic condition vector; and This represents the learnable projection matrix. Represents the linear rectification activation function. This represents a positive activation function. This represents the dynamic modulation weight.

8. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 7, characterized in that, The step of generating prompt information and conditional information based on the prior features, and using the conditional information to perform feature adaptation of the image embedding to the medical image to be processed, to obtain the adapted image embedding, further includes: The channel dimension of the image embedding is compressed using the lower projection matrix, and the channel dimension compression result is activated. The activation processing result is multiplied element-wise with the dynamic modulation weights, and the channel dimension is recovered using the up projection matrix. The channel dimension recovery result is residually fused with the image embedding to obtain the adapted image embedding, expressed as: in, Indicates image embedding, Represents the downward projection matrix. This represents the activation function. This represents element-wise multiplication. Represents dynamic modulation weights. Represents the projection matrix. Indicates the preset scaling factor. This indicates that the image is being adapted for embedding.

9. The automatic medical image segmentation method based on semantic prior-guided dynamic adaptation as described in claim 1, characterized in that, The steps of encoding the prompt information to obtain the prompt embedding, and generating a medical image segmentation mask based on the adapted image embedding and the prompt embedding include: The sparse and dense cues in the cues are input into the SAM cues encoder to obtain the sparse cues embedding and the dense cues embedding. The adapted image embedding, the sparse cue embedding, the dense cue embedding, and the position encoding are input into the SAM mask decoder to obtain a low-resolution segmentation mask; The low-resolution segmentation mask is interpolated, normalized, activated, and binarized to obtain a medical image segmentation mask corresponding to the medical image to be processed.

10. A medical image automatic segmentation system based on semantic prior-guided dynamic adaptation, characterized in that, The system includes: The image processing module is used to acquire the medical image to be processed and convert the medical image to be processed into a first-scale image and a second-scale image; The feature extraction module is used to encode the first-scale image to obtain the image embedding, and to extract the semantic prior from the second-scale image to obtain the prior features. The prior guidance module is used to generate prompt information and condition information based on the prior features, and use the condition information to perform feature adaptation of the image embedding to the medical image to be processed, so as to obtain the adapted image embedding. The segmentation output module is used to encode the prompt information to obtain the prompt embedding, and generate a medical image segmentation mask based on the adapted image embedding and the prompt embedding.

Images