A unified medical image segmentation method based on context hierarchical guidance

Abstract

The application discloses a unified medical image segmentation method based on context hierarchical guidance, comprising the following steps: acquiring a query image and a support image set; processing the query image and the support image set to obtain query image features and a support image feature set; comparing the query image features and the support image feature set to obtain a best support image set; constructing a semantic guidance signal by using the best support image set and the query image; obtaining an intermediate mask by using the best support image set and the query image; obtaining a segmentation mask by using a visual prompt constructed by the intermediate mask and the semantic guidance signal; and fusing the segmentation mask and the intermediate mask to obtain a final mask. The application obtains high-quality visual context information by using a retrieval strategy based on multi-layer feature similarity, designs a visual context guidance decoding strategy and an adaptive mask fusion mechanism, and improves the segmentation precision and robustness of a unified model when facing cross-domain style difference problems.

Description

A unified medical image segmentation method based on context hierarchical guidance Technical Field

[0001] This invention relates to the fields of image segmentation technology and artificial intelligence technology, and in particular to a cross-domain style adaptive unified medical image segmentation method based on context hierarchical guidance. Background Technology

[0002] Medical image analysis provides crucial technical support for accurate disease diagnosis and treatment planning, playing a vital role in modern medicine. Medical image segmentation, a key component of medical image analysis, assists doctors in accurately dividing anatomical structures and lesion regions, and is now widely used in disease monitoring, surgical navigation, and minimally invasive surgery. The rapid development of artificial intelligence technology has provided a powerful technological driving force for advancements in medical image segmentation. Early medical image segmentation methods were designed for single imaging modalities, specific organs, or lesions, lacking the ability to generalize across modalities and to new tasks, making it difficult to adapt to diverse clinical application needs.

[0003] In recent years, with the rapid evolution of large-scale visual foundational models, the "segmenting everything" approach, represented by the SegmentAnything Model (SAM) proposed by Meta AI, has achieved significant success in the field of natural image processing. This provides a framework for building an efficient and unified medical image segmentation model. Researching a unified medical segmentation model can improve the generalization ability of algorithms across different medical modalities and tasks, enabling efficient segmentation and recognition of new scenes and tasks.

[0004] However, current unified medical segmentation methods often suffer performance degradation when faced with new medical scenarios and tasks due to domain variability. Domain variability stems from the inconsistency in data distribution between the training data and the data encountered by the model in the actual deployment environment. This problem is particularly prominent and complex in the field of medical imaging, and can be caused by a variety of factors, including different scanning devices, different imaging protocols (such as scan sequences, slice thickness, and intervals), different image reconstruction algorithms, and individual patient biological differences (such as age, body size, and physiological state) and disease heterogeneity. Even if a unified model is trained on a wide variety of datasets, its performance may still significantly degrade when faced with previously unseen domain-specific data, severely limiting the model's robustness and deployability. Therefore, researching a unified medical image segmentation algorithm that can effectively adapt to the domain variability problem is a crucial issue that urgently needs to be addressed in the field of medical image segmentation.

[0005] Traditional methods for addressing domain dissimilarity typically require access to unlabeled or limited labeled data from the target domain for model tuning. An ideal unified model should be able to achieve good performance directly on unknown domain data without additional training or fine-tuning. In recent years, the context learning paradigm has demonstrated significant potential in artificial intelligence, offering a new approach to addressing the generalization challenge of medical image segmentation in the face of domain dissimilarity. Unlike traditional model fine-tuning or domain adaptation methods, the core idea of ​​context learning is to directly guide the model to understand the features and distribution of the current task or target domain by providing a small number of examples (i.e., context) during inference. This enables the model to adapt instantly to unknown data, improving its flexibility and ease of application without updating model parameters. Inspired by the success of context learning in large models, researchers have begun exploring its application in computer vision, gradually developing context-based visual segmentation methods. These methods typically utilize example images and their corresponding segmentation masks as visual context to guide the model in segmenting query images.

[0006] Recent methods have demonstrated the potential of contextual information in improving model generalization, but they still face several challenges in addressing the complex domain differences prevalent in medical imaging. First, the effectiveness of visual context learning is highly dependent on the quality and relevance of the selected contextual examples (supporting image set); poor-quality examples can severely mislead the model, leading to performance degradation. Second, existing methods struggle to accurately capture the fine-grained features of medical structures (especially subtle lesions or complex boundaries), resulting in limited accuracy in zero-shot or few-shot segmentation when the query image differs significantly from the supporting image set. Therefore, efficiently transferring and deeply integrating the domain-specific prior knowledge contained in the supporting image set into a powerful base segmentation model (such as SAM) to guide its segmentation process on the query image remains a core challenge.

[0007] Therefore, to address the challenges of inconsistent image quality and difficulty in capturing fine-grained features in supporting image sets, and to improve the performance of unified medical image segmentation, it is of urgent significance to research and develop a cross-domain style adaptive unified medical image segmentation method based on context-hierarchical guidance. This method not only holds promise for overcoming the performance bottlenecks and adaptability limitations of existing unified segmentation models when facing the prevalent domain differences in medical images, but will also drive medical image segmentation technology towards higher levels of intelligence and generalization, providing strong support for improving the efficiency and accuracy of medical diagnosis and helping the construction of a smart healthcare system reach a new level. Summary of the Invention

[0008] (1) Technical problems to be solved

[0009] This invention discloses a context-hierarchical guided cross-domain style adaptive unified medical image segmentation method, aiming to address the performance bottlenecks and insufficient adaptability issues when facing the prevalent domain differences in medical images. High-quality visual context information is obtained through a retrieval strategy based on multi-layer feature similarity. A visual context-guided decoding strategy and an adaptive mask fusion mechanism are designed to improve the segmentation accuracy and robustness of the unified model when facing domain differences.

[0010] (2) Technical solution

[0011] This invention discloses a unified medical image segmentation method based on context-hierarchical guidance, comprising the following steps:

[0012] Step 1: Obtain the query image and supporting image set;

[0013] Step 2: Process the query image and the supporting image set to obtain the query image features and the supporting image feature set;

[0014] Step 3: Compare the query image features and the supporting image feature set to obtain the optimal supporting image set;

[0015] Step 4: Construct semantic guidance signals using the optimal supporting image set and query images;

[0016] Step 5: Use the optimal support image set and query image to obtain the intermediate mask and construct visual cues;

[0017] Step 6: Obtain the segmentation mask using visual cues and semantic guidance signals;

[0018] Step 7: Use the segmentation mask and the intermediate mask to merge them to obtain the final mask.

[0019] Preferably, step 2 involves processing the query image and the supporting image set to obtain query image features and supporting image feature sets, including the following steps:

[0020] Step 201: Using the query image, obtain multi-level query image features through a pre-trained encoder;

[0021] Step 202: Using the support image set, obtain a multi-level support image feature set through a pre-trained encoder.

[0022] Preferably, step 3 compares the query image features and the supporting image feature set to obtain the optimal supporting image set, including the following steps:

[0023] Step 301: Calculate the similarity scores of each feature in the multi-level query image feature set and the multi-level support image feature set, respectively.

[0024] Step 302: Based on the calculated similarity scores, select the K supporting images with the highest scores to form the optimal supporting image set.

[0025] Preferably, step 301 calculates the similarity scores of each feature in the multi-level query image feature set and the multi-level support image feature set, including the following steps:

[0026] Step 3011, first calculate the similarity score:

[0027] ,

[0028] in This represents the score of the i-th supporting image. Indicates the weighting coefficient. This indicates the calculation of cosine similarity. It is a feature of the i-th supporting image. It is a query image feature;

[0029] Step 3012: Next, select the top K images with the highest scores to form the optimal support image set.

[0030] Preferably, step 4 involves constructing a semantic guidance signal using the optimal support image set and the query image, including the following steps:

[0031] Step 401: Select the best support image from the set of best support images;

[0032] Step 402: Using the optimal support image and the query image, and combining them with an unsupervised medical image registration algorithm, the deformation field from the optimal support image to the query image is obtained.

[0033] Step 403: Using the mask of the optimal support image, combined with the deformation field and spatial transformation network, the semantic guidance signal is calculated.

[0034] Preferably, step 5, which uses the optimal support image set and query image to obtain an intermediate mask and construct a visual cue, includes the following steps:

[0035] Step 501: Obtain the optimal supporting image set through step 3 and obtain the query image through step 1;

[0036] Step 502: Using the optimal support image set and query image, and combining the context segmentation model UniverSeg, an intermediate mask is generated.

[0037] Step 503: Using the intermediate mask, the edges of the intermediate mask are reduced by morphological operation erosion, and the mask visual cues are obtained by opening operation to reduce noise interference and remove unreliable areas.

[0038] Step 504: Using the intermediate mask, the minimum bounding box generation algorithm is used to generate the minimum bounding rectangle for the preprocessed intermediate mask as a box tooltip;

[0039] Step 505: Randomly sample 5 points in the intermediate mask as point prompts;

[0040] Step 506: The mask visual cues, box cues, and dot cues together constitute the visual cues.

[0041] Preferably, step 6, which utilizes visual cues and semantic guidance signals to obtain a segmentation mask, includes the following steps:

[0042] Step 601: Convert the semantic guidance signal into an attention bias matrix;

[0043] Step 602: Add the attention bias matrix to the last layer of the SAM mask decoder to obtain the modified SAM mask decoder;

[0044] Step 603: Using the query image and visual cues, output the segmentation mask through the modified SAM mask decoder.

[0045] Preferably, step 7 uses the segmentation mask and the intermediate mask to fuse them to obtain the final mask, including the following steps:

[0046] Step 701: Obtain the intermediate mask and the segmentation mask;

[0047] Step 702: Obtain the fusion mask using the intermediate mask and the segmentation mask.

[0048] ,

[0049] The middle mask is The segmentation mask is , merge mask is The integration process requires... To balance the effects of the intermediate mask and the segmentation mask,

[0050] ,

[0051] in The confidence score for SAM decoding. Here, m represents the temperature coefficient, and m represents the number of supporting images.

[0052] k is the weighting coefficient.

[0053] Preferably, step 601, which converts the semantic guidance signal into an attention bias matrix, includes the following steps:

[0054] First, the semantic guidance signal, i.e., the mask image, is sampled until it is the same size as the feature, and then flattened into a vector.

[0055] Next, the bias matrix is ​​calculated using the following operations.

[0056] ,

[0057] in The bias coefficient, and For intermediate semantic vectors, and This is the index of the intermediate semantic vector;

[0058] In the field of image segmentation, the region that needs to be segmented is called the foreground, and the region in the query image other than the foreground is called the background.

[0059] That is, the future. This is the background. Through this bias matrix, attention between foreground elements is enhanced, attention between background elements is suppressed, and attention between foreground and background elements is not processed.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] This invention proposes a unified medical image segmentation method based on context-layered guidance. By introducing a visual context style guidance mechanism, this invention performs multi-level feature extraction and similarity comparison between the query image and the supporting image set, selecting high-quality context examples, and constructing semantic guidance signals using an unsupervised medical image registration algorithm. Based on this, a context-aware model is fused to generate intermediate masks, and multimodal visual cues are designed to enhance the semantic understanding capability of the SAM decoder. An attention bias matrix injection strategy is used to strengthen the model's focus on foreground regions and suppress background interference. Simultaneously, an adaptive mask fusion mechanism is introduced to dynamically balance context guidance and basic model prediction results. This method significantly improves the robustness and generalization ability of the unified medical image segmentation model in the face of complex domain differences without relying on target domain annotation data, achieving accurate segmentation of fine-grained medical structures. It not only effectively mitigates the misleading risk caused by inconsistent quality of supporting image sets but also enhances the model's recognition accuracy for small lesions and complex boundary regions, promoting the development of medical image segmentation towards a more efficient, intelligent, and adaptive direction. It is applicable to various clinical application scenarios such as medical image analysis, assisted diagnosis, and surgical navigation. Attached Figure Description

[0062] Figure 1 is a flowchart of the present invention;

[0063] Figure 2 is a flowchart of step 4 of the present invention, which involves constructing a semantic guidance signal using the optimal support image set and query image.

[0064] Figure 3 is a flowchart of step 5 of the present invention, which involves obtaining an intermediate mask and constructing a visual cue using the optimal support image set and query image. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] For the sake of clarity and reference, the technical terms, abbreviations, or acronyms used below are summarized and explained as follows:

[0067] SAM: A general image segmentation model proposed by Meta, capable of accurately segmenting any object in an image at the instance or semantic level based on prompts.

[0068] Query Image: The target medical image used for segmentation tasks.

[0069] Supporting image set: A collection of multiple labeled medical images used to provide contextual information.

[0070] Query image features: a multi-level representation of the query image extracted by a pre-trained encoder.

[0071] Supporting image features: Multi-level features extracted from the supporting image set by a pre-trained encoder.

[0072] Intermediate mask: The initial segmentation result generated based on the context segmentation model.

[0073] Visual cues: Information derived from the intermediate mask to guide the model to focus on the target region.

[0074] Semantic guidance signals: semantic prior information mapped onto the query image through image registration and deformation field mapping.

[0075] Segmentation mask: The pixel-level segmentation result output by SAM, used to identify target regions in an image.

[0076] Deformation field: A spatial transformation parameter that describes the pixel displacement relationship between the supporting image and the query image.

[0077] Spatial Transformation Network: A differentiable image transformation module used to align images by applying deformation fields.

[0078] Context-based segmentation models: Models that use contextual information to generate segmentation results, such as UniverSeg.

[0079] Morphological operations: Image processing techniques used to optimize mask edges and remove noisy areas.

[0080] Attention bias matrix: A bias term introduced in the attention mechanism to enhance the attention of the foreground region.

[0081] The primary objective of this invention is to propose a unified medical image segmentation method based on context-layered guidance.

[0082] This invention discloses a unified medical image segmentation method based on context-hierarchical guidance, comprising: an image input module, an image processing module, an image selection module, a semantic guidance construction module, a context-aware module, a segmentation mask generation module, and a mask fusion module. The image input module acquires a query image and a set of supporting images; the image processing module processes the image and generates image features; the image selection module selects the optimal set of supporting images; the semantic guidance construction module constructs a semantic guidance signal; the context-aware module generates an intermediate mask; the segmentation mask generation module generates a segmentation mask; and the mask fusion module fuses the intermediate mask and the segmentation mask to construct the final mask. This invention addresses the performance bottlenecks and insufficient adaptability of current unified medical image segmentation methods when facing the prevalent domain differences in medical images. It is applicable to medical segmentation and other applications requiring image segmentation.

[0083] Refer to Figure 1;

[0084] A unified medical image segmentation method based on context hierarchical guidance includes the following steps:

[0085] Step 1: Obtain the query image and supporting image set;

[0086] Step 2: Process the query image and the supporting image set to obtain the query image features and the supporting image feature set;

[0087] Step 3: Compare the query image features and the supporting image feature set to obtain the optimal supporting image set;

[0088] Step 4: Construct semantic guidance signals using the optimal supporting image set and query images;

[0089] Step 5: Use the optimal support image set and query image to obtain the intermediate mask and construct visual cues;

[0090] Step 6: Obtain the segmentation mask using visual cues and semantic guidance signals;

[0091] Step 7: Use the segmentation mask and the intermediate mask to merge them to obtain the final mask.

[0092] The following example illustrates the process of a unified medical image segmentation method based on context-layered guidance.

[0093] Step 1: Obtain the query image and supporting image set. Specifically, the query image can be acquired by a camera or other acquisition device and processed by an upstream algorithm to obtain the image input. The supporting image set is generally a collection of relevant past case photos, labeled by doctors and other professionals. The process of obtaining the query image and supporting image set described in Step 1 is collectively referred to as the image input module.

[0094] Step 2: Process the query image and the supporting image set to obtain the query image features and the supporting image feature set;

[0095] Specifically, the following algorithm flow is included:

[0096] Step 201: Using the query image obtained from the image input module, multi-level query image features are obtained through a pre-trained encoder;

[0097] Step 202: Using the support image set obtained from the image input module, a multi-level support image feature set is obtained through a pre-trained encoder;

[0098] The relevant calculations can be expressed as:

[0099]

[0100] in Let represent the i-th supporting image, q represent the query image, and l represent the feature layer number.

[0101] The process described in step 2, which involves processing the query image and the support image set to obtain multi-level query image features and multi-level support image feature sets, is collectively referred to as the image processing module.

[0102] The most valuable supporting image is selected by calculating the similarity between the query image and the supporting image set. The reason for using multi-level features is that the similarity calculated by using multi-level supporting image feature sets and multi-level query image features is more accurate.

[0103] Further, in step 3, the query image features and the supporting image feature set are compared to obtain the optimal supporting image set, including the following steps:

[0104] Step 301: Using the query image features and supporting image feature sets obtained by the image processing module, calculate the similarity scores of each feature in the multi-level query image features and the multi-level supporting image feature sets respectively.

[0105] Step 302: Based on the calculated similarity scores, select the highest score. These support images constitute the optimal support image set.

[0106] Specifically, step 301 also includes the following steps:

[0107] Step 3011, first calculate the similarity score:

[0108] ,

[0109] in This represents the score of the i-th supporting image. Indicates the weighting coefficient. This indicates the calculation of cosine similarity. It is a feature of the i-th supporting image. It is a query image feature;

[0110] Step 3012, then select the highest scorer. The optimal supporting image set is composed of 3 images. The process described in step 3, which selects the optimal supporting image set by calculating the similarity of multi-level features, is called the image selection module.

[0111] Refer to Figure 2;

[0112] Furthermore, in step 4, a semantic guidance signal is constructed using the optimal support image set and the query image, including the following steps:

[0113] Step 401: Using the optimal support image set obtained through the image selection module, further select the best support image from the optimal support image set;

[0114] Step 402: Using the optimal support image and the query image, and combining the unsupervised medical image registration algorithm (HyperMorph), the deformation field from the optimal support image to the query image is obtained;

[0115] Step 403: Using the mask of the optimal support image, combined with the deformation field and spatial transformation network, the semantic guidance signal is calculated.

[0116] Specifically, in step 401, the image with the highest similarity score to the query image in the best supporting image set is selected as the best supporting image. .

[0117] Specifically, in step 402, since the number of labeled samples, especially masked samples, is limited, this step utilizes an unsupervised medical image registration algorithm (HyperMorph) to match the best supporting image set selected in the previous stage. and the corresponding query image Using the HyperMorph algorithm Calculate the deformation field ,

[0118] ;

[0119] Specifically, in step 403, the deformation field is transformed using a spatial transformation network. Application to supporting image set mask ,get semantic guidance signal :

[0120] ;

[0121] This will serve as a semantic guidance signal, subsequently used for feature enhancement during the SAM mask decoder decoding process, thereby strengthening the model's ability to perceive the target to be segmented.

[0122] The process described in step 4, which obtains the semantic guidance signal through the optimal support image, query image, and medical image registration algorithm, is called the semantic guidance construction module.

[0123] Spatial Transformation Network It is a non-learning neural network module used for spatial transformations. Specifically, it employs a mesh generation and sampling process based on the deformation field. Mask for supporting image set Each pixel is remapped and interpolated to obtain the semantic guidance signal. This process can be described by the following formula:

[0124] ;

[0125] ;

[0126] in, Represents the deformation field Medium pixel The displacement; Represents pixels The neighboring pixels; This represents the spatial dimension index, traversing the horizontal x and vertical y directions in a two-dimensional image. and Representing pixels The coordinates of q in dimension d.

[0127] Refer to Figure 3;

[0128] Further, in step 5, an intermediate mask is obtained using the optimal support image set and the query image, and a visual cue is constructed, including the following steps:

[0129] Step 501: Obtain the optimal supporting image set and query image;

[0130] Step 502: Using the optimal support image set and query image, and combining the context segmentation model UniverSeg, an intermediate mask is generated. ;

[0131] Step 503: Using the intermediate mask, the edge of the mask is reduced by morphological operation erosion, and the mask visual cues are obtained by opening operation to reduce noise interference and remove unreliable areas.

[0132] Step 504: Using the intermediate mask, the minimum bounding box generation algorithm is used to generate the minimum bounding rectangle for the preprocessed mask as a box tooltip;

[0133] Step 505: Randomly sample 5 points in the intermediate mask as point prompts;

[0134] Step 506: The mask visual cues, box cues, and dot cues together constitute the visual cues.

[0135] Specifically, in step 501, the image input module described in step 1 obtains the optimal supporting image set and the query image.

[0136] Specifically, in step 502, the classic context segmentation model UniverSeg is selected as the intermediate mask generation model, and an intermediate mask is obtained through this model. This intermediate mask is an initial, and possibly coarse, segmentation prediction generated based on its context understanding. To generate more accurate and comprehensive visual cues, three types of visual cues (points, boxes, and masks) for SAM will be obtained from the intermediate mask.

[0137] Specifically, in step 503, to minimize the impact of noise, morphological operations are used to preprocess the intermediate mask. Specifically, erosion is used to shrink the mask edges, and opening operations are used to reduce noise interference and remove unreliable regions. The preprocessed intermediate mask will then be used directly as a visual cues.

[0138] Specifically, in step 504, the bounding box hints obtained from the mask are used to generate a minimum bounding rectangle for the preprocessed mask, which serves as another bounding box hint.

[0139] Specifically, in step 505, five points are randomly sampled from the set of points constituting the intermediate mask as point prompts.

[0140] Specifically, in step 506, the mask visual cue, the box cue, and the dot cue constitute a visual cue group.

[0141] Step 5 can be represented as follows:

[0142] ;

[0143] ;

[0144] ;

[0145] in As a context-aware model, it accepts a supporting image set. and query images And output the intermediate mask , For morphological operations, obtain the processed mask. MinBox and RandomSample are the minimum bounding box generation algorithm and the random sampling point algorithm, respectively. The set of visual cues to be constructed includes three types of visual cues: masks, boxes, and dots.

[0146] Specifically, in step 6, a segmentation mask is obtained using visual cues and semantic guidance signals, including the following steps:

[0147] Step 601: Convert the semantic guidance signal into an attention bias matrix;

[0148] Step 602: Add the attention bias matrix to the last layer of the SAM mask decoder;

[0149] Step 603: Using the query image and visual cues, output the segmentation mask through the modified SAM mask decoder. .

[0150] Specifically, in step 601, the semantic guidance signal is converted into an attention bias matrix, which includes the following steps:

[0151] First, the semantic guidance signal, i.e. the mask image, is downsampled to the same size as the feature and flattened into a vector;

[0152] Next, the bias matrix is ​​calculated using the following operations;

[0153] ;

[0154] The bias matrix, That is, the future. This is the background. Through this bias matrix, attention between foreground elements can be enhanced, attention between background elements can be suppressed, and attention between foreground and background elements is not processed.

[0155] Specifically, in step 602, the attention score calculation method of the SAM decoder is modified based on the obtained bias matrix:

[0156] ;

[0157] in For attention scores, Q and Let d represent the query vector Q and the key-value vector Q, respectively, for the attention mechanism. Dimensions This represents the transpose of the key-value vector matrix K. This modification allows SAM to focus more effectively on foreground features when the bounding box is inaccurate, thereby improving overall segmentation accuracy.

[0158] Specifically, in step 603, a segmentation mask is generated that benefits from SAM's fine-grained boundary delineation capabilities and is guided by semantic and visual cues. Finally, it is processed by the modified decoder. Obtain the segmentation mask :

[0159] ,

[0160] The process described above, from semantic guidance signals to segmentation mask generation, is called the segmentation mask generation module. The segmentation mask generation module is the core component connecting the semantic guidance signals and the SAM model.

[0161] Specifically, in step 7, the final mask is obtained by fusing the segmentation mask and the intermediate mask, including the following steps:

[0162] Step 701: Obtain the intermediate mask and the segmentation mask;

[0163] Step 702: Obtain the fusion mask using the intermediate mask and the segmentation mask;

[0164] ;

[0165] The middle mask is The segmentation mask is , merge mask is The integration process requires... To balance the effects of the intermediate mask and the segmentation mask, The fusion coefficient is used to quantitatively weigh the contribution weights of the intermediate mask and the segmentation mask in the final mask.

[0166] ;

[0167] in The confidence score for SAM decoding. For temperature coefficient, To support the number of image sets, These are the weighting coefficients.

[0168] The process described in step 7, which involves fusing the segmentation mask and the intermediate mask to obtain the final mask, is called the mask fusion module.

[0169] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0170] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] The technical features of the above embodiments can be combined arbitrarily. Furthermore, the numbering of each step is not intended to constrain the order of the steps; their order is permissible as long as there are no strict constraints on the sequence. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, any combination of these technical features that does not contradict each other should be considered within the scope of this specification.

[0172] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A unified medical image segmentation method based on context-hierarchical guidance, characterized in that: Includes the following steps: Step 1: Obtain the query image and support image set; Step 2: Process the query image and support image set to obtain query image features and support image feature sets; Step 3: Compare the query image features and support image feature sets to obtain the optimal support image set; Step 4: Construct a semantic guidance signal using the optimal support image set and the query image; Step 5: Obtain an intermediate mask and construct visual cues using the optimal support image set and the query image; Step 6: Obtain a segmentation mask using the visual cues and semantic guidance signal; Step 7: Fuse the segmentation mask and the intermediate mask to obtain the final mask; In Step 4, the optimal support image set and query image set are used... The process of constructing a semantic guidance signal includes the following steps: Step 401, selecting the best supporting image from the best supporting image set; Step 402, using the best supporting image and the query image, combined with an unsupervised medical image registration algorithm, to obtain the deformation field from the best supporting image to the query image; Step 403, using the mask of the best supporting image, combined with the deformation field and a spatial transformation network, to calculate the semantic guidance signal; wherein, Step 5, using the best supporting image set and the query image to obtain an intermediate mask and construct a visual cue, includes the following steps: Step 501, obtaining the best supporting image set through Step 3 and obtaining the query image through Step 1; Step 502, using... Using the optimal support image set and query image, an intermediate mask is generated by combining the context segmentation model UniverSeg; Step 503: Using the intermediate mask, the edges of the intermediate mask are reduced by morphological erosion, and noise interference and unreliable regions are reduced by opening operations to obtain a mask visual cue; Step 504: Using the intermediate mask, a minimum bounding box generation algorithm is used to generate a minimum bounding rectangle for the preprocessed intermediate mask as a box cue; Step 505: Five points are randomly sampled in the intermediate mask as point cue; Step 506: The mask visual cue, box cue, and point cue together constitute the visual cue; among which, Step 6 The process involves obtaining a segmentation mask using visual cues and semantic guidance signals, including the following steps: Step 601: Converting the semantic guidance signal into an attention bias matrix; Step 602: Adding the attention bias matrix to the last layer of the SAM mask decoder to obtain the modified SAM mask decoder; Step 603: Using the query image and visual cues, outputting the segmentation mask through the modified SAM mask decoder; Step 7, which fuses the segmentation mask and intermediate mask to obtain the final mask, includes the following steps: Step 701: Obtaining the intermediate mask and segmentation mask; Step 702: Obtaining the fused mask using the intermediate mask and segmentation mask. The middle mask is The segmentation mask is , merge mask is The integration process requires... To balance the effects of the intermediate mask and the segmentation mask, ,in The confidence score for SAM decoding. is the temperature coefficient, m is the number of supporting images, and k is the weighting coefficient.

2. The unified medical image segmentation method based on context-hierarchical guidance according to claim 1, characterized in that: Step 2 processes the query image and the support image set to obtain query image features and support image feature set, including the following steps: Step 201, using the query image, obtain multi-level query image features through a pre-trained encoder; Step 202, using the support image set, obtain multi-level support image feature set through a pre-trained encoder.

3. The unified medical image segmentation method based on context-hierarchical guidance according to claim 1, characterized in that: Step 3 compares the query image features and the supporting image feature set to obtain the optimal supporting image set, including the following steps: Step 301, calculate the similarity score of each feature in the multi-level query image features and the multi-level supporting image feature set respectively; Step 302, based on the calculated similarity scores, select the K supporting images with the highest scores to form the optimal supporting image set.

4. The unified medical image segmentation method based on context-hierarchical guidance according to claim 3, characterized in that: Step 301 calculates the similarity scores of each feature in the multi-level query image feature set and the multi-level support image feature set, including the following steps: Step 3011, first calculate the similarity score: ,in This represents the score of the i-th supporting image. Indicates the weighting coefficient. This indicates the calculation of cosine similarity. It is a feature of the i-th supporting image. The query image features are used; step 3012, then the top K images with the highest scores are selected to form the best supporting image set.

5. The unified medical image segmentation method based on context-hierarchical guidance according to claim 1, characterized in that: Step 601 converts the semantic guidance signal into an attention bias matrix, which includes the following steps: First, the semantic guidance signal, i.e., the mask image, is sampled until it is the same size as the feature image, and then flattened into a vector. Next, the bias matrix is ​​calculated using the following operations. ,in The bias coefficient, and For intermediate semantic vectors, and This is the index of the intermediate semantic vector; in the field of image segmentation, the region to be segmented is called the foreground, and the region in the query image other than the foreground is called the background. That is, the future. This is the background. Through this bias matrix, attention between foreground elements is enhanced, attention between background elements is suppressed, and attention between foreground and background elements is not processed.

Images