Improved few-sample medical image target region segmentation method

Abstract

The invention discloses an improved few-sample medical image target region segmentation method, belongs to the technical field of medical image segmentation, and solves the technical problems of few samples, insufficient intra-class diversity, insufficient inter-class difference learning, poor cross-domain data set segmentation performance and the like in medical image segmentation. The segmentation method comprises the steps of support set processing, prototype production and optimization, cross prototype learning and optimization and query set processing and segmentation, an independent prototype self-attention module and a cross prototype attention module are introduced, fine learning and optimization are carried out on a foreground prototype and a background prototype in a support set image, and a segmentation result is obtained. Therefore, the segmentation capability of the model on the complex medical image under the condition of few samples is enhanced. According to the iterative optimization mechanism, foreground and background prototypes are updated through multiple iterations, the segmentation precision and the robustness of the model are continuously improved, and the method is particularly suitable for processing medical images with high noise and complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to an improved few-sample medical image target area segmentation method. Background Art

[0002] Medical image segmentation is an important part of medical image processing and is widely used in computer-aided diagnosis, surgical planning, and treatment planning. Its goal is to automatically identify and segment specific anatomical structures in medical images, thereby providing doctors with accurate diagnostic information and treatment references. In recent years, with the rapid development of deep learning technology, medical image segmentation methods based on deep learning have gradually become mainstream and have achieved remarkable results on a variety of medical image datasets.

[0003] Early medical image segmentation methods were usually based on low-level features of images (such as grayscale, texture, shape, etc.). Due to the complexity and variability of medical images, these traditional methods often have less than ideal segmentation performance when faced with high-noise, low-contrast, and complex morphological medical images.

[0004] With the improvement of computer hardware performance, deep learning, especially convolutional neural networks (CNN), has shown powerful feature extraction and expression capabilities in the field of computer vision. Although deep learning technology has made significant progress in medical image segmentation, its limitations of relying on a large amount of labeled data are gradually emerging. The labeling process of medical image data is not only time-consuming and labor-intensive, but also requires the participation of a large number of professionals, which makes it difficult to obtain large-scale, high-quality labeled data.

[0005] In response to the above data scarcity problem, few-shot learning methods have been proposed and gradually become a research hotspot. Few-shot learning uses a small amount of labeled data or unlabeled data to train a model with strong generalization ability. By extracting foreground and background prototypes from a small number of support images, these prototypes are used to guide the segmentation of new images. For example, models such as PANet and SSL-ALPNet generate and optimize prototypes through different strategies, thereby improving the effect of few-shot segmentation. These models obtain the features of the support set images through feature extraction networks and generate foreground and background prototypes. These prototypes are then matched with the features of the query image to achieve accurate image segmentation.

[0006] Although few-shot learning methods have alleviated the problem of scarce labeled data to a certain extent, they still face many challenges in practical applications:

[0007] The features of the target area in medical images are usually not distinct enough, which makes the labeling process complicated and difficult. Especially under the condition of few samples, traditional supervised learning algorithms can hardly cope with this challenge effectively.

[0008] Medical image processing faces a serious data scarcity problem. It is difficult to obtain high-quality datasets with detailed pixel-level annotations, which restricts the performance improvement of existing segmentation algorithms.

[0009] Existing technologies are inadequate in dealing with the diversity within the target area and the differences between the target area and the background, resulting in segmentation accuracy that is difficult to meet clinical needs. This problem is particularly prominent when the distinction between foreground and background is complex.

[0010] The significant differences between cross-domain datasets also further increase the difficulty of information transfer between support images and query images, resulting in unsatisfactory segmentation results. Summary of the invention

[0011] In view of the shortcomings and deficiencies in the prior art, the present invention introduces few-sample learning and prototype attention mechanism to improve the segmentation accuracy and robustness of the target area, thereby achieving efficient and accurate segmentation of medical images, and solving the challenges faced by current technology to provide an improved few-sample medical image target area segmentation method.

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions: the improved few-sample medical image target area segmentation method provided by the present invention includes support set processing, prototype production and optimization, cross-prototype learning and optimization, and query set processing and segmentation, and introduces independent prototype self-attention modules and cross-prototype attention modules to perform refined learning and optimization of foreground prototypes and background prototypes in the support set image, thereby enhancing the model's ability to segment complex medical images under few-sample conditions.

[0013] Preferably, the specific steps include:

[0014] S1 support set feature extraction: using the support prototype generation module, the feature extractor extracts high-dimensional features from the support set image, and then generates the initial prototypes of the foreground and background of the support image through the prototype generator;

[0015] S2 Prototype generation and optimization: Use an independent prototype self-attention module to optimize the self-attention layer of the preliminary foreground and background prototypes of the support image generated in step S1. By capturing subtle differences within the prototype, the processing ability of intra-class diversity is improved to enhance its expressiveness, so that the prototype can better express the core features of the support set category;

[0016] S3 Cross-prototype learning and optimization: The cross-prototype attention module is used to learn the difference between the foreground and background of the support image. Through the multi-head attention mechanism, the foreground and background prototypes of the support image optimized in step S2 are cross-matched.

[0017] By exploring the relationship between the supporting foreground and background prototypes, learning the difference knowledge between the foreground prototype and the background prototype for modeling, learning the correlation features between them, and effectively transferring the features to the query foreground and background, continuously iterating and updating, and gradually optimizing the foreground and background feature prototypes in the query image;

[0018] S4 query set processing and segmentation: iterative segmentation of the query image, applying the optimized foreground prototype and background prototype in step S3 to the query image, matching the pixel features in the query image with the optimized foreground and background prototypes, and determining whether each pixel belongs to the foreground or background by calculating the similarity between each pixel and the prototype. The category with the highest similarity is selected as the label of the pixel, thereby generating the corresponding segmentation mask.

[0019] Preferably, in step S1, the support prototype generation module specifically extracts high-dimensional feature representation from the support set image through a feature extractor, and generates prototypes of the foreground and background through a prototype generator. The prototype represents the core features of a specific category in the support set and is the basis for subsequent module optimization and learning;

[0020] In generating the foreground prototype and background prototype set of supporting images and After that, the initial foreground and background prototypes of the query image are estimated. The specific steps include the following:

[0021] Calculate the average of the foreground prototype and background prototype of the support image and Similarity with all pixel features of the query image to construct the initial query foreground mask

[0022] This process is quantified by the cosine similarity S(a, b) = cos(a, b), using the following formula:

[0023] Further based on Use EM operation to process query feature F q , generate initial query prospect prototype and background prototype Among them, R represents the real number space, R C Belonging to a C-dimensional feature vector, it is used to describe the high-dimensional embedding representation of the feature; therefore, and It is the foreground and background feature representation of the query image. Each representation is encoded into a C-dimensional vector. This C is determined by the feature extraction network and is the number of feature channels output by the network. These vectors are used as input to interact with other modules in subsequent processing (attention calculation or update);

[0024] The entire initial estimation process can be summarized as follows:

[0025] Preferably, in step S2, the independent prototype self-attention module has a structure consisting of a self-attention layer and a residual connection. The self-attention layer is used to learn the relationship between the foreground and background prototypes, and analyze the relationship within the prototype through the self-attention mechanism. The self-attention layer calculates the relationship weight between each prototype and other prototypes, emphasizing the prototype features that contribute more to the diversity within the class. The residual connection is used to maintain the original feature information to ensure the stability of the model optimization process.

[0026] Preferably, query the prospect prototype Specifically, it is projected into different spaces through three different linear weight layers to generate and Both R N×C size; among them, Query matrix, used to query attention weights; is the Key matrix, and Used together to calculate attention weights; is the Value matrix, which contains the important information to be extracted. The attention map A calculated by cosine similarity describes and The relationship between:

[0027]

[0028] Through this attention map, the model is able to adaptively extract and learn Important information in:

[0029] Furthermore, a residual structure and layer normalization LN are added to optimize the output sequence:

[0030]

[0031] Updated Then, the feed-forward neural network FFNN is used, which contains two linear layers for dimensionality increase and decrease, and a GELU activation layer, while using residual connections and layer normalization:

[0032]

[0033] Preferably, in step S3, a cross-prototype attention module, the structure of which includes a multi-head attention mechanism, a position encoder and a decoder, wherein the multi-head attention mechanism is used to capture the complex relationship between the foreground and background prototypes, the position encoder retains the spatial position information, and the decoder is responsible for mapping the optimized features back to the query image;

[0034] The cross-prototype attention module explores the relationship between the supporting foreground and background prototypes, learns the correlation features between them, and effectively transfers these features to the query foreground and background to achieve the correction effect. The specific algorithm is as follows:

[0035]

[0036] Among them, A: attention score matrix, used to measure the similarity between query feature Pq and key feature Pk; P q : Query feature, representing the feature vector of the target image (query set); P k : key feature, representing the feature vector of the support set (annotated data set); C: the dimension of the feature vector, used for normalization to avoid excessive values ​​caused by the dot product of high-dimensional features; the above formula functions: P is calculated by dot product q Similarity with Pk and generate attention score;

[0037] A=A+Mask,

[0038] Among them, Mask: Mask is used to cover areas that do not need to be paid attention to; weight restrictions are imposed on certain pixels or areas to ignore their impact on segmentation; the above formula functions: adding a mask based on the original attention score to make the model more focused on specific foreground or background areas;

[0039] P v =LN(softmax(A)·P v +P v ),

[0040] Among them, P v : The updated feature vector, the value vector P through the attention score A v Weighted; softmax(A): normalize the attention score to ensure that the weight sum is 1; P v :To P v The feature weighting in , +P v : Retain the original features to ensure model stability; LN (Layer Normalization): Normalize the updated feature vector to ensure stable distribution and avoid gradient explosion or disappearance;

[0041] The above formula works as follows: By using the attention mechanism to weight and optimize the features, the original information is retained, and the layer normalization is combined to enhance the expressiveness of the features, so that the model can more accurately capture the difference between the foreground and the background. This optimization method can improve the robustness and accuracy of segmentation under complex backgrounds or few sample conditions;

[0042] P v =LN(FFNN(P v )+Pv ),

[0043] Among them, FFNN (Feed-Forward Neural Network): for feature P v Perform nonlinear transformation to enhance its expressive power; the above formula functions to further optimize P v , extract deeper features through nonlinear activation;

[0044] Next, the self-update process of the query foreground prototype is completed by applying the softmax function to the attention score A and combining it with the value V:

[0045]

[0046] Where Q: query feature matrix; K: key feature matrix; The above formula is used to recalculate the attention score matrix according to the above principle formula for multiple optimizations;

[0047] The model performs global average pooling on the updated query foreground-background features and calculates cosine similarity with the pixel features of the query image to obtain a new query foreground-background prototype:

[0048]

[0049] Among them, P f l+1 and P b l+1 : Optimized foreground prototype and background prototype; F q : Query image features; GAP (Global Average Pooling): Globally pool the foreground and background features, extract global features, and reduce local noise interference; QPG: Prototype generation function (Query Prototype Generation), combines the pooled foreground and background features with the query image features to generate a new prototype; The above formula works: Through the interaction of global pooling and query features, the foreground and background prototypes are updated to make the segmentation more accurate.

[0050] Preferably, in step S4, query set processing and segmentation, iterative segmentation of query image, this module consists of a similarity calculator and a segmentation mask generator, the similarity calculator is used to calculate the similarity between the query image pixels and the foreground and background prototypes, and the segmentation mask generator generates the final segmentation result according to the similarity;

[0051] The final query image segmentation module is responsible for applying the optimized prototypes to the query image and generating the final segmentation mask by matching pixels in the query image with the optimized foreground and background prototypes.

[0052] Preferably, a control system is also included, and the control signal of the control system is used to manage the data flow and processing sequence between the modules to ensure that the whole process proceeds in an orderly manner.

[0053] The present invention provides an improved target segmentation method for few-sample medical images. It has the following beneficial effects:

[0054] Based on the existing medical image segmentation technology, this invention solves multiple technical problems by introducing independent prototype self-attention modules and cross prototype attention modules, and significantly improves the accuracy and robustness of segmentation. The specific main advantages are as follows:

[0055] (1) Independent prototype self-attention module, which refines the foreground and background prototypes through the self-attention mechanism. This method effectively enhances the expressive power of the prototype, especially when the target area is complex in shape and the number of samples is limited, and can better capture the subtle features of the target area. Compared with the prior art, this processing method of the present invention effectively solves the problem of insufficient intra-class diversity processing, ensuring that the segmentation model can still accurately distinguish different categories of target areas under the condition of few samples. Therefore, when processing complex target areas, the segmentation accuracy of the present invention is significantly higher than that of traditional methods.

[0056] (2) The cross-prototype attention module further optimizes the segmentation effect by learning the differences between the foreground and background prototypes. In the prior art, the differences between the foreground and background are often difficult to fully capture, which leads to misclassification during the segmentation process. The present invention uses a multi-head attention mechanism to interactively match and optimize the foreground and background prototypes, thereby enhancing the segmentation ability of the model in complex backgrounds. It can be seen that the segmentation accuracy and robustness of the present invention are significantly better than traditional methods when processing complex backgrounds and cross-domain data sets, especially in cross-domain applications, it shows stronger generalization ability.

[0057] (3) The present invention introduces an iterative optimization mechanism. By iteratively updating the foreground and background prototypes for multiple times, the segmentation results can be gradually optimized. This iterative optimization not only improves the accuracy of segmentation, but also makes the segmentation process more stable and reliable. Compared with the single optimization method of the prior art, the iterative optimization mechanism of the present invention shows higher segmentation accuracy in long-term applications, especially when processing high-noise data or irregular target areas, the advantages of iterative optimization are more obvious.

[0058] (4) The present invention has made significant technical progress in solving the problems of insufficient intra-class diversity, insufficient capture of foreground-background differences, and poor cross-domain segmentation performance by introducing independent prototype self-attention modules, cross-prototype attention modules, and iterative optimization mechanisms. Compared with the prior art, the present invention has higher segmentation accuracy, stronger robustness, and wider application adaptability, especially under conditions of few samples and complex backgrounds, showing superior performance and practicality.

[0059] (5) In terms of technical effects, the segmentation effect of the present invention under few-sample and cross-domain conditions is significantly improved compared with the prior art. Experimental verification shows that the present invention significantly improves the accuracy of medical image segmentation when data is scarce, especially in scenes with complex internal morphology of the target area and significant differences between the background and foreground. The segmentation accuracy is improved by more than 10%, and better generalization ability is demonstrated on cross-domain data sets. The iterative optimization mechanism further improves the segmentation accuracy. Experimental data show that after multiple iterations, the segmentation accuracy is improved by 15% to 20% compared with the first iteration. These results demonstrate the practicality and superiority of the present invention in medical image segmentation, and provide strong technical support for the high-precision segmentation needs in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of the model framework of the present invention;

[0061] Figure 2 Schematic diagram of the cross-prototype attention module;

[0062] Figure 3 Mask visualization data graph for cross-prototype attention;

[0063] Figure 4 Visualization data graph for SegRap target segmentation;

[0064] Figure 5 Visualization data diagram for HECKTOR target segmentation;

[0065] Figure 6 Model performance data graphs for different numbers of labeled datasets;

[0066] Figure 7 The impact of the number of prototypes on the data plot;

[0067] Figure 8 Iterative segmentation visualization of data graph;

[0068] Fig. 9 The impact of the number of independent prototype self-attention modules stacked;

[0069] Fig.10 The effect of the number of cross-prototype attention modules stacked. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0071] Example 1

[0072] See also Figure 1-3 The improved few-sample medical image target segmentation method provided by the present invention includes support set processing, prototype optimization, cross-prototype learning, and query set processing and segmentation. Through the close combination of these steps, high-precision segmentation of medical images is finally achieved. By introducing a more sophisticated prototype generation and optimization mechanism, the purpose is to further improve the accuracy and robustness of medical image segmentation, especially under few-sample and cross-domain conditions, to achieve more efficient segmentation.

[0073] Although advances in existing deep learning technologies have significantly improved the segmentation of medical images based on large-scale annotated data, these methods are highly dependent on a large amount of annotated data, which is a challenge in medical images because annotated data is usually difficult to obtain, resulting in poor segmentation results of existing methods under few-sample conditions. Existing few-sample learning methods are insufficient in dealing with diversity within the target area and differences between foreground and background, which affects the accuracy and robustness of segmentation. The purpose is to improve the segmentation performance under few-sample conditions, especially under complex backgrounds and cross-domain data sets, to improve the accuracy and robustness of segmentation. Through the close combination of these steps, the present invention achieves high-precision segmentation of medical images under few-sample conditions, overcomes the problems of existing methods in dealing with intra-class diversity, insufficient learning of inter-class differences, and limited cross-domain segmentation performance, and demonstrates high practical value and technical advantages, which are described in detail as follows.

[0074] like Figure 1 As shown, the improved few-sample medical image target area segmentation method of the present invention, the overall structure and workflow of the model, consists of multiple key modules such as support set feature extraction, prototype generation and optimization, foreground and background difference learning, and iterative segmentation of query images. Each module plays a different function in the overall solution and works together to achieve high-precision medical image segmentation. An independent prototype self-attention module and a cross-prototype attention module are introduced to perform refined learning and optimization of the foreground and background prototypes in the support set image, thereby enhancing the model's ability to segment complex medical images under few-sample conditions.

[0075] S1. The support prototype generation module extracts high-dimensional feature representations from the support set images through a feature extractor, and then generates initial prototypes of the foreground and background through a prototype generator.

[0076] S2. The independent prototype self-attention module further optimizes the foreground and background prototypes generated above, improves the processing capability of intra-class diversity, and enables the prototype to better express the core features that support the concentrated category. Specifically: the self-attention mechanism is used to analyze the relationship within the prototype. The self-attention layer calculates the relationship weight between each prototype and other prototypes, emphasizing the prototype features that contribute more to the intra-class diversity. The residual connection is used to maintain the original information, ensure the stability of the optimization process, and enhance the expressive power of the prototype.

[0077] S3. Figure 2 As shown in the figure, the cross-prototype attention module is responsible for optimizing the differences between the foreground and background prototypes, and improving the segmentation effect of the model on complex backgrounds and cross-domain data. Specifically, through the multi-head attention mechanism, the foreground prototype and the background prototype are cross-matched to learn the differences between them; this process is continuously iterated and updated to gradually optimize the foreground and background features in the query image, and finally generate a more accurate segmentation result.

[0078] S4. Final query image segmentation module: Use the optimized prototype to segment the query image to generate the final foreground and background segmentation masks. Specifically, the optimized prototype is matched with the pixels in the query image through similarity calculation, the category of each pixel is determined, and the corresponding segmentation mask is generated.

[0079] The improved few-sample medical image target segmentation method of the present invention has the following working principles:

[0080] Support set processing is the first step of the entire technical solution. The core of the support prototype generation module is to extract high-dimensional feature representations from the support set image through a feature extractor, and then generate initial prototypes of the foreground and background through a prototype generator. The above prototypes represent the core features of a specific category in the support set. Subsequently, the extracted features are processed through methods such as clustering or average pooling to generate foreground and background prototypes. These prototypes are simplified expressions of specific category features in the support set, which play a role in summarizing and condensing category features, laying the foundation for subsequent optimization and learning processes. In this way, the support prototype generation module preliminarily solves the problem of how to effectively represent category features under few sample conditions.

[0081] In the prototype optimization step, the generated preliminary foreground and background prototypes are input into the independent prototype self-attention module for optimization. The independent prototype self-attention module analyzes the relationship within the prototype through the self-attention mechanism. The self-attention layer calculates the relationship weight between each prototype and other prototypes, and highlights those prototype features that contribute more to the intra-class diversity. The residual connection is used to maintain the original information and ensure the stability of the optimization process. Through this mechanism, the independent prototype self-attention module effectively enhances the expressiveness of the prototype and solves the problem of insufficient processing of intra-class diversity in the existing technology.

[0082] like Figure 2 The schematic diagram of the cross-prototype attention module is shown. Cross-prototype learning is a key innovation of the present invention. The structure and working principle of the cross-prototype attention module. Through the multi-head attention mechanism. After being optimized by the independent prototype self-attention module, the foreground and background prototypes are input into the cross-prototype attention module for further learning and optimization. The cross-prototype attention module cross-matches the foreground prototype and the background prototype through the multi-head attention mechanism, captures the complex relationship between the foreground and background prototypes, and learns the differences between them. The cross-prototype attention module gradually optimizes the foreground and background prototypes in the query image by modeling the difference knowledge between the foreground and the background. In this process, the prototypes are continuously updated through iterations and interact with each other, and the optimization effect is gradually accumulated. The foreground and background features in the query image are gradually optimized, and finally an optimized prototype that can accurately reflect the difference between the foreground and the background is formed, thereby greatly improving the segmentation accuracy and finally generating a more accurate segmentation result. By learning the difference knowledge between the foreground and the background and paying attention to the correlation between the features, the model can more accurately divide the foreground and the background. The cross-prototype attention module improves the accuracy of segmentation, especially the segmentation effect under complex backgrounds has been significantly improved. Query set processing and segmentation are the last step of the technical solution and the key steps to achieve the ultimate goal. The pixel features of the query image are matched with the optimized foreground and background prototypes. By calculating the similarity between each pixel and the prototype, it is determined whether the pixel belongs to the foreground or the background. The category with the highest similarity is selected as the label of the pixel, and the corresponding segmentation mask is generated. The spatial position information is introduced to ensure that the model can consider the influence of different positions during the relationship learning process between the foreground and background. The role of the decoder is to map the optimized features back to the query image for the final segmentation operation.

[0083] In order to further improve the accuracy of segmentation, the system will perform multiple iterative updates. In each iteration, the optimized prototype will be reapplied to the query image, thereby gradually improving the accuracy of segmentation. Through this iterative update mechanism, the segmentation results of the query image can be continuously optimized, and finally achieve high-precision medical image segmentation, especially under conditions of few samples and complex backgrounds.

[0084] The improved few-sample medical image target segmentation method of the present invention, in the process of data flow, the data starts from the support set image input, firstly extracts high-dimensional features through the feature extractor, and generates foreground and background prototypes. The initially generated prototype flows to the independent prototype self-attention module, and after being optimized by the self-attention layer, it continues to flow to the cross-prototype attention module. In the cross-prototype attention module, the foreground and background prototypes are interactively learned through the multi-head attention mechanism, and the optimized prototypes are passed to the query image segmentation module. Finally, the pixels of the query image are matched with the optimized prototype, a segmentation mask is generated, and the segmentation result is output. The control signal of the system is mainly used to manage the data flow and processing order between the modules to ensure that the whole process is carried out in an orderly manner.

[0085] Example 2

[0086] like Figure 1-Figure 2 As shown, the improved few-sample medical image target area segmentation method of the present invention specifically includes the following steps:

[0087] S1 Support set feature extraction: The support prototype generation module is used to extract high-dimensional features from the support set image through the feature extractor, and then the initial prototype set of the foreground and background of the support image is generated through the prototype generator; next, the initial foreground and background prototypes of the query image are estimated.

[0088] First, the support prototype generation module extracts high-dimensional feature representations from the support set images through the feature extractor, and generates prototypes of the foreground and background through the prototype generator. The prototype represents the core features of a specific category in the support set and is the basis for subsequent module optimization and learning. In this way, the support prototype generation module preliminarily solves the problem of how to effectively represent category features under few-sample conditions.

[0089] A collection of foreground and background prototypes for generating support images and After that, the initial foreground and background prototypes of the query image are estimated. The specific steps include: calculating the average value of the supporting prototypes and Similarity with all pixel features of the query image to construct the initial query foreground mask

[0090] This process is quantified by the cosine similarity S(a, b) = cos(a, b), using the following formula:

[0091] Further based on Use EM operation to process query feature F q , generate initial query prospect prototype and background prototype in Among them, R represents the real number space, R C Belonging to a C-dimensional feature vector, it is used to describe the high-dimensional embedding representation of the feature; therefore, and It is the foreground and background feature representation of the query image. Each representation is encoded into a C-dimensional vector. This C is determined by the feature extraction network and is the number of feature channels output by the network. These vectors are used as input to interact with other modules in subsequent processing (attention calculation or update);

[0092] The entire initial estimation process can be summarized as:

[0093] S2 Prototype generation and optimization: Using an independent prototype self-attention module, after generating preliminary foreground and background prototypes of the supporting image in step S1, the prototype is optimized at the self-attention layer to capture subtle differences within the prototype and improve the ability to handle intra-class diversity to enhance its expressiveness.

[0094] The independent prototype self-attention module consists of a self-attention layer and a residual connection. The self-attention layer is used to learn the relationship between prototypes and analyze the internal relationship of the prototype through the self-attention mechanism. The self-attention layer calculates the relationship weight between each prototype and other prototypes, emphasizing the prototype features that contribute more to the diversity within the class. The residual connection is used to maintain the original feature information to ensure the stability of the model optimization process.

[0095] Query Prospect Prototype The expression is as follows:

[0096] It is projected into different spaces through three different linear weight layers to generate and Both size; among them, Query matrix, used to query attention weights; is the Key matrix, and Used together to calculate attention weights; is the Value matrix, which contains the important information to be extracted. The attention map A calculated by cosine similarity describes and The relationship between:

[0097]

[0098] Through this attention map, the model is able to adaptively extract and learn Important information in.

[0099] Furthermore, a residual structure and layer normalization LN are added to optimize the output sequence:

[0100] Updated Then, the feed-forward neural network FFNN is used, which contains two linear layers for dimensionality increase and decrease, and a GELU activation layer, while using residual connections and layer normalization:

[0101]

[0102] S3 Cross-prototype learning and optimization: The cross-prototype attention module is used to learn the differences between the foreground and background of the support image. Through the multi-head attention mechanism, the foreground and background prototypes of the support image optimized in step S2 are cross-matched. By exploring the relationship between the support foreground and background prototypes, the difference knowledge between the foreground prototype and the background prototype is learned for modeling, the correlation features between them are learned, and the features are effectively transferred to the query foreground and background. It is continuously iterated and updated, and the foreground and background feature prototypes in the query image are gradually optimized to achieve the correction effect. The optimized prototypes are passed to the query image segmentation module to finally generate more accurate segmentation results.

[0103] The cross-prototype attention module significantly improves the segmentation accuracy by enhancing the ability to distinguish between foreground and background; the function of the cross-prototype attention module is to further optimize the foreground and background prototypes in the query image by learning the differences between the foreground and background prototypes; the cross-prototype attention module significantly improves the segmentation accuracy by enhancing the ability to distinguish between foreground and background. The structure of this module includes a multi-head attention mechanism, a position encoder and a decoder. The multi-head attention mechanism is used to capture the complex relationship between the foreground and background prototypes, the position encoder retains the spatial position information, and the decoder is responsible for mapping the optimized features back to the query image.

[0104] The cross-prototype attention module explores the relationship between the supporting foreground and background prototypes, learns the correlation features between them, and effectively transfers these features to the query foreground and background to achieve the correction effect.

[0105]

[0106] Where A: attention score matrix, used to measure the query feature P q and key feature P k similarity; P q : Query feature, representing the feature vector of the target image (query set); P k : key feature, representing the feature vector of the support set (annotated data set); C: the dimension of the feature vector, used for normalization to avoid excessive values ​​caused by the dot product of high-dimensional features; the above formula functions: P is calculated by dot productq Similarity with Pk and generate attention score;

[0107] A=A+Mask,

[0108] Among them, Mask: Mask is used to cover areas that do not need to be paid attention to; weight restrictions are imposed on certain pixels or areas to ignore their impact on segmentation; the above formula functions: adding a mask based on the original attention score to make the model more focused on specific foreground or background areas;

[0109] P v =LN(softmax(A)·P v +P v ),

[0110] Among them, P v : The updated feature vector, the value vector P through the attention score A v Weighted; softmax(A): normalize the attention score to ensure that the weight sum is 1; P v :To P v The feature weighting in , +P v : Retain the original features to ensure model stability; LN layer normalization: Normalize the updated feature vector to ensure stable distribution and avoid gradient explosion or disappearance;

[0111] The above formula works as follows: By using the attention mechanism to weight and optimize the features, the original information is retained, and the layer normalization is combined to enhance the expressiveness of the features, so that the model can more accurately capture the difference between the foreground and the background. This optimization method can improve the robustness and accuracy of segmentation under complex backgrounds or few sample conditions;

[0112] P v =LN(FFNN(P v )+P v ),

[0113] Among them, FFNN feedforward neural network: for feature P V Perform nonlinear transformation to enhance its expressive power; the above formula functions to further optimize P V , extract deeper features through nonlinear activation;

[0114] Next, the self-update process of the query foreground prototype is completed by applying the softmax function to the attention score A and combining it with the value V:

[0115]

[0116] Where Q: query feature matrix; K: key feature matrix; The above formula is used to recalculate the attention score matrix according to the above principle formula for multiple optimizations;

[0117] The model performs global average pooling on the updated query foreground-background features and calculates cosine similarity with the pixel features of the query image to obtain a new query foreground-background prototype:

[0118]

[0119] Among them, P f l+1 and P b 1+1 : Optimized foreground prototype and background prototype; F q : Query image features; GAP global average pooling: Globally pool the foreground and background features, extract global features, and reduce local noise interference; QPG: Prototype generation function, combine the pooled foreground and background features with the query image features to generate a new prototype; The above formula functions: Through the interaction of global pooling and query features, the foreground and background prototypes are updated to make the segmentation more accurate.

[0120] Finally, if Figure 3 The Mask visualization diagram in the cross-prototype attention module shows that the Mask generated in the cross-prototype attention module is used to distinguish the foreground and background areas. The mask intuitively reflects the segmentation effect of the model on the foreground and background in the image, that is, which pixels belong to the foreground and which belong to the background. Through visualization, we can observe the model's ability to accurately segment complex areas, indicating the efficiency and accuracy of the cross-prototype attention module in distinguishing the foreground and background.

[0121] S4 Query Set Processing and Segmentation: Iterative segmentation of query images. This module consists of a similarity calculator and a segmentation mask generator. The similarity calculator is used to calculate the similarity between the query image pixels and the foreground and background prototypes, while the segmentation mask generator generates the final segmentation result based on the similarity.

[0122] The optimized foreground prototype and background prototype in step S3 are applied to the query image, and the pixel features in the query image are matched with the optimized foreground and background prototypes. By calculating the similarity between each pixel and the prototype, it is determined whether each pixel belongs to the foreground or the background. The category with the highest similarity is selected as the label of the pixel, thereby generating the corresponding segmentation mask, generating the final segmentation mask, and outputting the segmentation result.

[0123] Application Examples

[0124] The improved few-sample medical image target segmentation method of the present invention is applied to specific few-sample medical image target segmentation, SegRap target segmentation, and HECKTOR target segmentation. The specific results are as follows:

[0125]

[0126] Table 1 Comparative experimental results of SegRap dataset as query set

[0127] result Figure 4 The SegRap target segmentation visualization data diagram clearly shows the target segmentation effect of the model on the SegRap dataset. From the visualization results, we can see that the model accurately segments the target area and successfully identifies the boundaries of the target area in complex images. Under the condition of few-sample learning, the model can still achieve high segmentation accuracy, reflecting the strong adaptability of the model in data-scarce scenarios.

[0128] The comparative experimental results of the SegRap dataset in Table 1 as the query set further prove that the comparative experimental results of the present model and other segmentation methods on the SegRap dataset are mainly evaluated from the indicators such as segmentation accuracy and robustness.

[0129]

[0130] The model outperforms traditional methods in all indicators, especially showing excellent segmentation performance under the condition of few samples. This shows that the independent prototype self-attention module and cross-prototype attention module of the model effectively improve the segmentation accuracy of complex target areas, and can maintain high accuracy and generalization ability when the sample data is small.

[0131] Table 2 Comparative experimental results using the HECKTOR2022 dataset as a query set

[0132] Combination Figure 5 From the HECKTOR target segmentation visualization data graph shown, we can see that the segmentation performance of the model on the HECKTOR2022 dataset is similar to the results of the SegRap dataset. This visualization also verifies the generalization performance of the model. Figure 5 The results in show that the model can adapt to the target segmentation tasks in different datasets, which shows the robustness of the model in cross-domain segmentation tasks. The model performs well in identifying image boundaries and segmentation details, can accurately segment the target area, and avoid misjudging the background as the foreground.

[0133] Combined with the comparative experimental results of the HECKTOR2022 dataset as the query set in Table 2, it can be seen that the segmentation effect of the model of the segmentation method of the present invention is compared with other methods on the HECKTOR2022 dataset. The experimental results show that the model of the segmentation method of the present invention is significantly better than the comparison method in multiple evaluation indicators, especially in the case of complex backgrounds and diverse samples, showing higher segmentation accuracy and robustness. This shows that the multi-level attention mechanism of the model adapts to the differences in cross-domain datasets, helps to improve the generalization ability of the model, enables it to handle feature differences between different datasets, and enhances the practical application value of medical image segmentation.

[0134] Table 3 Effect of prototype position order

[0135]

[0136] * This indicates that there is a significant difference between the labeled results and the model results in the previous row at the 0.05 significance level.

[0137] From the data on the influence of the prototype position order in Table 3, it can be obtained that: This table analyzes the influence of the position order of the foreground and background prototypes on the model segmentation effect. The results show that although the adjustment of the position order in the segmentation method of the present invention has limited influence on the segmentation effect, the proper optimization of the prototype position order can improve the segmentation accuracy to a certain extent, especially in fine-grained segmentation tasks, it can improve the recognition accuracy of specific areas. This table shows the flexibility of the model of the segmentation method of the present invention in processing different samples and target area features, so that the optimization of the prototype position order in the segmentation process slightly improves the actual effect.

[0138] Depend on Figure 6 The model performance data graph under different numbers of labeled data sets shown in the figure can be obtained: the segmentation performance of the model under different numbers of labeled data conditions. As the number of labeled data increases, the segmentation accuracy of the model gradually improves, indicating that the model can effectively use more labeled data for optimization. However, even when the amount of labeled data is small, the model still shows a high segmentation accuracy, reflecting its advantage in few-sample learning. Overall, Figure 6 The consistency and robustness of the segmentation effect of the model under the conditions of scarce and sufficient labeled data are verified.

[0139] Depend on Figure 7The influence diagram of the number of prototypes shown in the figure can be used to obtain the influence of the number of foreground and background prototypes on the model performance in the segmentation method of the present invention. The experimental results show that appropriately increasing the number of prototypes can enhance the model's ability to express different foreground and background features, thereby improving the segmentation accuracy. However, when the number of prototypes is too large, the computational cost of the model will increase significantly, and even a slight decrease in segmentation accuracy will occur, indicating that the number of prototypes should strike a balance between accuracy and efficiency to optimize the segmentation effect.

[0140] Depend on Figure 8 The segmentation visualization shown in the figure shows the segmentation effect after multiple iterations of optimization, which shows the gradual improvement of the model in the iterative update. As the number of iterations increases, the segmentation results of the model become more accurate and the segmentation boundaries become clearer. This iterative optimization mechanism allows the model to improve the segmentation accuracy in the gradual correction, especially in complex backgrounds. The segmentation effect can be continuously improved through iteration, making the segmentation mask more accurate and stable.

[0141] Depend on Fig. 9 The data shown in the figure shows the impact of the number of independent prototype self-attention module stacks on the segmentation performance of the model. Increasing the number of stacked layers of this module can improve the model's ability to capture foreground and background features, thereby enhancing the accuracy of segmentation. However, when the number of stacked layers reaches a certain number, the performance improvement gradually slows down, or even decreases slightly, indicating that too much stacking will increase the complexity of the model and reduce the benefits. Therefore, when designing the model, it is necessary to reasonably set the number of stacked layers to achieve a balance between segmentation accuracy and computational cost.

[0142] Depend on Fig.10 The impact of the number of stacked cross-prototype attention modules shown in the data graph shows the impact of the number of stacked layers of the cross-prototype attention module on the segmentation performance of the model. Increasing the number of stacked layers of this module can significantly improve the distinction between foreground and background, especially in complex backgrounds. The multi-layer attention mechanism can better capture feature differences and improve segmentation accuracy. However, too many stacked layers will also lead to excessive computational overhead and may cause saturation or even degradation of model performance. Therefore, it is necessary to optimize the number of layers in practical applications to obtain the best segmentation effect.

[0143] The above data detection results show that the segmentation method of the present invention has the following advantages in terms of technical effects: the improved few-sample medical image target segmentation method of the present invention has significantly improved segmentation effects under few-sample and cross-domain conditions compared with the prior art. The present invention significantly improves the accuracy of medical image segmentation when data is scarce, especially in scenes with complex internal morphology of the target area and significant differences between the background and foreground. The segmentation accuracy is improved by more than 10%, and better generalization ability is demonstrated on cross-domain data sets. The iterative optimization mechanism further improves the segmentation accuracy. Experimental data show that after multiple iterations, the segmentation accuracy is improved by 15% to 20% compared with the first iteration. These results demonstrate the practicality and superiority of the present invention in medical image segmentation, and provide strong technical support for the high-precision segmentation needs in practical applications.

[0144] In the improved few-sample medical image target segmentation method of the present invention, support set processing is the first step of the entire technical solution. The core task of this step is to extract effective features from the support set image and generate preliminary prototypes of the foreground and background. Specifically, the support set image first passes through a feature extractor to extract high-dimensional deep features. These features represent the main feature information of each category in the support set. Subsequently, the extracted features are processed by methods such as clustering or average pooling to generate foreground and background prototypes. These prototypes are simplified expressions of specific category features in the support set, which play a role in summarizing and condensing category features, laying the foundation for subsequent optimization and learning processes.

[0145] In the prototype optimization step, the generated preliminary foreground and background prototypes are input into the independent prototype self-attention module for optimization. This module refines the foreground and background prototypes through the self-attention mechanism. Specifically, the self-attention mechanism enhances the expressiveness of the prototypes by calculating the relationship weights between each prototype and other prototypes, highlighting those prototype features that contribute more to the diversity within the class. In addition, the residual connection is used to retain the original feature information to ensure the stability of the optimization process. Through this optimization, the generated foreground and background prototypes can better express the diversity of intra-class features, thereby improving the accuracy of subsequent segmentation.

[0146] Cross-prototype learning is a key innovation of the present invention. After being optimized by the independent prototype self-attention module, the foreground and background prototypes are input into the cross-prototype attention module for further learning and optimization. The working principle of this module is to cross-match the foreground prototype and the background prototype through a multi-head attention mechanism to learn the differences between them. The cross-prototype attention module gradually optimizes the foreground and background prototypes in the query image by modeling the knowledge of the differences between the foreground and the background. In this process, the prototypes continuously interact with each other through iterative updates, and the optimization effect gradually accumulates, eventually forming an optimized prototype that can accurately reflect the difference between the foreground and the background, thereby greatly improving the segmentation accuracy, especially under complex background conditions.

[0147] Query set processing and segmentation is the last step of the technical solution and a key step to achieve the ultimate goal. The optimized foreground and background prototypes are used for the segmentation of the query set image. The specific operation is to match the pixel features of the query image with the optimized foreground and background prototypes, and determine whether the pixel belongs to the foreground or background by calculating the similarity between each pixel and the prototype. The category with the highest similarity is selected as the label of the pixel, and the corresponding segmentation mask is generated. In order to further improve the accuracy of the segmentation, the system will perform multiple iterative updates. In each iteration, the optimized prototype will be reapplied to the query image, thereby gradually improving the accuracy of the segmentation. Through this iterative update mechanism, the segmentation results of the query image can be continuously optimized, and ultimately high-precision medical image segmentation can be achieved.

[0148] In summary, the improved few-sample medical image target segmentation method of the present invention includes support set processing, prototype optimization, cross-prototype learning, and query set processing and segmentation. Through the close combination and synergy of these steps, high-precision segmentation of medical images is finally achieved. On the basis of existing medical image segmentation technology, by introducing independent prototype self-attention modules and cross-prototype attention modules, multiple technical problems are solved, and the accuracy and robustness of segmentation are significantly improved, especially under few-sample and cross-domain conditions, more efficient segmentation is achieved, which has significant practical value and technical advantages.

[0149] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An improved few-sample medical image target segmentation method, characterized in that: It includes support set processing, prototype production and optimization, cross-prototype learning and optimization, and query set processing and segmentation. It introduces independent prototype self-attention module and cross-prototype attention module to perform refined learning and optimization on the foreground prototype and background prototype in the support set image, thus enhancing the model's ability to segment complex medical images under few-sample conditions.

2. The improved few-sample medical image target segmentation method according to claim 1, characterized in that: The specific steps are as follows: S1 support set feature extraction: using the support prototype generation module, the feature extractor extracts high-dimensional features from the support set image, and then generates the initial prototypes of the foreground and background of the support image through the prototype generator; S2 Prototype generation and optimization: Use an independent prototype self-attention module to optimize the self-attention layer of the preliminary foreground and background prototypes of the support image generated in step S1. By capturing subtle differences within the prototype, the processing ability of intra-class diversity is improved to enhance its expressiveness, so that the prototype can better express the core features of the support set category; S3 Cross-prototype learning and optimization: The cross-prototype attention module is used to learn the differences between the foreground and background of the support image. Through the multi-head attention mechanism, the foreground and background prototypes of the support image optimized in step S2 are cross-matched; By exploring the relationship between the supporting foreground and background prototypes, learning the difference knowledge between the foreground prototype and the background prototype for modeling, learning the correlation features between them, and effectively transferring the features to the query foreground and background, continuously iterating and updating, and gradually optimizing the foreground and background feature prototypes in the query image; S4 query set processing and segmentation: iterative segmentation of the query image, applying the optimized foreground prototype and background prototype in step S3 to the query image, matching the pixel features in the query image with the optimized foreground and background prototypes, and determining whether each pixel belongs to the foreground or background by calculating the similarity between each pixel and the prototype. The category with the highest similarity is selected as the label of the pixel, thereby generating the corresponding foreground and background segmentation masks.

3. The improved few-sample medical image target segmentation method according to claim 2, characterized in that: In step S1, the support prototype generation module specifically extracts high-dimensional feature representation from the support set image through a feature extractor, and generates prototypes of the foreground and background through a prototype generator. The prototype represents the core features of a specific category in the support set and is the basis for subsequent module optimization and learning; In generating the foreground prototype and background prototype set of supporting images and After that, the initial foreground and background prototypes of the query image are estimated. The specific steps include the following: Calculate the average of the foreground prototype and background prototype of the support image and Similarity with all pixel features of the query image to construct the initial query foreground mask This process is quantified by the cosine similarity S(a, b) = cos(a, b), using the following formula: Further based on Use EM operation to process query feature F q , generate initial query prospect prototype and background prototype in Among them, R represents the real number space, R C Belonging to a C-dimensional feature vector, it is used to describe the high-dimensional embedding representation of the feature; therefore, and It is the foreground and background feature representation of the query image. Each representation is encoded into a C-dimensional vector. This C is determined by the feature extraction network and is the number of feature channels output by the network. These vectors are used as input to interact with other modules in subsequent processing (attention calculation or update); The entire initial estimation process can be summarized as follows:

4. The improved few-sample medical image target segmentation method according to claim 2, characterized in that: In step S2, the independent prototype self-attention module consists of a self-attention layer and a residual connection. The self-attention layer is used to learn the relationship between the foreground and background prototypes. The relationship within the prototype is analyzed through the self-attention mechanism. The self-attention layer calculates the relationship weight between each prototype and other prototypes, emphasizing the prototype features that contribute more to the diversity within the class. The residual connection is used to maintain the original feature information to ensure the stability of the model optimization process.

5. The improved few-sample medical image target segmentation method according to claim 4, characterized in that: Query Prospect Prototype Specifically, it is projected into different spaces through three different linear weight layers to generate and Both R N×C size; among them, Query matrix, used to query attention weights; is the Key matrix, and Used together to calculate attention weights; is the Value matrix, which contains the important information to be extracted. The attention map A calculated by cosine similarity describes and The relationship between: Through this attention map, the model is able to adaptively extract and learn Important information in: Furthermore, a residual structure and layer normalization LN are added to optimize the output sequence: Updated Then, the feed-forward neural network FFNN is used, which contains two linear layers for dimensionality increase and decrease, and a GELU activation layer, while using residual connections and layer normalization:

6. The improved few-sample medical image target segmentation method according to claim 2, characterized in that: In step S3, a cross-prototype attention module, the structure of which includes a multi-head attention mechanism, a position encoder and a decoder, wherein the multi-head attention mechanism is used to capture the complex relationship between the foreground and background prototypes, the position encoder retains the spatial position information, and the decoder is responsible for mapping the optimized features back to the query image; The cross-prototype attention module explores the relationship between the supporting foreground and background prototypes, learns the correlation features between them, and effectively transfers these features to the query foreground and background to achieve the correction effect. The specific algorithm is as follows: Where A: attention score matrix, used to measure the query feature P q and key feature P k similarity; P q : query feature, representing the feature vector of the target image (query set); Pk: key feature, representing the feature vector of the support set (annotated dataset); C: dimension of the feature vector, used for normalization to avoid excessive values ​​caused by the dot product of high-dimensional features; the above formula functions: calculate the similarity of Pq and Pk through the dot product and generate an attention score; A=A+Mask, Among them, Mask: Mask is used to cover areas that do not need to be paid attention to; weight restrictions are imposed on certain pixels or areas to ignore their impact on segmentation; the above formula functions: adding a mask based on the original attention score to make the model more focused on specific foreground or background areas; P v =LN(softmax(A)·P v +P v ), Among them, P v : The updated feature vector, the value vector P through the attention score A v Weighted; softmax(A): normalize the attention score to ensure that the weight sum is 1; P v :To P v The feature weighting in , +P v : Retain the original features to ensure model stability; LN: Layer normalization, normalizes the updated feature vector to ensure stable distribution and avoid gradient explosion or disappearance; The above formula works as follows: the features are weighted and optimized through the attention mechanism while retaining the original information. The layer normalization is combined to enhance the expressiveness of the features, so that the model can more accurately capture the difference between the foreground and the background. This optimization method can improve the robustness and accuracy of segmentation under complex backgrounds or few sample conditions. P v =LN(FFNN(P v )+P v ), Among them, FFNN: feedforward neural network, for feature P v Perform nonlinear transformation to enhance its expressive power; the above formula functions to further optimize P v , extract deeper features through nonlinear activation; Next, the self-update process of the query foreground prototype is completed by applying the softmax function to the attention score A and combining it with the value V: Where Q: query feature matrix; K: key feature matrix; the above formula is used to recalculate the attention score matrix according to the above principle formula for multiple optimizations; The model performs global average pooling on the updated query foreground-background features and calculates cosine similarity with the pixel features of the query image to obtain a new query foreground-background prototype: Among them, P f l+1 and P b l+1 : Optimized foreground prototype and background prototype; Fq: query image features; GAP global average pooling: global pooling of foreground and background features, extracting global features and reducing local noise interference; QPG: prototype generation function, combining the pooled foreground and background features with the query image features to generate a new prototype; The above formula functions: through the interaction of global pooling and query features, the foreground and background prototypes are updated to make the segmentation more accurate.

7. The improved few-sample medical image target segmentation method according to claim 2, characterized in that: In step S4, query set processing and segmentation, iterative segmentation of query image, this module consists of a similarity calculator and a segmentation mask generator, the similarity calculator is used to calculate the similarity between the query image pixels and the foreground and background prototypes, and the segmentation mask generator generates the final segmentation result according to the similarity; The final query image segmentation module is responsible for applying the optimized prototypes to the query image and generating the final segmentation mask by matching pixels in the query image with the optimized foreground and background prototypes.

8. The improved few-sample medical image target segmentation method according to claim 1, characterized in that: It also includes a control system, whose control signals are used to manage the data flow and processing sequence between modules to ensure that the entire process proceeds in an orderly manner.

Images