A small sample medical image segmentation method and system based on multi-scale skip connection and contrastive learning

Abstract

The application discloses a kind of based on multi-scale jump connection and contrast learning small sample medical image segmentation method, comprising: 1) obtaining and processing the MRI image data of different organs, the MRI image data of non-cardiac organ is as training set and verification set, the cardiac MRI image data is as test set;The MRI image data of non-cardiac organ is input to the double-branch network small sample learning segmentation model and is trained and verified;2) constructing double-branch network small sample learning segmentation model, and adding spatial excitation and channel compression interaction module (sSE);3) using sequential splicing mode obtains different levels of feature map, so that the feature map from the different scales of encoder is transmitted to the corresponding decoder.4) using the double-branch structure of model, introduce the contrast learning module based on multi-scale feature.5) input test set data to the model after training to realize the function of image segmentation.The application realizes the cross-domain data segmentation of medical image, and improves the model segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a small-sample medical image segmentation method and system based on multi-scale skip connections and contrastive learning. Background Technology

[0002] Medical image segmentation is a crucial task in medical image analysis, aiming to accurately distinguish regions of interest (such as organs and tumors) from the background in medical images. This process plays a vital role in clinical diagnosis, treatment planning, and efficacy evaluation. However, acquiring annotated medical image data often faces numerous challenges, such as the complexity and specialized nature of the annotation process, high costs, and issues related to patient privacy protection. These problems severely limit the development and application of medical image segmentation technology.

[0003] To address the scarcity of labeled data for medical images, Few-Shot Learning (FSL) does not require a large amount of labeled data, reducing computational resource consumption during training. While FSL holds potential value in medical image segmentation, current techniques still face several challenges. First, the complexity and diversity of medical images make it difficult to directly apply FSL techniques to real-world scenarios. Medical images exhibit high diversity, including different anatomical structures, pathological changes, and imaging patterns. FSL needs to be able to handle this diversity in order to learn generalized features from limited data. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings and deficiencies of the prior art by providing a small-sample medical image segmentation method and system based on multi-scale skip connections and contrastive learning.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] This invention provides a few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning, comprising:

[0007] MRI image data of different organs are acquired and processed. MRI image data of non-cardiac organs are used as training and validation sets, and cardiac MRI image data is used as test set. The training set, validation set, and test set include support set and query set, respectively.

[0008] The MRI image data of the non-cardiac organs are input into a two-branch network few-shot learning segmentation model. The support set feature vector is extracted using conditional branches. The support set feature vector is then passed to the segmentation branch after activation and compression to segment the query set images.

[0009] By using sequentially concatenated multi-scale skip connections instead of traditional skip connections, encoder feature maps of different scales are fused together, and then the feature maps are passed to the corresponding decoders through skip connections.

[0010] The loss function calculation module based on multi-scale features is used to calculate the contrastive learning loss, and combines the result of the contrastive learning loss function with the cross-entropy loss function to obtain the final mixed loss function.

[0011] The trained dual-branch network is used to learn the segmentation model using few samples to predict the segmentation results on the test set data.

[0012] As a preferred embodiment of the small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the method involves processing MRI image data of different organs, including...

[0013] The MRI image data of the different organs are filtered to remove slice data that do not contain the segmentation target, and geometric transformation and brightness transformation are performed on the MRI image data of the different organs.

[0014] As a preferred embodiment of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the method includes: extracting support set feature vectors using conditional branching; these support set feature vectors are then activated and compressed before being passed to the segmentation branch to segment the query set image, including...

[0015] Both the conditional branch and the split branch contain four encoders, one bottleneck block, and four decoders;

[0016] The support set feature vectors are excited and compressed as follows:

[0017]

[0018] Among them, u con u is the feature vector output by the conditional branch convolution block. seg To segment the feature vector output by the convolutional block, P(·) represents a 1*1*1 convolution operation, and σ(·) represents the sigmoid function. This is the output after excitation and compression.

[0019] As a preferred embodiment of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the method utilizes sequentially concatenated multi-scale skip connection modules to improve the segmentation ability of the dual-branch network few-sample learning segmentation model for small targets, including:

[0020] The feature maps e1, e2, e3, and e4 from the outputs of the four encoders at different scales are sequentially concatenated. Feature map e4 is reduced in dimensionality by a convolutional layer, then passed through a ReLU activation layer, and finally upsampled and concatenated with e3 in one channel to serve as the skip connection input feature map d4 for decoder D4. d4 contains information from both e4 and e3 scale feature maps. Then, d4 is upsampled to the same scale as e2 by a convolutional and activation layer, and finally concatenated with e2 in one channel to serve as the skip connection input feature map d3 for decoder D3. d3 contains information from e4, e3, and e2 scale feature maps. This process continues, and after being processed by three network layers, d3 is concatenated with e1 to serve as the skip connection input feature map d2 for decoder D2. d2 contains feature information from four different scales: e1, e2, e3, and e4.

[0021] Considering the network complexity and the fact that although the feature maps of the first shallow network contain rich details, they are relatively simple in semantics and do not use sequential concatenation operations, the skip connection input feature maps of decoder D1 use traditional skip connections.

[0022] It should also be noted that the network model adopts a two-branch structure, so the multi-scale jump connection module is used in both the conditional branch and the split branch, and they are exactly the same.

[0023] As a preferred embodiment of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the method includes: a contrastive learning module based on multi-scale features, and further includes...

[0024] This invention obtains three pairs of feature maps d4, d3, and d2 at three scales for conditional branches and splitting branches.

[0025] This invention inputs the d4 image features of the conditional branch and the segmentation branch into the contrastive learning model to obtain the contrastive learning loss.

[0026] As a preferred embodiment of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the following is included: Calculating the contrastive learning loss function includes:

[0027] A positive sample contrastive learning loss function was used.

[0028] The positive sample contrast loss function is expressed as follows:

[0029]

[0030] Among them, L MFCL It is a positive sample contrast loss function based on sequential concatenation, where N is the number of feature vectors, and d i It is a sequence of spliced ​​feature maps of segmented branches. Feature maps of conditional branches The Euclidean distance between the i-th pair of eigenvectors can be specifically expressed as:

[0031]

[0032] Where d(a, b) represents the sequentially concatenated feature map of the segmented branches. The vector a and the feature map of the conditional branch The Euclidean distance between vectors b, where n is the dimension of the vector and a j b j Let represent the values ​​of vectors a and b in the j-th dimension.

[0033] As a preferred embodiment of the small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning described in this invention, the final mixed loss function is obtained by combining the contrastive learning loss function result and the cross-entropy loss function, including:

[0034] The cross-entropy loss function is expressed as:

[0035]

[0036] Among them, y i For cardiac MRI labeled images, p(y) i () represents the predicted cardiac MRI map, where N is the number of labels and L is the cross-entropy loss function value;

[0037] By combining the contrastive learning function and the cross-entropy loss function, the hybrid loss function is expressed as follows:

[0038] L M =L+λL MFCL

[0039] Among them, L M Let L be the cross-entropy loss function value, where L is the mixed loss function. MFCL To compare the loss function values, λ is the weighting coefficient, which is a constant.

[0040] This invention provides a system for small-sample medical image segmentation based on multi-scale skip connections and contrastive learning, comprising:

[0041] The data preprocessing module is used to acquire and process MRI image data of different organs. MRI image data of non-cardiac organs are used as training and validation sets, and cardiac MRI image data are used as test sets. The training set, validation set, and test set include support set and query set, respectively.

[0042] The dual-branch network model module is used to input the MRI image data of the non-cardiac organs into the dual-branch network few-shot learning segmentation model, extract the support set feature vector using the conditional branch, and then pass the support set feature vector to the segmentation branch to segment the query set image after activation and compression.

[0043] The sequentially spliced ​​multi-scale skip connection module fuses encoder feature maps of different scales and then passes the feature maps to the decoder through skip connections, thereby improving the segmentation ability of the dual-branch network few-shot learning segmentation model for small targets.

[0044] The loss function calculation module based on multi-scale features is used to calculate the contrastive learning loss function, and combine the results of the contrastive learning loss function with the cross-entropy loss function to obtain the final hybrid loss function.

[0045] The prediction module is used to predict the segmentation results on the test set data using a pre-trained dual-branch network small sample learning segmentation model.

[0046] This invention provides a computing device, comprising:

[0047] Memory and processor;

[0048] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning.

[0049] The present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning.

[0050] Beneficial effects:

[0051] 1. The multi-scale skip connection and contrastive learning method based on sequential stitching proposed in this invention firstly helps the network capture small-sized features, ensuring that important details are not lost during the segmentation process, and can better adapt to various changes in the image, such as noise, contrast differences and occlusion, thereby improving not only the accuracy of segmentation, but also the robustness of segmentation.

[0052] 2. This invention utilizes the dual-branch structure of the model and introduces a contrastive learning module based on multi-scale features. It effectively utilizes the data information extracted by the conditional branch and uses it as the contrastive learning target of the segmentation branch. This strategy enables the model to extract more generalized feature representations from limited data, enhance the semantic correlation between samples, and thus further improve the segmentation accuracy. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] in:

[0055] Figure 1 This is a schematic diagram of the overall process of a small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of a dual-branch network small-sample learning network model for a small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to an embodiment of the present invention.

[0057] Figure 3 This is a schematic diagram of a multi-scale skip connection module based on multi-scale skip connections and contrastive learning in a sequential stitching according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of a contrastive learning module based on multi-scale skip connections and contrastive learning according to an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of some cardiac MRI prediction results of a small sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to an embodiment of the present invention. (a) is the segmentation result of LV (left ventricle), (b) is the segmentation result of RV (right ventricle), and (c) is the segmentation result of LV-MYO (left ventricular myocardium). The red lines in each image are the segmentation contours. Detailed Implementation

[0060] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0061] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0062] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0063] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0064] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0065] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0066] Example 1

[0067] Reference Figure 5 As an embodiment of the present invention, a few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning is provided, comprising:

[0068] MRI image data of different organs are acquired and processed. MRI image data of non-cardiac organs are used as training and validation sets, and cardiac MRI image data is used as test set. The training set, validation set, and test set include support set and query set, respectively.

[0069] Furthermore, the MRI image data of different organs are filtered to remove slice data that do not contain the segmentation target, and geometric transformation and brightness transformation are performed on the MRI image data of different organs.

[0070] It should be noted that the input consists of two medical image datasets, CHAOST2 and CMR, both MRI images but with different segmentation target locations. The CHAOST2 dataset is used as the training and validation set, and the CMR dataset is used as the test set. Five data points (5-shot) are sampled from the test set as the support set, and the rest are used as the query set.

[0071] Specifically, 80% of the MRI images in the CHAOST2 dataset are used as the training set and 20% as the validation set. After the training set and validation set are divided into support set and query set, a segmentation target (label) is randomly selected each time. Each time, a query image and a support image are input in pairs. This pair of images first has a segmentation target, and secondly, they come from two consecutive slice samples of the same volume in the data.

[0072] The CMR dataset is used as the test set. After selecting a segmentation target, 5 sample slices and their labels are taken as the support set data. These 5 samples come from the same volume. The selection method is to divide the volume (3D) containing the segmentation target into 5 smaller volumes, and then take the center slice of each of the 5 volumes as the 5 support samples.

[0073] Conversely, the test set query set comes from other volumes, which contain segmentation targets and are also divided into 5 parts, each part corresponding to 5 supporting samples. Each time, one supporting sample is input and the query sample is a query sample of the corresponding part.

[0074] It should also be noted that the data input method during testing is similar to that during training. Each time, a segmentation target is selected, and a query image and a support image are input in pairs.

[0075] MRI image data of non-cardiac organs are input into a two-branch network few-shot learning segmentation model. The model uses conditional branches to extract support set feature vectors. These feature vectors are then activated and compressed before being passed to the segmentation branch to segment the query set images. The two-branch network few-shot learning segmentation model is as follows: Figure 3 As shown;

[0076] Furthermore, both the conditional branch and the split branch contain four encoders, one bottleneck block, and four decoders;

[0077] The support set feature vectors are excited and compressed as follows:

[0078]

[0079] Among them, u con u is the feature vector output by the conditional branch convolution block. seg To segment the feature vector output by the convolutional block, P(·) represents a 1*1*1 convolution operation, and σ(·) represents the sigmoid function. The output after excitation and compression;

[0080] It should be noted that each decoder, bottleneck block, and encoder is composed of convolutional blocks. Each convolutional block consists of two repeated 3x3 convolutional blocks, followed by the ReLU activation function. Skip connections are used between the encoder and decoder blocks. Four downsampling operations use 2x2 max pooling with a stride of 2, and four upsampling operations use 4x4 deconvolution to restore image resolution. Finally, the network's segmentation map is output after a 1x1 convolution operation and the Sigmoid activation function.

[0081] It should also be noted that this scheme achieves efficient segmentation of MRI image data of non-cardiac organs by introducing a dual-branch network few-shot learning segmentation model. The conditional branch in this architecture is specifically used to extract support set feature vectors. These feature vectors, after activation and compression processing, effectively enhance key information and suppress irrelevant noise, thereby significantly improving the segmentation accuracy of the segmentation branch on the query set images. The dual-branch network design strategy ensures that the model maintains good generalization ability even in environments with scarce labeled data. This design not only reduces annotation costs but also greatly improves the practicality and popularity of medical image segmentation technology, enabling it to be more widely applied in clinical practice and providing more accurate support for medical diagnosis and treatment.

[0082] The ability of the dual-branch network few-shot learning segmentation model to segment small targets is improved by utilizing sequentially concatenated multi-scale skip connection modules, including:

[0083] The feature maps e1, e2, e3, and e4 from the outputs of the four encoders at different scales are sequentially stitched together. It should be noted that the resolution of e1 is 128*128, the resolution of e2 is 64*64, the resolution of e3 is 32*32, and the resolution of e4 is 16*16.

[0084] The 512-dimensional feature map e4 is reduced to 256 channels through a 3D convolution, which is comparable to the number of channels in the e3 feature map. It is then passed through a ReLU activation layer and finally upsampled to the same scale as the e3 feature map, so that it is concatenated with e3 in channels to serve as the skip connection input feature map d4 of the decoder D4. d4 has 512 channels and contains information from both the e3 and e4 feature maps.

[0085] Then, the d4 obtained in the first step is reduced to 128 dimensions through a 3D convolutional layer and an activation layer, and upsampled to a resolution of 64*64. It is then concatenated with e2 on the channel as the skip connection input feature map d3 of the decoder D3. The number of channels of d3 is 256, and d3 contains feature map information of three scales: e4, e3, and e2.

[0086] Similarly, after being processed by three network layers, d3 is concatenated with e1 to serve as the skip connection input feature map d2 for decoder D2. d2 has 128 channels and contains information from four different scale features.

[0087] The final layer of decoder D1 uses traditional skip connections for its input feature maps.

[0088] It should also be noted that the network model adopts a two-branch structure, so the multi-scale jump connection module is used in both the conditional branch and the split branch, and they are completely consistent.

[0089] To further explore the potential of the model's dual-branch structure, a contrastive learning module based on multi-scale features was introduced. The sequential concatenation module obtained three pairs of feature maps (d4, d3, d2) at three scales for the conditional branch and the segmentation branch, with d4 selected as the input to the contrastive learning model.

[0090] The image features of the conditional branch and the segmentation branch are input into the contrastive learning model to obtain the contrastive learning loss.

[0091] A positive sample contrastive learning loss function was used.

[0092] The positive sample contrast loss function is expressed as follows:

[0093]

[0094] Among them, L MFCL It is a positive sample contrast loss function based on sequential concatenation, where N is the number of feature vectors, and d i It is a sequence of spliced ​​feature maps of segmented branches. Feature maps of conditional branches The Euclidean distance between the i-th pair of eigenvectors can be specifically expressed as:

[0095]

[0096] Where d(a, b) represents the sequentially concatenated feature map of the segmented branches. The vector a and the feature map of the conditional branch The Euclidean distance between vectors b, where n is the dimension of the vector and a j b j Let represent the values ​​of vectors a and b in the j-th dimension.

[0097] The results of the contrastive learning loss function and the cross-entropy loss function are combined to obtain the final hybrid loss function, which is expressed as follows:

[0098]

[0099] Among them, y i For cardiac MRI labeled images, p(y) i () represents the predicted cardiac MRI map, where N is the number of labels and L is the cross-entropy loss function value;

[0100] By combining the contrastive learning function and the cross-entropy loss function, the hybrid loss function is expressed as follows:

[0101] L M =L+λL MFCL

[0102] Among them, L M Let L be the cross-entropy loss function value, where L is the mixed loss function. MFCL To compare the loss function values, λ is a weighting coefficient with a value of 0.8.

[0103] It should be noted that by calculating the contrastive learning loss function and combining it with the cross-entropy loss function, the final hybrid loss function is obtained, which more effectively guides the model training process. This not only fully utilizes the semantic information between pixels but also incorporates labeled data, enabling the model to consider both pixel similarity and label accuracy during training, thereby improving the model's segmentation accuracy. The contrastive learning loss function based on multi-scale skip connections introduced in this technical solution effectively utilizes the data information extracted from the conditional branches, using it as the contrastive learning target for the segmentation branch. This significantly improves the performance of the dual-branch network few-shot learning segmentation model, providing strong support for the development and application of medical image segmentation technology.

[0104] The trained dual-branch network is used to learn the segmentation model with few samples to predict the segmentation prediction results on the test set data.

[0105] Furthermore, the preprocessed CMR dataset is input into a trained dual-branch network few-shot learning segmentation model to obtain cardiac MRI image segmentation maps.

[0106] The above is an illustrative scheme of a few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to this embodiment. It should be noted that the technical solution of this few-sample medical image segmentation system based on multi-scale skip connections and contrastive learning belongs to the same concept as the technical solution of the aforementioned few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning. Details not described in detail in this embodiment can be found in the description of the aforementioned few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning.

[0107] This embodiment of the few-sample medical image segmentation system based on multi-scale skip connections and contrastive learning includes:

[0108] The data preprocessing module is used to acquire and process MRI image data of different organs. MRI image data of non-cardiac organs are used as training and validation sets, and cardiac MRI image data are used as test sets. The training set, validation set, and test set include support set and query set, respectively.

[0109] The dual-branch network model module is used to input the MRI image data of the non-cardiac organs into the dual-branch network few-shot learning segmentation model, extract the support set feature vector using the conditional branch, and then pass the support set feature vector to the segmentation branch to segment the query set image after activation and compression.

[0110] The sequentially spliced ​​multi-scale skip connection module fuses encoder feature maps of different scales and then passes the feature maps to the decoder through skip connections, thereby improving the segmentation ability of the dual-branch network few-shot learning segmentation model for small targets.

[0111] The loss function calculation module based on multi-scale features is used to calculate the contrastive learning loss function, and combine the results of the contrastive learning loss function with the cross-entropy loss function to obtain the final hybrid loss function.

[0112] The prediction module is used to predict the segmentation results on the test set data using a pre-trained dual-branch network small sample learning segmentation model.

[0113] This embodiment also provides a computing device suitable for small-sample medical image segmentation based on multi-scale skip connections and contrastive learning, including:

[0114] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning, as proposed in the above embodiments.

[0115] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning as proposed in the above embodiments.

[0116] The storage medium proposed in this embodiment belongs to the same inventive concept as the small sample medical image segmentation method based on multi-scale skip connections and contrastive learning proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0117] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0118] Example 2

[0119] Reference Figure 5 As an embodiment of the present invention, a small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning is provided. To verify its beneficial effects, experimental test results are provided.

[0120] First, input two preprocessed medical image datasets: abdominal MRI data CHAOST2 and cardiac MRI data CMR. Both datasets have a resolution of 256*256. CHAOST2 has 20 volumes with a total of about 3000 slices, and CMR has 35 volumes with a total of about 380 slices.

[0121] CHAOST2 has four parts that need to be divided: liver, right kidney, left kidney, and spleen; CMR has two parts that need to be divided: left ventricle (LV), right ventricle (RV), and left ventricular myocardium (LV-MYO).

[0122] CHAOST2 is divided into training and validation sets in a 4:1 ratio. The training and validation sets are further divided into support and query sets. First, a segmentation target (label) is randomly selected. During training, a query image and a support image are input in pairs. These images first have segmentation targets, which come from two consecutive slice samples with the same number in the same volume (medical image 3D voxel) in the data.

[0123] Secondly, the CMR dataset is used as the test set. After selecting a segmentation target, 5 sample slices and their labels are taken as the support set data. These 5 samples come from the same volume. The slice selection method is to divide the volume containing the segmentation target into 5 smaller volumes and select the center slice of these 5 volumes as 5 support samples. The corresponding query set comes from other volumes, which are also divided into 5 parts. The 5 support samples correspond one-to-one with each part. Each time, one support sample is input and the query sample is a query sample of the corresponding part.

[0124] Then, the abdominal MRI data CHAOST2 was input into a two-branch network small sample learning segmentation model with multi-scale sequential splicing skip connections for training and validation. Positive sample contrast learning was achieved based on multi-scale connections, and the mixed loss function was calculated and optimized.

[0125] Finally, after obtaining the trained dual-branch network few-shot segmentation model, the test set data (cardiac MRI) was input into the saved model for prediction, and the results were as follows. Figure 5 The image shown is a segmentation result of a portion of a cardiac MRI image;

[0126] Experimental statistics show that, according to the method of the present invention, after adding multi-scale skip connections, the Dice coefficient (Dice similarity coefficient) and IoU (Intersection over Union) of LV reach 66.14% / 77.77%, the Dice / IoU of RV reaches 60.74% / 74.85%, and the Dice / IoU of LV-MYO reaches 15.20% / 25.44%.

[0127] With the addition of the contrastive learning module, the Dice / IoU of LV reached 66.44% / 78.30%, the Dice / IoU of RV reached 62.75% / 76.43%, and the Dice / IoU of LV-MYO reached 17.28% / 28.37%.

[0128] Therefore, by introducing a dual-branch network few-sample learning segmentation model and a contrastive learning method, this invention effectively improves the accuracy and performance of medical image segmentation. Satisfactory results have been achieved in both the Dice coefficient and the IoU index, fully demonstrating its effectiveness in practical applications. These results not only provide strong support for the development of medical image segmentation technology but also offer more accurate and reliable auxiliary means for clinical diagnosis and treatment.

[0129] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A small-sample medical image segmentation method based on multi-scale skip connections and contrastive learning, characterized in that, include: MRI image data of different organs were collected and preprocessed. MRI image data of non-cardiac organs were used as training and validation sets, while cardiac MRI image data were used as test sets. Each dataset consisted of a support set and a query set to meet the needs of the few-shot learning framework. The MRI image data of the non-cardiac organs are input into a two-branch network few-shot learning segmentation model. In the conditional branch, the model focuses on extracting the feature vectors of the support set. The feature vectors of the support set are then passed to the segmentation branch to segment the query set images after activation and compression. The Sequential Concatenation of Multiscale SkipConnections (SC-MSC) method replaces the traditional skip connections, effectively integrating features from different scales of the encoder and passing them to the corresponding decoder using a sequential concatenation approach. Based on the adoption of multi-scale skip connections, the model's dual-branch structure is utilized to introduce a multi-scale feature contrastive learning module (MFCL), which combines the contrastive learning loss function and the cross-entropy loss function to obtain the final hybrid loss function. The trained dual-branch network is used to learn the segmentation model using few samples to predict the segmentation results on the test set data.

2. The method for segmenting small-sample medical images based on multi-scale skip connections and contrastive learning according to claim 1, characterized in that, Processing MRI image data from different organs, including, The MRI image data of the different organs are filtered to remove slice data that do not contain the segmentation target, and the MRI image data of the different organs are augmented by geometric transformation and brightness transformation.

3. The method for segmenting small-sample medical images based on multi-scale skip connections and contrastive learning according to claim 1, characterized in that, The support set feature vector is extracted using conditional branching. This support set feature vector is then activated and compressed before being passed to the segmentation branch to segment the query set image. Both the conditional branch and the split branch contain four encoders, one bottleneck block, and four decoders; The support set feature vectors are excited and compressed as follows: Among them, u con u is the feature vector output by the conditional branch convolution block. seg To segment the feature vector output by the convolutional block, P(·) represents a 1*1*1 convolution operation, and σ(·) represents the sigmoid function. This is the output after excitation and compression.

4. The method for segmenting small-sample medical images based on multi-scale skip connections and contrastive learning according to claim 1, characterized in that, The ability of the dual-branch network few-shot learning segmentation model to segment small targets is improved by utilizing sequentially concatenated multi-scale skip connection modules, including: The feature maps e1, e2, e3, and e4 from the outputs of the four encoders at different scales are sequentially concatenated. Feature map e4 is passed through a convolutional layer to reduce the number of channels, making it comparable to the number of channels in feature map e3. It then passes through a ReLU activation layer and is finally upsampled to the same scale as feature map e3, thus concatenating it with e3 on the channel. This concatenation serves as the skip connection input feature map d4 for decoder D4, which contains information from feature maps at both scales e3 and e4. d4 is then passed through a convolutional layer and an activation layer, upsampled to the same scale as e2, and finally concatenated with e2 on the channel. This concatenation serves as the skip connection input feature map d3 for decoder D3, which contains information from feature maps at three scales e2, e3, and e4. This process continues, with d3 processed through three network layers and then concatenated with e1 to serve as the skip connection input feature map d2 for decoder D2, which contains information from feature maps at four scales e1, e2, e3, and e4. The skip connection input feature map of decoder D1 uses conventional skip connections; The network model adopts a two-branch structure, so the multi-scale jump connection module is used in both the conditional branch and the split branch, and they are exactly the same.

5. The few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to claim 4, characterized in that, A contrastive learning module based on multi-scale features includes, To further explore the potential of the bi-branch structure of the few-sample model, a contrastive learning module based on multi-scale features was introduced, which obtained three pairs of feature maps d4, d3, and d2 at three scales for the conditional branch and the splitting branch. d4 was selected as the input of the contrastive learning model. Input the d4 image features of the conditional branch and the segmentation branch into the contrastive learning model to obtain the contrastive learning loss.

6. The few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to claim 1, characterized in that, Calculate the contrastive learning loss function, including: The positive sample contrastive learning loss function was used, which is expressed as follows: Among them, L MFCL It is a contrastive loss function based on sequential concatenation, where N is the number of feature vectors, and d i It is the Euclidean distance between the i-th pair of eigenvectors: Where d(a, b) represents the sequentially concatenated feature map of the segmented branches. The vector a and the feature map of the conditional branch The Euclidean distance between vectors b, where n is the dimension of the vector and a j b j Let represent the values ​​of vectors a and b in the j-th dimension.

7. The few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning according to claim 1, characterized in that, The results of the contrastive learning loss function and the cross-entropy loss function are combined to obtain the final hybrid loss function, including: The cross-entropy loss function is expressed as: Among them, y i For cardiac MRI labeled images, p(y) i () represents the predicted cardiac MRI map, where N is the number of labels and L is the cross-entropy loss function value; By combining the contrastive learning function and the cross-entropy loss function, the hybrid loss function is expressed as follows: THE M =L+λL MFCL Among them, L M Let L be the cross-entropy loss function value, where L is the mixed loss function. MFCL To compare the loss function values, λ is the weighting coefficient, which is a constant.

8. A system for small-sample medical image segmentation based on multi-scale skip connections and contrastive learning, characterized in that, include: The data preprocessing module is used to acquire and process MRI image data of different organs. MRI image data of non-cardiac organs are used as training and validation sets, and cardiac MRI image data are used as test sets. The training set, validation set, and test set include support set and query set, respectively. The dual-branch network model module is used to input the MRI image data of the non-cardiac organs into the dual-branch network few-sample learning segmentation model, extract the support set feature vector using conditional branches, and then pass the support set feature vector to the segmentation branch to segment the query set image after activation and compression. A sequentially concatenated multi-scale skip connection module replaces the traditional skip connection, fusing encoder feature maps of different scales, and then passing the feature maps to the decoder through skip connections. This is used to improve the segmentation ability of the dual-branch network few-shot learning segmentation model for small targets. The loss function calculation module based on multi-scale features is used to calculate the contrastive learning loss function, and combines the results of the contrastive learning loss function with the cross-entropy loss function to obtain the final hybrid loss function. The prediction module is used to predict the segmentation results on the test set data using a pre-trained dual-branch network small-sample learning segmentation model.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the small sample medical image segmentation method based on multi-scale skip connections and contrastive learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the few-sample medical image segmentation method based on multi-scale skip connections and contrastive learning as described in any one of claims 1 to 7.

Images