A medical image segmentation method and system based on self-supervised learning

Abstract

The application discloses a kind of medical image segmentation method and system based on self-supervised learning, belong to artificial intelligence and digital medical technology field.The method described in the application first pre-processes medical image data, secondly designs a medical image segmentation method based on self-supervised learning that can effectively utilize unlabeled medical image data, the method can learn to contain more subtle lesion information the vector representation of medical image data, and is segmented to medical image by vector representation, to improve the accuracy of medical image segmentation.The system described in the application is used to implement the above method, including computer processor, memory and graphics processor;Medical image data storage unit;Medical image data pre-processing unit;Medical image representation model training unit;Medical image data segmentation unit.The application improves the accuracy of medical image subtle lesion segmentation, so that early patients have greater treatment opportunities, avoid further deterioration of the disease or cause other complications.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and digital medical technology, and in particular to a medical image segmentation method and system based on self-supervised learning. Background Technology

[0002] With the continuous development of algorithms and technologies in the field of computer vision, especially the popularization of medical imaging equipment and the maturity of digital technology, a large amount of medical image data has been collected, stored and shared, and medical image segmentation has gradually become possible and widely used. Medical image segmentation is the process of dividing a medical image into multiple regions, each containing the same semantic information, and accurately separating the target region (damaged tissue, cells, nucleus, organ, etc.). Among them, medical images usually include CT scans, MRI images, X-rays, PET images, etc., which provide doctors with key information needed for diagnosing and treating diseases. Image segmentation plays an important role in the medical field. (1) Accurate segmentation results can help doctors obtain quantitative feature information such as the location, size, shape and density of lesions, thereby accurately assessing the condition and lesion characteristics and formulating the most appropriate treatment plan to improve the treatment effect; (2) Medical image segmentation can help doctors make preoperative plans, better understand the patient's structure, and guide the operation in real time during surgery, improving the accuracy and safety of surgery; (3) Medical image segmentation automates and simplifies the doctor's workflow and improves work efficiency.

[0003] Traditional medical image segmentation often relies on image processing and computer vision techniques, requiring manual model design. However, medical images vary significantly among different patients and organs, making manual modeling cumbersome. Recently, the application of deep neural networks has automated medical image description, leading to significant progress in medical image segmentation. However, training deep neural networks requires massive amounts of data, and efficiently generating labels for such large datasets necessitates specialized biomedical analysts and substantial manual work—a lengthy and expensive task, also susceptible to human error. Self-supervised learning, on the other hand, can design auxiliary tasks to mine the inherent features of medical images as supervisory information without requiring labels. Contrastive learning, as a method of self-supervised learning, learns the representation of medical images by designing tasks that show similar representations of different views of the same medical image and separate representations of different medical images. Using the trained model, representations of any medical image can be obtained, and diseased tissues and cells can be segmented from the background using medical image segmentation techniques. Ultimately, the medical image segmentation results are delivered to doctors and other healthcare professionals, providing more comprehensive, accurate, and convenient information for medical diagnosis and treatment, while also improving medical efficiency and the effectiveness of multidisciplinary collaboration.

[0004] Existing methods for medical image segmentation include: "BT-Unet: A self-supervised learning framework for biomedical image segmentation using barlow twins with U-net models" proposes a self-supervised learning framework, BT-Unet, which pre-trains the encoder of the U-Net model in an unsupervised manner by reducing redundancy to learn data representations, and then fine-tunes the complete network for actual segmentation. Furthermore, "Swin-Unet: Unet-like pure Transformer for medical image segmentation" implements self-attention from local to global perspectives in medical images based on the SwinTransformer encoder, and upsamples global features to the input resolution for corresponding pixel-level segmentation prediction.

[0005] The patent with publication number CN113763399A, entitled "A Medical Image Segmentation Method Based on Weakly Supervised Learning and a Computer-Readable Storage Medium," utilizes a limited medical image dataset and weakly labeled information for training and learning to establish a weakly supervised learning model that can benefit from a partially labeled dataset. The method includes: (1) collecting medical image data and volume prior data, and preprocessing the medical image data and volume prior data to form a raw dataset for training; (2) constructing a convolutional neural network based on weakly supervised learning based on the raw dataset, training and alternately updating the convolutional neural network model and its weight parameters; and (3) segmenting the medical images and obtaining the medical image segmentation results.

[0006] In summary, medical image segmentation methods consist of two stages. First, a model is trained using medical image data, which yields a representation of the medical images. This representation is then applied to image segmentation to obtain the final segmentation result. However, existing methods require a large amount of medical image data for training, necessitating manual annotation by multiple medical professionals to obtain reliable labels. This annotation process is costly and challenging. Furthermore, for medical image datasets, the differences between different medical images are relatively small. Existing methods for obtaining image representations focus on the global information of the images, failing to capture subtle details and thus struggling to segment subtle lesion regions, potentially affecting doctors' ability to make accurate diagnoses. Summary of the Invention

[0007] To address the aforementioned problems, this invention designs a medical image segmentation method and system based on self-supervised learning. First, it utilizes unlabeled medical image data to learn the vector representation of medical images through a medical image representation model. This mainly includes constructing the medical image representation model, initializing model parameters, obtaining the vector representation of the medical image data, reconstructing the vector representation, and designing a loss function to optimize network parameters. Second, it obtains medical image segmentation results based on the vector representation of the medical images. Finally, it uses the segmentation results to assist doctors in obtaining more specific and comprehensive image information, making diagnostic conclusions more accurate and reliable. The medical image vector representation of this invention can uncover detailed information in medical images, thus making it easier to segment small lesion areas. Furthermore, this invention obtains the medical image vector representation through self-supervised contrastive learning, thereby avoiding the problem of difficulty in obtaining data labels.

[0008] The parameters involved in this invention include: temperature parameters, medical image mask ratio, medical image block size, momentum update parameters, optimizer parameters, and the number of model training epochs. Specifically, the temperature parameter adjusts the "hardness" or "softness" of sample similarity; the medical image mask ratio adjusts the proportion of the mask region; the medical image mask block size adjusts the size of each mask block; the momentum update parameter controls the speed of momentum network updates; the optimizer parameters include optimizer type, initial learning rate, and learning rate decay method; and the number of model training epochs sets the termination conditions for model training.

[0009] To achieve the above objectives, the present invention employs the following technical solutions:

[0010] A medical image segmentation method based on self-supervised learning includes the following steps:

[0011] S1. Preprocess medical image data, convert medical image data to a standard medical image format, clean medical image data, and construct a new medical image dataset.

[0012] S2. The new medical image dataset constructed in step S1 is processed sequentially through data augmentation and image masking.

[0013] S3. Construct and initialize the medical image representation model;

[0014] S4. Train the medical image representation model. Use the medical image dataset obtained in step S2 to train the medical image representation model constructed and initialized in step S3.

[0015] S5. The medical image representation model trained by S4 encodes the medical image data to be segmented, obtains its vector representation, and obtains the medical image segmentation result based on the vector representation.

[0016] Furthermore, step S1 includes the following specific steps:

[0017] S11. Collect medical image datasets, convert the medical image data into the standard DICOM format, and perform subsequent processing and analysis;

[0018] S12. Clean the medical image dataset: Use Slicer software to view and filter clear, frontal medical image data; remove noise from the medical images; use interpolation to fill in missing areas of the medical images, and finally obtain a new medical image dataset. Choose any N samples, and denote the medical imaging data as X = (x1, ..., x2) i ,…x N ).

[0019] Furthermore, step S2 includes the following specific steps:

[0020] S21. Augmenting medical imaging data, for medical imaging datasets Data augmentation methods include random cropping and scaling of medical images to a fixed size, medical image flipping, random color jittering of medical images, medical image data normalization, and conversion of medical image data format to Tensors. These data augmentation methods increase the amount and diversity of training data, improve the model's generalization ability and robustness, and address the problems of insufficient or imbalanced data. Taking any N samples, the medical image data X is denoted as X′=(x′1,…x′) after two data augmentations. i ,…x′ N ), X”=(x″1,…x″ i ,…x″ N );

[0021] S22. Perform the augmented medical image data masking operation in step S21, and arbitrarily select medical image x′. i Divide it into M medical image blocks of the same size, denoted as x′. i ={x′ i(1) ,x′ i(2) ,…,x′ i(j) ,…,x′ i(M)}, where x′ i(j) Representing the j-th image block of the i-th medical image, a mask sequence m′ is randomly generated. i ={m′ i(1) ,m′ i(2) ,…m′ i(j) ,…m′ i(M)}, where m′ i(j) ={0,1}, m′ i(j)=1 indicates that the j-th block of the i-th medical image is masked. Asymmetric masking is used for the medical images, meaning only the medical image X′ from one of the two augmentations is masked. This avoids ambiguity in vector representation caused by simultaneously masking both augmented images, thus reducing the quality of the vector representation. By masking the medical images, the amount of coding is reduced, thereby improving the model's generalization ability and robustness, while also increasing computational efficiency. The masked medical image data is denoted as... in It only includes medical image blocks that are not masked.

[0022] Furthermore, the medical image representation model in step S3 includes an online network and a target network, specifically including: a medical image online encoder f′, a medical image target encoder f″, a medical image online projection head p′, a medical image target projection head p″, a medical image online decoder d′, and a medical image target decoder d″.

[0023] Furthermore, step S3, constructing and initializing the medical image representation model, specifically includes the following steps:

[0024] S31. Construct and initialize the online medical image encoder f′; specifically: use VisionTransformer as the network of the online medical image encoder f′, which contains 12 Transformer blocks, replace the last classification linear layer of Vision Transformer with the online medical image projection head p′, and initialize the parameters of the online medical image encoder f′.

[0025] S32. Construct and initialize the medical image target encoder f”; specifically: the medical image target encoder f” is constructed using Vision Transformer, which contains 12 Transformer blocks, and the parameter θ” ​​of f” is updated by the exponential moving average of the parameter θ′ of the online medical image encoder f’, as shown in formula (1):

[0026] θ″=αθ″+(1-α)θ#(1)

[0027] Where α is the parameter for momentum update of the medical image target encoder;

[0028] S33. Construct and initialize the online medical image projection head p'; specifically: the online medical image projection head is a multilayer nonlinear perceptron network containing 5 linear layers, where the 1st, 3rd, and 4th linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer, and initialize the parameters of the online medical image projection head p'.

[0029] S34. Construct and initialize the medical image target projection head p”; Specifically: the medical image target projection head is a multi-layer nonlinear perceptron network, p” contains 2 linear layers, between which there is a BatchNormal normalization layer and a ReLU nonlinear activation layer. The parameters of p” are updated by the momentum of the parameters of the first two linear layers of p”, and the update method is the same as that of formula (1). Initialize the network parameters of the medical image target projection head p”.

[0030] S35. Construct and initialize the online medical image decoder d'; specifically: the online medical image decoder d' is constructed from the Vision Transformer model. d' contains one linear layer and 8 Transformer blocks, and the last linear classification layer is removed. Initialize the parameters of the online medical image decoder d'.

[0031] S36. Construct and initialize the medical image target decoder d”; specifically: construct the medical image target decoder d” using the Vision Transformer model, which includes one linear layer and 8 Transformer blocks. The network parameters of d” are updated by the parameter momentum of d’.

[0032] Furthermore, step S4 specifically includes the following steps:

[0033] S41. The medical image data after masking in step S22 Input the online medical image encoder f' constructed in step S31, and input the augmented medical image data X” from step S21 into the target encoder f” constructed in step S32. Through encoding, online vector representations are obtained respectively. With target vector representation in, N represents the number of randomly selected samples. in, This represents the number of medical image blocks remaining after masking. Where M represents the total number of medical image blocks, and Corresponding to online network input With the target network input x' i The medical image vector block representation, with h″ i(M+1) These are the online [CLS] vector representation and the target [CLS] vector representation, respectively, representing the medical image x' i With x″ i The global vector representation is denoted as h' for convenience. For the online [CLS] vector representation and the target [CLS] vector representation are respectively denoted as h'. i ,h″i Let the online [CLS] block vector representation and the target [CLS] block vector representation of any N medical image data be denoted as H', H'', respectively.

[0034] S42. Input the online [CLS] block vector representation H' from step S41 into the medical image online projection head p' constructed in step S33, and input the target [CLS] block vector representation H” from step S41 into the medical image target projection head p” constructed in step S34. Through projection mapping, the online low-dimensional embedding Z' and the target low-dimensional embedding Z” are obtained respectively, where Z' = (z'1, ..., z') i ,…z' N ), Z″=(z″1,…z″) i ,…z″ N );

[0035] S43. Calculate the low-dimensional contrast loss; for any medical image sample x' i , to embed z' in online low-dimensional i As the anchor point, its corresponding target low-dimensional embedding z″ i With z' i The samples are mutually positive, and the remaining targets have low-dimensional embeddings z″. k With z' i The low-dimensional contrastive loss is defined as shown in formula (2) when the samples are negative to each other:

[0036]

[0037] Among them, z' i ,z″ i ,z″ k Mapped to the hypersphere space via L2 normalization, where τ is the temperature parameter, N represents the number of randomly selected samples, and e (·) Represents an exponential function;

[0038] S44. Randomly initialize the vector representation of the masked medical image block in step S22, and combine it with the online vector representation in step S41. By combining these methods, a new online vector representation can be obtained. The online vector representation of medical images and the target vector representation Online high-dimensional embeddings are obtained through the online medical image decoder d' in step S35 and the medical image target decoder d” in step S36, respectively. and target high-dimensional embedding in, Where N represents the number of randomly selected samples, and M represents the number of medical image blocks. in and Corresponding to medical image x' i With x″ i The high-dimensional embedding of M medical image blocks, r' i(M+1 ) and r″ iM+1 These are online [CLS] high-dimensional embedding and target [CLS] high-dimensional embedding, respectively, representing medical image x' i With x″ i For convenience, the global high-dimensional embedding is denoted as r' for the online [CLS] high-dimensional embedding and the target [CLS] high-dimensional embedding, respectively. i ,r″ i Let the online [CLS] high-dimensional embedding and the target [CLS] high-dimensional embedding of any N medical image data be denoted as R' and R' respectively. Calculate the reconstruction loss of the online high-dimensional embedding to ensure that the masked medical image blocks are accurately predicted. In this way, the representation of medical image microlesions can be learned through the relationship between medical image blocks. The reconstruction loss is shown in Equation (3):

[0039]

[0040] Where M represents the number of medical image blocks, The number of medical image blocks that are masked, i.e. Where N is the number of visible blocks in the medical image, and N represents the number of randomly selected samples.

[0041] S45. Calculate the high-dimensional contrast loss; for any medical image sample x' i , with online high-dimensional embedding r' i As the anchor point, its corresponding high-dimensional target embedding r″ i With r' i The samples are mutually positive, and the remaining targets are high-dimensional embeddings r″ k With r' i The high-dimensional contrastive loss is defined as follows: (4) where the samples are mutually negative.

[0042]

[0043] S46. Calculate the total loss as shown in formula (5):

[0044]

[0045] S47. Minimize the loss using gradient descent. The entire network is then optimized using the LARS optimizer, and steps S41-S46 are repeated until the set termination condition is met, resulting in a trained medical image representation model.

[0046] Furthermore, step S5 specifically includes the following steps:

[0047] S51. Augment the medical image data to be segmented. The data augmentation process involves scaling the medical image to a fixed size, converting the medical image data format to Tensor, and then inputting the medical image data into the trained medical image representation model to obtain the vector representation of the medical image.

[0048] S52. Construct a medical image reducer g; the reducer includes a linear mapping layer, as well as 2× upsampling and 1×1 convolutional layers, and iterates the upsampling and convolutional layer operations four times to achieve the final medical image resolution.

[0049] S53. Input the vector representation from step S51 into the reducer g to obtain an integer value matrix A, where different tissues or organs are different digital labels. Calculate the cross-entropy loss between the obtained digital labels and the medical image labels. Optimize the reducer g through gradient descent. After optimization, obtain the medical image segmentation result.

[0050] S54. Present the segmentation results of medical images to doctors to help them locate and segment structures or lesions of interest. Through the segmentation results, doctors can more accurately assess the size, shape, and location of lesions, especially the discovery of tiny lesions, thereby enabling accurate diagnosis and treatment planning for the disease.

[0051] This invention also provides a medical image segmentation system based on self-supervised learning. The medical image segmentation system is used to implement the above-described medical image segmentation method based on self-supervised learning, and includes a computer processor, memory and graphics processor; a medical image data storage unit; a medical image data preprocessing unit; a medical image representation model training unit; and a medical image data segmentation unit.

[0052] Furthermore, the medical image data storage unit is used to store the preprocessed medical image dataset and the medical image data to be segmented obtained from medical institutions, and inputs it into computer memory; the medical image data preprocessing unit is used to perform medical image data augmentation and masking operations in step S2 on the medical image data stored in computer memory, and load the obtained medical image samples into the graphics processor; the medical image representation model training unit uses the augmented and masked data in the graphics processor to perform steps S3 and S4 to train and obtain a medical image representation model; the medical image data segmentation unit performs step S5 on the medical image data to be segmented in computer memory to obtain the segmentation result of the medical image; the specific data processing and calculation work in all units is completed by the computer processor, and all units interact with the data in the computer memory.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] 1. The medical image segmentation method and system based on self-supervised learning designed in this invention does not require a large amount of manual annotation of medical image data. Specifically, the method utilizes unlabeled medical image data during the training of the image representation model, and only uses a small number of limited data labels during fine-tuning, thus saving on manual annotation.

[0055] 2. This invention projects medical image representations onto a low-dimensional embedding space and reconstructs them into a high-dimensional embedding space, respectively. This not only learns the common information contained in the same type of lesions in medical images, but also provides the possibility of discovering subtle lesions for medical image segmentation, giving early-stage patients a greater chance of treatment and avoiding further deterioration of the disease or other complications.

[0056] 3. This invention uses a mask prediction method to learn information between adjacent medical image blocks, providing better medical image representation, improving the accuracy of medical image segmentation, and helping doctors make more accurate diagnoses and decisions. Attached Figure Description

[0057] Figure 1 This is a structural diagram of the medical image representation model in the self-supervised learning-based medical image segmentation method described in this invention;

[0058] Figure 2 This is a structural diagram of the medical image segmentation module in the self-supervised learning-based medical image segmentation method described in this invention;

[0059] Figure 3 This is a computer system architecture diagram of the medical image segmentation method based on self-supervised learning described in this invention.

[0060] Figure 4 This is a flowchart of the medical image segmentation method based on self-supervised learning described in this invention. Detailed Implementation

[0061] The self-supervised learning-based medical image segmentation method described in this invention is implemented through a computer program. Figure 3 The diagram shown illustrates the system architecture of a computer implementation. The term "Graphics Processing Unit" represents only the type of computing resource; it can be a standalone graphics card supporting GPU computing, or a local server or cloud server that supports GPU computing. The following section will combine... Figure 1 , 2 The model structure diagram shown and Figure 4 The method flowchart, using the BUSIS dataset as an example, provides a detailed explanation of the technical solutions in this embodiment of the invention. The implementation methods mainly include the following key aspects:

[0062] S1. Preprocessing the medical image dataset, specifically including the following steps:

[0063] S11. This invention utilizes the BUSIS dataset to train a medical image representation model. Specifically, the BUSIS (Baidu Ultrasound Speckle Imaging Segmentation) dataset contains a total of 3017 medical ultrasound images, including 2696 images for training and 321 images for testing. These images come from three different types of diseases: breast tumors, benign and malignant liver lesions, and thyroid nodules. The ultrasound images in the BUSIS dataset all contain "speckle," which is a high-frequency signal enhancement randomly distributed in the image. This is a common phenomenon in ultrasound imaging and also a major challenge in ultrasound image segmentation.

[0064] S12. Process the medical image dataset. View and analyze the medical images using the software RadiAnt DICOMViewer. Measure the size and angle of the medical images. Select medical images with a resolution of 600×800 or higher, as high-resolution images are beneficial for image segmentation. Remove noise from the medical images using Gaussian filtering. Complete missing areas of the medical images using interpolation. Measure the angles between different structures on the image and the human coordinate axes. Select frontal medical image data. The processed medical image data is denoted as X=(x1,…,x…). i ,…x 2658 ), where 2658 is the sample size.

[0065] S2. The medical image data constructed in step S1 is processed sequentially through data augmentation and image masking. The specific process is as follows:

[0066] S21. Augmenting medical image data: For a medical image dataset X, the data augmentation method of this invention is as follows: randomly cropping the medical image within the range of 0.08-1, and readjusting the size of the medical image to 224×224; color jittering of the medical image with a probability of 0.8, configured as (0.4, 0.4, 0.2, 0.1); horizontally and vertically flipping the medical image with a probability of 0.2; converting the medical image data format to Tensor, and performing the above data augmentation operations twice to obtain X' and X''. The final dimension of the medical image is [64, 3, 224, 224], where 64 is the number of samples processed each time, 3 is the number of channels, and 224×224 is the pixel size of each channel.

[0067] S22. Perform a masking operation on the augmented medical image data X' from step S21; randomly select a medical image sample x'. iThe image is divided into 196 medical image blocks of equal size, each block having a pixel size of 16×16. This invention masks the medical image sample x' with a 50% probability. i Therefore, for each medical image sample, 98 image blocks are randomly masked, and 98 image blocks are visible image blocks. This invention uses a minimum masking unit of 16×16 to randomly mask the medical image blocks, thereby learning information between smaller medical image blocks. Therefore, it is easier to locate smaller lesions. The dimensions after masking are [64,3,98,224,224]. For the medical image X” in step S21, it is simply divided into 196 medical image blocks of the same size, with dimensions of [64,3,196,224,224].

[0068] S3. Construct and initialize the medical image representation model, specifically including: online medical image encoder, medical image target encoder, online medical image projector, medical image target projector, online medical image decoder, and medical image target decoder. The specific models are attached. Figure 1 As shown:

[0069] S31. Construct and initialize the online medical image encoder f'. Specifically, this invention uses VisionTransformer as the network of the online medical image encoder f', which contains 12 Transformerblocks. This invention replaces the last classification linear layer of VisionTransformer with the online medical image projection head p'. The parameters of the online medical image encoder f' are randomly initialized. The output vector of f' has a dimension of [64, 99, 768], where 99 is the sum of the number of visible image blocks (98) and [CLS] blocks, 768 is the vector dimension, and [64, 768] is the online [CLS] vector representation dimension, representing the global vector representation of the medical image.

[0070] S32. Construct and initialize the medical image target encoder f”; Specifically, the medical image target encoder f” is constructed using Vision Transformer, which contains 12 Transformer blocks. The output vector of f” has a dimension of [64, 197, 768], where 99 is the sum of the number of all medical image blocks (196) and [CLS] blocks, 768 is the vector dimension, and [64, 768] is the dimension of the target [CLS] vector representation, representing the global vector representation of the medical image. The parameter θ” ​​of f” is updated by the exponential moving average (EMA) of the parameter θ’ of the online medical image encoder f’, as shown in formula (1):

[0071] θ″=αθ″+(1-α)θ′#(1)

[0072] Where α is the parameter for momentum update of the medical image target encoder, and α is set to 0.99;

[0073] S33. Construct and initialize the online medical image projection head p'. Specifically, the online medical image projection head is a multi-layer nonlinear perceptron network containing 5 linear layers. The 1st, 3rd, and 4th linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer. The output dimension of each linear layer is set to [2048, 256, 2048, 2048, 256]. The input dimension of the online medical image projection head p' is a vector representation of [CLS] blocks with a dimension of [64, 768], and the output dimension is [64, 256], where 256 is the low-dimensional embedding dimension. Randomly initialize the parameters of the online medical image projection head p'.

[0074] S34. Construct and initialize the medical image target projection head p”; Specifically, the medical image target projection head is a multi-layer nonlinear perceptron network. p” contains two linear layers. Between these two linear layers is a BatchNormal normalization layer and a ReLU nonlinear activation layer. The output dimensions of the linear layers are [2048, 256], and the output dimension of p” is [64, 256]. Among them, 256 is the low-dimensional embedding dimension. The parameters of p” are updated by the momentum of the first two linear layers of p’. The update method is the same as that of formula (1).

[0075] S35. Construct and initialize the online medical image decoder d'. Specifically, the online medical image decoder d' is constructed using the Vision Transformer model. d' contains one linear layer and eight Transformer blocks, and the last linear classification layer is removed. The output high-dimensional embedding dimension of d' is [64, 197, 768], where 197 is the sum of the number of visible image blocks (196) and [CLS] blocks, 768 is the embedding dimension, and [64, 768] is the online [CLS] high-dimensional embedding, representing the global high-dimensional embedding of medical images.

[0076] S36. Construct and initialize the medical image target decoder d”; Specifically, construct the medical image target decoder d” using the Vision Transformer model, which includes one linear layer and 8 Transformer blocks. The network parameters of d” are updated by the parameter momentum of d’. The output dimension of d” is [64,197,768].

[0077] S37. Initialize the LARS parameters of the image representation model optimizer, setting the initial learning rate lr to 0.6 and the weight decay rate W to 10. -6 .

[0078] S4. Train the medical image representation model using the medical images masked in step S2. The medical image representation model for training step S3, which involves augmenting medical images with data, is trained as follows:

[0079] S41. Mask the medical image after step S22. Input the online medical image encoder f′ constructed in step S31, The medical image X” with dimensions [64,3,98,16,16] is input into the target encoder f” constructed in step S32 after augmentation processing in step S21. The dimensions of X” are [64,3,196,16,16]. Online vector representations are obtained through the online medical image encoder f’ and the target medical image encoder f”, respectively. With target vector representation in The dimensions are [64, 99, 768], where 64 represents the number of samples processed each time, 99 represents the sum of the number of visible medical image blocks remaining after masking (98) and the number of [CLS] blocks, and 768 represents the dimension of each block vector representation. The dimensions are [64, 197, 768], where 197 represents the sum of the total number of medical image blocks and the number of [CLS] blocks, and h' i(99) with h″ i(197) These are the online [CLS] vector representation and the target [CLS] vector representation, respectively, representing the medical image x' i With x″ i The global representation of the 64 medical image samples is denoted as H' and H', respectively, with dimensions of [64, 768].

[0080] S42. Input the online [CLS] block vector representation H' from step S41 into the online medical image projection head p' constructed in step S33, and input the target [CLS] block vector representation H” from step S41 into the target medical image projection head p” constructed in step S34. Through projection mapping, the online low-dimensional embedding Z' and the target low-dimensional embedding Z” are obtained respectively, where Z' = (z'1, ..., z') i ,…z' N ), Z”=(z″1,…z″ i ,…z″ N The dimensions of Z' and Z” are both [64, 256], where 64 represents the number of samples processed each time and 256 represents the dimension of the low-dimensional embedding of each sample.

[0081] S43. Calculate the low-dimensional contrast loss; for any medical image sample x' i , to embed z' in online low-dimensional i As the anchor point, its corresponding target low-dimensional embedding z″ i With z' i The samples are mutually positive, and the remaining targets have low-dimensional embeddings z″. k With z' i The low-dimensional contrastive loss is defined as shown in formula (2) when the samples are negative to each other:

[0082]

[0083] Among them, z' i ,z″ i ,z″ k Mapped to the hypersphere using l2 normalization, the temperature parameter τ is set to 0.99; N represents the number of randomly selected samples, e (·) Represents an exponential function;

[0084] S44. The vector representation of the medical image block masked in step S22 with random initialization of dimensions [64, 98, 768] is compared with the online vector representation of step S41 with dimensions [64, 99, 768]. By combining these elements, a new online vector representation with dimensions [64, 197, 768] is obtained. Online and target representation of medical images Online high-dimensional embeddings are obtained through the online medical image decoder d' in step S35 and the medical image target decoder d” in step S36, respectively. The dimensions are [64,197,768], and the target is a high-dimensional embedding. The dimensions are [64, 197, 768], where 64 represents the number of samples processed each time, and 197 represents the sum of the number of medical image blocks and the number of [CLS] blocks. The target high-dimensional embedding also has a similar structure. r' i(197 ) and r″ i(197) These are online [CLS] high-dimensional embedding and target [CLS] high-dimensional embedding, representing medical image x' respectively. i With x″ i For convenience, the global high-dimensional embedding is denoted as r' for the online [CLS] high-dimensional embedding and the target [CLS] high-dimensional embedding, respectively. i ,r″ i Let the online [CLS] high-dimensional embedding and the target [CLS] high-dimensional embedding of 64 medical image samples be denoted as R' and R”, respectively, with dimensions of [64, 768]. The reconstruction loss is calculated for the target high-dimensional embedding to ensure the restoration of the masked image block. The reconstruction loss is shown in formula (3):

[0085]

[0086] Where 64 represents the number of samples processed each time, and 98 represents the number of masked medical image blocks;

[0087] S45. Calculate the high-dimensional contrast loss; for any medical image sample x' i , with online high-dimensional embedding r' i As the anchor point, its corresponding high-dimensional target embedding r″ i With r' i The samples are mutually positive, and the high-dimensional embeddings of the remaining targets are r' i The high-dimensional contrastive loss is defined as follows: (4) where the samples are mutually negative.

[0088]

[0089] Where, r' i ,r″ i ,r″ k The temperature parameter τ is set to 0.99 and mapped to the hypersphere space using l2 normalization.

[0090] S46. Calculate the total loss as shown in formula (5):

[0091]

[0092] S47. Minimize the loss using gradient descent. The entire network is then optimized using the LARS optimizer. This process is repeated continuously, with the number of iterations for the medical image representation model set to 400. Training stops once the number of iterations is reached, and the trained medical image representation model is obtained.

[0093] Furthermore, step S5 includes the following process:

[0094] S51. Augmenting medical image data: The data augmentation process involves scaling the medical images to a fixed size of 224×224, converting the medical image format to Tensor, and then inputting the medical image data into the trained medical image representation model to obtain the vector representation of the medical images. The input dimensions are [64, 3, 196, 768], and the output vector representation is [64, 196, 768]. Here, 64 represents the number of samples processed each time, 3 represents the number of medical image channels, 196 represents the number of medical image blocks, and 768 represents the vector representation dimension of each medical image block.

[0095] S52. Construct a medical image restorer g; the restorer includes a linear mapping layer, and 2× upsampling and 1×1 convolutional layers. The upsampling and convolutional layer operations are repeated four times to achieve the final medical image resolution. The input dimension of the linear layer is [64,196,768], and the output dimension is [64,196,1024]. The linear output is transformed back to [64,14,14,1024]. The output dimensions of the four upsampling and convolutional layers are [64,28,28,256], [64,56,56,256], [64,112,112,256], and [64,224,224,19], respectively. Here, 19 is the number of channels, and 224×224 is the medical image resolution.

[0096] S53. The vector representation from step S51 is passed through the reducer g to obtain an integer matrix A, where different tissues or organs are different digital labels. The obtained digital labels and medical image labels are used to calculate the cross-entropy loss. The reducer g is optimized by gradient descent. Steps S51-S53 are repeated until the set 100 rounds are met. After optimization, the medical image segmentation result is obtained.

[0097] S54. Present the segmentation results of medical images to doctors to help them locate and segment structures or lesions of interest. Through the segmentation results, doctors can more accurately assess the size, shape, and location of lesions, especially the discovery of tiny lesions, thereby enabling accurate diagnosis and treatment planning for the disease.

[0098] like Figure 3 As shown, a medical image segmentation system based on self-supervised learning includes a computer processor, memory, and a graphics processor; a medical image data storage unit; a medical image data preprocessing unit; a medical image representation model training unit; and a medical image data segmentation unit. The medical image data storage unit stores the preprocessed medical image dataset and the medical image data to be segmented obtained from medical institutions, and inputs it into the computer memory. The medical image data preprocessing unit performs step S2 (medical image data augmentation and masking) on ​​the medical image data stored in the computer memory and loads the obtained medical image samples into the graphics processor. The medical image representation model training unit uses the augmented and masked data in the graphics processor to perform steps S3 and S4 to train and obtain a medical image representation model. The medical image data segmentation unit performs step S5 on the medical image data to be segmented in the computer memory to obtain the segmentation result of the medical image. The specific data processing and calculation work in all units is completed by the computer processor, and all units interact with the data in the computer memory.

[0099] Finally, the above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical image segmentation method based on self-supervised learning, characterized in that, Includes the following steps: S1. Preprocess medical image data, convert medical image data to a standard medical image format, clean medical image data, and construct a new medical image dataset. S2. The new medical image dataset constructed in step S1 is processed sequentially through data augmentation and image masking. S3. Construct and initialize the medical image representation model; S4. Train the medical image representation model. Use the medical image dataset obtained in step S2 to train the medical image representation model constructed and initialized in step S3. S5. The medical image representation model trained by S4 encodes the medical image data to be segmented, obtains its vector representation, and obtains the medical image segmentation result based on the vector representation. In step S3, the medical image representation model is divided into an online network and a target network, specifically including: an online medical image encoder. Medical image target encoder Online medical imaging projector Medical imaging target projection head Online medical image decoder Medical image target decoder ; Step S3, which involves constructing and initializing the medical image representation model, specifically includes the following steps: S31. Construct and initialize the online medical image encoder. Specifically, Vision Transformer is used as an online encoder for medical images. The network contains 12 Transformer blocks, replacing the last classification linear layer of the Vision Transformer with a medical imaging online projection head. Initialize the online encoder for medical images Parameters; S32. Construct and initialize the medical image target encoder. Specifically, a medical image target encoder was constructed using Vision Transformer. It contains 12 Transformer blocks, and parameters Medical Imaging Online Encoder parameters The exponential moving average is updated as shown in formula (1): ； in, These are parameters for momentum updates in medical image target encoders; S33. Construct and initialize the online medical imaging projection head. Specifically, the online medical imaging projection head is a multilayer nonlinear perceptron network containing 5 linear layers. The 1st, 3rd, and 4th linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer to initialize the online medical imaging projection head. Parameters; S34. Construct and initialize the medical imaging target projection head. Specifically, the medical imaging target projection head is a multilayer nonlinear perceptron network. It consists of two linear layers, with a BatchNormal normalization layer and a ReLU nonlinear activation layer in between. The parameters are determined by The momentum of the first two linear layers is updated in the same way as in formula (1), and the medical image target projection head is initialized. Network parameters; S35. Construct and initialize the online medical image decoder. Specifically: Online medical image decoder Constructed from the VisionTransformer model, It contains one linear layer and eight Transformer blocks, with the last linear classification layer removed, and initializes the online medical image decoder. Parameters; S36. Construct and initialize the medical image target decoder. Specifically, this involves constructing a medical image target decoder using the Vision Transformer model. It contains one linear layer and eight Transformer blocks. Network parameters are determined by The parameter momentum is updated.

2. The medical image segmentation method based on self-supervised learning according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Collect medical image datasets, convert the medical image data into the standard DICOM format, and perform subsequent processing and analysis; S12. Clean the medical image dataset: Use Slicer software to view and filter clear, frontal medical image data; remove noise from the medical images; use interpolation to fill in missing areas of the medical images, and finally obtain a new medical image dataset. Choose any One sample, medical imaging data recorded as .

3. The medical image segmentation method based on self-supervised learning according to claim 2, characterized in that, Step S2 specifically includes the following steps: S21. Augmenting medical imaging data, for medical imaging datasets Choose any Each sample will contain medical imaging data. The medical imaging data after two data augmentations are denoted as , ; S22. Perform the augmented medical image data masking operation in step S21, and randomly select a medical image. Cut it into equal-sized pieces A medical imaging block, denoted as ,in, Indicates the first The first medical image Each image block is randomly generated with a mask sequence. ,in Indicates the first The first medical image Each block is masked, and the masked medical image data is denoted as... ,in, Contains unmasked medical image blocks.

4. The medical image segmentation method based on self-supervised learning according to claim 3, characterized in that, Step S4 specifically includes the following steps: S41. The medical image data after masking in step S22 Input the online medical image encoder constructed in step S31 And the augmented medical image data from step S21 Input the target encoder constructed in step S32 Through encoding, online vector representations are obtained respectively. With target vector representation ,in, , Represents the number of randomly selected samples. ,in, This represents the number of medical image blocks remaining after masking. , ,in, The number representing all medical image blocks. and Corresponding to online network input With target network input The medical image vector block representation, and They are online Vector representation and target Vector representation, representing medical images respectively. and Global vector representation, online Vector representation and target Vector representations are denoted as follows: Choose any Online medical imaging data Block vector representation and target The block vector representations are denoted as follows: ,in , ; S42, the online process in step S41 Block vector representation Input the online medical image projection head constructed in step S33 And the target in step S41 Block vector representation Input the medical image target projection head constructed in step S34 Through projection mapping, online low-dimensional embeddings were obtained respectively. Low-dimensional embedding of the target ,in, , ; S43. Calculate the low-dimensional contrast loss; for any medical image sample Online low-dimensional embedding As an anchor point, its corresponding target low-dimensional embedding and The samples are mutually positive, and the remaining targets are low-dimensional embeddings. and The low-dimensional contrastive loss is defined as shown in formula (2) when the samples are negative to each other: ； in, pass Normalization mapping to hypersphere, For temperature parameters, This indicates the number of randomly selected samples. Represents an exponential function; S44. Randomly initialize the vector representation of the masked medical image block in step S22, and combine it with the online vector representation in step S41. By combining these methods, a new online vector representation can be obtained. The online vector representation of medical images and the target vector representation are combined. The medical image online decoder in step S35 is used respectively. and the medical image target decoder in step S36 Get online high-dimensional embedding and target high-dimensional embedding ,in, , ,in Indicates the number of randomly selected samples. Indicates the number of medical image blocks. , ,in and Corresponding to medical imaging and of High-dimensional embedding of a medical image block and They are online High-dimensional embeddings and targets High-dimensional embedding, respectively representing medical images and Global high-dimensional embedding, for convenience, online High-dimensional embedding and target High-dimensional embeddings are denoted as follows: Choose any Online medical imaging data High-dimensional embeddings and targets High-dimensional embeddings are denoted as follows: The reconstruction loss of online high-dimensional embedding is calculated to ensure that the masked medical image blocks are accurately predicted, thereby learning the representation of small lesions in medical images through the relationship between medical image blocks. The reconstruction loss is shown in Equation (3): ； in, The number of medical imaging blocks. The number of medical image blocks that are masked, i.e. , The number of visible blocks in a medical image. Indicates the number of randomly selected samples; S45. Calculate the high-dimensional contrast loss; for any medical image sample Online high-dimensional embedding As an anchor point, its corresponding high-dimensional embedding of the target and The samples are mutually positive, and the remaining targets are high-dimensional embeddings. and The high-dimensional contrastive loss is defined as follows: (4) where the samples are mutually negative. ； S46. Calculate the total loss as shown in formula (5): ； S47. Minimize the loss using gradient descent. And through the optimizer Optimize the entire network and repeat steps S41-S46 until the set termination condition is met to obtain the trained medical image representation model.

5. The medical image segmentation method based on self-supervised learning according to claim 4, characterized in that, Step S5 specifically includes the following steps: S51. Augment the medical image data to be segmented. The data augmentation process involves scaling the medical image to a fixed size, converting the medical image data format to Tensor, and then inputting the medical image data into the trained medical image representation model to obtain the vector representation of the medical image. S52, Constructing a Medical Image Reconstructor The reducer includes a linear mapping layer and 2 Upsampling and 1 A convolutional layer of 1 is used, and upsampling and convolutional layer operations are repeated four times to achieve the final medical image resolution; S53. Input the vector representation from step S51 into the deconstructor. Obtain an integer value matrix Different tissues or organs are assigned different digital labels. The obtained digital labels are compared with medical image labels to calculate cross-entropy loss, and the reducer is optimized using gradient descent. After optimization, the medical image segmentation results are obtained; S54. Present the segmentation results of medical images to doctors to help them locate and segment structures or lesions of interest. Through the segmentation results, doctors can more accurately assess the size, shape, and location of lesions, especially the discovery of tiny lesions, thereby enabling accurate diagnosis and treatment planning for the disease.

6. A medical image segmentation system based on self-supervised learning, characterized in that, The medical image segmentation system is used to implement the self-supervised learning-based medical image segmentation method according to any one of claims 1-5, and includes a computer processor, memory and graphics processor; a medical image data storage unit; and a medical image data preprocessing unit. Medical image representation model training unit; medical image data segmentation unit.

7. A medical image segmentation system based on self-supervised learning according to claim 6, characterized in that: The medical image data storage unit stores preprocessed medical image datasets and medical image data to be segmented obtained from medical institutions, and inputs them into computer memory. The medical image data preprocessing unit performs medical image data augmentation and masking operations in step S2 on the medical image data stored in computer memory, and loads the obtained medical image samples into the graphics processor. The medical image representation model training unit uses the augmented and masked data in the graphics processor to perform steps S3 and S4 to train and obtain a medical image representation model. The medical image data segmentation unit performs step S5 on the medical image data to be segmented in computer memory to obtain the segmentation result of the medical image. The specific data processing and calculation work in all units is completed by the computer processor.

Images