Medical image segmentation method based on lightweight visual basic model

Abstract

The invention discloses a medical image segmentation method based on a lightweight visual basis model, and the method comprises the steps: achieving the cross-modal and cross-anatomical region universal feature extraction through introducing a multi-scale token aggregation and a lightweight decoder; interlayer tokens are fused in a layered mode, global semantics and local textures are considered, and small target and boundary precision is remarkably improved. According to the multi-stage field adaptive pre-training, firstly, a fine-grained structure is captured by self-distillation, reconstruction and regularization composite loss, then, teacher-student distribution is aligned by Gram matrix anchoring loss, and finally, field migration and representation enhancement under the non-labeling condition are realized through high-resolution data refining. The lightweight compression selectively removes part of self-attention and retains MLP, and reduces parameter quantity and calculation quantity on the premise of almost no precision loss, so that the model can perform real-time reasoning on edge equipment. The end-to-end process reduces the annotation dependence, shortens the fine adjustment period, can support multi-organ and multi-focus segmentation through a unified frame, and improves the clinical deployment efficiency and generalization ability.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and specifically to a medical image segmentation method based on a lightweight visual basic model. Background Technology

[0002] Medical image segmentation is a key technology for assisting disease diagnosis, surgical planning, and efficacy evaluation. Its goal is to accurately segment regions of interest such as organs, lesions, or tumors from medical images such as CT, MRI, and ultrasound. Traditional methods mostly rely on convolutional neural networks (CNNs), which perform well in local feature extraction, but are limited by the receptive field and are difficult to effectively model long-distance dependencies and global contextual information. In recent years, the Visual Transformer (ViT) has made significant breakthroughs in natural image tasks due to its powerful global modeling capabilities. However, directly applying ViT to medical image segmentation faces three major challenges: (1) High model complexity: Standard ViT contains a large number of self-attention calculations and parameters, resulting in high inference latency and large memory consumption, making it difficult to deploy on resource-constrained clinical devices (such as mobile terminals and embedded systems); (2) Significant domain differences: Natural image pre-trained models lack the low contrast, blurred boundaries, and heterogeneous textures unique to medical images, leading to a decline in transfer performance; (3) Scarcity of labeled data: Medical image labeling relies on professional physicians, which is costly and results in extremely limited high-quality labeled data, while a large amount of unlabeled data is not effectively utilized. While existing lightweight methods (such as knowledge distillation, channel pruning, and low-rank decomposition) can reduce model size, they often neglect the high accuracy requirements of medical images for dense prediction tasks during compression, leading to problems such as blurred boundaries and missed detection of small targets. Furthermore, most methods do not fully exploit unsupervised signals, making it difficult to maintain robustness in scenarios with scarce annotations. Therefore, there is an urgent need for a medical image segmentation method that balances high segmentation accuracy, strong domain adaptability, and low computational cost to meet the needs of practical clinical deployment. Summary of the Invention

[0003] The technical problems to be solved by this invention are high model complexity, lack of medical image-specific features in pre-trained models, scarcity of labeled data, and poor segmentation ability. The purpose is to provide a medical image segmentation method based on a lightweight visual basic model. Through multi-stage domain adaptive pre-training, the unsupervised representation of medical images is fully explored. Combined with a progressive structure-aware compression strategy, the method significantly reduces the computational burden of the model while maximizing the preservation of segmentation performance and improving image segmentation accuracy.

[0004] This invention is achieved through the following technical solution:

[0005] This invention provides a medical image segmentation method based on a lightweight visual foundation model, comprising the following specific steps:

[0006] We acquire multi-source medical image data, construct a pre-trained visual foundation model, introduce a multi-scale token aggregation mechanism and a lightweight decoder, extract image patch tokens of different levels from the intermediate layer of the visual foundation model encoder, and then stitch and fuse them to construct a visual foundation model for medical images.

[0007] The visual foundation model is pre-trained on a large medical image dataset using a multi-stage domain-adaptive pre-training strategy to obtain a medical vision foundation model.

[0008] Among them, multi-stage domain adaptive pre-training strategies include:

[0009] Phase 1: Construct a composite loss function that includes self-distillation loss, image patch reconstruction loss, and uniform distribution regularization loss to perform self-supervised training on the model. At the same time, capture the global structural information and local detail representation of medical images to obtain the model for the first training phase.

[0010] The second stage: Based on the model in the first training stage, an anchoring loss based on the Gram matrix is ​​introduced to constrain the differences between the student model and the teacher model on the Gram matrix of image block-level features.

[0011] The third stage involves inputting medical images with higher resolution than those in the first and second stages into the basic vision model, performing adaptive training, and outputting the basic medical vision model.

[0012] The medical vision basic model is lightweighted and compressed, and the self-attention sub-modules in some Transformer modules of the medical vision basic model are selectively removed while some multilayer perceptron sub-modules are retained.

[0013] Input the image to be segmented into the compressed, lightweight medical vision basic model, and output the segmentation result.

[0014] Furthermore, the first phase of the training process specifically includes:

[0015] The input medical image is randomly cropped to obtain a first cropped image corresponding to the global view and a second cropped image corresponding to multiple local views;

[0016] Input the first cropped image into the teacher model to obtain the first CLS classification token feature;

[0017] The second cropped image and the corresponding second cropped image after random masking are input into the student model to obtain the second CLS classification token features and the predicted features of the masked image block.

[0018] Based on the first CLS classification token features and the second CLS classification token features, a self-distillation loss is calculated to align the student model with the teacher model in the feature space, wherein the weights of the teacher model are updated by an exponential moving average of the weights of the student model.

[0019] Based on the predicted features of the masked image patch and the corresponding real image patch features, the image patch reconstruction loss is calculated, enabling the student model to learn local patch-level semantic correspondences.

[0020] Calculate the uniform distribution regularization loss for all features within a batch;

[0021] The self-distillation loss, image patch reconstruction loss, and uniform distribution regularization loss are weighted and summed to obtain the total loss. The total loss is then used to update the student model parameters, constraining the difference between the student model and the teacher model in the image patch-level feature Gram matrix.

[0022] Furthermore, the second phase of the training process specifically includes:

[0023] From the complete archive of the visual base model at a certain moment in the first stage of training, an intermediate model is extracted as the Gram teacher model.

[0024] The first-resolution global cropped image is input into the Gram teacher model, and the image patch features output by the Gram teacher model are normalized to obtain the image patch feature matrix.

[0025] The global cropped image at the second resolution is input into the student model, and the image patch features output by the model are normalized to obtain the image patch feature matrix, where the second resolution is lower than the first resolution;

[0026] Calculate the anchoring loss of the Gram matrix;

[0027] The total loss for the second stage is calculated as the weighted sum of the first-stage loss and the Gram matrix anchoring loss.

[0028] Furthermore, the third stage of the training process specifically includes:

[0029] Input medical images with higher resolution than those in the first and second stages into the visual baseline model;

[0030] A hybrid resolution cropping strategy is adopted to generate global cropped images and local cropped images of different sizes from the same batch of input image data, thereby obtaining multi-scale image block packets;

[0031] The multi-scale image patch package is input into the teacher model and the student model respectively to obtain the corresponding image patch-level features;

[0032] High-resolution adaptive training is performed based on the total loss in the second stage to update the student model parameters and obtain the medical image representation model.

[0033] Furthermore, the lightweight compression process includes:

[0034] The number of Transformer modules to be compressed and the node retention ratio of the multilayer perceptron submodules are determined based on the target computational load.

[0035] Synthetic images are used as a metric dataset to evaluate the medical vision foundation model with different compressed modules;

[0036] Based on the evaluation results, the module with the highest recoverability was selected for compression and fine-tuning.

[0037] Furthermore, obtaining the synthesized image as a metric dataset specifically includes:

[0038] Initialize a set of images composed of random Gaussian noise, and randomly assign a pseudo-label to each synthesized image;

[0039] Using a pre-trained medical vision model as a fixed teacher, the pixel values ​​of the synthesized image are directly optimized through gradient backpropagation, and the optimization objective is to minimize the comprehensive loss function.

[0040] After multiple iterations and optimizations, a set of synthetic images that can effectively activate the medical vision basic model and carry its inherent knowledge is obtained, forming a metric dataset for evaluating the recoverability of the module.

[0041] Furthermore, the minimized comprehensive loss function includes:

[0042] Cross-entropy loss is used to encourage the model to predict the synthesized image as the specified pseudo-label;

[0043] Image prior regularization term is used to constrain the visual plausibility of the synthesized image;

[0044] The image prior regularization term is composed of a weighted average of L2 norm regularization and total variation regularization.

[0045] Furthermore, the recoverability of the assessment module specifically includes:

[0046] Based on the lightweight compressed medical vision basic model, candidate compressed models corresponding to the modules to be evaluated are generated.

[0047] The candidate compressed model was fine-tuned using an unlabeled training set and a feature mimicry loss function;

[0048] The loss function during fine-tuning is defined as the mean squared error between all output tokens of the compressed model and the original model:

[0049] The metric set of the synthesized image is input into the fine-tuned candidate compression model, and the cross-entropy loss between the synthesized image and the original medical vision basic model on the output probability distribution is calculated as the recoverability score of this module.

[0050] Furthermore, the selection of the module with the highest recoverability for compression and fine-tuning specifically includes:

[0051] Based on recoverability scores, all modules to be compressed are sorted, and the module with the highest score is selected. Each module is selected as the target compression module, and their compression order is determined.

[0052] The first round of compression is performed: the self-attention submodule of the top-ranked module is removed from the original medical vision basic model, and its multilayer perceptron submodule is retained in a certain proportion to form the first intermediate compressed model;

[0053] The intermediate compression model is fine-tuned, and feature mimicry loss is used in the fine-tuning process;

[0054] The fine-tuning process only updates the network parameters from the model input layer to the next module after the current compressed module, while the parameters of subsequent uncompressed modules remain frozen;

[0055] Based on the current fine-tuned intermediate compression model, the next target module is compressed in a predetermined order, with local fine-tuning performed, until all modules are compressed. All target modules were compressed, resulting in a lightweight medical vision basic model that meets the target computational requirements.

[0056] Furthermore, after obtaining the segmentation results, the medical vision basic model is trained based on the segmentation results, including:

[0057] The first part of the loss function is the Yinzi coefficient loss, which measures the difference between the predicted segmentation result and the actual segmentation result in terms of spatial overlap.

[0058] The second part of the loss function is the cross-entropy loss, which measures the difference between the predicted segmentation result and the actual segmentation result in the pixel category probability distribution.

[0059] The two parts of the loss function are weighted and summed using preset weight coefficients to form the final segmentation loss function for model training.

[0060] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0061] Through multi-stage self-supervised pre-training, the model fully learns the global semantics and local details of medical images, especially excelling in tumor boundary and small lesion segmentation, thus improving segmentation performance. The progressive structure-aware compression strategy significantly outperforms traditional pruning methods by reducing the number of parameters and decreasing the Dice coefficient. Efficient and lightweight deployment: It makes full use of unlabeled data for pre-training, reducing the dependence on expensive annotations. Multi-scale token aggregation and high-resolution training enable the model to handle large organs and small structures simultaneously. The compressed model can achieve real-time inference on ordinary GPUs or edge devices. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0063] Figure 1 This is a medical image segmentation process in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0065] As one possible implementation method, such as Figure 1 As shown, this embodiment provides a medical image segmentation method based on a lightweight visual foundation model. By introducing multi-scale token aggregation and a lightweight decoder, it achieves universal feature extraction across modalities and anatomical regions. Layered fusion of intermediate layer tokens balances global semantics and local texture, significantly improving the accuracy of small targets and boundaries. Multi-stage domain-adaptive pre-training first captures fine-grained structures using a composite loss of self-distillation, reconstruction, and regularization, then aligns the teacher-student distribution using Gram matrix anchoring loss, and finally refines on high-resolution data to achieve domain transfer and representation enhancement in unlabeled cases. Lightweight compression selectively removes some self-attention while retaining the MLP, reducing the number of parameters and computational cost with almost no loss of accuracy, enabling the model to infer in real time on edge devices. The end-to-end process reduces label dependence and shortens the fine-tuning cycle. A unified framework can support multi-organ and multi-lesion segmentation, improving clinical deployment efficiency and generalization ability. It significantly improves the generalization, robustness, and segmentation accuracy in real-world clinical scenarios of medical image recognition, image extraction, and image semantic segmentation.

[0066] The specific implementation process of this embodiment is as follows:

[0067] Step 1: Data Preparation and Training Set Construction

[0068] This embodiment first collects multi-source medical image data from multiple medical imaging modalities (such as CT, MRI, ultrasound, etc.) to construct a hybrid training set, which includes labeled and unlabeled data. The labeled data is generated by professional radiologists who perform pixel-level fine-grained annotations on target organs or tumor regions, forming a supervised training set; the unlabeled data is used for subsequent self-supervised and semi-supervised training, constituting an unsupervised training set. This hybrid training set provides fundamental support for subsequent domain-adaptive pre-training and lightweight fine-tuning of the model.

[0069] Step 2: Construction of the Basic Medical Vision Model

[0070] This embodiment builds a visual foundation model for medical image tasks based on the DINOv3 model pre-trained on large-scale natural images. Specifically, it adopts the Visual Transformer (ViT) as the encoder backbone and introduces a multi-scale token aggregation mechanism and a lightweight decoder to adapt to the high requirements of spatial detail and boundary accuracy in medical image segmentation tasks.

[0071] Among them, the multi-scale token aggregation mechanism extracts image patch tokens from different intermediate layers of the ViT encoder (such as layers 4, 8, and 12), and performs channel splicing and cross-scale fusion of these feature tokens from different levels, thereby providing rich spatial context information for the lightweight decoder and effectively alleviating the problem of local detail loss caused by global attention in ViT in dense prediction tasks.

[0072] Step 3: Multi-stage domain adaptive pre-training

[0073] To ensure the model fully adapts to the distribution characteristics of medical images, this embodiment performs multi-stage domain-adaptive pre-training on large medical image datasets (such as MedicalNet, BraTS, LiTS, etc.) to gradually improve its ability to understand medical semantics and anatomical structures.

[0074] Phase 1: Self-supervised representation learning

[0075] In the first stage, a loss function combining self-distillation (DINO), image patch mask reconstruction (iBOT), and uniform distribution regularization (Kolco) is used to guide the model in learning global and local semantic representations of medical images. The specific process is as follows:

[0076] The input medical image is randomly cropped to generate a global view cropped image and multiple local view cropped images.

[0077] The globally cropped image is input into the teacher model (i.e., the exponential moving average EMA of the student model weights), and the locally cropped image and its masked version are input into the student model.

[0078] Calculate self-distillation loss This aligns the [CLS] token of the student model with that of the teacher model.

[0079] Calculate image patch reconstruction loss This enables the model to predict the true features of the masked image patch;

[0080] Introducing uniform distribution regularization loss To prevent feature collapse and improve feature diversity within a batch;

[0081] The self-distillation loss, image patch reconstruction loss, and uniform distribution regularization loss are weighted and summed to obtain the total loss, which is:

[0082] ;

[0083] in Preset regularization weights (e.g.) ).

[0084] Phase 2: Gram matrix anchoring and stabilization training

[0085] In the second stage, a model with strong dense feature representation ability is selected from the intermediate checkpoints saved in the first stage as the Gram teacher model. During training:

[0086] The high-resolution globally cropped image is input into the Gram teacher model to obtain the normalized image patch feature matrix. ;

[0087] Input a standard resolution image into the student model to obtain ;

[0088] The Gram matrix anchoring loss is calculated, defined as the squared Frobenius norm between the Gram matrices of the image patch features of the student model and the Gram teacher model, and its expression is:

[0089] ;

[0090] The total loss function for the second stage is a weighted sum of the loss terms from the first stage and the anchoring loss of the Gram matrix, and its expression is:

[0091] ;

[0092] in Gram loss weights (e.g.) ).

[0093] Phase 3: High-resolution adaptive fine-tuning

[0094] The third stage further increases the resolution of the input images (e.g., from 224×224 to 384×384 or higher), providing a series of medical images with significantly improved resolution as input for both the teacher and student models;

[0095] It also employs a hybrid resolution cropping strategy, which includes global and local cropped images of different scales within the same batch, to enhance the model's adaptability to multi-scale anatomical structures.

[0096] This stage retains the Gram anchoring loss and further optimizes it to ensure that the similarity structure of image patch features remains stable after resolution improvement. The final output is a high-quality dense feature map, significantly improving organ boundary segmentation accuracy. The total loss function used in this training stage remains consistent with the second stage. Continued training on high-resolution data further enhances the quality of the dense feature maps output by the model, revealing clearer anatomical structure boundaries.

[0097] The entire pre-training process employs a phased decreasing learning rate scheduling: the first phase uses a base learning rate (e.g., ...). The second stage decays to The third phase further decreased to .

[0098] After the above multi-stage training, a dedicated medical vision foundation model (MVFM) is obtained, whose backbone is based on a finely tuned DINOv3 architecture.

[0099] Step 4: Lightweight Model Compression

[0100] To meet the requirements of clinical deployment in terms of computational efficiency and memory usage, this embodiment performs lightweight compression of MVFM, specifically including the following steps:

[0101] Step 401: Compression Target Setting

[0102] Based on the computing resources of the target device (such as a mobile device or embedded GPU), determine the number of Transformer modules to be compressed and the node retention ratio of the MLP sub-modules (e.g., retain 70% of neurons).

[0103] Step 402: Construction of Synthetic Image Metric Set and Assessment of Recoverability

[0104] To avoid relying on real labeled data to evaluate compression effectiveness, this embodiment proposes a method for constructing a synthetic image metric set:

[0105] Initialize a set of random Gaussian noise images And assign pseudo-labels to each image. ;

[0106] Using a pre-trained MVFM as the teacher model, optimization is performed through gradient backpropagation. The objective function is:

[0107]

[0108] in, The cross-entropy loss encourages the model to predict the synthesized image as the specified pseudo-label. For image prior regularization, TV represents the total variation term, used to improve the rationality of image structure. and These are the weight coefficients for two regularization operations. The ℓ2 norm regularization promotes the stability of image pixel values, while the total variation regularization controls the local smoothness and structure of the synthesized image.

[0109] After multiple rounds of optimization, It becomes a knowledge distillation image set that can effectively activate model knowledge.

[0110] Based on this metric set, for each candidate compression module Conduct a recoverability assessment:

[0111] Build a compression model ;

[0112] Small-scale unlabeled medical image sets Feature mimicry loss Quick fine-tuning:

[0113]

[0114] The generated synthetic image metric set Input the fine-tuned candidate compressed model, calculate its cross-entropy loss with the original medical vision baseline model on the output probability distribution, and use this as the recoverability score for the module; the lower the loss value, the better the recoverability. The formula for calculating the cross-entropy loss is:

[0115]

[0116] in, The model represents the categories The predicted probability.

[0117] Step 403: Gradual trimming and local fine-tuning

[0118] Based on recoverability scores, all modules to be compressed are sorted, and the module with the highest score is selected. Each module is selected as the target compression module, and their compression order is determined.

[0119] Perform the first round of compression: from the original medical vision basic model The self-attention submodule of the top-ranked module is removed, and its multilayer perceptron submodule is retained in a determined proportion to form the first intermediate compressed model. That is, only one self-attention submodule of a module is removed each time, while its MLP submodule is retained (according to a preset ratio).

[0120] For the first intermediate compression model Fine-tuning is performed, and feature imitation loss is used during the fine-tuning process. MSE ensures intermediate feature alignment and only updates the network parameters from the model input layer to the next module after the current compressed module, while the parameters of subsequent uncompressed modules remain frozen.

[0121] Repeat the above steps, using the current fine-tuned intermediate compressed model as a basis, to compress the next target module in a predetermined order, and perform local fine-tuning, until all k target modules have been compressed, finally obtaining a lightweight medical vision basic model that meets the target computational requirements (e.g., FLOPs reduced by 50%). The final result is the Lightweight Medical Vision Basic Model (LightweightMVFM).

[0122] Step 5: Deployment of Medical Image Segmentation Task

[0123] The lightweight model described above was applied to a practical medical image segmentation task. After inputting the medical image to be segmented, the model extracts multi-scale features through an encoder, and then upsamples and outputs a pixel-level segmentation mask through a lightweight decoder.

[0124] The training phase uses a composite segmentation loss function:

[0125]

[0126] The first term is the Dice loss, and the second term is the cross-entropy loss. The number of categories to be divided. The total number of pixels in each category. For the first in the real mask The pixel in the first Values ​​on a class To predict the value at the corresponding position in the mask, These are the weighting coefficients used to balance the two types of losses. This loss function addresses both class imbalance and boundary accuracy, achieving excellent performance in medical segmentation tasks such as tumor, liver, and brain tissue.

[0127] Furthermore, experimental verification on multiple publicly available medical image segmentation datasets such as BraTS2023, LiTS, and ACDC shows that the method in this embodiment maintains segmentation accuracy (Dice coefficient above 0.92) while reducing the number of model parameters by approximately 45% and increasing inference speed by 2.3 times, which is significantly better than existing lightweight segmentation models (such as nnU-Net-Lite, MedFormer, etc.) and has good potential for clinical deployment.

[0128] As one possible implementation, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a medical image segmentation method based on a lightweight visual model.

[0129] As one possible implementation, this embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a medical image segmentation method based on a lightweight visual foundation model.

[0130] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical image segmentation method based on a lightweight visual fundamental model, characterized in that, The specific steps include the following: We acquire multi-source medical image data, construct a pre-trained visual foundation model, introduce a multi-scale token aggregation mechanism and a lightweight decoder, extract image patch tokens of different levels from the intermediate layer of the visual foundation model encoder, and then stitch and fuse them to construct a visual foundation model for medical images. The visual foundation model is pre-trained on a large medical image dataset using a multi-stage domain-adaptive pre-training strategy to obtain a medical vision foundation model. Among them, multi-stage domain adaptive pre-training strategies include: Phase 1: Construct a composite loss function that includes self-distillation loss, image patch reconstruction loss, and uniform distribution regularization loss to perform self-supervised training on the model. At the same time, capture the global structural information and local detail representation of medical images to obtain the model for the first training phase. The second stage: Based on the model in the first training stage, an anchoring loss based on the Gram matrix is ​​introduced to constrain the differences between the student model and the teacher model on the Gram matrix of image block-level features. The third stage involves inputting medical images with higher resolution than those in the first and second stages into the basic vision model, performing adaptive training, and outputting the basic medical vision model. The medical vision basic model is lightweighted and compressed, and the self-attention sub-modules in some Transformer modules of the medical vision basic model are selectively removed while some multilayer perceptron sub-modules are retained. Input the image to be segmented into the compressed, lightweight medical vision basic model, and output the segmentation result.

2. The medical image segmentation method based on a lightweight visual fundamental model according to claim 1, characterized in that, The first phase of the training process specifically includes: The input medical image is randomly cropped to obtain a first cropped image corresponding to the global view and a second cropped image corresponding to multiple local views; Input the first cropped image into the teacher model to obtain the first CLS classification token feature; The second cropped image and the corresponding second cropped image after random masking are input into the student model to obtain the second CLS classification token features and the predicted features of the masked image block. Based on the first CLS classification token features and the second CLS classification token features, a self-distillation loss is calculated to align the student model with the teacher model in the feature space, wherein the weights of the teacher model are updated by an exponential moving average of the weights of the student model. Based on the predicted features of the masked image patch and the corresponding real image patch features, the image patch reconstruction loss is calculated, enabling the student model to learn local patch-level semantic correspondences. Calculate the uniform distribution regularization loss for all features within a batch; The self-distillation loss, image patch reconstruction loss, and uniform distribution regularization loss are weighted and summed to obtain the total loss. The total loss is then used to update the student model parameters, constraining the difference between the student model and the teacher model in the image patch-level feature Gram matrix.

3. The medical image segmentation method based on a lightweight visual fundamental model according to claim 2, characterized in that, The second phase of the training process specifically includes: From the complete archive of the visual base model at a certain moment in the first stage of training, an intermediate model is extracted as the Gram teacher model. The first-resolution global cropped image is input into the Gram teacher model, and the image patch features output by the Gram teacher model are normalized to obtain the image patch feature matrix. The global cropped image at the second resolution is input into the student model, and the image patch features output by the model are normalized to obtain the image patch feature matrix, where the second resolution is lower than the first resolution; Calculate the anchoring loss of the Gram matrix; The total loss for the second stage is calculated as the weighted sum of the first-stage loss and the Gram matrix anchoring loss.

4. The medical image segmentation method based on a lightweight visual fundamental model according to claim 3, characterized in that, The third phase of the training process specifically includes: Input medical images with higher resolution than those in the first and second stages into the visual baseline model; A hybrid resolution cropping strategy is adopted to generate global cropped images and local cropped images of different sizes from the same batch of input image data, thereby obtaining multi-scale image block packets; The multi-scale image patch package is input into the teacher model and the student model respectively to obtain the corresponding image patch-level features; High-resolution adaptive training is performed based on the total loss in the second stage to update the student model parameters and obtain the medical image representation model.

5. The medical image segmentation method based on a lightweight visual fundamental model according to claim 1, characterized in that, The lightweight compression process includes: The number of Transformer modules to be compressed and the node retention ratio of the multilayer perceptron submodules are determined based on the target computational load. Synthetic images are used as a metric dataset to evaluate the medical vision foundation model with different compressed modules; Based on the evaluation results, the module with the highest recoverability was selected for compression and fine-tuning.

6. The medical image segmentation method based on a lightweight visual fundamental model according to claim 5, characterized in that, The acquisition of the synthetic image as a metric dataset specifically includes: Initialize a set of images composed of random Gaussian noise, and randomly assign a pseudo-label to each synthesized image; Using a pre-trained medical vision model as a fixed teacher, the pixel values ​​of the synthesized image are directly optimized through gradient backpropagation, and the optimization objective is to minimize the comprehensive loss function. After multiple iterations and optimizations, a set of synthetic images that can effectively activate the medical vision basic model and carry its inherent knowledge is obtained, forming a metric dataset for evaluating the recoverability of the module.

7. The medical image segmentation method based on a lightweight visual fundamental model according to claim 6, characterized in that, The minimized comprehensive loss function includes: Cross-entropy loss is used to encourage the model to predict the synthesized image as the specified pseudo-label; Image prior regularization term is used to constrain the visual plausibility of the synthesized image; The image prior regularization term is composed of a weighted average of L2 norm regularization and total variation regularization.

8. The medical image segmentation method based on a lightweight visual fundamental model according to claim 6, characterized in that, The recoverability of the assessment module specifically includes: Based on the lightweight compressed medical vision basic model, candidate compressed models corresponding to the modules to be evaluated are generated. The candidate compressed model was fine-tuned using an unlabeled training set and a feature mimicry loss function; The loss function during fine-tuning is defined as the mean squared error between all output tokens of the compressed model and the original model: The metric set of the synthesized image is input into the fine-tuned candidate compression model, and the cross-entropy loss between the synthesized image and the original medical vision basic model on the output probability distribution is calculated as the recoverability score of this module.

9. The medical image segmentation method based on a lightweight visual fundamental model according to claim 8, characterized in that, The process of selecting the module with the highest recoverability for compression and fine-tuning specifically includes: Based on recoverability scores, all modules to be compressed are sorted, and the module with the highest score is selected. Each module is selected as the target compression module, and their compression order is determined. The first round of compression is performed: the self-attention submodule of the top-ranked module is removed from the original medical vision basic model, and its multilayer perceptron submodule is retained in a certain proportion to form the first intermediate compressed model; The intermediate compression model is fine-tuned, and feature mimicry loss is used in the fine-tuning process; The fine-tuning process only updates the network parameters from the model input layer to the next module after the current compressed module, while the parameters of subsequent uncompressed modules remain frozen; Based on the current fine-tuned intermediate compression model, the next target module is compressed in a predetermined order, with local fine-tuning performed, until all modules are compressed. All target modules were compressed, resulting in a lightweight medical vision basic model that meets the target computational requirements.

10. The medical image segmentation method based on a lightweight visual fundamental model according to claim 1, characterized in that, After obtaining the segmentation results, the basic medical vision model is trained based on the segmentation results, including: The first part of the loss function is the Yinzi coefficient loss, which measures the difference between the predicted segmentation result and the actual segmentation result in terms of spatial overlap. The second part of the loss function is the cross-entropy loss, which measures the difference between the predicted segmentation result and the actual segmentation result in the pixel category probability distribution. The two parts of the loss function are weighted and summed using preset weight coefficients to form the final segmentation loss function for model training.

Images