A sam-med2d model quantization method and related apparatus

Abstract

This application relates to a model quantization method. Addressing the challenge of existing data-free quantization techniques failing to meet the stringent requirements of boundary accuracy and semantic consistency in medical image segmentation, this application provides a SAM-Med2D model quantization method and related apparatus. When acquiring a pseudo-dataset, the objective function is optimized by jointly applying a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of pseudo-samples and the response feature distribution of real medical images. The semantic matching loss function, based on distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable pseudo-samples to acquire anatomical structures and semantic regions. This method achieves the learning and calibration of SAM-Med2D model quantization parameters without original data, significantly reducing the computational and storage overhead of large-scale segmentation models while ensuring medical data privacy and security. This enables efficient operation and intelligent diagnosis of medical image segmentation models on resource-constrained edge devices.

Description

Technical Field

[0001] This application pertains to a model quantization method, specifically a SAM-Med2D model quantization method and related apparatus. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning models have achieved remarkable results in the field of medical image analysis, especially in key tasks such as lesion detection, organ segmentation, and assisted diagnosis, demonstrating outstanding performance and providing important support for the intelligent upgrading of healthcare. In recent years, general segmentation models, represented by the Segment Anything Model (SAM), have become the core foundation for promoting intelligent medical image processing due to their powerful visual understanding and segmentation capabilities. SAM-Med2D (SAM-Med2D, a medical 2D version of the SAM-Med2D model) is a typical medical segmentation model in this field. However, the SAM-Med2D model suffers from a massive parameter scale, with hundreds of millions of parameters, resulting in high computational and storage costs. This deficiency severely restricts the practical application of the model in scenarios with limited computing power and network conditions, such as primary healthcare institutions, mobile diagnostic terminals, and telemedicine systems, forming a core contradiction between model performance and deployment environment.

[0003] To resolve the aforementioned contradictions, model quantization technology has become the mainstream lightweight solution. By compressing floating-point parameters into low-bit integers, it can effectively reduce storage consumption and computational complexity, enabling rapid model inference. However, existing quantization methods generally rely on raw real training data for parameter calibration to determine the quantization scaling factor and bias parameters, which places strict requirements on access to and processing of real medical image data. However, medical data has extremely strong privacy and compliance restrictions, making it difficult to use freely in open environments, and cross-institutional data sharing poses significant security and ethical risks; data leaks may lead to serious legal and social problems. Therefore, data-free quantization techniques have gradually emerged, achieving parameter calibration by generating pseudo-data or utilizing prior features within the model, thus avoiding dependence on real data. However, existing data-free quantization methods are mostly suitable for classification and detection tasks, and cannot meet the stringent requirements of boundary accuracy and semantic consistency for medical image segmentation. Summary of the Invention

[0004] This application addresses the technical problem that existing data-free quantization techniques are difficult to meet the stringent requirements of boundary accuracy and semantic consistency in medical image segmentation, and provides a SAM-Med2D model quantization method and related apparatus.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application proposes a SAM-Med2D model quantization method, including: Using Gaussian noise images and random masks as initial inputs to the SAM-Med2D model, and performing iterative optimization with the goal of minimizing the objective function, a pseudo dataset is obtained. The objective function combines a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo samples and the response feature distribution of the real medical images. The semantic matching loss function, based on distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo samples to acquire anatomical structures and semantic regions. The pseudo samples serve as the input samples for the SAM-Med2D model. The SAM-Med2D model's quantization parameters were calibrated using a pseudo-dataset.

[0006] Furthermore, the expression for the distribution matching loss function is:

[0007] in, For distribution matching loss function, The differential entropy of the probability density function of the similarity distribution. For the first Patch similarity matrix of layer self-attention modules For the generated set of pseudo samples, , This represents the number of layers in the self-attention module.

[0008] Furthermore, the differential entropy of the probability density function of the similarity distribution... The calculation methods include:

[0009]

[0010] in, Let be the probability density function of the similarity distribution. ( ) is the kernel function. For sample points in the kernel function, , The number of sample points. This is the current test point.

[0011] Differential entropy of the probability density function of similarity distribution The calculation methods include:

[0012] in, For the first In the layer self-attention module and Patch similarity matrix, For the first i Each patch feature vector For the first j Each patch feature vector This represents the number of feature vectors in the patch.

[0013] Furthermore, the pseudo-positive sample label evolution mechanism includes: Through multiple iterations, the semantic consistency between the input and output of the SAM-Med2D model is gradually optimized. Each iteration performs the following steps: Step 1: Input samples into the SAM-Med2D model as pseudo samples. The SAM-Med2D model generates corresponding segmentation response results. Segmentation masks with confidence values ​​greater than a preset value in the segmentation response results are selected as pseudo positive samples and added to the pseudo label set. In the first iteration, the pseudo samples input are distribution-level samples. Step 2: Based on the mask selection mechanism of confidence and region constraints, the pseudo-label set is optimized to obtain the optimized pseudo-label set; Step 3: Using the optimized pseudo-label set as a benchmark, compare the current pseudo-sample and calculate the semantic bias according to the semantic matching loss function; and determine whether the semantic bias meets the preset requirements. If it does not meet the requirements, proceed to step 4. If it does meet the requirements, complete the iteration and use the current pseudo-sample as a semantic-level sample to obtain the anatomical structure and semantic region. The semantic matching loss function takes into account both mask overlap and category confidence. Step 4: Adjust the pseudo-samples in reverse according to the semantic bias, and use them as pseudo-samples for the input of the SAM-Med2D model in the next iteration.

[0014] Furthermore, the method for optimizing the pseudo-label set using the mask selection mechanism based on confidence and region constraints includes: Using the segmentation mask regions in the current pseudo-label set as candidate segmentation regions, the candidate segmentation regions are filtered according to the confidence threshold and the connected region size threshold of the segmentation mask to obtain the optimized pseudo-label set.

[0015] Furthermore, the expression for the semantic matching loss function is:

[0016]

[0017]

[0018] in, This represents the semantic matching loss function. This represents a balance coefficient, used to adjust the importance ratio between spatial structure and category semantics. Represents the set of prediction masks. This represents the set of pseudo-labels used as a baseline. Indicates belonging to a category c The segmentation mask, This indicates the loss of overlap between the segmented masks. Indicates the loss for mask category determination. Represents pixel ( h , w ) belongs to category c The confidence level.

[0019] Furthermore, the method for calibrating the quantization parameters of the SAM-Med2D model using a pseudo-dataset includes: Low-bit quantization is performed on the image encoder in the SAM-Med2D model, and a post-training quantization strategy is adopted when performing low-bit quantization.

[0020] Secondly, this application proposes a SAM-Med2D model quantization system, comprising: A pseudo-dataset acquisition module is used to take Gaussian noise images and random masks as the initial input to the SAM-Med2D model, and to perform optimization iterations with the goal of minimizing the objective function to obtain a pseudo-dataset. The objective function jointly employs a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo-samples and the response feature distribution of the real medical images. The semantic matching loss function, based on the distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo-samples to acquire anatomical structures and semantic regions. The pseudo-samples are the input samples of the SAM-Med2D model. The calibration module is used to calibrate the quantization parameters of the SAM-Med2D model using a pseudo dataset.

[0021] Thirdly, this application proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the SAM-Med2D model quantization method described above.

[0022] Fourthly, this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the SAM-Med2D model quantization method described above.

[0023] Compared with the prior art, this application has the following beneficial effects: This application proposes a SAM-Med2D model quantization method. When acquiring a pseudo-dataset, it optimizes the objective function by jointly using a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of pseudo-samples and the response feature distribution of real medical images. The semantic matching loss function, based on distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable pseudo-samples to acquire anatomical structures and semantic regions. This application achieves the learning and calibration of SAM-Med2D model quantization parameters without original data, significantly reducing the computational and storage overhead of large-scale segmentation models while ensuring the privacy and security of medical data. This enables efficient operation and intelligent diagnostic applications of medical image segmentation models on resource-constrained edge terminals.

[0024] This application also proposes a SAM-Med2D model quantization system, an electronic device, and a computer-readable storage medium, all of which possess all the advantages of the aforementioned SAM-Med2D model quantization method. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a SAM-Med2D model quantization method according to this application; Figure 2 This is another flowchart illustrating the SAM-Med2D model quantization method of this application; Figure 3 A visual diagram illustrating the iterative process of a pseudo-sample generated for an embodiment of this application; Figure 4 This is a schematic diagram of the visualization results of the pseudo-samples generated in the embodiments of this application; Figure 5 This is a comparison diagram between the quantization method of this application and the real data calibration method in the embodiments of this application; Figure 6 This is a comparison diagram of the quantization method and the full-precision model in the embodiments of this application; Figure 7 The images show the segmentation results of different methods obtained in the embodiments of this application. Figure 8 This is a comparison chart of the inference speed of the pre-trained model and the quantization method of this application in the embodiments of this application; Figure 9This is a schematic diagram of the SAM-Med2D model quantization system of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0030] In the description of the embodiments of this application, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0032] In the description of the embodiments of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0033] With the rapid development of artificial intelligence technology, deep learning models are increasingly widely and maturely applied in the field of medical image analysis. They have demonstrated performance advantages far exceeding traditional technologies in core clinical tasks such as lesion detection, organ segmentation, and assisted diagnosis, providing crucial support for the precision and efficiency of medical diagnosis. Among these, general segmentation models, with their powerful visual understanding and zero-shot segmentation capabilities, have become the core technological foundation for promoting intelligent medical image processing. The SAM-Med2D model is a typical medical segmentation model derived from this technology, specifically adapted to the segmentation processing needs of two-dimensional medical images, and has extremely high application value in clinical diagnostic assistance scenarios. Currently, these medical segmentation models are gradually being implemented in various levels of medical institutions, mobile diagnostic terminals, and telemedicine systems, aiming to alleviate the problem of uneven distribution of medical resources and improve diagnostic efficiency in primary healthcare and special scenarios through technological empowerment.

[0034] While the SAM-Med2D model performs exceptionally well in medical image segmentation tasks, its parameter scale typically reaches hundreds of millions due to its architecture design, resulting in extremely high computational and storage costs. In real-world clinical scenarios, many primary healthcare institutions, mobile diagnostic devices, and telemedicine systems face limitations in hardware costs and network conditions, generally lacking sufficient computing power, storage resources, and network bandwidth. These limitations prevent them from fully supporting the operation of the SAM-Med2D model, hindering its widespread adoption in primary healthcare settings and severely restricting its practical application and promotion in broader clinical settings. Consequently, the technological value cannot be fully translated into clinical benefits.

[0035] To address the conflict between model size and limited computing resources, model quantization has gradually become the mainstream solution for lightweighting deep learning models. Model quantization refers to compressing high-precision floating-point parameters in a model into low-bit integer forms. This significantly reduces model storage space and computational complexity with a slight sacrifice in model accuracy, enabling rapid inference and deployment, and adapting to the needs of low-computing-power devices. However, most existing quantization methods rely on raw, real medical image training data for parameter calibration. Statistical analysis of real data determines the quantization scaling factor and bias parameters to ensure the accuracy of the quantized model. But medical image data differs from ordinary data, possessing strong privacy and strict compliance requirements. It involves patients' personal health information and privacy, and is subject to relevant laws, regulations, and ethical norms, preventing free access, processing, and sharing in open environments. Furthermore, data sharing across medical institutions faces data security and ethical risks. Data leaks not only infringe on patient privacy but may also trigger serious legal disputes and social problems, making quantization solutions relying on real data difficult to implement in the medical field.

[0036] To reduce reliance on real medical data, data-free quantization techniques are gaining traction in research. These techniques eliminate the need for real training data, generating pseudo-data with statistical distribution characteristics of the target data or directly utilizing prior information from pre-trained models to calibrate quantization parameters, thus mitigating privacy and compliance risks associated with medical data from the outset. However, existing data-free quantization methods are primarily designed for relatively simple visual tasks such as image classification and object detection, with lower requirements for task accuracy and detail, making them unsuitable for direct adaptation to medical image segmentation tasks. Medical image segmentation demands extremely high accuracy in segmentation boundaries and semantic consistency, requiring precise differentiation between lesions and normal tissues, as well as the anatomical structures of different organs. Pseudo-data generated by existing methods often lacks the fine semantic structure and anatomical features of real medical images, failing to meet the accuracy requirements of SAM-Med2D model quantization calibration. This leads to a significant decrease in the segmentation performance of the quantized model, making it difficult to meet the rigorous requirements of clinical diagnosis and treatment. In summary, there is an urgent need for a SAM-Med2D model quantization method and device specifically designed for medical image segmentation tasks that does not require real data calibration. This method would achieve high-precision and lightweight models while strictly protecting medical data privacy and compliance, reducing deployment costs, and promoting the widespread application of this technology in various medical scenarios.

[0037] like Figure 1 The diagram shown illustrates a flowchart of a SAM-Med2D model quantization method, which may include: S101 uses a Gaussian noise image and a random mask as the initial input to the SAM-Med2D model, and performs iterative optimization with the objective function minimization as the goal, resulting in a pseudo dataset. The objective function combines a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo samples and the response feature distribution of the real medical images. The semantic matching loss function, based on the distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo samples to acquire anatomical structures and semantic regions. The pseudo samples are the input samples of the SAM-Med2D model.

[0038] This application, based on the core idea of ​​data-free quantization, utilizes the feature prior knowledge of pre-trained SAM-Med2D. Using initial noise and a mask as a basis, and through the collaborative constraint of dual loss functions, it allows pseudo-samples to gradually approximate the distribution characteristics and semantic structure of real medical images during iteration. Simultaneously, it relies on the pseudo-positive sample label evolution mechanism to complete semantic information, ultimately generating a qualified pseudo dataset. This approach generates pseudo data with both statistical distribution consistency and semantic structural integrity without relying on any real medical images, thus avoiding medical data privacy leaks and compliance risks from the source. Furthermore, through dual-loss collaborative optimization, it solves the problems of semantic ambiguity and structural distortion in traditional data-free pseudo-samples, providing high-quality data support for subsequent quantization calibration.

[0039] It's important to note that the joint distribution matching loss function and semantic matching loss function establish a unified optimization benchmark. This allows the two losses to work synergistically and mutually constrain each other within the same iterative framework, avoiding the distortion of pseudo-samples caused by optimizing a single loss. If only a single loss is optimized, either the samples only have statistical distribution features without semantic structure, or the semantic structure is complete but the distribution deviates from the real image. The joint design enables simultaneous achievement of the benchmark in both dimensions. The purpose of the distribution matching loss function is to quantify the gap, characterizing the degree of difference between the response feature distribution of the synthetic sample (pseudo-sample) and the response feature distribution of the real medical image. It defines the true boundary at the statistical level for the pseudo-sample, ensuring that the generated pseudo-samples closely match the feature distribution patterns of the real medical image, providing a foundation of samples that conform to the real statistical features for subsequent semantic optimization. The semantic matching loss function uses the distribution-level samples as the initial basis, but does not directly generate semantic information. Instead, it guides the pseudo-samples to obtain clear anatomical structures and semantic regions through a pseudo-positive sample label evolution mechanism. This precisely compensates for the core shortcoming of distribution-level samples, which only have statistical features and lack semantic information.

[0040] S102 calibrates the quantization parameters of the SAM-Med2D model using a pseudo dataset.

[0041] It should be noted that model quantization requires parameter determination through data statistics to ensure accuracy. This application utilizes a generated high-quality pseudo-dataset to simulate the calibration process of real data, providing a precise basis for converting floating-point parameters of the SAM-Med2D model to low-bit integer parameters, ensuring that the model performance does not significantly decrease after quantization. This achieves high-precision quantization parameter calibration under conditions without real data, significantly reducing the storage overhead and computational complexity of the SAM-Med2D model, making the model adaptable to low-computing-power scenarios such as primary healthcare equipment and mobile terminals. Simultaneously, it avoids the loss of segmentation accuracy caused by quantization, ensuring the reliability of the model in clinical tasks.

[0042] This application overcomes the bottleneck of traditional medical model quantization's reliance on real data. Through the synthesis of data-free pseudo-samples, it achieves high-precision quantization of the SAM-Med2D model while strictly adhering to medical data privacy protection and compliance requirements, balancing model lightweighting with segmentation performance stability. Compared to existing technologies, the pseudo-data generated in this application possesses both distribution consistency and semantic integrity, effectively solving the problems of semantic distortion and insufficient accuracy in medical segmentation tasks caused by traditional data-free quantization methods. It significantly reduces model storage and computational overhead, enabling models with hundreds of millions of parameters, which were previously difficult to run on low-computing devices, to be adapted to scenarios such as primary healthcare institutions, mobile diagnostic terminals, and telemedicine systems, significantly improving the clinical universality and practicality of medical segmentation models. Furthermore, the approach of this application can be transferred to quantization tasks of other medical image segmentation models, possessing strong scalability and providing a feasible path for the implementation of data-free quantization technology in the field of medical artificial intelligence.

[0043] This application proposes a SAM-Med2D model quantization method that does not require real data calibration. The core idea is to utilize the implicit semantic priors and distribution features in the pre-trained SAM-Med2D model to achieve data-free calibration of quantization parameters through pseudo-sample synthesis. The method comprises three key steps: distribution-level sample generation, semantic-level sample generation, and overall process integration. Distribution-level sample generation primarily reconstructs the data distribution of the SAM-Med2D model's input space, making the synthesized samples statistically approximate the distribution of real medical images. Semantic-level sample generation leverages the masking segmentation capability and semantic consistency mechanism of the SAM-Med2D model to imbue the pseudo-samples with identifiable medical structural semantics. Finally, through multi-stage adaptive optimization, both methods are fused to form a high-quality quantization calibration sample set, achieving high-precision quantization and deployment of the SAM-Med2D model without real data.

[0044] like Figure 2 The diagram shown illustrates another flowchart of the SAM-Med2D model quantization method of this application, which may specifically include: S201, sample generation at the distribution level.

[0045] To achieve medical image distribution reconstruction under data-free conditions, this application designs a distribution-matching driven pseudo-sample synthesis mechanism based on a pre-trained SAM-Med2D model. This mechanism analyzes the response characteristics of multi-layer self-attention modules in the Transformer architecture, measures the distribution difference between input noise and real images in the internal feature space, and thus guides pseudo-samples to gradually approximate the statistical characteristics of real medical images.

[0046] First, extract the output features of the self-attention modules in each layer of the SAM-Med2D model, denoted as... ,in , This represents the number of layers in the self-attention module. Subsequently, the output features are normalized along the patch dimension, and the cosine similarity between feature vectors from different patches is calculated to construct a patch similarity matrix.

[0047] in, Indicates the first In the layer self-attention module and Patch similarity matrix, Indicates the first i Each patch feature vector Indicates the first j Each patch feature vector This represents the number of patch feature vectors. The patch similarity matrix characterizes the relative structural similarity of the SAM-Med2D model across different regions.

[0048] To measure and optimize the distribution diversity of pseudo-samples, this application further introduces a patch similarity diversity entropy metric. The probability density function of the similarity distribution is obtained through kernel density estimation (KDE). And calculate its differential entropy:

[0049]

[0050] in, The differential entropy of the probability density function of the similarity distribution. Used to simplify representation , for the first Patch similarity matrix of layer self-attention modules ( ) represents the kernel function, such as the Gaussian kernel. h For bandwidth parameters, For sample points in the kernel function, For the current test point, , The number of sample points. For the generated pseudo data set, Let be the probability density function of the similarity distribution. This differential entropy reflects the diversity of patch similarity in the current layer.

[0051] Finally, the differential entropy of all layers is summed in weights to obtain the overall distribution matching loss function. :

[0052] By minimizing the distribution matching loss function, the generation process of pseudo-samples is gradually guided, making the response feature distribution of the SAM-Med2D model to pseudo-samples approximate its response feature distribution to real medical images, thereby achieving distribution alignment at the statistical level.

[0053] S202, semantic-level sample generation.

[0054] In the process of achieving data-free quantization of medical images, simple distribution alignment is insufficient to fully recover the structural semantic information in real images. To address this, this application further proposes a semantic-level sample generation method. By introducing a pseudo-positive sample label evolution mechanism, it utilizes the segmentation response features of a pre-trained model to adaptively generate pseudo-labels with real semantic structure, achieving a gradual semantic approximation of pseudo-samples. This significantly improves the medical interpretability and segmentation task adaptability of synthetic data.

[0055] (1) Generation of pseudo-tags guided by semantic information.

[0056] In traditional classification tasks, predefined category labels, such as "liver" and "kidney," can guide the semantic optimization of pseudo-samples, gradually imbuing the generated images with specific semantics. However, in medical image segmentation tasks, labels not only contain category information but also need to accurately describe the shape, size, and relative spatial position of the mask region. Therefore, direct manual pre-setting is difficult to achieve. To address this issue, this application proposes a dynamic pseudo-label generation strategy: In each iteration, the SAM-Med2D model automatically selects segmentation masks with high confidence as pseudo-positive samples based on the segmentation response of the currently generated image, and these masks are gradually superimposed to form a pseudo-label set. This strategy effectively utilizes the zero-shot segmentation capability of the SAM-Med2D model, ensuring that the semantic features of the generated samples are consistent with the prior knowledge of the SAM-Med2D model.

[0057] (2) Label evolution based on confidence and regional constraints.

[0058] To ensure the effectiveness of pseudo-labels in terms of spatial structure and category semantics, this application introduces a mask selection mechanism based on confidence and region constraints. Specifically, it first utilizes the segmentation mask regions generated by the SAM-Med2D model. As candidate segmentation regions, the average class confidence of pixels inside the segmentation mask is calculated using an auxiliary classification model, such as TransUNet.

[0059] in, Represents pixel (h , w ) belongs to category c confidence level Indicates the first t The pseudo-positive sample segmentation mask obtained in the next iteration is calculated from the average class confidence of the pixels within the mask. Indicates category c The prediction mask is determined based on the confidence threshold of the segmentation mask during the iteration process. With connected region size threshold The candidate segmentation regions are filtered in real time to remove noise and low-confidence regions, ultimately forming an optimized set of pseudo-labels. :

[0060] The optimized pseudo-label set evolves and updates continuously during the iteration, gradually enabling the pseudo-samples to possess anatomically reasonable organ boundaries and spatial structural features.

[0061] (3) Design of semantic matching loss function.

[0062] To further improve the semantic consistency of pseudo-samples, this application defines a semantic matching loss function. Taking into account both mask overlap and class confidence:

[0063]

[0064]

[0065] in, This represents the semantic matching loss function. This represents a balance coefficient, used to adjust the importance ratio between spatial structure and category semantics. Represents the set of prediction masks. This represents the set of pseudo-labels used as a baseline. Indicates belonging to a category c The segmentation mask, This indicates the loss of overlap between the segmented masks. This represents the semantic matching loss function. Represents pixel ( h , w ) belongs to category c The confidence level.

[0066] In practical applications, the optimized pseudo-label set is used as a benchmark, and the current pseudo-samples are compared according to the semantic matching loss function. The semantic deviation is calculated; and it is determined whether the semantic deviation meets the preset requirements. If it does not meet the requirements, the pseudo-samples are adjusted in reverse according to the semantic deviation and used as pseudo-samples for the input of the SAM-Med2D model in the next iteration. If it meets the requirements, the iteration is completed and the current pseudo-samples are used as samples at the semantic level. During the optimization process, the evolution of pseudo-labels and image generation alternate: the synthesized images provide the model response basis for the evolution of pseudo-labels, while the updated pseudo-labels, in turn, guide the semantic optimization of the images. As the iteration progresses, the two gradually converge to a stable state with high semantic consistency, so that the pseudo-samples not only possess the spatial structure of real images, but also reflect the semantic associations of medical organs.

[0067] This semantic-level sample generation mechanism can adaptively construct high-quality pseudo-samples that conform to medical semantic features under conditions where there is no real data.

[0068] S203, Overall Process Integration.

[0069] The SAM-Med2D model quantization method proposed in this application, which does not require real data calibration, consists of two parts: a pseudo-sample synthesis stage and a quantization calibration stage. High-quality pseudo-data with both semantic and distributional alignment is generated in the pseudo-sample synthesis stage, and this pseudo-data is used to complete the quantization parameter calibration in the quantization calibration stage, thereby achieving high-precision compression and deployment of the model in the absence of real medical images.

[0070] (1) Pseudo-sample synthesis stage.

[0071] This application uses a pre-trained SAM-Med2D model as its core, comprehensively utilizing its distributed response features and semantic prior knowledge to achieve self-supervised synthesis of random noise into pseudo-medical images. Specifically: (1.1) Optimize the design of the objective function.

[0072] With the goal of generating synthetic images, the distribution matching loss function is jointly optimized. semantic matching loss function This guides the synthesized images to simultaneously approximate real medical images in terms of both statistical features and semantic structure. The optimization objective function for generating the synthesized images is... for:

[0073] Here, β is a tradeoff coefficient used to control the relative importance of semantic and distribution alignment.

[0074] (1.2) Optimization and evolution process.

[0075] The initial input consists of a Gaussian noise image and a random mask. As optimization iterates, the SAM-Med2D model progressively updates the input image through backpropagation, ensuring that its response statistics in the multi-layer attention module align with the distribution of real medical images. Simultaneously, a pseudo-positive sample label evolution mechanism dynamically generates semantic labels, allowing pseudo-samples to gradually acquire clear anatomical structures and semantic regions. Ultimately, the resulting synthetic image achieves high fidelity in both statistical features and semantic representation, and can be directly used for SAM-Med2D model quantization and calibration.

[0076] This stage requires no real image data input; it generates a pseudo dataset solely based on the prior knowledge and feature responses of the pre-trained model, achieving complete data desensitization.

[0077] (2) Quantitative calibration stage.

[0078] After synthesizing pseudo-samples and obtaining a pseudo-dataset, this application uses this pseudo-dataset to replace real medical data for automatic calibration of SAM-Med2D model quantization parameters. Specifically, the following methods can be used: (2.1) Selection of calibration object.

[0079] Since the image encoder accounts for the largest proportion of computation in the SAM-Med2D model, this application focuses on low-bit quantization of this part, while maintaining full precision for the lightweight mask decoder, so as to balance performance and accuracy.

[0080] (2.2) Quantitative calibration.

[0081] In this embodiment, a common post-training quantization strategy is adopted for quantization calibration. The entire calibration process does not involve parameter retraining or energy-intensive reconstruction steps. It is based entirely on statistical calibration of previously generated pseudo-samples, achieving rapid compression and low-resource deployment, and significantly reducing computing and storage costs.

[0082] This application aims to address the problem of existing medical image segmentation models struggling to achieve high-precision quantization without real data. Specifically, it includes four aspects: First, establishing a data-free quantization process. Addressing the issue that existing quantization technologies generally rely on real data for distribution calibration, making them unsuitable for scenarios where medical data is restricted or prohibited from external transmission, this application mines the statistical priors and feature distribution information of pre-trained models, combined with inverse reconstruction of the model's internal activation space, to achieve self-generation of samples and self-calibration of quantization parameters. The entire process can complete quantization parameter calibration without any input of real image data. Second, constructing a sample synthesis mechanism for the SAM-Med2D model structure. Adapting to the complex structure, large number of parameters, and deep feature layers of the SAM-Med2D model, this application models the feature mapping relationship between the encoder and mask decoder of the SAM-Med2D model, utilizing local feature similarity and global semantic consistency to generate multi-layered pseudo-features, ensuring that the distribution of pseudo-samples is consistent with that of real samples. The system achieves high-quality simulation data support for quantitative calibration by matching the feature domain of real medical images. Thirdly, it realizes pseudo-sample generation and annotation evolution for segmentation tasks, meeting the stringent requirements of medical image segmentation for boundary accuracy and semantic consistency. A pseudo-positive sample label evolution mechanism is introduced in pseudo-sample synthesis, combined with local image patch similarity measurement and semantic constraint optimization, to adaptively iterate the pseudo-sample labels, making their semantic structure and spatial distribution approximate real segmentation annotations, thus improving model calibration accuracy in scenarios without data quantization. Fourthly, it completes the application of the quantized SAM-Med2D model in the medical field. For multimodal medical imaging scenarios such as CT and MRI, it constructs SAM-Med2D inference systems with low-bit quantization such as 4-bit and 8-bit, which can effectively adapt to the intelligent diagnostic needs of low-power platforms such as primary hospitals and mobile terminals, ensuring that medical data does not leave the local area and privacy is not leaked, ultimately achieving safe, efficient, and inclusive intelligent medical services.

[0083] To verify the effectiveness and practicality of the SAM-Med2D model quantization method in this application, the following systematic experimental environment and parameter configuration were constructed: (1) Pre-trained model and framework environment.

[0084] The base model used in the experiments was SAM-Med2D, which demonstrates excellent general segmentation capabilities and cross-modal transfer performance in the field of medical image segmentation. All experiments were implemented using the PyTorch deep learning framework to ensure the portability and scalability of the algorithm modules. The experiments were conducted on an NVIDIA RTX A6000 GPU, which has sufficient video memory resources to support the entire process of image synthesis and model quantization.

[0085] (2) Quantitative settings.

[0086] The quantization scheme used here is as follows: channel-level quantization for weights and hierarchical quantization for activations. Both weight and activation quantization employ asymmetric quantization, and the quantization precision is set to 4 bits (INT4). Furthermore, the quantization process only compresses the image encoder portion, while the lightweight mask decoder maintains full precision to balance model performance and computational efficiency.

[0087] (3) Pseudo-data synthesis and parameter optimization.

[0088] A pseudo-image sample is synthesized for each dataset, with a resolution of 256×256 pixels, consistent with the training input size of the SAM-Med2D model, for quantization parameter calibration. The Adam optimization algorithm is used for gradient updates during the pseudo-sample synthesis stage, with a total of 1500 iterations, the first 500 iterations used for the dynamic evolution of pseudo-positive sample labels. Semantic matching loss function weights... =0.5, weights of the distribution matching loss function =0.05, determined through grid search. This process alternates between updating the images and evolving the labels of the pseudo-samples in each iteration, gradually aligning the semantics and distribution of the pseudo-samples.

[0089] (4) Calibration strategy and model deployment.

[0090] After generating pseudo-samples, these pseudo-samples are used to calibrate the model's quantization parameters. The entire calibration phase employs a post-training quantization strategy, eliminating the need for retraining or reconstruction, thus significantly reducing computational resource consumption and accelerating the compression process. The quantized SAM-Med2D model exported after calibration can be directly deployed on medical edge terminals, portable imaging diagnostic devices, or primary hospital computing platforms, enabling secure and efficient intelligent segmentation inference.

[0091] By implementing the aforementioned embodiments, not only can the model size and computing power requirements be effectively reduced while ensuring data privacy and security, but the practicality and universality of the intelligent medical image segmentation system in edge environments are also significantly improved, giving it broad value in medical diagnosis and telemedicine applications.

[0092] like Figure 3 The image shown is a visualization of the iterative process of a pseudo-sample generated in an embodiment of this application. Figure 3 The term "iteration" in this context refers to the number of iterations. As the iterations proceed, the semantic information of the pseudo-labels and pseudo-samples gradually becomes richer. For example... Figure 4The diagram shows a visualization of multiple pseudo-samples generated through the embodiments of this application, with a resolution of 256×256 pixels. The first two rows are grayscale images, and the third row is an RGB image. This synthesis process relies solely on the pre-trained SAM-Med2D model, requiring no original medical image data or external prior information to generate high-quality pseudo-samples. The synthesized images achieve significant results in both semantic consistency and distribution alignment, providing a reliable data foundation and semantic support for subsequent model quantization and calibration.

[0093] Table 1 below shows the quantization results of the SAM-Med2D model under different data calibrations. Using the synthetic image calibration method of this application and INT4 quantization, performance comparable to the full-precision model can be achieved, which is superior to methods using real data calibration.

[0094] Table 1. Quantization results of the SAM-Med2D model under different data calibrations.

[0095] like Figure 5 The figure shows a performance comparison between the quantization method of this application and the real data labeling method. It can be seen that the quantization method of this application outperforms the real data labeling method on most datasets when real data is not required. Figure 5 The horizontal axis represents different datasets, and the vertical axis represents the Intersection over Union (IoU). The different datasets on the horizontal axis are as follows: AbdomenCT1K: Includes a dataset of 1000 abdominal CT scans.

[0096] FLARE: Fast and Low-resource Abdominal Organ Segmentation, a challenge dataset for fast and low-resource abdominal organ segmentation.

[0097] Synapse: Includes the Synapse multi-organ segmentation dataset.

[0098] ACDC: Automated Cardiac Diagnosis Challenge dataset.

[0099] MSD: Medical Segmentation Decathlon, a dataset of ten medical segmentation challenges.

[0100] ATLAS: Anatomical Tracings of Lesions After Stroke, a dataset for anatomical annotation of stroke lesions.

[0101] AutoPET: Includes an automated PET tumor segmentation dataset.

[0102] SA-Ultrasound: Segment Anything Ultrasound Dataset.

[0103] SA-Xray: Segment Anything X-ray Dataset.

[0104] ISIC: International Skin Imaging Collaboration.

[0105] EndoVis: Endoscopic Vision Challenge Dataset.

[0106] SA-Endoscopy: Segment Anything Endoscopy Dataset, an endoscopic segmentation dataset.

[0107] like Figure 6 The figure shown is a comparison between the quantization method of this application and the full-precision model. It can be seen that the quantization method of this application, under INT4 quantization, can achieve performance comparable to the full-precision model. Figure 7 The image shows a visualization of the segmentation results obtained by different methods. Figure 7 In the table, the first row corresponds to the pre-trained full-precision model, the second row corresponds to Gaussian noise calibration, and the third row corresponds to the quantization method of this application.

[0108] In addition, to verify the inference acceleration effect of the quantization method in actual deployment scenarios, the 4-bit quantization model obtained in this application was deployed on different models of NVIDIA GPUs (including Titan RTX, RTX 3090 and RTX A6000) for testing. Figure 8 The figure shows a comparison of inference speed between the pre-trained model and the quantization method of this application. Test results show that, compared with the full-precision model, the quantization method of this application achieves an inference speedup of approximately 3.4 to 3.6 times on the aforementioned hardware platforms.

[0109] like Figure 9 The diagram shown is a schematic of one possible SAM-Med2D model quantization system of this application, which may include: A pseudo-dataset acquisition module is used to take Gaussian noise images and random masks as the initial input to the SAM-Med2D model, and to perform optimization iterations with the goal of minimizing the objective function to obtain a pseudo-dataset. The objective function jointly employs a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo-samples and the response feature distribution of the real medical images. The semantic matching loss function, based on the distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo-samples to acquire anatomical structures and semantic regions. The pseudo-samples are the input samples of the SAM-Med2D model. The calibration module is used to calibrate the quantization parameters of the SAM-Med2D model using a pseudo dataset.

[0110] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of each block is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple blocks may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0111] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0112] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0113] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0114] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the SAM-Med2D model quantization method described above.

[0115] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0116] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0117] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the SAM-Med2D model quantization method described above.

[0118] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0119] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for quantizing a SAM-Med2D model, characterized in that, include: Using Gaussian noise images and random masks as initial inputs to the SAM-Med2D model, and performing iterative optimization with the goal of minimizing the objective function, a pseudo dataset is obtained. The objective function combines a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo samples and the response feature distribution of the real medical images. The semantic matching loss function, based on distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo samples to acquire anatomical structures and semantic regions. The pseudo samples serve as the input samples for the SAM-Med2D model. The SAM-Med2D model's quantization parameters were calibrated using a pseudo-dataset.

2. The SAM-Med2D model quantification method of claim 1, wherein, The expression for the distribution matching loss function is: wherein, is a distribution matching loss function, is a differential entropy of a probability density function of a similarity distribution, is the number of layers of the self-attention module, is a Patch similarity matrix of the layer self-attention module, is a generated pseudo sample set, , is the number of layers of the self-attention module.

3. The SAM-Med2D model quantization method according to claim 2, characterized in that, Differential entropy of a probability density function of a similarity distribution The method for calculating the differential entropy of a probability density function of a similarity distribution comprises: wherein, is a probability density function of the similarity distribution, is a kernel function, is a sample point in the kernel function, is a number of sample points, is a current test point;​​ Differential entropy of the probability density function of similarity distribution The calculation methods include: in, For the first In the layer self-attention module and Patch similarity matrix, For the first i Each patch feature vector For the first j Each patch feature vector This represents the number of feature vectors in the patch.

4. The SAM-Med2D model quantization method according to claim 1, characterized in that, The pseudo-positive sample label evolution mechanism includes: Through multiple iterations, the semantic consistency between the input and output of the SAM-Med2D model is gradually optimized. Each iteration performs the following steps: Step 1: Input samples into the SAM-Med2D model as pseudo samples. The SAM-Med2D model generates corresponding segmentation response results. Segmentation masks with confidence values ​​greater than a preset value in the segmentation response results are selected as pseudo positive samples and added to the pseudo label set. In the first iteration, the pseudo samples input are distribution-level samples. Step 2: Based on the mask selection mechanism of confidence and region constraints, the pseudo-label set is optimized to obtain the optimized pseudo-label set; Step 3: Using the optimized pseudo-label set as a benchmark, compare the current pseudo-sample and calculate the semantic bias according to the semantic matching loss function; and determine whether the semantic bias meets the preset requirements. If it does not meet the requirements, proceed to step 4. If it does meet the requirements, complete the iteration and use the current pseudo-sample as a semantic-level sample to obtain the anatomical structure and semantic region. The semantic matching loss function takes into account both mask overlap and category confidence. Step 4: Adjust the pseudo-samples in reverse according to the semantic bias, and use them as pseudo-samples for the input of the SAM-Med2D model in the next iteration.

5. The SAM-Med2D model quantization method according to claim 4, characterized in that, The mask selection mechanism based on confidence and region constraints, and the method for optimizing the pseudo-label set, include: Using the segmentation mask regions in the current pseudo-label set as candidate segmentation regions, the candidate segmentation regions are filtered according to the confidence threshold and the connected region size threshold of the segmentation mask to obtain the optimized pseudo-label set.

6. The SAM-Med2D model quantization method according to claim 4, characterized in that, The expression for the semantic matching loss function is: in, This represents the semantic matching loss function. This represents a balance coefficient, used to adjust the importance ratio between spatial structure and category semantics. Represents the set of prediction masks. This represents the set of pseudo-labels used as a baseline. Indicates belonging to a category c The segmentation mask, This indicates the loss of overlap between the segmented masks. Indicates the loss for mask category determination. Represents pixel ( h , w ) belongs to category c The confidence level.

7. The SAM-Med2D model quantization method according to claim 1, characterized in that, The method for calibrating the quantization parameters of the SAM-Med2D model using a pseudo-dataset includes: Low-bit quantization is performed on the image encoder in the SAM-Med2D model, and a post-training quantization strategy is adopted when performing low-bit quantization.

8. A SAM-Med2D model quantization system, characterized in that, include: A pseudo-dataset acquisition module is used to take Gaussian noise images and random masks as the initial input to the SAM-Med2D model, and to perform optimization iterations with the goal of minimizing the objective function to obtain a pseudo-dataset. The objective function jointly employs a distribution matching loss function and a semantic matching loss function. The distribution matching loss function characterizes the difference between the response feature distribution of the pseudo-samples and the response feature distribution of the real medical images. The semantic matching loss function, based on the distribution-level samples, uses a pseudo-positive sample label evolution mechanism to enable the pseudo-samples to acquire anatomical structures and semantic regions. The pseudo-samples are the input samples of the SAM-Med2D model. The calibration module is used to calibrate the quantization parameters of the SAM-Med2D model using a pseudo dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the SAM-Med2D model quantization method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the SAM-Med2D model quantization method as described in any one of claims 1-7.

Images