A method for constructing a cervical cancer screening pathological large model based on DINOv3-TCT

Abstract

The application relates to the technical field of medical image processing, and particularly discloses a cervical cancer screening pathological large model construction method based on DINOv3-TCT. The technical problem to be solved by the application is that when a general visual Transformer model is directly applied to a cervical cell instance segmentation task, there are problems of mismatch between model internal feature representation and instantiation target and high calculation redundancy. Therefore, the application proposes to transform the DINOv3 encoder: inject a learnable query token at a specific layer, and innovatively construct an instance decoupling loss term acting on an attention weight matrix to explicitly guide the encoder to realize instance-level feature separation during training. The method optimizes the internal attention distribution of the model, improves the accuracy of dense cell segmentation, and simultaneously reduces the model parameter quantity and calculation overhead through integrated design, so that the balance between high accuracy and high efficiency is realized.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for constructing a large pathological model for cervical cancer screening based on DINOv3-TCT. Background Technology

[0002] Thinprep Cytologic Test (TCT) is the primary method for early screening of cervical cancer. With the development of digital pathology and artificial intelligence technologies, automated cell analysis models based on deep learning have become an important research direction for improving screening efficiency and consistency. In recent years, large-scale visual pre-training techniques, represented by VisionTransformer (ViT) and its derivative models (such as DINOv3), have shown significant potential in the field of medical image analysis due to their powerful general feature extraction capabilities. Existing technical solutions typically employ a pre-training-fine-tuning paradigm, utilizing a ViT model pre-trained on natural images as an encoder, adapting it with a specially designed decoder (such as a Transformer-based or convolutional neural network structure), and completing specific medical image segmentation tasks through supervised fine-tuning.

[0003] However, directly applying the aforementioned general architecture to TCT cervical cell instance segmentation still presents significant technical shortcomings. First, the separate encoder and decoder architecture leads to loss and redundancy of feature information during cross-module transmission. The high-level semantic features output by the encoder require a complex decoding process to reconstruct pixel-level masks, introducing numerous additional parameters and computational overhead, and making it difficult to guarantee the integrity of instance-level features. A more fundamental flaw lies in the lack of an effective instance decoupling mechanism in the core feature encoding stage of existing methods. Whether based on convolutional neural networks or standard Transformers, the encoder's design aims to extract hierarchical features of the overall image, rather than explicitly modeling and separating multiple independent target instances within the image. Second, existing methods lack effective guidance for instance perception during the feature encoding stage. The learning objectives of general pre-trained models (such as DINOv3) deviate from the dense instance segmentation task; their encoding process does not explicitly model the discriminability between different cell instances. Specifically, in standard attention mechanisms, all tokens (including later-added task queries) equally focus on the global context, making it difficult for the model to spontaneously bind different feature channels or tokens to different physical instances. Therefore, even with the introduction of attention mechanisms, the model still struggles to achieve early decoupling and full representation of instance features within the encoder when dealing with complex scenarios such as cell overlap and adhesion. This limits further improvement in the final segmentation accuracy and also affects the feasibility of deploying the model in environments with limited computing resources.

[0004] Therefore, this invention proposes a method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT. Summary of the Invention

[0005] This invention provides a method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT. Addressing the fundamental problem of the lack of instance decoupling capability within existing encoders, it proposes a novel mechanism of synergistic early injection and attention supervision. Early injection refers to introducing a set of learnable query tokens in the middle layers (rather than the end) of the encoder, giving it sufficient network depth to fuse and evolve with image features. Attention supervision innovatively constructs an instance decoupling loss term that directly acts on the multi-head self-attention weight matrix within the encoder. During training, this loss term forces the attention distribution of different query tokens to tend towards dispersion or concentration, thereby guiding each query token to autonomously and stably focus on different potential cell instance regions in the image during the feature encoding stage (i.e., forward propagation). This synergistic mechanism is equivalent to implanting an instance-aware controller within the encoder, realizing a paradigm shift from general feature extraction to instance-specific feature binding.

[0006] This invention provides a method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT, comprising: Construct an encoder-mask integrated segmentation network; the integrated segmentation network uses the visual Transformer model DINOv3 as the encoder, and after the Lth Transformer block of the DINOv3 encoder, a set of learnable query tokens is concatenated and injected to form a hybrid token sequence, where L is an integer greater than 0 and less than N, and N is the total number of Transformer blocks of the DINOv3 encoder. The injection of query tokens aims to achieve early binding of instance features in collaboration with subsequent encoding layers. In the L+1 to Nth Transformer blocks of the DINOv3 encoder, multi-head self-attention computation is performed on the mixed token sequence, and an instance decoupling loss term is constructed based on the attention weight matrix. The instance decoupling loss term is used to guide the attention weight distribution of different query tokens to produce differences during training, so that each query token focuses on different cell instances in the image. Based on the overall loss function that includes an instance decoupling loss term and cervical cell labeled data, the integrated segmentation network is trained end-to-end to obtain a large pathological model that can directly output instance segmentation masks.

[0007] Preferably, the instance decoupling loss term is constructed as follows: Calculate the attention weight matrix of the output of the L+1 to Nth Transformer blocks, and extract the attention weight distribution of each query token to all image block tokens; Based on the extracted attention weight distribution, the Jensen-Shannon divergence between the distributions corresponding to different query tokens is calculated, and the negative value of the divergence is used as the instance decoupling loss term. Alternatively, calculate the entropy of the attention weight distribution for a single query token and use the sum of the entropy values ​​as the instance decoupling loss term.

[0008] Preferably, an instance-aware self-supervised pre-training step is included before end-to-end training: The DINOv3 encoder is pre-trained based on the unlabeled TCT image dataset. The pre-training adopts a student-teacher network framework and defines a token-level alignment loss. The token-level alignment loss requires that the query token features extracted from different augmented views of the same original image maintain a one-to-one correspondence in the feature space.

[0009] Preferably, the token-level alignment loss is defined as follows: Use the optimal transmission algorithm or the Hungarian matching algorithm to establish the correspondence between the query token feature set output by the student network and the query token feature set output by the teacher network. Calculate the feature distance between corresponding query token pairs and construct a token-level alignment loss based on the average feature distance of all pairs.

[0010] Preferably, the number M of learnable query tokens injected is set according to the average cell count of cervical cell smears in the target dataset.

[0011] Preferably, after obtaining the trained large pathological model, the step of generating an instance segmentation mask is also included: The query token is input into a lightweight mask generation head; the mask generation head maps the query token to a mask prototype vector through linear projection, calculates the dot product of the prototype vector and the image patch token features output by the encoder to generate a low-resolution mask map, and obtains the instance segmentation mask of the final resolution through bilinear upsampling.

[0012] Preferably, after obtaining the trained large pathological model, the process also includes a cell classification step: Input the query token into a parallel classification head; the classification head outputs the pathological category probability distribution of the cell instance represented by each query token.

[0013] Preferably, a parameter freezing strategy is adopted during end-to-end training: Freeze the parameters of the first to Lth Transformer blocks of the instance-aware self-supervised pre-trained DINOv3 encoder; Supervised fine-tuning is performed only on the L+1 to Nth Transformer blocks, the learnable parameters related to the instance decoupling loss term, and the parameters of the mask generation head and the classification head.

[0014] Preferably, the overall loss function is a weighted sum of the instance decoupling loss term and the standard segmentation loss term; the standard segmentation loss term includes the Dice loss of the segmentation mask and the cross-entropy loss.

[0015] Preferably, it also includes model validation and optimization steps: The trained pathological model was evaluated using an independent validation set, and the average precision and average recall of the model on the cell instance segmentation task were calculated. Based on the evaluation results, the weight coefficients of the instance decoupling loss term, the number of query tokens M, or the injection layer L of query tokens in the encoder are adjusted in reverse to optimize model performance and complete the large model construction process.

[0016] The beneficial effects of this invention compared to the prior art are as follows: First, at the model architecture level, this method achieves structural simplification and efficiency improvement by constructing an integrated encoder-mask network. Traditional segmentation models employ an encoder-independent decoder architecture, which inherently suffers from information loss and computational redundancy during feature transfer and reconstruction. This invention eliminates the need for a separate, parameter-intensive decoder module. Instead, it directly injects learnable query tokens into the encoder (after the Lth block of DINOv3) and performs joint attention calculations with image features in subsequent encoding layers. This integrated design directly reduces the total number of model parameters and the depth of the computation graph. Specifically, this results in significantly reduced model file size, lower latency for single-image inference on GPUs and other hardware, and optimized memory footprint, thereby greatly improving the feasibility and real-time performance of the model on computationally limited clinical pathology workstations or edge devices.

[0017] Second, at the level of internal model representation and training mechanisms, this method enhances the model's ability to distinguish dense instances by introducing and optimizing an instance decoupling loss term. Existing methods typically only apply supervision at the final output level of training, lacking targeted guidance for the model's intermediate attention mechanisms. The core innovation of this invention lies in actively constructing an instance decoupling loss term based on the attention weight matrix during training (e.g., maximizing the Jensen-Shannon divergence of attention distributions for different query tokens). This loss term acts as a continuous, fine-grained optimization signal, forcing each query token to focus on and bind to different potential cell instance regions within the image in the feature space. This mechanism enables the model to possess powerful instance perception and feature decoupling capabilities even during the encoding stage. Its direct and quantifiable technical effect is that, when facing challenges such as cell overlap, adhesion, and morphological variation that are common in TCT images, the instance segmentation mask output by the model has been significantly improved in terms of boundary clarity, instance integrity, and anti-adhesion performance. Specifically, this is reflected in a significant improvement in core segmentation metrics such as mean precision (mAP) and mean recall on the standard test set, as well as a significant reduction in segmentation errors (such as instance merging and background misjudgment).

[0018] Third, the aforementioned effects of simplified architecture and enhanced representation are not isolated but mutually reinforcing. The lightweight architecture reduces optimization difficulty, allowing the model to focus more on learning key discriminative features; while the powerful instance decoupling capability ensures that model performance does not degrade while parameters are simplified. Therefore, the final technical result obtained by this method is a specialized pathological analysis model that achieves an excellent balance between accuracy and efficiency. It inherits the powerful general visual representation capabilities of DINOv3, and through targeted modifications, it is particularly suitable for the specific clinical task of cervical cell instance segmentation, which requires high accuracy and high efficiency. This provides a core model foundation for building a next-generation automated and widely deployable cervical cancer screening assistance system.

[0019] Through the above scheme, the present invention achieves the decoupling of the main instances during the encoding stage. The subsequent mask generation head only needs to perform a light linear projection and upsampling, thereby achieving structural integration and computational lightweighting while maintaining high precision.

[0020] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a diagram of the encoder-mask integrated segmentation network architecture based on DINOv3-TCT in an embodiment of the present invention. Figure 2 This is a schematic diagram of the working mechanism of the decoupling loss term in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the complete large model construction process in an embodiment of the present invention. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] To fully illustrate the specific embodiments of the present invention, verify its technical effects, and demonstrate its advancement over the prior art, a detailed and data-driven example is provided below.

[0025] First, regarding the experimental setup and baseline comparison.

[0026] This embodiment uses the publicly available cervical cell liquid-based thin-layer cytology image dataset SIPaKMeD. This dataset contains 966 images with a total of 4049 cell instances, each with pixel-level segmentation and cell category annotations. To ensure the fairness of the evaluation, the training, validation, and test sets were randomly divided in a 7:1:2 ratio.

[0027] To objectively evaluate the performance of this invention, the following three representative prior art types were selected as comparative baselines: the first type is the classic instance segmentation model MaskR-CNN (a representative convolutional neural network based on region proposal); the second type is the advanced medical image segmentation model Hover-Net (specifically designed for instance segmentation of histopathological images); and the third type is a method based on fine-tuning of a general visual Transformer model, namely, a standard fine-tuning scheme that uses DINOv2-ViT-Base as an encoder and connects to a Deformable DETR decoder, referred to as the DINOv2+Detr scheme.

[0028] The specific configuration of the method in this invention is as follows: the backbone network adopts the DINOv3-Base (ViT-B / 14) model; the key hyperparameters are set as follows: query token injection layer L=8; number of query tokens M=64; instance decoupling loss term weight λ=0.1; the Dice loss weight α and cross-entropy loss weight β in the standard segmentation loss are both set to 1.0. All comparison methods use the same AdamW optimizer, are trained on the same training set for 150 epochs, and the input images are uniformly scaled to 1024×1024 resolution. In the instance-aware self-supervised pre-training stage of the method in this invention, an additional 50,000 unlabeled TCT images are used.

[0029] Second, regarding the results of the quantitative performance comparison.

[0030] The core performance metrics of each method on the cell instance segmentation task are compared below on independent test sets: The classic instance segmentation model Mask R-CNN (with ResNet-50 as its backbone) has an average precision (AP@0.5:0.95) of 71.3%, an average recall (AR@100) of 74.8%, and an aggregated Dice coefficient of 0.816.

[0031] The advanced medical image segmentation model Hover-Net has an average precision of 74.6%, an average recall of 77.9%, and an aggregated Dice coefficient of 0.832.

[0032] The general vision Transformer fine-tuning solution DINOv2+Detr has an average precision of 76.8%, an average recall of 79.5%, and a converged Dice coefficient of 0.841.

[0033] The method of this invention achieved an average precision of 81.2%, an average recall of 83.7%, and a Dice coefficient of 0.872.

[0034] In terms of model efficiency, MaskR-CNN has 44.1 million parameters and an inference speed of 12.5 FPS on a standard GPU; Hover-Net has 32.9 million parameters and an inference speed of 15.2 FPS; the DINOv2+Detr scheme has 112.5 million parameters and an inference speed of 8.7 FPS; while the method of this invention has 88.4 million parameters and an inference speed of 18.3 FPS.

[0035] Third, regarding the analysis of technical effects.

[0036] The aforementioned quantitative data confirms the significant technical effects of this invention: in terms of segmentation precision and recall, compared to the suboptimal DINOv2+Detr baseline method, this invention achieves an absolute improvement of 4.4 percentage points in average precision and a relative improvement of approximately 5.7%; the average recall also reaches the highest level. This demonstrates that this invention can detect cell instances more completely with a lower false negative rate. Simultaneously, this invention achieves the highest aggregated Dice coefficient. Combined with qualitative observations of the segmentation results (such as clear separation of overlapping cells in the image), this result directly proves that this invention effectively solves the mask adhesion problem in the segmentation of dense cervical cell instances through the synergistic mechanism of early query token injection and instance decoupling loss term.

[0037] In achieving a balance between accuracy and efficiency, this invention significantly improves accuracy while having fewer model parameters than the DINOv2+Detr baseline, which also uses the Transformer architecture, and its inference speed is faster than all comparison methods. This is thanks to the integrated network design that eliminates redundant decoders, achieving a synergistic optimization of high accuracy and high efficiency.

[0038] Fourth, regarding the contribution of core components in ablation experiments.

[0039] To verify the necessity of each core component of this invention, we conducted ablation experiments using the complete model as a benchmark. The average accuracy of the complete model (including query token injection, instance decoupling loss, and self-supervised pre-training) was 81.2%. When the instance decoupling loss term was removed (i.e., λ=0), the average accuracy dropped to 77.1%, demonstrating that this loss term is crucial for guiding the model's internal attention to achieve instance decoupling. When the self-supervised pre-training step was removed, the average accuracy dropped to 79.5%, demonstrating that this pre-training stage can effectively utilize unlabeled data to improve model robustness. If the query token is injected in the last layer of the encoder (i.e., L=11), the average accuracy drops to 78.9%, demonstrating that the early injection strategy provides sufficient network optimization depth for instance feature decoupling. If the integrated design is abandoned and a separate Transformer decoder is used instead, the average accuracy drops to 78.0%, and the number of model parameters increases, demonstrating the superiority of the integrated design. The ablation experiments show that each design element of this invention contributes positively to the final performance, and their combination produces a synergistic effect.

[0040] like Figure 1 , Figure 2 , Figure 3 As shown, this invention provides an embodiment of a method for constructing a large pathological model for cervical cancer screening based on DINOv3-TCT, including: Step S101: Construct an integrated encoder-mask segmentation network.

[0041] Specifically, the integrated segmentation network uses the visual Transformer model DINOv3 as the encoder. The DINOv3 encoder consists of multiple sequentially connected Transformer blocks. In a specific configuration of this embodiment, the DINOv3-Base model is used, with a total of 12 Transformer block layers. Structural expansion is performed after a certain Transformer block; this layer position is denoted as L, and is a configurable hyperparameter. Experiments have shown that L values ​​in the later parts of the encoder (e.g., layers 6 to 10) yield better results; in this embodiment, L=8 is preferred. The expansion method involves concatenating a set of learnable query tokens to the end of the image block token sequence output by the Lth block. The query token is a learnable parameter matrix; its number of rows represents the total number of query tokens, and its number of columns represents the feature dimension, which is the same as the feature dimension of the image block tokens, for example, 768 dimensions. After concatenation, a new hybrid token sequence is formed, the length of which is the original number of image block tokens plus the total number of query tokens. The purpose of this design is to introduce the instantiated query concept into the encoding process in advance, providing a carrier for subsequent instance decoupling, such as... Figure 1 As shown.

[0042] Step S102: Perform joint coding with instance decoupling constraints.

[0043] The hybrid token sequence obtained in step S101 is input into the L+1 to the last Transformer block of the DINOv3 encoder. In these blocks, query tokens and image block tokens interact within the same multi-head self-attention mechanism. To achieve instance decoupling in this process, this invention innovatively introduces an instance decoupling loss term. This loss term does not act on the network output but is directly calculated and optimized based on the attention weight matrix generated from the L+1 to the last Transformer block. Specifically, during the forward propagation of training, these attention weights are monitored; during backpropagation, the gradient generated by this loss term directly affects the learning of the attention weights, thereby guiding the attention focus of different query tokens to automatically disperse to different semantic regions of the image, avoiding multiple query tokens focusing on the same cell instance. This is the core mechanism for achieving instance decoupling within the encoder, and its working principle is as follows: Figure 2 As shown.

[0044] Step S103: Perform end-to-end training to obtain the final model.

[0045] Define an overall loss function that equals the product of the standard segmentation loss, the instance decoupling loss term, and a balanced weight. The standard segmentation loss can be, for example, the Dice loss and the cross-entropy loss. The unified segmentation network is trained end-to-end using a dataset with cervical cell instance-level annotations. By optimizing the overall loss function, the network learns both accurate segmentation and internal instance decoupling. Upon training, a large-scale pathological model is obtained, which can directly input TCT images and output a segmentation mask for each cell instance. The complete construction process is as follows: Figure 3 As shown.

[0046] To force the model to focus on different cell instances during training and achieve early feature decoupling, a further instance decoupling loss term is proposed, specifically: Step S201: Extract the attention distribution.

[0047] During training iterations, the multi-head self-attention weight matrix of the output from the (L+1)th to the last Transformer block is obtained. The dimension of this matrix is ​​determined by the batch size, the number of attention heads, and the length of the mixed token sequence. For each query token, the attention weight vector corresponding to all image block tokens is extracted from this matrix. This vector is then subjected to Softmax normalization to obtain the attention probability distribution of the query token.

[0048] Step S202: Calculate the instance decoupling loss.

[0049] This invention provides two preferred calculation methods: Method 1 (Based on Distribution Difference): Calculate the Jensen-Shannon divergence between the attention probability distributions of any two different query tokens. First, calculate the intermediate distribution of these two distributions, i.e., their average. Then, the Jensen-Shannon divergence is equal to half multiplied by (the KL divergence of the first distribution relative to the intermediate distribution plus the KL divergence of the second distribution relative to the intermediate distribution). Define the instance decoupling loss term as the negative of the sum of the Jensen-Shannon divergences between all different query token pairs. Minimizing this loss is equivalent to maximizing the difference in attention distributions between different query tokens.

[0050] Method 2 (Based on Distribution Concentration): Calculate the entropy of the attention probability distribution for each query token. This entropy is equal to the negative summation of all elements in the distribution (each element multiplied by its logarithm). Define the instance decoupling loss term as the sum of the entropy values ​​of the attention distributions for all query tokens. Minimizing this loss is equivalent to encouraging the attention of each query token to be as concentrated as possible.

[0051] These two approaches achieve the same technical objective from different perspectives. In a specific implementation, the first approach (based on Jensen-Shannon divergence) is more effective in promoting attentional separation in the early stages of training.

[0052] To leverage massive amounts of unlabeled data to inject prior knowledge of instance discovery into the model and address the problem of scarce annotations, a further instance-aware self-supervised pre-training step is proposed before end-to-end training: Before step S101, a pre-training phase is performed (e.g., Figure 3 (As shown in Phase A). This phase does not rely on any manual annotations and only uses a large number of unannotated TCT images.

[0053] Step S301, construct the framework. Construct student and teacher networks with identical structures but independent parameters. Their backbones are both DINOv3 encoders containing query token injection structures (i.e., the extended encoder described in step S101). The parameters of the teacher network are updated using an exponential moving average of the student network parameters. The update rule is: the new teacher network parameter equals the momentum coefficient multiplied by the old teacher network parameter, plus (1 minus the momentum coefficient) multiplied by the student network parameter, where the momentum coefficient is typically 0.99 or higher.

[0054] Step S302: Generate views and extract features. Sample an image from the dataset and apply two different random augmentations to it, resulting in two views: a student view and a teacher view. Input the student view into the student network and the teacher view into the teacher network.

[0055] Step S303: Define and calculate the token-level alignment loss. The core objective of this loss is to align the features of all query tokens output by the student network and the teacher network on an instance-by-instance basis in the feature space.

[0056] To accurately establish cross-view instance correspondences during self-supervised pre-training and enhance the robustness of feature alignment, a token-level alignment loss is further proposed, including: Step S401: Establish the correspondence. This is crucial for achieving token-level alignment. Obtain the query token feature sets output by the student network and the teacher network, respectively. Each set contains the same number of feature vectors as the total number of query tokens, and each feature vector has a dimension of D. Since the two views have identical content but different appearances, and the initialization order of the query tokens is random, direct index matching is unreasonable. Therefore, this invention employs the optimal transmission algorithm or the Hungarian matching algorithm to calculate a cost matrix based on feature similarity. The number of rows and columns in this matrix equals the total number of query tokens. Each element of this cost matrix equals 1 minus the cosine similarity between the corresponding student network feature and the teacher network feature. Then, an optimal one-to-one correspondence mapping is solved, which maps each feature index of the student network to a feature index of the teacher network, minimizing the sum of costs for all corresponding indices. This mapping indicates that a query token feature of the student network and the corresponding query token feature of the teacher network most likely correspond to the same instance.

[0057] Step S402: Calculate the alignment loss. For each pair of matching query token features found in step S401, calculate the squared Euclidean distance between their features. The token-level alignment loss is defined as the average of the squared feature distances of all matching pairs. By minimizing this alignment loss, the model learns, in unsupervised mode, to associate the same instance in different views and encode it into features of the same query token, thus gaining a strong instance discovery prior.

[0058] To ensure that the model capacity matches the actual cell number distribution and to avoid query token redundancy or insufficiency, a further proposal is made to determine the number of learnable query tokens injected through splicing, based on the average cell number statistics of cervical cell smears in the target dataset.

[0059] Step S501, Data Statistics. Before model building, statistical analysis is performed on the target TCT dataset used for training and deployment. Specifically, a sufficient number of samples are randomly selected, the number of cell instances in the clear field of view of each smear is counted, and their mean and standard deviation are calculated.

[0060] Step S502, set the quantity value. The total number of query tokens is set to the average value mentioned above plus a coefficient k multiplied by the standard deviation mentioned above, where the coefficient k is an adjustable constant, typically ranging from 1.0 to 2.0. For example, in a specific dataset, if the average is approximately 50 and the standard deviation is approximately 10, and the coefficient k is set to 1.5, then the total number of query tokens is set to 65. The purpose of this method is to ensure that the number of instance slots owned by the model has a high probability of covering the number of cells in most images, while avoiding resource waste. The total number of query tokens is a hyperparameter that remains fixed during training and inference.

[0061] To efficiently decode high-level instance features into pixel-level segmentation masks while preserving boundary details, a further step is proposed: after obtaining the trained large pathological model, the step of generating instance segmentation masks is also included. During the model inference phase, the following steps are performed: Step S601, Linear Projection. The instance-aware query token matrix, the final output of the encoder, is input into a lightweight mask generation head. This generation head is first an unbiased linear projection layer that maps the query tokens from the feature dimension D to another dimension C, obtaining a mask prototype vector matrix, where dimension C is typically equal to the feature dimension of the image patch tokens.

[0062] Step S602: Generate the mask logic graph. Simultaneously, obtain the image patch token feature matrix output by the encoder. Calculate the matrix product between the mask prototype vector matrix and the transpose of the image patch token feature matrix to obtain the low-resolution mask logic graph matrix. Each row of this matrix corresponds to a query token, and each column corresponds to an image patch position. The elements represent the relevance score between the corresponding query token and the corresponding image patch position.

[0063] Step S603, Upsampling and Activation. Since the number of image patches is much smaller than the number of original pixels, the above mask logic diagram is low-resolution. It is upsampled to the spatial resolution of the original input image through a bilinear upsampling operation. Then, typically along the instance dimension, the Softmax activation function is applied to obtain the final instance segmentation mask.

[0064] To achieve accurate segmentation and automatic classification of the pathological type for each cell instance, a further step is proposed: after obtaining the trained large-scale pathological model, a cell classification step is also included. In parallel with mask generation, the classification step is performed: Step S701, Classification Prediction. Input the same set of instance-aware query token matrices into a parallel classification head. This classification head typically consists of a linear layer and a softmax activation function. The output is a probability matrix with rows equal to the total number of query tokens and columns equal to the total number of pathology categories. Each row of this matrix corresponds to the probability distribution of a query token belonging to each category.

[0065] Step S702, result association. Each query token corresponds to both a segmentation mask and a classification probability distribution. Therefore, for each output instance, the user can simultaneously obtain its accurate pixel-level contour and pathology type prediction, achieving true instance-level analysis.

[0066] To efficiently utilize pre-trained features during the fine-tuning phase, prevent catastrophic forgetting, and accelerate convergence, a parameter freezing strategy is further proposed during end-to-end training. During the end-to-end supervised fine-tuning in step S103, a phased parameter update strategy is adopted: Step S801: Freeze the underlying parameters. Completely freeze the parameters of the 1st to Lth Transformer blocks in the instance-aware self-supervised pre-trained DINOv3 encoder. These parameters have already learned rich general visual features and preliminary instance-aware capabilities. Freezing them protects this knowledge from being destroyed by supervised fine-tuning and significantly reduces the number of parameters that need to be updated.

[0067] Step S802, fine-tune the high-level layers and task heads. Gradient updates and learning are performed only on the following parameters: (1) parameters of the L+1th to the last Transformer block; (2) learnable parameters that may be involved in the instance decoupling loss term itself; (3) all parameters of the mask generation head and classification head. This strategy ensures the efficiency and stability of training.

[0068] To balance the instance decoupling objective and the final segmentation accuracy objective during training and achieve synergistic optimization, a weighted sum of the instance decoupling loss term and the standard segmentation loss term is proposed. The standard segmentation loss term includes the Dice loss of the segmentation mask and the cross-entropy loss.

[0069] Step S901, define the standard segmentation loss. For the predicted instance segmentation mask and the ground truth annotation, calculate: Dice loss: The value of this loss function is equal to 1 minus (the sum of the products of the corresponding elements of the predicted mask and the true mask) divided by (the sum of the elements of the true mask plus the sum of the elements of the predicted mask plus a very small constant).

[0070] Cross-entropy loss: This loss function value is equal to the inner product between the negative true label and the logarithm of the predicted probability.

[0071] The standard segmentation loss is usually a weighted sum of the two losses mentioned above. That is, the standard segmentation loss is equal to the weight α multiplied by the Dice loss plus the weight β multiplied by the cross-entropy loss, where the weights α and β can usually be set to 1.0.

[0072] Step S902, Combine the total loss. The final overall loss function used for backpropagation is: the overall loss equals the standard segmentation loss plus the balancing weight λ multiplied by the instance decoupling loss term. Here, the balancing weight λ is an important hyperparameter used to balance segmentation accuracy and instance decoupling strength. Experiments show that adjusting the balancing weight λ within the range of 0.05 to 0.2 yields good results. By adjusting the balancing weight λ, the emphasis placed on the internal attention decoupling mechanism during optimization can be controlled, allowing the two objectives to work synergistically to improve the final performance.

[0073] To construct a complete model production closed loop with quantifiable performance and iteratively optimized parameters, ensuring model optimality, further steps including model validation and optimization are proposed. After model training is complete, the evaluation and optimization phase begins: Step S1001, Performance Evaluation. The trained pathology model is systematically evaluated using an independent validation set that has never been used in training. Input validation set images; the model outputs segmentation masks and classification results. Calculate key quantitative metrics such as mean precision, mean recall, classification accuracy, and F1 score.

[0074] Step S1002, Hyperparameter Optimization and Analysis. Based on the evaluation results of step S1001, diagnostic analysis is performed. If the model performance does not meet expectations, a feedback optimization loop can be initiated: If instance segments are severely stuck together, you can try increasing the balancing weight λ of the instance decoupling loss term.

[0075] If the number of instances in some images exceeds the number of model outputs, try increasing the total number of query tokens.

[0076] If the model feels that feature fusion is insufficient, you can try adjusting the query token injection layer L.

[0077] After adjustment, the fine-tuning starting from step S103 or the complete process starting from step S301 can be re-executed, and then evaluated again to form an iterative closed loop until the model performance meets the predetermined requirements, and finally the entire pathological model construction process is completed.

[0078] In other embodiments, the technical solution of the present invention can also be embodied in the following forms: As a deployable large-scale pathology model product: The core achievement of this invention is a trained large-scale pathology model that can be used for practical reasoning. This model product includes a restructured DINOv3 encoder, a lightweight mask generation head integrated with it, and an optional parallel classification head.

[0079] The modified encoder's core feature is that it persistently incorporates a set of query token parameters learned during training into the output of the Lth Transformer block (e.g., the 8th layer). These query tokens are no longer variable during training but are part of the model's fixed parameters.

[0080] The model's intrinsic characteristics: After training using the method described in claim 1, especially with optimization of the instance decoupling loss term, the query tokens in this model product possess specific, decoupled internal representations. When a new TCT image is input, during forward propagation, different query tokens spontaneously and stably focus on different semantic regions in the image, thereby directly outputting separate cell instance masks. This instance-aware capability is embedded in the model parameters.

[0081] Product Form: The model can be stored and distributed as a file (such as .pth, .onnx format) and can be loaded into any compatible inference framework for use.

[0082] As an electronic device: The present invention can also be implemented as an electronic device. This electronic device includes at least one memory, at least one processor, and a computer program stored in the memory and executable on the processor.

[0083] The memory is used to store instructions and data, such as the aforementioned pathological large model file and the TCT image data to be analyzed. When executing the program, the processor is configured to perform all or part of the steps of the cervical cancer screening pathological large model construction method based on DINOv3-TCT as described in any of the embodiments above, or to load and run the aforementioned pathological large model product to perform instance segmentation and classification analysis on the input TCT images.

[0084] The electronic device may be, but is not limited to: a workstation computer in a pathology department, a server deployed in the cloud, a microscope imaging system with computing capabilities, or a mobile diagnostic device.

[0085] As a computer-readable storage medium: The present invention can also be implemented as a computer-readable storage medium on which a computer program (or instructions) is stored.

[0086] When the program is read and executed by the processor of a computing device, it can cause the computing device to perform all or part of the steps of the DINOv3-TCT-based cervical cancer screening pathological model construction method as described in any of the preceding embodiments. Alternatively, the program directly includes the parameter and structure definitions of the aforementioned pathological model product, enabling the computing device to perform instance-level analysis and processing of TCT images when running the program.

[0087] Computer-readable storage media include, but are not limited to: USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, optical disks, and various media capable of storing program code.

[0088] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT, characterized in that, include: Construct an encoder-mask integrated segmentation network; the integrated segmentation network uses the visual Transformer model DINOv3 as the encoder, and after the Lth Transformer block of the DINOv3 encoder, a set of learnable query tokens is concatenated and injected to form a hybrid token sequence, where L is an integer greater than 0 and less than N, and N is the total number of Transformer blocks of the DINOv3 encoder. The injection of query tokens aims to achieve early binding of instance features in collaboration with subsequent encoding layers. In the L+1 to Nth Transformer blocks of the DINOv3 encoder, multi-head self-attention computation is performed on the mixed token sequence, and an instance decoupling loss term is constructed based on the attention weight matrix. The instance decoupling loss term is used to guide the attention weight distribution of different query tokens to produce differences during training, so that each query token focuses on different cell instances in the image. Based on the overall loss function that includes an instance decoupling loss term and cervical cell labeled data, the integrated segmentation network is trained end-to-end to obtain a large pathological model that can directly output instance segmentation masks.

2. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, Construct the instance decoupling loss term, specifically as follows: Calculate the attention weight matrix of the output of the L+1 to Nth Transformer blocks, and extract the attention weight distribution of each query token to all image block tokens; Based on the extracted attention weight distribution, the Jensen-Shannon divergence between the distributions corresponding to different query tokens is calculated, and the negative value of the divergence is used as the instance decoupling loss term. Alternatively, calculate the entropy of the attention weight distribution for a single query token and use the sum of the entropy values ​​as the instance decoupling loss term.

3. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, Prior to end-to-end training, an instance-aware self-supervised pre-training step is also included: The DINOv3 encoder is pre-trained based on the unlabeled TCT image dataset. The pre-training adopts a student-teacher network framework and defines a token-level alignment loss. The token-level alignment loss requires that query token features extracted from different augmented views of the same original image maintain a one-to-one correspondence in the feature space.

4. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 3, characterized in that, Define the token-level alignment loss, including: Use the optimal transmission algorithm or the Hungarian matching algorithm to establish the correspondence between the query token feature set output by the student network and the query token feature set output by the teacher network. Calculate the feature distance between corresponding query token pairs and construct a token-level alignment loss based on the average feature distance of all pairs.

5. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, The number M of learnable query tokens injected is set based on the average cell count of cervical smears in the target dataset.

6. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, After obtaining the trained large pathology model, the next step is to generate instance segmentation masks: The query token is input into a lightweight mask generation head; the mask generation head maps the query token to a mask prototype vector through linear projection, calculates the dot product of the prototype vector and the image patch token features output by the encoder to generate a low-resolution mask map, and obtains the instance segmentation mask of the final resolution through bilinear upsampling.

7. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, After obtaining the trained pathological model, the next step is to perform cell classification: Input the query token into a parallel classification head; the classification head outputs the pathological category probability distribution of the cell instance represented by each query token.

8. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 3, characterized in that, When performing end-to-end training, a parameter freezing strategy is used: Freeze the parameters of the first to Lth Transformer blocks of the instance-aware self-supervised pre-trained DINOv3 encoder; Supervised fine-tuning is performed only on the L+1 to Nth Transformer blocks, the learnable parameters related to the instance decoupling loss term, and the parameters of the mask generation head and the classification head.

9. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, The overall loss function is a weighted sum of the instance decoupling loss term and the standard segmentation loss term; the standard segmentation loss term includes the Dice loss of the segmentation mask and the cross-entropy loss.

10. The method for constructing a large-scale pathological model for cervical cancer screening based on DINOv3-TCT according to claim 1, characterized in that, It also includes model validation and optimization steps: The trained pathological model was evaluated using an independent validation set, and the average precision and average recall of the model on the cell instance segmentation task were calculated. Based on the evaluation results, the weight coefficients of the instance decoupling loss term, the number of query tokens M, or the injection layer L of query tokens in the encoder are adjusted in reverse to optimize model performance and complete the large model construction process.

Images