End-to-end cervical cell classification integrated framework

Abstract

The invention discloses an end-to-end cervical cell classification integrated framework. The system mainly comprises four modules: an enhancement module for expanding and preprocessing an original data set; the fusion module is used for training an improved primary learner on the data set and fusing, outputting and generating a new data set; the purification module is used for purifying the new data set to obtain an integration result of the primary learner and retaining misclassified samples; and the relearning module is used for carrying out relearning by using a secondary learner and carrying out final classification. In addition, a self-adaptive data enhancement method is provided in the training stage of the primary learner and the secondary learner. According to the method, through stage-by-stage fine-grained optimization, the problems that existing cervical cell data sets are unbalanced in category distribution and rare in number are solved, and meanwhile, the classification precision is remarkably improved by emphatically learning easily-confused image features through an integrated framework. Experiments show that the method can effectively assist in cervical cancer screening, the workload of pathologists is greatly reduced, and the method has good application value and huge social benefits.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image diagnosis, and in particular to an end-to-end cervical cell classification integrated framework. Background Art

[0002] Cervical cancer is a common malignant tumor that poses a serious threat to women's health. Traditional cervical cancer screening mainly relies on cytological examinations (such as Pap smear examinations and liquid-based thin-layer cell tests, etc.). However, these methods require experienced pathologists to manually screen under a microscope, which is time-consuming and labor-intensive, with high subjectivity in manual interpretation and limited accuracy. In recent years, computer-assisted screening technology has gradually emerged, which can provide more accurate means of cervical cancer detection.

[0003] Deep learning technology has performed well in the field of medical image analysis and has provided new possibilities for cervical cancer screening. Through deep convolutional neural networks (CNN), patients' cervical cell images can be efficiently extracted and classified. These deep learning models can automatically learn and identify cell morphological features, improving the accuracy of cervical cancer diagnosis.

[0004] Although deep learning models have performed well in image classification tasks, there are still many problems with cervical cell recognition. The first is the complexity and diversity of the data set. There are problems such as the small number of data sets and the uneven distribution of categories, and the very similar pathological features between some categories. The second is that the single network is limited by the architecture, with limited recognition accuracy, easy overfitting when trained on low-quality data sets, and poor robustness and generalization capabilities. How to deal with the above challenges to achieve accurate recognition of cervical cells is a hot topic of current research. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention proposes an end-to-end cervical cell classification integrated framework that combines the advantages of deep learning and integrated learning. The framework can automatically extract the features of cervical cell images, and by integrating the outputs of multiple models, achieve more accurate and reliable cervical cancer screening and help pathologists make efficient diagnoses. At the same time, the framework is further optimized for the scarcity and data imbalance of medical image data to ensure the feasibility and practicality of the model in practical applications.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0007] An end-to-end integrated framework for cervical cell classification, consisting of four modules:

[0008] Module 1: Enhancement module. The cervical cell dataset is enhanced using an improved diffusion model. Specifically, the model generates new cervical cell images to expand the original dataset to generate a new dataset, and performs basic preprocessing operations on the images in the new dataset.

[0009] Module 2: Fusion module: Three improved Inception family neural network models will be used as primary learners, trained on the expanded dataset, and the network outputs will be fused to generate a new dataset.

[0010] Module 3: Purification module. Purify the newly generated data set to obtain the prediction results after the integration of the primary learner in module 2. The samples with correct predictions are directly output, and the samples with incorrect predictions are retained to generate a secondary data set.

[0011] Module 4: Relearning module: Use a machine learning model as a secondary learner, relearn on the secondary dataset, and use the trained model for the final prediction.

[0012] Compared with the existing technology, the beneficial effect of the present invention is that it combines data enhancement, deep learning, ensemble learning and other methods to effectively identify various types of cervical cells, meet the needs of modern medicine for high-quality and efficient cytological classification, and promote the development of the field of medical image processing.

[0013] An adaptive data enhancement method is used in the training phase of primary learners and secondary learners. By analyzing the four factors that affect model training: sample confidence, sample difficulty, sample uncertainty, and training phase, a hybrid method based on dynamically adjusted mixing ratio is proposed to achieve data enhancement. The specific steps are described as follows:

[0014] Step 1: During the training process, two pictures are randomly selected from the data set, and their sample confidence, sample difficulty, sample uncertainty, and training stage are calculated respectively.

[0015] Step 2: Calculate the mixing ratio based on the four indicators in step 1, and mix on the vector scale according to the mixing ratio to generate a new graph and the corresponding new category.

[0016] Step 3: Use the generated new images to train the network.

[0017] Compared with the prior art, the beneficial effect of the present invention is that data is dynamically mixed according to the learning performance during the model training process, thereby enhancing the robustness and generalization performance of the model.

[0018] Further, the enhancement module has the following specific steps:

[0019] Step 1: Obtain a cervical cell dataset. The source of the dataset can be a public dataset on the Internet or a private dataset of a hospital (or institution). The dataset contains conventional image formats (png, jpg, or bmp), in which the category of each image needs to be manually labeled and stored by category.

[0020] Step 2: Use the cervical cell dataset to train the diffusion model, and then use the trained diffusion model to generate new cervical cell images. The original dataset and the newly generated cervical cell images are mixed to generate a new cervical cell dataset.

[0021] Step 3: Preprocess the images in the new dataset, including scaling to a uniform size, random flipping, random rotation, etc., to obtain a primary dataset for training the module 2 network.

[0022] Compared with the prior art, the beneficial effect of the present invention is that it effectively solves the problems of unbalanced category distribution and scarce quantity in cervical cell datasets.

[0023] Further, the fusion module, the specific steps are described as follows:

[0024] Step 1: Select three Inception family convolutional neural network models as primary learners, namely InceptionV3, InceptionResNetV2 and Xception, and add an attention module to improve the network structure.

[0025] Step 2: Use the primary data set obtained in module 1 to train three neural network models, and use a K-fold splitting and splicing strategy to fuse the output of the primary learner to generate a new data set.

[0026] Compared with the prior art, the beneficial effect of the present invention is that the errors that may occur in a single network are reduced through ensemble learning, and the classification accuracy is further improved. In addition, by fusing the outputs of the primary learners, a more diverse data set is generated, reducing the risk of overfitting in the training of the secondary learners in the next stage.

[0027] Further, the purification module, the specific steps are described as follows:

[0028] Step 1: Use a distance-weighted voting strategy to purify the new data set and obtain the integration result of the primary learner.

[0029] Step 2: For correctly classified cell images, directly filter the output; for incorrectly classified cell images, retain and generate a secondary dataset.

[0030] Compared with the prior art, the beneficial effect of the present invention is that confusing images that cannot be recognized by the primary learner are retained for re-learning by the secondary learner, so that it can focus more on identifying sample features that are more difficult.

[0031] Further, the specific steps of the resuming learning module are described as follows:

[0032] Step 1: Use a multivariate linear regression model as the secondary learner, design an adaptive weighted softmax function to replace its loss function, and train it on the secondary dataset.

[0033] Step 2: Use the trained model for final prediction.

[0034] Compared with the prior art, the beneficial effect of the present invention is that accurate identification of cervical cells is achieved through re-learning of difficult features by the secondary learner.

[0035] Furthermore, step 2 of the enhancement module proposes a diffusion model, which mainly includes an encoder and a decoder, introduces a 1×1 convolutional layer and a residual connection, and generates a cervical cell image through a diffusion process.

[0036] Compared with the prior art, the beneficial effect of the present invention is that high-quality cell images are generated through a diffusion model, allowing the model to learn more effective features.

[0037] Furthermore, step 1 in the fusion module proposes an attention mechanism based on kernel normalization, which mainly includes a channel attention module and a spatial attention module. The specific steps are described as follows:

[0038] Step 1: Use the channel attention module to normalize the input features along the channel scale kernel and calculate the weight of each channel. After multiplication, the output features are calculated through the activation function sigmoid.

[0039] Step 2: Multiply the input features and output features of step 1 to get the input of the spatial attention module, normalize along the pixel scale kernel and calculate the weight of each channel. After multiplication, the output features are calculated through the activation function sigmoid.

[0040] Step 3: Multiply the input and output of step 2 to obtain the final refined features.

[0041] Compared with the prior art, the beneficial effect of the present invention is that the neural network model pays more attention to the local features of the cervical cell image, thereby further improving the model performance.

[0042] Furthermore, step 2 in the fusion module proposes a K-fold splitting and splicing strategy, and the specific steps are described as follows:

[0043] Step 1: Divide the dataset into training set and test set in a ratio of 7:3.

[0044] Step 2: Divide the training set into K parts and perform K rounds of training. In each round, select one part as the test set of this round and the rest as the training set of this round. Train the three improved convolutional neural network models on the training set of this round and test them on the test set of this round. Horizontally splice to obtain the training set blocks generated in this round. Test on the test set and average the outputs of the three improved convolutional neural network models to obtain the test set blocks generated in this round.

[0045] Step 3: vertically concatenate the training set blocks generated by K rounds to obtain a newly generated training set; horizontally concatenate the test set blocks generated by K rounds to obtain a newly generated test set. The number of newly generated training sets and test sets is the same.

[0046] Compared with the prior art, the present invention has the beneficial effect of increasing the diversity of data sets and reducing the risk of overfitting during model training.

[0047] Furthermore, step 1 in the purification module proposes a distance-weighted voting strategy, and the specific steps are described as follows:

[0048] Step 1: Get the output of each cytology image on each primary learner, which is a predicted probability vector, and calculate the Euclidean distance between the predicted probability vector and the true label.

[0049] Step 2: Calculate the weight of each primary learner based on the calculated distance.

[0050] Step 3: The predicted probability vector of the primary learner integration is obtained by weighted summation, and the final prediction result is obtained by the argmax function. If the predicted result is consistent with the true label, it is directly output, otherwise it is retained.

[0051] Compared with the prior art, the beneficial effect of the present invention is that the newly generated training set and test set are screened, on the one hand, the integration result of the primary learner is accurately obtained, and on the other hand, the characteristics of the misclassified samples are retained.

[0052] Furthermore, step 1 in the relearning module proposes a loss function of adaptive weights, and the specific steps are described as follows:

[0053] Step 1: Set the initial weight matrix to the category distribution of the dataset.

[0054] Step 2: In each round of training, the precision vector and recall vector of the category are calculated, and the F1 vector is calculated based on these two vectors, and then the weight matrix is ​​updated based on the F1 vector.

[0055] Step 3: Substitute the weight matrix into the loss function of the original multivariate linear regression model and calculate the loss function for each round.

[0056] Compared with the prior art, the beneficial effect of the present invention is that the weights and loss functions are adjusted according to the performance of the model during training, thereby further improving the classification performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is the overall flow chart of the present invention.

[0058] Figure 2 It is a schematic diagram of the diffusion process.

[0059] Figure 3 It is a schematic diagram of the diffusion model.

[0060] Figure 4 Schematic diagram of the attention mechanism based on kernel normalization.

[0061] Figure 5 This is a schematic diagram of the improved Inception module.

[0062] Figure 6 It is a schematic diagram of the K-fold splitting and splicing strategy.

[0063] Figure 7 This is a schematic diagram of the distance-based weighted voting strategy.

[0064] Figure 8 It is the loss function flow chart of adaptive weights.

[0065] Fig. 9 It is a schematic diagram of the adaptive data enhancement method.

[0066] Fig.10 This is a sample diagram of the cervical cell dataset used for training in the present invention. DETAILED DESCRIPTION

[0067] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] See also Figure 1-8 The present invention provides the following technical solution: an end-to-end cervical cell classification integrated framework, including four modules: an enhancement module, a fusion module, a purification module and a re-learning module, which are in a sequential and progressive relationship.

[0069] Before describing the specific implementation of the present invention in detail, some preliminary mathematical descriptions of the data set are given: Assume that the total number of cell images contained in the data set is v, there are a total of c categories, and each image x i The corresponding label is y i ,y i ∈{1,2,…,c}, function f(y i )=index(y i )∈{1,2,…,c} is responsible for converting labels into category indices. The dataset can be simply represented as

[0070] Enhancement Module

[0071] The enhancement module uses the designed diffusion model to enhance the original data set and preprocess the newly generated data set. The diffusion model mainly includes two processes: forward process and reverse process, such as Figure 2 As shown in the figure. The forward process gradually adds Gaussian noise to the data until it becomes random noise. Each diffusion step introduces more complexity to the data, capturing the complex patterns and details of the original distribution. The reverse process generates high-quality images with fine details and realistic textures by gradually removing noise. Through the diffusion method, more samples can be effectively generated to supplement the dataset.

[0072] This paper proposes a new diffusion model, named Cytofusion, and the model architecture is as follows: Figure 3 The following is a detailed description of the structure of Cytofusion. A is the overall overview of the model, x and are the input image and output image respectively. B is the details of the autoencoder, where the latent space is sampled at the end of the encoder through the reparameterization trick, and there is a channel (dashed line) into the decoder (only valid when training the autoencoder). C is a detailed view of the denoising U-Net. The part inside the thick dashed box shows the details of the submodules inside the autoencoder and U-Net.

[0073] The autoencoder includes an encoder and a decoder. The encoder maps the input data to a low-dimensional space, which is a compressed representation of the data, containing key features. The decoder maps the low-dimensional representation of the encoder output back to the original data space. The autoencoder is mainly composed of ResBlock, Up and Down layers. The ResBlock layer is used to solve the problem of gradient disappearance or gradient explosion. The present invention uses ConvBlock (including a 3×3 convolution layer, a batch normalization layer and a ReLU activation function) to extract valuable information from the input features. The use of 1×1 convolution can reduce parameters and model complexity while enhancing nonlinearity, and the introduction of residual connections allows the gradient to be directly passed to deeper networks during training, thereby avoiding the problem of gradient disappearance or gradient explosion. In addition, the present invention adds a 1×1 convolution layer to the output part of both the encoder and the decoder, which is mainly used to adjust the number of channels of the feature map to match the dimensions between different layers.

[0074] The training of the diffusion model consists of two stages. In the first stage, the autoencoder is trained using the Adam optimizer with a learning rate of 0.001. It encodes the image space into an 8x compressed latent space of size 32×32 for a 256×256 input space and 64×64 for a 512×512 input space. During the training phase, the latent space is directly decoded back to the image space and supervised by a reconstruction loss function. In the second stage, the pre-trained autoencoder encodes the image space into a latent space, which is then diffused into Gaussian noise. The latent space is denoised by training a U-Net model using the AdamW optimizer with a learning rate of 0.001. The difference between the true noise distribution and the predicted noise distribution is supervised using the L1 loss function. The weights of the autoencoder are frozen during the training phase.

[0075] After using the diffusion model to expand the original data set, we can get the primary data set, and then preprocess the primary data set. Data preprocessing will resize the image to the input size of the convolutional neural network and perform some random processing. First, the image is uniformly scaled to 299×299 (the input size of the Inception family network). After uniform scaling, some images are randomly rotated and flipped to generate the final primary data set.

[0076] Fusion Module

[0077] The main task of the fusion module is to train multiple primary learners and generate a new dataset based on their outputs using a K-fold splitting and splicing strategy. The selection and improvement of primary learners and the strategy for generating new datasets are described in detail below.

[0078] The fusion module adopts an integrated learning strategy, under which the primary learners can be homogeneous or heterogeneous. When using heterogeneous classifiers, the weight of each classifier must be carefully considered in the experiment. In addition, due to the large number of existing models, it takes a lot of work to select an integrated combination with excellent performance. In contrast, homogeneous networks are more efficient in deployment and training because they have similar structures and parameter settings. In addition, they have almost the same hyperparameter configuration and adjustment strategies. By integrating homogeneous classifiers, the risk of errors in categories with higher confidence can be reduced. Therefore, a homogeneous model is used as a primary learner in the present invention.

[0079] In cytological classification, the morphological size of cells is the key basis for classification. Therefore, a model that can extract features of various scales from images is needed. The Inception module can just meet this purpose. It captures features of different scales and levels by combining convolution kernels of different sizes (such as 1×1, 3×3 and 5×5), which helps to more comprehensively extract structural and texture information in cell images. In addition, cytological images can be very complex, containing different cell types, shapes and arrangements. The multi-branch structure of the Inception module helps to cope with this complexity, enabling the model to better understand the diversity of images. The present invention selects three convolutional neural network models of the Inception family as primary learners, namely InceptionV3, InceptionResNetV2 and Xception.

[0080] In order to further improve the classification performance, this paper proposes a new attention mechanism, called Kernel Normalization-based Attention Module (KNAM), such as Figure 4 As shown in the figure, this module redesigns the channel attention and spatial attention submodules based on the convolutional attention mechanism (CBAM). First, the concept of kernel normalization (KN) is introduced, which normalizes in the channel, height and width dimensions. The formula is described as follows:

[0081] U′=D p (U)#(1)

[0082]

[0083] Among them, p is the dropout probability, D p is the dropout operation, U is the normalization unit, U' is the dropout unit, μ u' and are the mean and variance of the dropout unit, is the normalized unit.

[0084] Then, the scaling factors α and β are introduced, which are inspired by batch normalization (BN) and are used to measure the importance of each channel in channel attention and the importance of each pixel in spatial attention, respectively. The formulas are described as follows:

[0085]

[0086] Where F represents the input feature, W c and W s They represent the weight vectors, respectively, and are defined as the proportion of the scaling factor.

[0087] The present invention adds the attention module to the original Inception module through residual connection and designs the following Figure 5 In the improved architecture shown in the figure, the residual connection can effectively alleviate the negative impact of the attention module and enhance the robustness of the model. In the subsequent process, the three Inception family convolutional neural networks will use the improved Inception module to replace the original Inception module.

[0088] In the training and testing phases of the primary learner, the present invention designs a K-Fold Split and Concatenation (KFSC) strategy to create a new data set using the output of the model. Assume that the target data set is constructed directly using the primary data set. In this case, there is a risk of overfitting, that is, the secondary learner is too dependent on the primary data set and cannot generalize well to new data.

[0089] First, the dataset is divided into a training set D and a test set The initial training set is further divided into k subsets of the same size, denoted as D1, D2, …, D k Let D i and Represent the validation set and training set of the i-th fold respectively. Train T primary learners M1, M2, ..., M T , then in D i Verify them on.

[0090] During the i-th round of training, each primary learner is trained on the training set Train on to obtain the corresponding classifier Where j∈{1,2,…,T}. In the validation set D iThe results obtained above are expressed as follows:

[0091] D′ ij =C j (D i ),j∈{1,2,…,T}#(5)

[0092] By horizontally concatenating the above results, the generated training set block for round i is expressed as follows:

[0093] D′ i =(D′ i1 ,D′ i2 ,…,D′ iT )#(6)

[0094] The final training set can be obtained by vertically splicing the training set blocks of each round, which is expressed as follows:

[0095] D′=(D′1;D′2;…;D′ k )#(7)

[0096] Obviously, the generated training set has the same dimension on the x-axis as the original training set, which means the number of generated samples remains the same.

[0097] The process of generating a new test set is similar. In the i-th round, the test set The test results of each classifier are shown as follows:

[0098]

[0099] In order to maintain the consistency of the dimension, the test results of each classifier need to be averaged, so the j-th test set block is expressed as follows:

[0100]

[0101] The complete test set can be obtained by horizontally concatenating the test set blocks, as shown below:

[0102]

[0103] Finally, we can get a new training set D' and a new test set Prepare for training and testing of secondary learners in the relearning module.

[0104] Purification module

[0105] The purpose of the purification module is to screen the newly generated data set of the fusion module to obtain the integrated results of the primary learner and the secondary data set. The data set obtained by the fusion module has the same size as the original data set. If it is still used to train the secondary learner, the time cost will increase significantly. Therefore, the present invention screens and retains the misclassified samples. In other words, the present invention introduces a new concept called "purity", which refers to the proportion of misclassified samples in the data set. The purpose of purification is to retain the easily confused image features in the data set and enhance the diversity of the data set.

[0106] Taking the purification of the training set as an example, the newly generated training set is defined as in The definition is as follows:

[0107]

[0108] Among them, P ij The probability vector representing the j-th classifier for the i-th image can be expressed as follows:

[0109]

[0110] in, Represents the classifier C j is the i-th image x i The probability of assigning the kth class.

[0111] This paper proposes a distance-weighted voting strategy. Given the unique label encoding T of the i-th image i , the probability vector P of each classifier can be calculated ij With the actual label T i Distance between:

[0112]

[0113]

[0114] The distance indirectly reflects the performance of the classifier. A smaller distance means that the predicted value is closer to the true label, which means that the classifier has a stronger ability to distinguish the category. Therefore, in the subsequent voting process, the weight of the classifier should be appropriately increased. By calculating the proportion of each classifier distance r ij , we can get the corresponding weight w ij , which is expressed as follows:

[0115]

[0116] The final prediction can be calculated by weighted summation, expressed as follows:

[0117]

[0118]

[0119] After filtering out samples whose predicted values ​​match the true labels, a purified secondary training set can be obtained for use by the relearning module. The secondary training set can be described as follows:

[0120]

[0121] The purification of the test set follows the same process as above. Note that the samples filtered by the voting strategy in the test set can be considered to be correctly predicted by the primary learner, so the testing phase of the secondary learner only needs to focus on the samples that are misclassified.

[0122] Return to school module

[0123] In the relearning module, a multinomial linear regression (MLR) model is used as a secondary learner for relearning. In order to further improve the model performance, the present invention designs an adaptive weight loss function, the principle of which is to dynamically adjust the weight of each category according to the performance of each round of the model. The specific steps are described as follows:

[0124] For the secondary training dataset Each category r∈{1,2,…,c} has a corresponding weight vector a r (i.e., the parameters of the model). Sample The probability of belonging to class r can be calculated as follows:

[0125]

[0126] Therefore, the secondary learner has The final predicted probability vector can be expressed as follows:

[0127]

[0128] Define the loss function L in the jth round of training j as follows:

[0129]

[0130] in, represents the weight vector, which can be obtained by recursive derivation.

[0131] Assume that the weight vector of the previous round is defined as Based on the predictions of each round, the precision P, recall R and F1 score F of each category r∈{1,2,…,c} can be calculated:

[0132]

[0133] The F1 score is used to provide a more comprehensive evaluation of model performance, which balances precision and recall and reflects the classification performance of the model. When dealing with unbalanced data, the F1 score is usually more meaningful and more reliable. The present invention uses the following formula to define the weight vector

[0134]

[0135] Among them, z-score is a standard normalization function to ensure that the sum of weights is 1 and prevent overflow. A lower F1 score indicates that the model has poor recognition ability for a specific category. In this case, their weights should be appropriately increased to shift the focus of the model in the next round of training to the easily confused categories.

[0136] In addition, for the recursive formula, an initial weight needs to be defined. In view of the uneven distribution of the data set, the present invention calculates the weight of each category r according to its proportion p r Define initial weights It is expressed as follows:

[0137]

[0138]

[0139] For classes with fewer samples, the weights should be increased moderately to prevent the model from being biased in subsequent training, and vice versa. Once the loss function is determined, the weights can be updated using the gradient descent method during the training process. The trained secondary learner will be used for the final prediction.

[0140] Adaptive Data Augmentation Methods

[0141] In order to further enhance the robustness and generalization ability of the model, the present invention adopts the mixup method in the training stage of both the primary learner and the secondary learner. As an important research result in computer vision, the mixup method lays the foundation for many data enhancement techniques. The principle is as follows: given two training samples x i and x j , and their corresponding labels y i and j , the mixup method generates a new training sample by mixing two samples and the corresponding labels It is expressed as follows:

[0142]

[0143] Among them, λ∈[0,1] is a parameter randomly sampled from the Beta distribution, which controls the weight of the two original samples of the synthetic sample. On large-scale datasets (such as ImageNet), mixup performs well. However, on small datasets (such as cytology datasets), due to the fixed and unchanging parameter λ, the performance of mixup is not stable. Therefore, the present invention explores and proposes improvements to mixup, and proposes a method called CytoMix to dynamically adjust the mixing ratio λ to enhance the effect and applicability of data augmentation, and further improve the generalization ability and robustness of the model. The present invention considers four important indicators and uses them as the basis for parameter adjustment: sample confidence, sample difficulty, uncertainty, and training stage, all of which are strongly correlated with the performance of the model. Below, the steps of CytoMix are described in detail, assuming that the mixing ratio of the i-th sample is expressed as λ i .

[0144] The following is a detailed description of sample confidence. In deep learning, sample confidence refers to the degree of trust the model has in the sample prediction, which directly reflects the performance of the model. The present invention uses the predicted probability to measure sample confidence. Specifically, for the i-th sample (x i ,y i ), the output is represented by z i =(z1,z2,...,z c ), where z j∈{1,2,…,c} represents the raw score of the i-th sample. The softmax function can be applied to convert these scores into probability distributions:

[0145]

[0146] Among them, p j∈{1,2,…,c} represents the probability that the i-th sample belongs to the j-th category. By using these probabilities, the confidence of the i-th sample is expressed as follows:

[0147]

[0148] When a sample has a high confidence level, it means that the model has a strong ability to distinguish the category. Therefore, during the training process, the value of λ should be appropriately reduced. On the contrary, the mixing ratio needs to be appropriately increased so that the model can learn the features of the image more comprehensively. It can be assumed that there is an inverse relationship between the sample confidence level and the mixing ratio, which can be expressed as follows:

[0149]

[0150] The sample difficulty is described in detail below. Sample difficulty refers to the relative difficulty level of the sample in the classification task, and it can also reflect the learning ability of the model. The present invention measures the sample difficulty by calculating the distance between the sample label and the predicted value. In order to avoid relying on a single distance metric, three commonly used distance calculation methods are adopted: Euclidean distance, Manhattan distance and cosine distance. There are two purposes for using multiple distance measurement methods: first, a single distance metric may introduce errors, and combining multiple methods can significantly reduce the impact of errors on the results. Secondly, considering the diversity of tasks, different categories of cervical cell images may have different requirements for distance metrics, so the use of multiple distance measurement methods can make the model more flexible, enabling it to adapt to the specific needs of different tasks.

[0151] For the i-th sample (x i ,y i ), define the predicted probability distribution P i and the truth vector G i (where the true label corresponds to 1 and the rest are 0), as shown below:

[0152] P i =(p1,p2,…,p c )#(34)

[0153]

[0154] The Euclidean distance is defined as follows:

[0155]

[0156] Manhattan distance is defined as follows:

[0157]

[0158] The cosine distance is defined as follows:

[0159]

[0160] In order to combine the distances obtained from the three distance measurement methods, the present invention utilizes a multiplication rule, which considers the weight of each part and normalizes the range of values ​​to a single value, which the addition rule or voting method cannot do. The formula of the multiplication rule is described as follows:

[0161] D=E(P i ,T i )×M(P i ,T i )×C(P i ,T i )#(39)

[0162] When the distance between the predicted value and the true value is far, the sample can be considered more difficult. In this case, it is appropriate to increase the mixing ratio appropriately. On the contrary, the mixing ratio should be reduced to make the model more focused on other samples. Therefore, it can be assumed that there is a positive relationship between sample difficulty and mixing ratio:

[0163] λ i =w2·D#(40)

[0164] Sample uncertainty is described in detail below. Sample uncertainty refers to the degree of uncertainty in the model's prediction of a sample, which is another measure of the model's learning ability. The present invention introduces a concept from information theory to measure sample uncertainty, namely mutual information. Mutual information is a measurement method that quantifies the degree of mutual dependence between two random variables. For a sample, its predicted label can be regarded as one random variable, and its true label can be regarded as another random variable. By measuring the degree of correlation between these two variables, the uncertainty of the sample can be obtained.

[0165] Assume that the true label vector of a batch of data from dataset Y is Y and the predicted label vector is It is expressed as follows:

[0166] Y=(index(y1),index(y2),…,index(y n ))#(41)

[0167]

[0168] First, we need to calculate the marginal probability distribution of the true label, expressed as P Y (y). The probability of each class k appearing in the sample can be obtained by dividing the number of samples of class k by the total number of samples:

[0169]

[0170] Secondly, we need to calculate the marginal probability distribution of the predicted label, which is expressed as Similarly, each category k has The probability of occurrence can be obtained by dividing the number of samples with a prediction result of k by the total number of samples:

[0171]

[0172] Next, we need to calculate the joint probability distribution of the true label and predicted label of the sample, which is expressed as It can be obtained by dividing the number of samples where both the true label and the predicted result are equal to k by the total number of samples:

[0173]

[0174] Finally, using the probability distribution mentioned above, we can calculate the mutual information between the true label and the predicted label, which is expressed as

[0175]

[0176] If the value of mutual information is small, it means that the correlation between the true label and the predicted label is low. In this case, a larger λ value is selected to rely more on the original features of the sample. On the contrary, a smaller λ value can be selected to better utilize the mixed information between samples. Therefore, it can be assumed that there is an inverse relationship between mutual information and the mixing ratio:

[0177]

[0178] The training phase is described in detail below. In CytoMix, the present invention also considers the influence of the training phase on the mixing ratio. The adjustment can be divided into three stages:

[0179] Initial stage: In the early stages of training, λ can be set to a large fixed value, such as 0.8, which helps the model focus more on the original features of the samples and avoids over-reliance on mixed samples too early.

[0180] Reduction stage: As training progresses, λ is gradually reduced to amplify the influence of mixed samples.

[0181] Stable stage: When the training enters a stable stage or is close to convergence, λ can be fixed to a smaller value, such as 0.2, which allows more effective use of the information of mixed samples and enhances the generalization ability of the model.

[0182] The training phase can be defined as the current epoch number N current Divide by the total number of epochs N epoch :

[0183]

[0184] Assume that there is an inverse relationship between the epoch number and the mixing ratio:

[0185]

[0186] Through the above analysis, four independent indicators are established to influence the mixture ratio λ. By assigning weights to each variable, the final mixture ratio can be obtained by weighted sum:

[0187]

[0188] w1+w2+w3+w4=1#(51)

[0189] Since the importance of each part is uncertain, the present invention assumes that the weights of all indicators are equal, that is, w = (w1, w2, w3, w4) = (0.25, 0.25, 0.25, 0.25). During the training process, w is regarded as a parameter and integrated into the network so that the model learns the optimal weight distribution.

[0190] In the back propagation process of updating parameters, due to optimization considerations, weights may disappear or explode. In order to prevent the model from ignoring specific weight components during adaptive learning, upper and lower bounds are set for the weight w, that is, min(w i∈{1,2,3,4} )=0.2,max(w i∈{1,2,3,4} )=0.8, which helps maintain stability and avoid extreme fluctuations in weights during training.

[0191] In addition, a bias term b is introduced to handle data imbalance. The proportion of each category k can be calculated and a bias value b is assigned accordingly. k :

[0192]

[0193] After the above derivation, we can get the complete formula of CytoMix:

[0194]

[0195]

Claims

1. An end-to-end integrated framework for cervical cell classification, characterized by: It includes four modules: Module 1: Enhancement module. The cervical cell dataset is enhanced using an improved diffusion model. Specifically, the model generates new cervical cell images to expand the original dataset to generate a new dataset, and performs basic preprocessing operations on the images in the new dataset. Module 2: Fusion module: Use three improved Inception family neural network models as primary learners, train them on the expanded dataset, and fuse the network outputs to generate a new dataset. Module 3: Purification module. Purify the newly generated data set to obtain the prediction results after the integration of the primary learner in module 2. The samples with correct predictions are directly output, and the samples with incorrect predictions are retained to generate a secondary data set. Module 4: Relearning module: Use a machine learning model as a secondary learner, relearn on the secondary dataset, and use the trained model for the final prediction.

2. An adaptive data enhancement method is used in the training phase of the primary learner and the secondary learner. By analyzing the four factors that affect model training: sample confidence, sample difficulty, sample uncertainty, and training phase, a hybrid method based on dynamically adjusted hybrid ratio is proposed to achieve data enhancement, which is characterized by: The specific steps are as follows: Step 1: During the training process, two pictures are randomly selected from the data set, and their sample confidence, sample difficulty, sample uncertainty, and training stage are calculated respectively. Step 2: Calculate the mixing ratio based on the four indicators in step 1, and mix on the vector scale according to the mixing ratio to generate a new graph and the corresponding new category. Step 3: Use the generated new images to train the network.

3. The enhancement module according to module 1 in claim 1, characterized in that: The specific steps are as follows: Step 1: Obtain a cervical cell dataset. The source of the dataset can be a public dataset on the Internet or a private dataset of a hospital (or institution). The dataset contains conventional image formats (png, jpg, or bmp), in which the category of each image needs to be manually labeled and stored by category. Step 2: Use the cervical cell dataset to train the diffusion model, and then use the trained diffusion model to generate new cervical cell images. The original dataset and the newly generated cervical cell images are mixed to generate a new cervical cell dataset. Step 3: Preprocess the images in the new dataset, including scaling to a uniform size, random flipping, random rotation, etc., to obtain a primary dataset for training the module 2 network.

4. The fusion module according to module 2 in claim 1, characterized in that: The specific steps are as follows: Step 1: Select three Inception family convolutional neural network models as primary learners, namely InceptionV3, InceptionResNetV2 and Xception, and add an attention module to improve the network structure. Step 2: Use the primary data set obtained in module 1 to train three neural network models, and use a K-fold splitting and splicing strategy to fuse the output of the primary learner to generate a new data set.

5. The purification module according to module 3 in claim 1, characterized in that: The specific steps are as follows: Step 1: Use a distance-weighted voting strategy to purify the new data set and obtain the integration result of the primary learner. Step 2: For correctly classified cell images, directly filter the output; for incorrectly classified cell images, retain and generate a secondary dataset.

6. The re-learning module according to module 4 in claim 1, characterized in that: The specific steps are as follows: Step 1: Use a multivariate linear regression model as the secondary learner, design an adaptive weighted softmax function to replace its loss function, and train it on the secondary dataset. Step 2: Use the trained model for final prediction.

7. Based on step 2 of the enhancement module described in claim 3, a diffusion model is proposed, which mainly includes an encoder and a decoder, introduces a 1×1 convolutional layer and a residual connection, and generates a cervical cell image through a diffusion process.

8. Based on step 1 of the fusion module in claim 4, a kernel normalized attention mechanism is proposed, which mainly includes a channel attention module and a spatial attention module, and is used to improve the traditional neural network architecture. It is characterized in that: The specific steps are as follows: Step 1: Use the channel attention module to normalize the input features along the channel scale kernel and calculate the weight of each channel. After multiplication, the output features are calculated through the activation function sigmoid. Step 2: Multiply the input features and output features of step 1 to get the input of the spatial attention module, normalize along the pixel scale kernel and calculate the weight of each channel. After multiplication, the output features are calculated through the activation function sigmoid. Step 3: Multiply the input and output of step 2 to obtain the final refined features.

9. Based on step 2 of the fusion module in claim 4, a K-fold splitting and splicing strategy is proposed to generate a more diverse data set by combining the outputs of the primary learners to solve the overfitting problem that may occur during the training process. It is characterized in that: The specific steps are as follows: Step 1: Divide the dataset into training set and test set in a ratio of 7:

3. Step 2: Divide the training set into K parts and perform K rounds of training. In each round, select one part as the test set of this round and the rest as the training set of this round. Train the three improved convolutional neural network models on the training set of this round and test them on the test set of this round. Horizontally splice to obtain the training set blocks generated in this round. Test on the test set and average the outputs of the three improved convolutional neural network models to obtain the test set blocks generated in this round. Step 3: vertically concatenate the training set blocks generated by round K to obtain a newly generated training set; Horizontally concatenate the test set blocks generated by round K to obtain a newly generated test set. The number of newly generated training sets and test sets is consistent.

10. Based on step 1 of the purification module of claim 5, a distance-weighted voting strategy is proposed to filter the newly generated data set. It is characterized in that: The specific steps are as follows: Step 1: Get the output of each cytology image on each primary learner, which is a predicted probability vector, and calculate the Euclidean distance between the predicted probability vector and the true label. Step 2: Calculate the weight of each primary learner based on the calculated distance. Step 3: The predicted probability vector of the primary learner integration is obtained by weighted summation, and the final prediction result is obtained by the argmax function. If the predicted result is consistent with the true label, it is directly output, otherwise it is retained.

11. Based on step 1 of the relearning module of claim 6, an adaptive weight loss function is proposed, which can automatically adjust the weight of each category according to the performance of each round of the model. It is characterized in that: The specific steps are as follows: Step 1: Set the initial weight matrix to the category distribution of the dataset. Step 2: In each round of training, the precision vector and recall vector of the category are calculated, and the F1 vector is calculated based on these two vectors, and then the weight matrix is ​​updated based on the F1 vector. Step 3: Substitute the weight matrix into the loss function of the original multivariate linear regression model and calculate the loss function for each round.

Images