A cervical cell classification method fusing differentiated cell relations and smear context

Abstract

This invention relates to a cervical cell classification method that integrates differential cell relationships and smear background. The problem of low accuracy in cervical cell classification stems from the similarity of cells within a sample and the differences between cells in different samples. Due to different pathological or physiological states between samples, cervical cells exhibit differences in morphology and staining. Simultaneously, within a sample, normal cells and lesion cells without clear diagnostic significance are similar, and low-grade and high-grade lesion cells show intraclass similarity. These factors contribute to the low accuracy of existing deep learning classification methods. To improve this problem, this invention proposes a cervical cell classification method that integrates differential cell relationships and smear background. Experiments show that this method effectively utilizes the smear background and differential cell relationships within a sample, reducing inter-sample differences, improving the distinguishability of cells within a sample, and enhancing the accuracy, sensitivity, and specificity of classification. This invention is applied to cervical cell classification in cytopathology.

Description

Technical Field

[0001] This invention relates to cervical cell classification. Background Technology

[0002] Cervical cancer is one of the most serious malignant tumors threatening women's health. According to the World Health Organization in 2020, there were approximately 604,127 cases of cervical cancer worldwide, with 341,831 deaths. Once cervical cancer progresses to an advanced stage, it becomes very difficult to cure. Therefore, early detection and treatment are effective means of combating cervical cancer. For the past half-century, pathological diagnosis of cervical cancer cells has typically involved collecting exfoliated cervical cells to prepare a Pap smear, which is then examined by pathologists under a microscope to search for lesion cells. However, the smear contains a large number of cells, often with only a few lesion cells, making the search process extremely difficult. Furthermore, doctors are prone to fatigue from long hours of work, which can easily lead to missed or incorrect diagnoses.

[0003] In recent years, with the development of image processing and deep learning technologies, automated screening systems have emerged, greatly improving screening efficiency by detecting abnormal cells and determining their category based on descriptive diagnostic criteria. Cervical cell classification is a crucial task in automated screening systems. However, due to different pathological or physiological states among samples, cervical cells exhibit variations in morphology and staining. Furthermore, within a sample, normal cells and lesion cells without clear diagnostic significance are similar, and low-grade and high-grade lesion cells show intraclass similarity. These factors contribute to the low accuracy of existing deep learning classification methods. To address this issue, this invention proposes a cervical cell classification method that integrates differential cell relationships and smear background. Specifically, firstly, this invention proposes a graph attention-based cell differential feature modeling method, effectively extracting differential features between different cell categories and enhancing the model's ability to represent cells. Secondly, this invention proposes a smear background feature enhancement method, enabling the method to effectively capture relevant information from the background and mitigate significant style differences between different samples. Finally, this invention fuses cell features and background features, improving classification accuracy. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of low cervical cell classification accuracy caused by intra-sample cell similarity and inter-sample cell differences, and to propose a cervical cell classification method that integrates differential cell relationships and smear background.

[0005] The aforementioned objectives are primarily achieved through the following technical solutions:

[0006] S1. Prepare cervical cell classification data and perform data augmentation;

[0007] First, positive cervical cytology smears from different institutions, of different ages, and with different staining styles were collected. Then, the smears were scanned using a digital slide scanner with a 20x objective to obtain a panoramic view of the cervical cytology smears. Next, normal cells and four types of abnormal cells were marked on the panoramic view of the cervical cytology smears using ASAP software. Then, the marked cells were cropped from the panoramic view and subjected to different degrees of data enhancement, such as random flipping, random saturation, and contrast. Finally, each sample was divided into a training set and a test set.

[0008] S2. Construct a differential cell relationship map;

[0009] For two dissimilar cells in the same smear, the distance between them in the latent space should be relatively large; therefore, we first use the Swin-Transformer pre-trained on the ImageNet dataset as a cell feature extractor to generate a vector for each cell image. Where D (0) =1024; Then, calculate the cosine similarity between each cell and other cells, select cell nodes with cosine similarity less than a set threshold as their neighbor nodes, and create an edge between neighbor nodes to obtain a differential cell relationship map of the smear; Differential cell relationship map G i =(V i E i (Adjacency matrix can be used) It means that V i Represents N nodes v in a differential cell relationship graph j ∈V i The set of (j = 1, ..., N), E i represent edges(v) j ,v k )∈E i The set is constructed using the following formula:

[0010]

[0011] In the formula, A i,j,k This indicates whether there is an edge between the j-th cell node and the k-th cell node; ε is the threshold for cosine similarity. and The feature vectors representing the j-th cell node and the k-th cell node; The eigenvectors were calculated. sum vector Cosine similarity between them.

[0012] S3. Cut different painted backgrounds;

[0013] First, draw a rectangular area around the center of the smear, with its size being half the length and half the width of the smear; then, randomly select several locations within this rectangular area, and use these locations as centers to cut out areas of size W. p ×H p The tile map is prepared to ensure that it includes the most representative areas of the smear and contains information relevant to the background; finally, the cropped tile map is saved according to the sample.

[0014] S4. Construct a cervical cell classification framework and train the classification model using the data generated in S2 and S3.

[0015] S4-1, the cervical cell classification framework uses a graph attention network to improve the representational power of the relationship graph, thereby more accurately capturing the complex relationships between nodes; given a differential cell relationship graph G generated in S2 from a smear. i =(V i E i After passing through the l-th GATv2 layer, a set of dimensions D is generated. (l) Enhanced node feature vectors Specifically, GATv2 takes the constructed differential cell relationship map as input and first calculates the attention scores of cell nodes j and k in the l-th GATv2 layer. The formula represents the importance of the features of cell node j to k, as shown below:

[0016]

[0017] In the formula, and It is a learnable linear transformation. That is, all neighboring nodes of node j; || represents the vector concatenation operation; to make the coefficients between different nodes comparable, the softmax(·) function is used to score the attention. Normalization is performed, and the normalization function is defined as follows:

[0018]

[0019] Using normalized attention coefficients, a weighted sum is applied to each cell node, and then averaged using M attention heads. Information from other nodes is aggregated, and the weights of edges and features are updated. Finally, a nonlinear transformation σ is applied to obtain the enhanced feature representation of cell node j.

[0020]

[0021] S4-2, the cervical cell classification framework uses the EfficientNetv2 model pre-trained on the ImageNet dataset to extract the feature vector P from the tile images; finally, a multi-head self-attention layer is used to obtain the feature representation vector P′ of the enhanced background; in the multi-head self-attention layer, we first initialize three weight matrices W. q W k W v These will be updated as the model is trained and optimized, with the weight matrices respectively compared to... Multiplying P and P' yields the query matrix Q, the index matrix K, and the content matrix V, as shown in the following formula:

[0022]

[0023] Then calculate the self-attention output head for each head. p The calculation formula is as follows:

[0024]

[0025] In the formula, softmax(·) normalizes the calculated attention weights; The scaling factor is used to make the normalization result more stable. Then, M′ self-attention operations are performed in parallel, the self-attention matrices obtained from each head are concatenated, and then multiplied by the weight matrix W. P′ The final enhanced feature P′ is obtained, and its calculation formula is as follows:

[0026] P′=||(head1,head2,…,head M′ W P′ (7)

[0027] In the formula, || denotes the vector concatenation operation; W P′ M is the linear transformation matrix; M′ is the number of heads in the multi-head self-attention layer;

[0028] S4-3, The cervical cell framework fuses enhanced cell features with background features through addition, given cell features. Combined with the background feature P′, the final fused feature vector X is obtained:

[0029]

[0030] Then, we obtain the final prediction score s = (s1, s2, ..., s) of the cells by using l′ layers of MLP modules combined with linear transformation and the LeakyReLU(·) function, and then by using the softmax(·) function. C ):

[0031] z p′ =Xp′ =LeakyReLU(X p′-1 W p′ +b p′ (9)

[0032] s = (s1, s2, ..., s C ) = softmax(z l′ (10)

[0033] In the formula, C is the number of cervical cell types; p′ represents the p′th layer of the MLP module (p′=1,2,…,l′); b is the weight of the linear transformation of the p′th layer. p′ It is the bias of the p′th layer; then we use cross-entropy loss to train the entire network:

[0034]

[0035] In the formula, The true label representing the cell image.

[0036] Invention Effects

[0037] This invention provides a cervical cell classification method that integrates differential cell relationships and smear background. First, it proposes a graph attention-based method for modeling differential cell features. This method first uses a Swin-Transformer pre-trained on ImageNet as a cell feature extractor to extract patient cell features, calculates edges using cosine similarity, and uses these edges as nodes to construct a differential cell relationship graph. Then, the constructed differential cell relationship graph is input into a graph attention network GATv2 to expand the relationships between nodes and generate richer and more accurate node representations. Next, this invention proposes a smear background feature enhancement method, enabling the model to effectively capture useful information from the background and alleviate significant style differences between different samples. Specifically, this method randomly crops multiple tile images of the same size from the smear, inputs these cropped tile images into an EfficientNetv2 pre-trained on ImageNet to extract features from the smear background region, and uses multi-head attention to enhance useful information on the background features. Finally, the enhanced cell features and background features are fused to enrich the model's ability to express features of cells between different samples and similar cells within the same sample, mitigating the significant style differences between different samples and improving classification accuracy. Experiments show that this invention can accurately classify cervical cells and can be effectively applied to automated cervical cancer screening systems. Attached Figure Description

[0038] Figure 1 Here is the main flowchart of the algorithm;

[0039] Figure 2 A flowchart for constructing a differential cell relationship diagram;

[0040] Figure 3 A structural diagram of a cervical cell classification method model that integrates differential cell relationships and smear background. Specific implementation methods Specific implementation method one:

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, this paper presents a cervical cell classification method that integrates differential cell relationships and smear background, which mainly includes the following steps:

[0044] S1. Prepare cervical cell classification data and perform data augmentation;

[0045] S2. Construct a differential cell relationship map;

[0046] S3. Cut different painted backgrounds;

[0047] S4. Construct a cervical cell classification framework and train the classification model using the data generated in S2 and S3.

[0048] In this embodiment of the invention, firstly, cervical cytology smears from different institutions, of different ages, and with different staining styles are collected. Normal cells and four types of abnormal cells are labeled. Then, the labeled cells are cropped and subjected to different degrees of data augmentation to create a cervical cell classification dataset. Next, this invention proposes a graph attention-based method for modeling differential cell features, effectively extracting the differential features between different cell categories and enhancing the model's ability to represent cells. Then, this invention proposes a smear background feature enhancement method, enabling the model to effectively capture relevant information from the background and mitigating significant style differences between different samples. Finally, this invention fuses the enhanced cell features and background features, improving accuracy.

[0049] The embodiments of the present invention will be described in detail below:

[0050] like Figure 1 The implementation of the algorithm shown includes the following steps:

[0051] S1. Prepare cervical cell classification data and perform data augmentation;

[0052] First, positive cervical cytology smears from different institutions, of different ages, and with different staining styles were collected. Then, the smears were scanned using a digital slide scanner with a 20x objective to obtain a panoramic view of the cervical cytology smears. Next, normal cells and four types of abnormal cells were marked on the panoramic view of the cervical cytology smears using ASAP software. Then, the marked cells were cropped from the panoramic view and subjected to different degrees of data enhancement, such as random flipping, random saturation, and contrast. Finally, the cells of each sample were divided into training and test sets at an 8:2 ratio.

[0053] S2. Construct a differential cell relationship map;

[0054] For two dissimilar cells in the same smear, the distance between them in the latent space should be relatively large; therefore, we first use the Swin-Transformer pre-trained on the ImageNet dataset as a cell feature extractor to generate a vector for each cell image. Where D (0) =1024; Then, calculate the cosine similarity between each cell and other cells, select cell nodes with cosine similarity less than a set threshold as their neighbor nodes, and create an edge between neighbor nodes to obtain a differential cell relationship map of the smear; Differential cell relationship map G i =(V i E i (Adjacency matrix can be used) It means that V i Represents N nodes v in a differential cell relationship graph j ∈V i The set of (j = 1, ..., N), E i represent edges(v) j ,v k )∈E i The set is constructed using the following formula:

[0055]

[0056] In the formula, A i,j,k This indicates whether there is an edge between the j-th cell node and the k-th cell node; ε is the threshold for cosine similarity. and The feature vectors representing the j-th cell node and the k-th cell node; The eigenvectors were calculated. sum vector Cosine similarity between them.

[0057] S3. Cut different painted backgrounds;

[0058] First, draw a rectangular area around the center of the smear, with a size equal to half the length and width of the smear; then, randomly select several locations within this rectangular area and cut out a region of size W centered on these locations. p ×H p The tile map is generated to ensure that it includes the most representative area of ​​the smear and contains background-related information; finally, the cropped tile map is saved according to the sample; in this embodiment, W p and H p Both are 384.

[0059] S4. Construct a cervical cell classification framework, the structure of which is as follows: Figure 3 As shown, the classification model is trained using the data generated in S2 and S3.

[0060] S4-1, the cervical cell classification framework uses a graph attention network to improve the representational power of the relationship graph, thereby more accurately capturing the complex relationships between nodes; given a differential cell relationship graph G generated in S2 from a smear. i =(V i E i After passing through the l-th GATv2 layer, a set of dimensions D is generated. (l) Enhanced node feature vectors Specifically, GATv2 takes the constructed differential cell relationship map as input and first calculates the attention scores of cell nodes j and k in the l-th GATv2 layer. The formula represents the importance of the features of cell node j to k, as shown below:

[0061]

[0062] In the formula, and It is a learnable linear transformation. That is, all neighboring nodes of node j; || represents the vector concatenation operation; to make the coefficients between different nodes comparable, the softmax(·) function is used to score the attention. Normalization is performed, and the normalization function is defined as follows:

[0063]

[0064] Using normalized attention coefficients, a weighted sum is applied to each cell node, and then averaged using M attention heads. Information from other nodes is aggregated, and the weights of edges and features are updated. Finally, a nonlinear transformation σ is applied to obtain the enhanced feature representation of cell node j.

[0065]

[0066] In this embodiment, l is 2 and M is 4;

[0067] S4-2, the cervical cell classification framework uses the EfficientNetv2 model pre-trained on the ImageNet dataset to extract the feature vector P from the tile images; finally, a multi-head self-attention layer is used to obtain the feature representation vector P′ of the enhanced background; in the multi-head self-attention layer, we first initialize three weight matrices W. q W k W v These will be updated as the model is trained and optimized, with the weight matrices respectively compared to... Multiplying P and P' yields the query matrix Q, the index matrix K, and the content matrix V, as shown in the following formula:

[0068]

[0069] Then calculate the self-attention output head for each head. p The calculation formula is as follows:

[0070]

[0071] In the formula, softmax(·) normalizes the calculated attention weights; The scaling factor is used to make the normalization result more stable. Then, M′ self-attention operations are performed in parallel, the self-attention matrices obtained from each head are concatenated, and then multiplied by the weight matrix W. P′ The final enhanced feature P′ is obtained, and its calculation formula is as follows:

[0072] P′=||(head1,head2,…,head M′ W P′ (7)

[0073] In the formula, || denotes the vector concatenation operation; W P′ M is the linear transformation matrix; M′ is the number of heads in the multi-head self-attention layer; in this embodiment, M′ is 4;

[0074] S4-3, The cervical cell framework fuses enhanced cell features with background features through addition, given cell features. Combined with the background feature P′, the final fused feature vector X is obtained:

[0075]

[0076] Then, we obtain the final prediction score s = (s1, s2, ..., s) of the cells by using l′ layers of MLP modules combined with linear transformation and the LeakyReLU(·) function, and then by using the softmax(·) function. C ):

[0077] z p′ =X p′ =LeakyReLU(X p′-1 W p′ +b p′ (9)

[0078] s = (s1, s2, ..., s C ) = softmax(z l′ (10)

[0079] In the formula, C is the number of cervical cell types; p′ represents the p′th layer of the MLP module (p′=1,2,…,l′); b is the weight of the linear transformation of the i-th layer. p′ It is the bias of the p′th layer; then we use cross-entropy loss to train the entire network:

[0080]

[0081] In the formula, The true label representing the cell image.

[0082] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the scope of this invention.

Claims

1. A method of classifying cervical cells that fuses the differential cellular relationships and the smear context, characterized by, Includes the following steps: S1. Prepare cervical cell classification data and perform data augmentation; S2. Construct a differential cell relationship map; S3. Cut different painted backgrounds; S4. Construct a cervical cell classification framework. This framework is trained using the differential cell relationship map data generated in step S2 and the smear background tile data generated in step S3 as inputs. The cervical cell classification framework uses a graph attention network to enhance the representational power of the relationship graph, thereby more accurately capturing the complex relationships between nodes; given an S2-generated differential cell relationship graph of a smear. After the first Each GATv2 layer produces a set of dimensions. Enhanced node feature vectors , Specifically, GATv2 takes the constructed differential cell relationship map as input and first calculates the... Cell nodes of GATv2 layer and Attention score , representing cell nodes Features The importance is shown in the following formula: (1) In the formula, , and It is a learnable linear transformation. , , i.e., node All neighboring nodes; This represents the concatenation operation of vectors; to make the coefficients between different nodes comparable, use... Function for attention score Normalization is performed, and the normalization function is defined as follows: (2) Using the normalized attention coefficients, a weighted sum is applied to each cell node, and then... The attention heads are averaged, information from other nodes is aggregated, and the weights of edges and features are updated. Finally, a nonlinear transformation is performed. , obtain cell nodes Enhanced feature representation: (3) The cervical cell classification framework uses an EfficientNetv2 model pre-trained on the ImageNet dataset to extract feature vectors from tile images. Finally, a multi-head self-attention layer is used to obtain the feature representation vector of the enhanced background. In the multi-head self-attention layer, we first initialized three weight matrices. , , These will be updated as the model is trained and optimized, with the weight matrices respectively compared to... , , Multiply to obtain the query matrix. index matrix Content Matrix The formula is as follows: (4) Then calculate the self-attention output for each head. The calculation formula is as follows: (5) In the formula, This involves normalizing the calculated attention weights; The scaling factor is used to make the normalization result more stable; subsequently, parallel computation is performed. The secondary self-attention method concatenates the self-attention matrices obtained from each head together and then multiplies them by the weight matrix. The final enhanced features are obtained. The calculation formula is as follows: (6) In the formula, Represents the concatenation operation of vectors; It is a linear transformation matrix; The number of heads in the multi-head self-attention layer; The cervical cell framework fuses enhanced cellular features with background features through additive processing, given cellular features. and background features The final fused feature vector is obtained. : (7) Then, we use linear transformation, The combination of functions Each layer of MLP module, and then through The function obtains the final predicted score for the cell. : (8) (9) In the formula, It refers to the number and type of cervical cells; Represents the first MLP module layer ; It is the first Weights of the layer linear transformation It is the first Layer bias; then we use cross-entropy loss to train the entire network: (10) In the formula, The true label representing the cell image.

2. The cervical cell classification method according to claim 1, which integrates differential cell relationships and smear background, is characterized in that... The specific steps for preparing cervical cell classification data as described in step S1 are as follows: First, positive cervical cytology smears from different institutions, of different ages, and with different staining styles were collected. Then, the smears were scanned using a digital slide scanner with a 20x objective to obtain a panoramic view of the cervical cytology smears. Next, normal cells and four types of abnormal cells were marked on the panoramic view of the cervical cytology smears using ASAP software. Then, the marked cells were cropped from the panoramic view and subjected to different degrees of data enhancement, such as random flipping, random saturation, and contrast. Finally, each sample was divided into a training set and a test set.

3. The cervical cell classification method according to claim 1, which integrates differential cell relationships and smear background, is characterized in that... The method for constructing the differential cell relationship map described in step S2 includes the following: First, a Swin-Transformer pre-trained on the ImageNet dataset is used as a cell feature extractor to generate vectors for each cell image. ,in ; Then, the cosine similarity between each cell and other cells is calculated. Cell nodes with a cosine similarity less than a set threshold are selected as their neighbor nodes, and an edge is created between the neighbor nodes to obtain a differential cell relationship map of the smear. Adjacency matrix can be used It means that among them Representing differential cell relationship diagrams Nodes The set, Representing an edge The set is constructed using the following formula: (11) In the formula, Indicates the first The cell node and the first Does an edge exist between each cell node? It is the threshold for cosine similarity; and Indicates the first The cell node and the first The feature vector of each cell node; The eigenvectors were calculated. sum vector Cosine similarity between them.

4. The cervical cell classification method as described in claim 1, which integrates differential cell relationships and smear background, is characterized in that... The method for cutting different smear backgrounds described in step S3 is as follows: First, draw a rectangular area around the center of the smear, with its size being half the length and half the width of the smear; then, randomly select several locations within this rectangular area, and use these locations as centers to cut out areas of a size... The tile map is prepared to ensure that it includes the most representative areas of the smear and contains information relevant to the background; finally, the cropped tile map is saved according to the sample.

Images