A Cervical Tissue Pathology Image Classification Method Based on Hybrid CNN-ViT Architecture
By combining the CNN-ViT architecture with the LACCSA module and the ExpReLU function, the problems of insufficient feature extraction and data distribution offset in cervical pathology image classification are solved, achieving efficient and accurate pathology image classification.
Patent Information
- Application Number
- CN202411807456.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2044-12-10
AI Technical Summary
Existing technologies for classifying cervical pathology images suffer from problems such as insufficient feature extraction, high feature similarity between categories, and poor model performance consistency due to data distribution bias, leading to decreased classification accuracy and low efficiency.
A hybrid CNN-ViT architecture is adopted, combining a linear angle cross-covariance self-attention module (LACCSA) and a novel activation function ExpReLU. CNN extracts local detail features, ViT captures global features, the LACCSA module uses the cross-covariance matrix and linear angle similarity function to capture dynamic relationships, and ExpReLU enhances the non-linear expressive power of the model and reduces the impact of data distribution bias.
It significantly improves the accuracy and efficiency of cervical pathology image classification, effectively distinguishes subtle differences between categories, enhances the robustness and adaptability of the model on different datasets, and reduces computational costs.
Smart Images
Figure CN119832297B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a method for classifying cervical tissue pathology images based on a hybrid CNN-ViT architecture. Background Technology
[0002] Cervical cancer is a significant disease affecting women's health worldwide, and its early diagnosis and accurate classification are crucial for patient treatment and recovery. Traditional cervical pathology image classification methods rely on manual annotation by experts. This method is not only time-consuming and labor-intensive, but also prone to subjective influence due to the complexity of pathological images and the variability between samples, resulting in low accuracy. Therefore, developing automated and intelligent cervical pathology image classification methods has become a current research hotspot.
[0003] In recent years, deep learning technology has made significant progress in the field of medical image processing. Convolutional Neural Networks (CNNs) can effectively extract local features and are widely used in medical image classification.
[0004] However, due to the limitations of local receptive fields, CNNs have shortcomings in handling global relationships and capturing complex feature interactions. To address this issue, researchers have proposed a method based on Visual Transformer (ViT), which can capture global features of images and exhibits good classification performance. However, ViT has high computational complexity and resource requirements, and its effectiveness in feature extraction is still insufficient when processing complex medical images (such as cervical pathology images).
[0005] Some existing studies attempt to combine CNNs with ViTs to balance the ability to extract local features and model global relationships.
[0006] For example, the literature Burmeister CA, Khan SF, G, et al. Cervical cancertherapies: Current challenges and future perspectives[J]. Tumour Virus Research, 2022, 13: 200238. proposed a multimodal deep learning framework that combines CNN with attention mechanism for the classification task of pathological images, and achieved certain results.
[0007] However, the current method still has the following shortcomings:
[0008] (1) The feature extraction stage failed to fully capture the dynamic relationship between features, resulting in a decrease in classification accuracy;
[0009] (2) In complex pathological images (such as cervical tissue pathological images), the feature similarity between categories is high, and existing methods are not good at distinguishing subtle differences.
[0010] (3) Due to the influence of data distribution offset, the model's performance is inconsistent across different datasets. Therefore, a new solution is needed to address these issues. Summary of the Invention
[0011] The purpose of this invention is to provide a cervical tissue pathology image classification method based on a hybrid CNN-ViT architecture, which aims to improve the accuracy and efficiency of complex pathology image classification tasks and solve the technical problems mentioned in the background art.
[0012] To achieve the above objectives, the present invention provides the following technical solution: a cervical tissue pathology image classification method based on a hybrid CNN-ViT architecture, comprising at least the following steps:
[0013] S1: The cervical tissue pathology image is input into the convolutional neural network module, which first extracts the local detail features of the image. The convolutional neural network module is the CNN module.
[0014] S2: Input the features proposed in S1 into the visual transformer module for further processing. The visual transformer module is the ViT module. The ViT module captures the dynamic relationship between global features through the linear angle cross covariance self-attention module, which is the LACCSA module.
[0015] S3: The LACCSA module combines the cross-covariance matrix and the linear angular similarity function to effectively extract common features of images of the same category, while distinguishing subtle differences between categories;
[0016] S4: The novel activation function ExpReLU is adopted in the ViT module to enhance the nonlinear expressive power of the model and reduce the impact of data distribution shift on model performance, thereby maintaining robustness and accuracy on image data from different sources.
[0017] S5: Experiments on the LDCH and CAISHI datasets demonstrate the efficiency of this method in multi-class and binary classification tasks.
[0018] Furthermore, the design of the LACCSA module includes at least the following steps:
[0019] S200: By calculating the cross-covariance matrix C = K' T·Q', where K′ and Q′ are the L2-normalized feature key and query matrices, respectively, and T represents the matrix transpose operation. The L2 normalization operation ensures the consistency of feature dimensions and avoids the imbalance problem caused by differences in feature scale.
[0020] S201: Calculating the attention weight matrix based on the cross-covariance matrix Where τ is a temperature coefficient used to adjust the smoothness of the Softmax function and ensure the stability of the weight distribution;
[0021] S202: Generate the final output features using the attention weight matrix: O = V·A XC The output feature O achieves dynamic capture of global features through weighted summation, which significantly improves the model's ability to understand complex pathological images. V is the value matrix.
[0022] Furthermore, in S3, the LACCSA module combines a linear angular similarity function to capture the angular relationship between global features;
[0023] The LACCSA module, when combined with the linear angular similarity function, includes at least the following steps:
[0024] Angle is defined as:
[0025]
[0026] Where ||.|| represents the Euclidean distance, <,> represents the inner product, and x i and x j Representing the i-th and j-th vectors in the query and key vectors, the output range of θ is [0,π]. Such an angle can be used as a way to measure similarity.
[0027] In this design, an angle kernel is defined as the similarity measurement function between query Q and key value K:
[0028]
[0029] Sim(Q i ,K j The output range is [0,1], when Q i and K j When they tend to be consistent, θ approaches 0, and the similarity approaches 1; conversely, if Q... i and K j They have opposing characteristics, θ approaches π, and the similarity approaches 0;
[0030] One characteristic of angle kernels is that replacing similarity measurements with angle kernels provides an efficient way to implicitly map input data to a high-dimensional feature space;
[0031] In a high-dimensional feature space, distance / angle is calculated based on the rich feature structure, denoted as the implicit mapping caused by this kernel, and the norm of the mapped input data is:
[0032]
[0033] That is, all data in the input space is mapped onto a sphere of radius 1 in the feature space, and the distance between two input features is:
[0034]
[0035] The Euclidean distance and spectral angle are positively correlated, and the distance range is [0,2]. Considering the quadratic complexity of the input label length;
[0036] To design linear attention, revisiting the trigonometric identities and the infinite series expansion of the arccos function, the similarity function is reconstructed as:
[0037]
[0038] Equivalent to:
[0039]
[0040] in: Represents the normalized linear kernel function (Q) i ,K j ) / (||Q i ||,||K j ||), which is equivalent to if Q i and K j The inner product when the vectors are unit vectors;
[0041] The first two linear terms are used directly as similarity measures in linear attention, while higher-order terms of infinite series introduce higher complexity. Therefore, a parameterized deep neural network (DNN) module is used to approximate higher-order residual terms.
[0042] The linear angle attention module can then be expressed as:
[0043]
[0044] in, M is a linear term with a complexity of O(N). DW ·V is the matrix form of DWConv, so the overall complexity of linear angular attention is linearly related to the length of the input tags.
[0045] Furthermore, the expression for the activation function ExpReLU is:
[0046] ExpReLU(x) = s·(ReLU(x)) 2 +e x +b
[0047] Where s∈R and b∈R are scalars of scale and bias, which are shared across all channels and can be set to constants or learnable to obtain different ExpReLU variants. The computational cost is much less than GELU, but the performance is better.
[0048] ReLU is an activation function widely used in neural networks. It is defined as ReLU(x) = max(0,x).
[0049] e is the natural constant, approximately equal to 2.71828.
[0050] Compared with the prior art, the beneficial effects of the present invention are:
[0051] This invention, based on the CNN and ViT architecture, addresses the shortcomings of existing methods in feature extraction, classification performance, and model adaptability, as well as the problem of dataset distribution bias. It proposes a Linear Angle Cross-Covariance Self-Attention Module (LACCSA) and a novel activation function, ExpReLU. The LACCSA module combines cross-covariance attention and linear angle similarity to effectively capture the dynamic relationships between global features, distinguishing subtle differences between categories and extracting common features within categories, significantly improving the accuracy of complex pathological image classification. The ExpReLU activation function enhances non-linear expressive power, reduces the impact of data distribution bias on model performance, and optimizes computational efficiency. By combining the local feature extraction capabilities of CNN and the global feature capture capabilities of ViT, this invention achieves a good balance between detailed representation and global relationship modeling. Attached Figure Description
[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 This is a diagram of the overall network model of a novel cervical tissue pathological image classification method disclosed in this invention.
[0054] Figure 2 This is a LACCSA diagram of the present invention;
[0055] Figure 3 This is a diagram illustrating the visualization results;
[0056] Figure 4 This is a flowchart. Detailed Implementation
[0057] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0058] Please see Figure 4 A cervical tissue pathology image classification method based on a hybrid CNN-ViT architecture includes at least the following steps:
[0059] S1: Input the cervical tissue pathology image into the convolutional neural network module, first extract the local detail features of the image; the convolutional neural network module is the CNN module.
[0060] S2: Input the features proposed in S1 into the visual transformer module for further processing. The visual transformer module is the ViT module. The ViT module captures the dynamic relationship between global features through the linear angle cross covariance self-attention module, which is the LACCSA module.
[0061] S3: The LACCSA module combines the cross-covariance matrix and the linear angular similarity function to effectively extract common features of images of the same category, while distinguishing subtle differences between categories;
[0062] S4: The novel activation function ExpReLU is adopted in the ViT module to enhance the nonlinear expressive power of the model and reduce the impact of data distribution shift on model performance, thereby maintaining robustness and accuracy on image data from different sources.
[0063] S5: Experiments on the LDCH and CAISHI datasets demonstrate the efficiency of this method in multi-class and binary classification tasks.
[0064] This model combines the advantages of CNN and ViT, and introduces a Linear Angle Cross Covariance Self-Attention Module (LACCSA) and a novel activation function, ExpReLU. The LACCSA module enables the model to capture the dynamic relationships between global features, effectively distinguishing subtle differences between categories; the ExpReLU activation function provides greater robustness to data distribution shifts, thereby improving classification performance and adaptability.
[0065] The design of a LACCSA module should include at least the following steps:
[0066] S200: By calculating the cross-covariance matrix C = K' T·Q', where K′ and Q′ are the L2-normalized feature key and query matrices, respectively, and T represents the matrix transpose operation. The L2 normalization operation ensures the consistency of feature dimensions and avoids the imbalance problem caused by differences in feature scale.
[0067] S201: Calculating the attention weight matrix based on the cross-covariance matrix Where τ is a temperature coefficient used to adjust the smoothness of the Softmax function and ensure the stability of the weight distribution;
[0068] S202: Generate the final output features using the attention weight matrix: O = V·A XC The output feature O achieves dynamic capture of global features through weighted summation, which significantly improves the model's ability to understand complex pathological images. V is the value matrix.
[0069] In S3, the LACCSA module combines a linear angular similarity function to capture the angular relationships between global features;
[0070] The steps of combining the LACCSA module with the linear angular similarity function include at least the following:
[0071] Angle is defined as:
[0072]
[0073] Where ||.|| represents the Euclidean distance, <,> represents the inner product, and x i and x j Representing the i-th and j-th vectors in the query and key vectors, the output range of θ is [0,π]. Such an angle can be used as a way to measure similarity.
[0074] In this design, an angle kernel is defined as the similarity measurement function between query Q and key value K:
[0075]
[0076] Sim(Q i ,K j The output range is [0,1], when Q i and K j When they tend to be consistent, θ approaches 0, and the similarity approaches 1; conversely, if Q... i and K j They have opposing characteristics, θ approaches π, and the similarity approaches 0;
[0077] One characteristic of angle kernels is that replacing similarity measurements with angle kernels provides an efficient way to implicitly map input data to a high-dimensional feature space;
[0078] In a high-dimensional feature space, distance / angle is calculated based on the rich feature structure, denoted as the implicit mapping caused by this kernel, and the norm of the mapped input data is:
[0079]
[0080] That is, all data in the input space is mapped onto a sphere of radius 1 in the feature space, and the distance between two input features is:
[0081]
[0082] The Euclidean distance and spectral angle are positively correlated, and the distance range is [0,2]. Considering the quadratic complexity of the input label length;
[0083] To design linear attention, revisiting the trigonometric identities and the infinite series expansion of the arccos function, the similarity function is reconstructed as:
[0084]
[0085] Equivalent to:
[0086]
[0087] in: Represents the normalized linear kernel function (Q) i ,K j ) / (||Q i ||,||K j ||), which is equivalent to if Q i and K j The inner product when the vectors are unit vectors;
[0088] The first two linear terms are used directly as similarity measures in linear attention, while higher-order terms of infinite series introduce higher complexity. Therefore, a parameterized deep neural network (DNN) module is used to approximate higher-order residual terms.
[0089] The linear angle attention module can then be expressed as:
[0090]
[0091] in, M is a linear term with a complexity of O(N). DW ·V is the matrix form of DWConv, so the overall complexity of linear angular attention is linearly related to the length of the input tags.
[0092] The expression for the activation function ExpReLU is:
[0093] ExpReLU(x) = s·(ReLU(x)) 2 +e x +b
[0094] Where s∈R and b∈R are scalars of scale and bias, which are shared across all channels and can be set to constants or learnable to obtain different ExpReLU variants. The computational cost is much less than GELU, but the performance is better.
[0095] ReLU is an activation function widely used in neural networks. It is defined as ReLU(x) = max(0,x).
[0096] e is the natural constant, approximately equal to 2.71828.
[0097] See Figure 1 This invention presents a schematic diagram of the overall network architecture of the novel cervical tissue pathology image classification method. In the encoding stage, a Linear Angle Cross-Covariance Self-Attention (LACCSA) module is constructed, which captures the dynamic relationships between global features by combining cross-covariance attention and linear angle similarity. In the activation function design, the ExpReLU activation function is introduced to enhance the model's non-linear expressive power and reduce the impact of data distribution bias on model performance. Furthermore, by combining a hybrid architecture of CNN and ViT, local feature extraction and global semantic modeling are effectively balanced, improving the performance and efficiency of the classification task. The specific implementation steps are as follows:
[0098] The technical approach can be briefly described as follows:
[0099] This invention is based on a hybrid architecture of CNN and ViT, employing a holistic design flow of convolution -> transformer encoder -> classifier. The encoding stage uses three consecutive convolutional modules (containing convolution and pooling operations) to extract local features; these features are then fed into the ViT module, where LACCSA captures global dynamic relationships. Within the ViT module, a linear angular similarity function and a cross-covariance matrix are combined to enhance intra-class features and distinguish inter-class differences. Finally, the robustness of feature representation is enhanced using the ExpReLU activation function to generate the final classification result.
[0100] See Figure 2 The LACCSA module design is briefly described as follows:
[0101] The LACCSA module is a crucial component of this invention, used to capture the interactions and dynamic relationships between different features in an image. This module consists of two branches: a covariance attention branch and a linear angular attention branch. The covariance attention branch models the interactions between feature channels, rather than operating between labels, providing a computationally efficient method that focuses on general intra-class features. This branch effectively extracts features more important for the classification task while significantly reducing computational complexity. The linear angular attention branch, based on the angular similarity between feature vectors, evaluates the relationships between features using an angular kernel function, thus accurately distinguishing subtle differences between categories, particularly suitable for differentiating visually very similar cervical lesion categories. By combining these two attention branches, the LACCSA module can capture the dynamic interactions of features globally, while simultaneously distinguishing minute changes in detailed features through angular relationships. This design is particularly important for handling complex cervical histopathology image classification tasks.
[0102] The proposed Linear Angle Cross-Covariance Self-Attention Module (LACCSA) combines covariance attention and linear angle attention branches to simultaneously capture global statistical relationships and angular similarities between features. Input features are first processed through linear projection to generate a query (Q), key (K), and value (V), which are then processed in two separate branches. In the covariance attention branch, the cross-covariance matrix between the query and key is calculated using matrix multiplication, and then temperature-adjusted and Softmax normalized to generate an attention weight matrix. This weight is used to perform a weighted summation of the value (V) to obtain the branch's output features. In the linear angle attention branch, feature similarity is calculated based on the angular relationship between the query and key, combined with an angle kernel function. Subsequently, multi-scale features are generated using depthwise convolution (DWConv), and the expressive power of the features is enhanced through global pooling. The outputs of the two branches are fused element-wise and normalized using a sigmoid activation function to generate the final feature output. The LACCSA module effectively captures global dynamic relationships and angular information through this dual-branch design, making it particularly suitable for distinguishing intra-class detail differences and inter-class similarities in complex pathological images.
[0103] The ExpReLU activation function is briefly described below:
[0104] ExpReLU is a novel activation function designed to address the shortcomings of existing activation functions in handling complex data distributions. Traditional ReLU is simple and efficient, but its linear nature limits the model's expressive power. While GELU offers superior performance, it comes with high computational costs. SquaredReLU enhances non-linearity through a squaring operation, but its performance remains insufficient for complex tasks. Therefore, ExpReLU further optimizes SquaredReLU by using normalization to mitigate the performance degradation caused by data distribution shifts and adding an exponential term e. x This significantly enhances the nonlinear expressive power. Unlike other activation functions, ExpReLU introduces learnable scale and bias parameters, enabling it to adaptively adjust according to specific tasks and data, thereby improving the model's robustness and adaptability. In the cervical pathology tissue image classification task, ExpReLU not only reduces computational costs but also outperforms traditional activation functions in performance, especially demonstrating higher stability and superiority when handling different data distributions.
[0105] Figure 3 The document presents representative visualizations, comparing heatmaps of CIN I, CIN II, and CIN III to demonstrate the changes in the feature importance region of the model before and after applying the LACCSA module (3(a) is the original cervical pathology tissue image, 3(b) is the feature importance region without LACCSA, and 3(c) is the feature importance region after applying LACCSA). The visualizations show that although there is inter-category similarity between CIN II and CIN III, the LACCSA module guides the model to focus more on the subtle differences between them, contributing to the accuracy of classification decisions.
[0106] In summary:
[0107] This invention addresses the limitations of existing deep learning models in cervical tissue pathology image classification by proposing a novel hybrid model architecture, CervicalFormer. During the encoding process, this method effectively solves the problem of insufficient feature representation capabilities through modular design. In the encoding stage, this invention designs a Linear Angle Cross-Covariance Self-Attention (LACCSA) module, combining cross-covariance attention and linear angle attention mechanisms to effectively capture the dynamic relationships between global features while maintaining computational efficiency. This module is particularly suitable for extracting global features from complex pathological images and can significantly improve the discriminative power between categories. Furthermore, this invention proposes a novel activation function, ExpReLU, which enhances nonlinear expressive power and further reduces the impact of data distribution bias on model performance, enabling the model to maintain high robustness on pathological image data from different sources. Experimental results show that this method significantly outperforms existing models in classification on datasets such as LDCH and CAISHI, while significantly reducing the number of parameters and computational cost, demonstrating its potential application value in the field of cervical tissue pathology image classification.
[0108] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A cervical tissue pathology image classification method based on a hybrid CNN-ViT architecture, characterized in that: At least the following steps are included: S1: The cervical tissue pathology image is input into the convolutional neural network module, which first extracts the local detail features of the image. The convolutional neural network module is the CNN module. S2: Input the features proposed in S1 into the visual transformer module for further processing. The visual transformer module is the ViT module. The ViT module captures the dynamic relationship between global features through the linear angle cross covariance self-attention module, which is the LACCSA module. S3: The LACCSA module combines the cross-covariance matrix and the linear angular similarity function to effectively extract common features of images of the same category, while distinguishing subtle differences between categories; S4: The novel activation function ExpReLU is adopted in the ViT module to enhance the nonlinear expressive power of the model and reduce the impact of data distribution shift on model performance, thereby maintaining robustness and accuracy on image data from different sources. S5: Experiments on the LDCH and CAISHI datasets demonstrate the efficiency of this method in multi-class and binary classification tasks; The LACCSA module in S3 combines a linear angle similarity function to capture the angle relationship between global features; The LACCSA module, when combined with the linear angular similarity function, includes at least the following steps: Angle is defined as: ； in, Represents Euclidean distance. Indicates the inner product. and Representing the i-th and j-th vectors in the query and key vectors, the output range of θ is [0, π]. Such an angle can be used as a way to measure similarity. In this design, an angle kernel is defined as the similarity measurement function between query Q and key value K: ； The output range is [0, 1]. and When they tend to be consistent, θ approaches 0, and the similarity approaches 1; conversely, if... and They have opposing characteristics, θ approaches π, and the similarity approaches 0; One characteristic of angle kernels is that replacing similarity measurements with angle kernels provides an efficient way to implicitly map input data to a high-dimensional feature space; In a high-dimensional feature space, distance / angle is calculated based on the rich feature structure, denoted as the implicit mapping caused by this kernel, and the norm of the mapped input data is: ； That is, all data in the input space is mapped onto a sphere of radius 1 in the feature space, and the distance between two input features is: ； The Euclidean distance and spectral angle are positively correlated, and the distance range is [0, 2]. Considering the quadratic complexity of the input label length; To design linear attention, revisiting the trigonometric identities and the infinite series expansion of the arccos function, the similarity function is reconstructed as: ； Equivalent to: ； in: Represents the normalized linear kernel function / This is equivalent to if and The inner product when the vectors are unit vectors; The first two linear terms are used directly as similarity measures in linear attention, while higher-order terms of infinite series introduce higher complexity. Therefore, a parameterized deep neural network module is used to approximate higher-order residual terms. The linear angle attention module is then expressed as: ； in, The complexity is linear terms, It is in matrix form of DWConv, therefore the overall complexity of linear angle attention is linearly related to the length of the input label; The expression for the activation function ExpReLU is: ； Where: s ∈ R and b ∈ R are scalars of scale and bias, respectively, shared across all channels, set to constant or learnable to obtain different ExpReLU variants, with a computational cost far less than GELU, but better performance; It is an activation function; It is a natural constant.
2. The cervical tissue pathology image classification method based on a hybrid CNN-ViT architecture according to claim 1, characterized in that: The design of the LACCSA module includes at least the following steps: S200: By calculating the cross-covariance matrix , where K′ and Q′ are the L2 normalized feature key and query matrices, respectively, and T represents the matrix transpose operation. The L2 normalization operation ensures the consistency of feature dimensions and avoids the imbalance problem caused by differences in feature scale. S201: Calculating the attention weight matrix based on the cross-covariance matrix ,in, It is a temperature coefficient used to adjust the smoothness of the Softmax function and ensure the stability of the weight distribution; S202: Generating the final output features using the attention weight matrix: The output feature O achieves dynamic capture of global features through weighted summation, significantly improving the model's ability to understand complex pathological images. It is a value matrix.