A spatial perception enhanced pathological image classification method

Abstract

The application provides a spatial perception enhancement and pathological image classification method for processing class imbalance, aiming to enhance the model's perception of spatial information and process class imbalance. The specific steps are as follows: in the preprocessing stage, the image is segmented, the background is filtered out, and the image block is cut, and the coordinates are recorded; in the feature extraction and fusion stage, the pathological features are extracted through a pre-trained convolutional model, then the coordinates are normalized to form position features, and the pathological features and position features are fused to form comprehensive features with spatial information. In the reasoning stage, a multi-scale feature enhancement network is introduced, which can capture local features through convolution operations and model spatial relationships through attention mechanisms, thus achieving more comprehensive processing of local and spatial information. Finally, the parallel classifier is used to solve the problems of class imbalance and sample shortage, and the final pathological classification result is obtained. This method can better assist pathologists in improving the diagnosis accuracy and reducing subjective differences in the diagnosis of chondroma.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and pathological images, and in particular to a pathological image classification method with enhanced spatial perception. Background Technology

[0002] Chondroma is a relatively common bone tumor, most frequently occurring in adolescents and children. Its incidence varies by region and population, but is generally quite common and has a high prevalence. Although chondroma is a benign tumor, it can cause complications in certain situations, such as pathological fractures, nerve compression, or vascular complications. Furthermore, chondroma can sometimes transform into chondrosarcoma, especially in patients with multiple chondromas. Globally, the diagnosis and treatment of chondroma rely in part on imaging examinations and surgical interventions. With the development of imaging technology and science, deep learning technology has made significant progress in the field of medical image analysis, showing great potential, especially in the diagnosis of chondroma. For example, some studies have used deep learning to classify chondromas in CT images, automatically extracting image features through convolutional neural networks (CNNs), significantly improving the accuracy and efficiency of diagnosis.

[0003] However, in the medical field, pathological examination, as the "gold standard" for cancer diagnosis, provides the most reliable basis for clinical diagnosis by observing the cellular structure and morphology of tissues under a microscope, playing a crucial role in the diagnosis of chondroma. However, the pathological image features of chondroma are highly similar to those of other benign or malignant bone tumors, making the diagnostic process complex and challenging. Furthermore, pathological examination is highly dependent on the experience and expertise of pathologists, and subjective differences exist among different pathologists, which to some extent affects the accuracy and consistency of diagnosis. More importantly, tissue structural features are extremely critical in the diagnosis of chondroma; for example, the identification of osteochondroma requires attention to obvious pathological manifestations such as the periosteum, cartilage, and mature bone structures, as well as the scarcity of data, which further increases the difficulty of diagnosing chondroma in the field of pathological images. However, currently, deep learning classification tasks for chondroma in pathological images are almost unexplored. This indicates that developing a chondroma classification method that can enhance the model's perception of spatial information and handle class imbalance is urgently needed. This would not only assist pathologists in improving diagnostic efficiency but also effectively reduce subjective differences, possessing significant clinical significance and research value. Summary of the Invention

[0004] The purpose of this invention is to assist pathologists in improving diagnostic accuracy and reducing subjective differences in the diagnosis of chondroma. It proposes a chondroma classification method that enhances the model's ability to perceive spatial information and handle class imbalance. This method primarily utilizes a fusion mechanism of pathological and locational features, as well as a parallel classifier mechanism based on latent coding, for the classification of chondroma histopathology. Compared with the traditional transformation-based WSI classification MIL method, this method innovatively enhances the model's ability to perceive spatial information in pathological images and handle class imbalance in classification tasks. The technical solution adopted in this invention is as follows: A spatial perception enhancement and class imbalance handling method for pathological image classification includes the following steps: (1) In the data preprocessing stage, the model uses the OTSU algorithm to segment the whole slide of the pathological image to filter out the background area. After the background removal is completed, the processed image is cut into image blocks of the same size, 256×256 pixels; (2) Record the coordinates of the image blocks; then, extract depth features from the pathological images using a pre-trained convolutional model, and then normalize the extracted coordinates to form position features; (3) Integrate pathological features with location features to form a comprehensive feature with spatial information; (4) Introduce a multi-scale feature enhancement network. On the one hand, use convolution operations to capture local features, and on the other hand, use attention mechanisms to model spatial relationships, thereby achieving more comprehensive processing of local and spatial information. (5) Classification is performed using a parallel classifier to solve the problems of data class imbalance and small sample size, while obtaining the final pathological classification result; The beneficial effects of this invention are: (1) This invention proposes a method that can enhance the model’s perception of spatial information, assisting pathologists in improving diagnostic accuracy and reducing subjective differences in the diagnosis of chondroma.

[0005] (2) This invention proposes a fusion mechanism of pathological features and location features, which can comprehensively capture the tissue structure information between the whole slides of pathological images to enhance the model's perception of spatial information, making it more suitable for the diagnosis of chondroma.

[0006] (3) A parallel classifier mechanism is proposed, which can more accurately identify and extract the unique features of each category. In addition, the design of this mechanism allows for specialized optimization for each category, which is very effective for classification tasks in cases of class imbalance or large differences in category features. Attached Figure Description

[0007] Figure 1 This is a comprehensive flowchart of the present invention; Figure 2 This is the multi-scale feature enhancement network of the present invention. Detailed Implementation

[0008] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. The present invention is implemented in accordance with the following steps, and the comprehensive flowchart is as follows: Figure 1 As shown: (1) Data Acquisition This invention is based on a H&E-stained pathological image dataset of chondromas, which includes six types: osteochondroma, enchondroma, chondromycinoid fibroma, subungual exostosis, chondroblastoma, and chondrosarcoma. Due to the limited amount of data, to ensure full utilization of the data, reduce model bias, improve the model's generalization ability, and enhance the reliability of the evaluation results, five-fold cross-validation was adopted, with a patient-level ratio of 8:1:1. The training set was used for model training, and the validation set was used for model performance validation. Finally, the model's performance was evaluated on the test set. (2) Data preprocessing In the data preprocessing stage, the model uses the OTSU algorithm to segment the whole slide of the pathological image to filter out background regions. After background removal, the processed image is cut into image blocks of the same size (256×256 pixels), and their coordinates are recorded. Therefore, It can be represented as: in , , yes In quantity, Indicates the first One patch, Indicates the size of the patch. express The corresponding center position coordinates; (3) Integration of location features and pathological features To more comprehensively capture the tissue structure information between chondromas and enhance the model's ability to capture spatial information, the specific method consists of three steps. The first step is to form location features. as follows: First, input the original position coordinate matrix. : Then, calculate the row coordinate vector. and column coordinate vector Maximum and minimum values: Secondly, calculate the height of the organization. and width and the scale of coordinate transformation : ; Finally, normalize the coordinates: Obtain the normalized position features : The second step, feature fusion, is as follows: First, utilize pre-trained... The model applies to preprocessed full-slice images Perform feature extraction to obtain 3D deep feature vector Then, the normalized position features with depth features The features are fused together to form a comprehensive feature vector. : Finally, return to the integrated features after fusion. : The third step involves integrating features through a multi-scale feature enhancement network, enabling the entire model to perceive structural information more effectively. First, the input features are processed through an attention mechanism module. First, activation values ​​are generated using two parallel fully connected layers. and gating signals : , in, , and , These are learnable weights and bias parameters.

[0009] The activation value is controlled through a gating mechanism. and gating signals Multiplying them together yields the weighted feature representation: The weighted features are then passed through another fully connected layer to obtain each... attention weights : in, and These are learnable weights and bias parameters.

[0010] Then, to more effectively identify local patterns in the input features, a convolutional layer module was further integrated after applying the attention mechanism. The second convolutional layer module is as follows: in This represents a one-dimensional convolutional layer.

[0011] Finally, after obtaining each attention weights and convolutional features Then, they are weighted and summed to obtain the aggregated features. The third feature weighting module is as follows: First, attention weights application Normalize it to represent a probability distribution: Then, the normalized attention weights are summed with the convolutional features using a weighted average: in, yes The total number, It is the first indivual Attention weights It is the first indivual The convolutional features.

[0012] (4) Parallel classifier A parallel independent classifier mechanism is employed to address the issues of data sparsity and class imbalance in multi-class classification. By configuring a dedicated classifier for each class, the unique features of each class can be identified and extracted more accurately. Furthermore, this independent classifier design allows for specialized optimization for each class, which is highly effective in classification tasks under conditions of class imbalance or significant differences in class features. The specific method is as follows: First, for each category An independent linear classifier was designed. The weighted feature representation obtained through the convolutional attention module These classifiers are used to evaluate the weighted features M, thereby generating a corresponding predicted score for each category: in, Represents the number of categories and Defined as: in, and The learnable weight vector and bias term of the classifier.

[0013] Then, through The function transforms the predicted scores into a probability distribution, yielding the final classification result:

Claims

1. A pathological image classification method with enhanced spatial perception, characterized in that: It includes two parts: a mechanism for fusing pathological and locational features, and a parallel classifier mechanism based on latent coding. The steps include: (1) Tissue sections stained by H&E program are digitized into WSI, and then irrelevant background areas are filtered out by OTSU method. The processed image is then cut into image patches of the same size, 256×256 pixels. (2) Obtain their corresponding pathological features by using a pre-trained CNN model. Then, input the patch coordinates into the location encoding module to obtain location features; (3) Pathological features and location features are fused to obtain comprehensive features, which are then input into a multi-scale feature enhancement network; (4) In the multi-scale feature enhancement network, the attention weights are first calculated by the attention mechanism. Then, the convolutional layer is applied to process the features to better capture local features and patterns. Finally, the convolutional features and the attention weights generated by each patch are weighted and summed to obtain the aggregated features. (5) Use a parallel classifier based on latent encoding to predict the aggregated features.

2. The spatially enhanced pathological image classification method according to claim 1, characterized in that: The fusion mechanism of pathological and location features, obtained in steps (2), (3), and (4), feature fusion, and through a multi-scale feature enhancement network: The first step is to segment the image to obtain images of the same size. Then to Extract the position coordinates, and then process the input position coordinate matrix. Normalization is performed to obtain location features : ； Calculate the maximum and minimum values ​​of the row and column coordinate vectors: ； ； Determine the overall height of the organization and width : ； Set the scale of coordinate transformation : ； Normalize each coordinate: ； Generated normalized location features : ； The second step is to fuse pathological features and location features. First, we use a pre-trained CNN model to extract features from the pre-processed WSI, obtaining a 512-dimensional deep feature vector. Then, we integrate the two-dimensional location features into these features, resulting in a fused composite feature. The dimensions have been expanded to 514. ； The third step, feature integration, utilizes a multi-scale feature enhancement network to enable the entire model to perceive structural information more effectively. First, the input features are processed through an attention mechanism module. First, activation values ​​are generated using two parallel fully connected layers. and gating signals : . ； By activating value and gating signals Multiplication yields weighted features: ； The weighted features are then passed through another fully connected layer to obtain the attention weights for each patch. : ； Then, to more effectively identify local patterns in the input features, we further integrate a convolutional layer module after applying the attention mechanism: ； ； Finally, obtain the attention weights. and convolutional features The aggregated features of each patch are obtained by weighting them together. : 。 3. The spatially enhanced pathological image classification method according to claim 1, characterized in that: A parallel classifier mechanism based on latent encoding, wherein step (5) is performed for each class Design an independent linear classifier. Then, weighted features are obtained through convolutional attention. Using a parallel classifier to weight features An evaluation is performed to generate a corresponding predicted score for each category, which do not affect each other: ； ； Then, through The function transforms the predicted scores into a probability distribution, yielding the final classification result: 。

Images