A Multilayer Perceptron-Based Auxiliary Classification Method for Lung Tissue Pathological Images

Abstract

This invention relates to a multilayer perceptron-based auxiliary classification method for lung tissue pathological images, belonging to the field of pathological data image processing technology. It addresses the problems of existing convolutional neural networks and VisionTransformers, such as overly complex convolutional operations and self-attention mechanisms, resulting in long training times and complex classification methods. This invention achieves auxiliary classification of pathological image types through steps including obtaining a mixed dataset, building a multilayer perceptron model, and evaluating the model's performance. The multilayer perceptron model in this invention does not use any complex convolutional operations or self-attention mechanisms, resulting in a simple model structure, low computational cost, and fast processing speed. A dual-data-stream input method is designed, enabling the model to capture both global and local information, fully extracting semantic information from the image, and improving the model's classification performance for lung tissue pathological images.

Description

Technical Field

[0001] This invention relates to the field of pathological data image processing technology, specifically to an auxiliary classification method for lung tissue pathological images based on a multilayer perceptron. Background Technology

[0002] Lung cancer has become one of the leading causes of cancer worldwide. It ranks first in incidence among men and second among women. While the incidence of squamous cell carcinoma has been declining in recent years, accounting for approximately 30%–40% of lung cancer cases, the incidence of adenocarcinoma has been rising, accounting for approximately 40%–55% of lung cancer cases. Therefore, adenocarcinoma is the major subtype of lung cancer. Adenocarcinoma can be further divided into invasive adenocarcinoma and minimally invasive adenocarcinoma.

[0003] The most reliable scientific basis for doctors to diagnose lung cancer is through pathological examination. The specific subtype of lung cancer is determined by observing the size, shape, and location of cells in pathological images. Accurate pathological diagnosis is crucial for patient treatment. However, pathological diagnosis is also a complex task for pathologists, not only time-consuming and labor-intensive, but also prone to missed diagnoses and misdiagnoses.

[0004] With the rapid development of artificial intelligence, deep learning has been widely applied in various fields. Increasing research demonstrates the reliability of deep learning algorithms in medical image analysis, particularly in the identification of lung cancer tissue pathology images.

[0005] However, the algorithms used in the aforementioned studies to process lung cancer tissue pathology images are mostly Convolutional Neural Networks (CNNs) and Vision Transformers (VTs). CNNs target local information in the image, while VTs capture global information. The convolution operations in CNNs and the self-attention mechanism in VTs are both very complex operations, which leads to the complexity of the models and makes them unsuitable for deployment in assisted diagnostic systems.

[0006] Multilayer perceptron (MPB) models have the advantages of simple network structure and easy operation; however, MPBs can only capture global information, and existing technologies rarely use MPB algorithms. Therefore, there is an urgent need for an MPB algorithm based on both local and global information to process lung cancer pathology data. Summary of the Invention

[0007] To address the problems of overly complex convolutional operations and self-attention mechanisms in current mainstream algorithms such as convolutional neural networks and Vision Transformers, which lead to long training times and complex diagnostic systems, this invention provides an auxiliary classification method for lung tissue pathological images based on multilayer perceptrons.

[0008] A multilayer perceptron-based method for assisting in the classification of lung tissue pathological images includes the following steps:

[0009] Step 1: Obtain lung tissue pathology image data and collect publicly available lung tissue pathology data to create a hybrid dataset, and divide the hybrid dataset into a training set and a test set;

[0010] Step 2: Process the mixed dataset by adjusting the image size to a uniform dimension, normalizing the background color of the data, and performing data augmentation.

[0011] Step 3: Build a multilayer perceptron model. Input the mixed dataset processed in Step 2 into the multilayer perceptron model, extract feature maps in the feature extraction layer, and finally obtain the image classification results through global pooling and fully connected layers.

[0012] Step 4: Use the test set to test the performance of the multilayer perceptron model trained in Step 3, and complete the performance evaluation of the multilayer perceptron model.

[0013] The beneficial effects of this invention are:

[0014] The pathological image type-assisted classification method described in this invention uses a multilayer perceptron model that does not employ any complex convolution operations or self-attention mechanisms. This results in a simple model structure, low computational cost, and fast processing speed. A dual-data-stream input method is designed, enabling the model to capture both global and local information, fully extracting semantic information from the image and improving the model's classification performance for lung tissue pathological images. Attached Figure Description

[0015] Figure 1 This is a flowchart of the lung tissue pathological image type auxiliary classification method based on multilayer perceptron described in this invention;

[0016] Figure 2 This is a schematic diagram of the processing flow based on the multilayer perceptron model. Detailed Implementation

[0017] Specific Implementation Method 1: Combination Figure 1 and Figure 2 This embodiment describes a lung tissue pathological image type-assisted classification method based on a multilayer perceptron. The specific process of this method is as follows:

[0018] 1. The acquired lung tissue pathology data is processed into usable pathology data, and a hybrid lung pathology dataset is created by combining it with the lung tissue pathology data collected from the public dataset LC25000, and then divided into training and testing sets.

[0019] 2. Perform data processing on the hybrid dataset, adjust the image size to a uniform size, normalize the background color of the data, and perform data enhancement processing;

[0020] The hybrid lung pathology dataset includes five types of data: squamous cell carcinoma, adenocarcinoma, invasive adenocarcinoma, microinvasive adenocarcinoma, and normal lung tissue. These five types of data were resized to a uniform 256×256 pixels, and the background color was standardized to avoid interference during model training. Data augmentation was performed on the hybrid dataset, increasing its size fivefold (using five operations: horizontal flipping, rotation, scaling, height shifting, and width shifting) to fully train the model and allow for more thorough learning.

[0021] 3. Establish a multilayer perceptron model. Input the pathological image data (training set) processed in step 2 into the multilayer perceptron model, extract feature maps in the feature extraction layer, and finally obtain the image classification results through global pooling and fully connected layers.

[0022] In this embodiment, there is a large amount of local information in the pathological data. In order for the model to fully capture this local information, the input pathological image is divided into large blocks and then further divided into small blocks. The large blocks and small blocks together construct a dual data stream.

[0023] Semantic information is extracted from the data: We use two standard multilayer perceptron feature extraction layers (Mixer Layers) to extract local and global information from large and small blocks. The first Mixer Layer processes local information from several small blocks, and the second Mixer Layer processes global information between several large block sequences.

[0024] like Figure 2 As shown, the workflow of the multilayer perceptron model specifically includes:

[0025] Step S1: The input 3×256×256 image is divided into 16×16 large blocks, and the large blocks are further divided into 4×4 small blocks. Then, the three-dimensional large block sequence and the three-dimensional small block sequence are linearly mapped to form a two-dimensional list of large blocks and a two-dimensional list of small blocks. The two two-dimensional lists are used as the final input of the model.

[0026] Step S2: Input the two-dimensional list of small blocks into the first Mixer Layer in the feature extraction layer. The Mixer Layer mainly consists of two multilayer perceptron blocks. First, a multilayer perceptron is used to mix the information between different spatial locations, which is equivalent to a 1×1 convolution in a convolutional neural network. Then, a multilayer perceptron is used to fuse the information between different channels, which is equivalent to a generalized depthwise separable convolution. The Mixer Layer also uses residual structures and normalization operations to output the extracted feature map containing local information.

[0027] Step S3: In order to further and better process the information in the small block sequence, the two-dimensional list of small blocks and the two-dimensional list of large blocks in S1 are merged. The two-dimensional list of small blocks is linearly mapped to the same size as the two-dimensional list of large blocks, and then an additive fusion is performed.

[0028] Step S4: The second Mixer Layer processes the fused two-dimensional list in the same way as in Step S2. It still uses a multilayer perceptron to extract spatial features and then uses a multilayer perceptron to fuse the channel information. This also involves layer normalization and residual connection operations.

[0029] Step S5: The two Mixer Layers in the feature extraction layer output two types of feature maps containing different information: a global information feature map between large blocks and a local information feature map of small blocks within a large block. The feature maps are compressed by a global pooling layer in the classification layer and then input into a fully connected layer in the classification layer. The number of categories in the fully connected layer is set to the five types in the dataset. Finally, the fully connected layer outputs the specific results.

[0030] Fourth, the performance of the multilayer perceptron model trained in step three is tested using a test set to complete the performance evaluation of the multilayer perceptron model.

[0031] In this implementation, the model performance is evaluated using four metrics: accuracy, precision, sensitivity, and specificity.

[0032] Specific Implementation Method Two: This implementation method is an example of the lung tissue pathological image type-assisted classification method based on multilayer perceptron described in Specific Implementation Method One:

[0033] The hybrid dataset contains five data types: squamous cell carcinoma of the lung, adenocarcinoma of the lung, invasive adenocarcinoma of the lung, microinvasive adenocarcinoma of the lung, and normal lung tissue. The dataset is divided into training and testing sets according to a certain ratio. The model is trained on the training set and its learning ability is tested on the testing set. The model training process adopts a start-from-the-begins approach, which requires a relatively long time. The training epochs are set to 200 epochs. The larger the batch size, the shorter the training time per epoch. The training machine uses a single NVIDIA RTX 3090 GPU with 24GB of video memory. The batch size can be set according to the GPU memory size of the training machine. In this embodiment, the batch size is... The number of reads is set to 128. In this invention, the dimension of the input data X is (128, 3, 256, 256). After multiple training experiments, the feature extraction layer is stacked with 8 layers, each containing two MixerLayers. The finer the segmentation granularity of the blocks, the more information is extracted. However, the computational complexity of the model is linearly related to the number of block sequences. To balance this, the size of the large blocks is set to 16×16, and the size of the small blocks is set to 4×4. In the feature extraction layer, in order to incorporate non-linear components into the model, the GELU activation function is used, and the Adam optimizer is used. To prevent the model from learning too much and overfitting, the method of randomly killing neurons is used, and its parameter is set to 0.1.

[0034] In this embodiment, the pathological slide image of the patient is processed. In response to the large amount of local information in the pathological image, a dual data stream input is constructed in the original multilayer perceptron model that can capture global information. The image is divided into large blocks and small blocks. The feature extraction layer uses two Mixer Layers to process the global information between large blocks and the local information between small blocks within large blocks. The classification layer uses a global pooling layer and a fully connected layer to output the specific cancer type.

[0035] Two classic convolutional neural network models (InceptionV2, EfficientNet), two classic VisionTransformer models (SwinTransformer, ConViT), and two classic multilayer perceptron models (MLPMixer, gMLP) were selected and compared with the model (MIM) designed in this invention, as shown in Table 1, a comparison table of lung cancer pathological image diagnosis results. Four evaluation indicators, namely accuracy, precision, specificity, and sensitivity, were used to compare the model performance.

[0036] Table 1

[0037] Model accuracy accuracy Outliers Sensitivity MIM (this model) 98.48％ 97.41％ 97.12％ 99.69％ gMLP 96.51％ 96.25％ 95.39％ 99.34％ MLPMixer 95.62％ 93.10％ 92.57％ 99.05％ InceptionV2 96.83％ 95.31％ 95.42％ 99.45％ EfficientNet 97.08％ 95.34％ 95.68％ 99.49％ Swin 97.83％ 96.31％ 95.99％ 99.62％ ConViT 97.96％ 96.89％ 96.58％ 99.69％

[0038] As shown in Table 1, our method achieves the best results in all metrics compared to the other six existing mainstream algorithms by using only a simple multilayer perceptron algorithm without any complex convolution operations or attention mechanisms, and by extracting local and global information from the image.

[0039] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0040] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A lung tissue pathological image type-assisted classification method based on multilayer perceptron, characterized by: This method is implemented by the following steps: Step 1: Obtain lung tissue pathology image data and collect publicly available lung tissue pathology data to create a hybrid dataset, and divide the hybrid dataset into a training set and a test set; Step 2: Process the mixed dataset by adjusting the image size to a uniform dimension, normalizing the background color of the data, and performing data augmentation. In step two, the pathological image data is enhanced by performing horizontal flipping, rotation, scaling, height shifting, and width shifting operations, thereby expanding the amount of data in the dataset to five times its original size. Step 3: Build a multilayer perceptron model. Input the mixed dataset processed in Step 2 into the multilayer perceptron model, extract feature maps in the feature extraction layer, and finally obtain the image classification results through global pooling and fully connected layers. In step three, the multilayer perceptron model includes an input layer, a feature extraction layer, and a classification layer; The 3×256×256 image is divided into several 16×16 large blocks, and then the large blocks are divided into 4×4 small blocks. The large blocks and small blocks are linearly mapped to obtain a two-dimensional list of a large block and a two-dimensional list of a small block. The two two-dimensional lists are used as the final input of the multilayer perceptron model. The two-dimensional list of small patches is input into the first Mixer Layer in the feature extraction layer through the input layer. The first Mixer Layer obtains the local information feature map of the small patches in the two-dimensional list of small patches. The small two-dimensional list is linearly mapped to the same size as the large two-dimensional list, and then the small two-dimensional list and the large two-dimensional list are merged to obtain a merged large two-dimensional list. The second Mixer Layer in the feature extraction layer is used to obtain the global information feature map between large blocks in the fused large two-dimensional list; The classification results of the pathological image are obtained by passing the local information feature map of the small block and the global information feature map between the large blocks through a classification layer; Step 4: Use the test set to test the performance of the multilayer perceptron model trained in Step 3, and complete the performance evaluation of the multilayer perceptron model.

Images