Layer-by-layer lesion classification method for gastric cancer pathological diagram

Abstract

A lesion layer-by-layer classification method for a gastric cancer pathological diagram comprises the following steps: 1) image preprocessing and effective Patch cutting are carried out, and an area containing enough tissue information is extracted to improve a model training effect; 2) cancer pathological features are extracted by using cross-gastric-cancer-level supervised contrast learning, and the confusion influence between adjacent levels of pathological features is reduced; 3) scoring the Patch by using a gating attention mechanism to obtain importance distribution of the key lesion area; 4) constructing a multi-level classification structure of a Patch level based on a cancer severity priority; 5) predicting a pathological picture by using the constructed model; and 6) performing post-processing on a model prediction result to generate a thermodynamic diagram and a visual image of the canceration region to assist doctors in diagnosis. According to the method, canceration regions of different levels can be distinguished more accurately, and the interpretability and robustness of an intelligent pathological diagram diagnosis system in practical application are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image classification, and particularly relates to a lesion hierarchical classification method for gastric cancer pathological images. BACKGROUND

[0002] Gastric cancer is the fifth most common cancer worldwide and the fourth leading cause of cancer-related deaths, with a global 5-year survival rate hovering around 20%. Despite extensive research into the pathobiology of the disease, predicting its progression remains difficult, which leads to persistently low survival rates. In addition, the complexity and time-consuming nature of medical diagnosis can result in the omission of critical details during microscopic examination, which can potentially lead to misdiagnosis. With the development of deep learning technology, deep learning combined with gastric cancer diagnosis has made great progress. In practical applications, CT images, clinical diagnosis information, gastroscopy images, whole slide images, and advanced image processing and deep learning algorithms are mainly used to improve the accuracy of gastric cancer diagnosis technology, thereby reducing the probability of misdiagnosis and the workload of doctors. However, traditional gastric cancer diagnosis methods ignore different levels of gastric cancer, which gradually worsen from low to high. At the current level of gastric cancer, the tissue block contains cancer at a lower level or even two lower levels, which leads to a large amount of noise in the gastric cancer image dataset during deep learning training. Therefore, there are still a large number of misjudgments in gastric cancer diagnosis. At the same time, the diagnosis of the entire pathological picture does not locate the lesion area of the tissue region, so doctors still consume a large amount of time in checking the entire pathological picture. Therefore, it is necessary to propose a method that can reduce dataset noise, fully utilize disease feature differences, and locate possible lesion areas.

[0003] Existing pathological image classification methods mostly adopt general deep learning architectures, such as global classification models based on convolutional neural networks (CNN), which aim to diagnose and judge the entire pathological image through feature extraction and classifier design. However, due to the high resolution, complex structure, and uneven distribution of lesion areas of pathological images, traditional single-label or multi-label classification methods often struggle to accurately model the priority relationship of multiple coexisting diseases in tissue sections. In recent years, Multiple Instance Learning (MIL) has been widely applied to the classification task of Whole Slide Images (WSI), which treats an entire image as a bag containing multiple image patches. By extracting features from each patch and aggregating them, the method achieves prediction on the entire image.

[0004] Although MIL has achieved certain results in pathological image classification, the current methods still have several key problems. First, most existing MIL methods ignore the clinical priority relationship between different lesion types in the patch-level classification process, and usually adopt independent classification or multi-label strategies, which makes the model prone to misjudgment when faced with the coexistence of multiple lesions in the same tissue block. Secondly, when dealing with multi-level diseases, existing methods often adopt a hard division method in the feature extraction stage, forcibly distinguishing lesions of adjacent levels, ignoring the continuity and progressive characteristics in the disease evolution process, thereby affecting the generalization ability of the model and limiting the interpretability and clinical practicality of the model. In the present invention, medical prior knowledge is combined with traditional model training, and adjacent levels of symptoms are not forced to be separated, reducing the impact of noise in model training. When classifying images, priority is arranged according to the severity of the disease to better prevent serious cases from being ignored, improve the clinical interpretability and clinical practicality of the model, and improve the generalization ability of the model. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a hierarchical classification method for gastric cancer pathology images.

[0006] One of the purposes of the present invention is to provide a hierarchical classification method for gastric cancer pathology images that is less susceptible to noise, has high reliability, and is versatile.

[0007] A second object of the present invention is to provide a thermal imaging method for image block classification and pathology image classification.

[0008] The present invention provides a method for classifying gastric cancer pathology images hierarchically, comprising the following steps:

[0009] S1. Obtain an existing dataset;

[0010] S2. Preprocess the pathological images in step 1 to construct a training dataset;

[0011] S3. Based on the Swin-Transformer network, a hierarchical classification network for lesions in pathology images was constructed.

[0012] A feature extractor is built based on Swin-Transformer to extract features from the input pathological image blocks and output the feature vector of each image block;

[0013] An attention weight allocation module is built based on the gated attention mechanism. The attention weight allocation module is used to assign an attention value to each feature vector and linearly aggregate it into a feature vector for the entire pathological image, reducing the impact of non-critical information on the model.

[0014] Two classification modules are constructed based on the full connection layer; the classification modules are used for classifying the feature vectors of the entire pathological picture obtained by the gating attention module weight block and the feature vectors of the pathological image blocks obtained by the feature extractor;

[0015] S4. Using the training data set constructed in step 2, training the various modules of the pathological image classification constructed in step S3 according to different training strategies to obtain a pathological image classification model;

[0016] S5. Using the pathological image classification model obtained in step S4, performing actual entire pathological image and pathological image block classification.

[0017] The existing pathological image in step S1 specifically includes the following contents:

[0018] The existing pathological image is a pathological image containing one or more pathological tissues, and has accurate annotations for the pathological tissues. The entire pathological image needs to be labeled with label information, and the pathological tissues need to be cropped and labeled.

[0019] The preprocessing in step S2 specifically includes the following steps:

[0020] Image block annotation area range filtering: for each annotated tissue area, perform binaryzation processing, i.e. the annotation area is white and the non-annotation area is black. Crop the pathological image block from this part, and require that the proportion of white pixels in the cropped image block under the condition of binaryzation annotation area is not less than a set value, to complete the reading and filtering of the image block;

[0021] Image block effective information set value filtering: extract each image block; for each image block, analyze the red channel in the image block, count the proportion of pixel values less than a set value, and determine that the pixel proportion less than the set value is background, to complete the reading and filtering of the image block;

[0022] Image block edge information set value filtering: extract each image block; for each image block, analyze the edge information in the image block, count the edge information value, and determine that the image block with edge information less than a set value lacks sufficient information, to complete the reading and filtering of the image block.

[0023] The Swin-Transformer network based feature extraction module in step S3 is constructed, specifically including the following steps:

[0024] The image blocks obtained in step S2 are rearranged;

[0025] Each image block is divided into a plurality of non-overlapping small windows according to the set requirements, and each window contains a fixed number of image patches;

[0026] The image blocks in each window are encoded by using a window-in attention mechanism;

[0027] The small window after encoding is moved in the spatial dimension to form a cross-window interaction mechanism, thereby realizing the interaction of context information between image blocks;

[0028] The image features are input into a pre-trained Swin Transformer model for processing, and the Swin Transformer model is composed of a plurality of hierarchical Transformer modules, each module encoding the image blocks through a window-in multi-head attention mechanism and a feedforward network;

[0029] The processing process of each layer of the Transformer network includes the following operations: LayerNorm(x+WindowAttention(x,x,x)) and LayerNorm(x+FeedForward(x)), wherein LayerNorm() is a layer normalization operation, x is an input feature, WindowAttention() is a multi-head attention mechanism based on a local window, and FeedForward() is a fully connected network processing function;

[0030] After processing by each layer of the Transformer module, the feature map is down-sampled and the channel number is increased through a Patch Merging operation; the receptive field is gradually expanded and global feature information is obtained through layer-by-layer stacking;

[0031] Finally, the encoded image block features are reshaped into a two-dimensional feature map, and feature aggregation is performed to obtain a high-level semantic representation of each image block.

[0032] The attention distribution module based on the gating attention mechanism described in step S3 specifically includes the following steps:

[0033] The feature vector of the input image block is processed by a fully connected layer for dimension reduction, and the processing process of the fully connected layer is represented as out=Wx+b, wherein x is the input of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer.

[0034] The dimension-reduced feature vector is mapped using two independent fully connected layers, and the processes are a=tanh(W a x+b a ) and b=σ(W b x+b b ), respectively, wherein tanh is the hyperbolic tangent activation function, σ is the sigmoid activation function, W a , and W bare the weight matrices in the two fully connected layers, x is the input of the fully connected layer after dimension reduction, b a and b b are the bias vectors in the two fully connected layers, respectively.

[0035] The two mapping results are multiplied element by element to obtain the gated attention representation A = a © b; then an linear layer a = W c A + b c , wherein W c is a weight matrix.

[0036] The values in the final obtained a represent the attention scores of each image block, and the features of each image block are linearly summed according to the corresponding attention scores, as follows: z = sum (a x x), wherein z is the feature vector of the entire pathological picture, a is the attention score matrix, and x is the input feature vector matrix.

[0037] The two classification modules constructed based on the fully connected layers in step 3 have the following specific structures:

[0038] The first classification module is for classifying the feature vector of the entire image, and includes a fully connected layer, the processing process of which is represented as y = Wx + b, y is the output of the fully connected layer, x is the input of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer.

[0039] The second classification module is a multi-level classifier composed of multiple single-disease binary classifiers for multi-classification. Each binary classifier has the same structure and classifies the feature vector of each image block, including a first full connection layer, a first activation layer, a second full connection layer, a second activation layer, and a third full connection layer. The first full connection layer is used to map the input features to a space with a set dimension, the first activation layer is used to activate the output of the first full connection layer to eliminate the linear relationship from the first full connection layer to the second full connection layer, the second full connection layer is used to compress the input features to a space with a set dimension, the second activation layer is used to activate the output of the second full connection layer to eliminate the linear relationship from the second full connection layer to the third full connection layer, and the third full connection layer is used to map the input features to a space with a set dimension. The processing process of the first full connection layer is represented as out1=W1x1+b1, where out1 is the output of the first full connection layer, x1 is the input of the first full connection layer, and b1 is the bias vector of the first full connection layer. The processing process of the second full connection layer is represented as out2=W2x2+b2, where out2 is the output of the second full connection layer, x2 is the input of the second full connection layer, and b2 is the bias vector of the second full connection layer. The processing process of the third full connection layer is represented as out3=W3x3+b3, where out3 is the output of the third full connection layer, x3 is the input of the third full connection layer, and b3 is the bias vector of the third full connection layer.

[0040] The training in step S4 specifically includes the following steps:

[0041] The feature extractor constructed based on the Swin-Transformer is trained using cross-cancer level supervised contrast learning, and the following formula is used as the loss function of each image block: where i, j, and k represent the indices of different image blocks, q i represents the feature obtained after the i-th image block is extracted by the network, q k represents the feature obtained after the k-th image block is extracted by the network, q j represents the feature obtained after the j-th image block is extracted by the network. S(i) represents a set of indices with the same class as the i-th image block, F(i) represents a set of indices that need to be compared with the i-th image block in terms of features, and τ represents a temperature parameter.

[0042] The following formula is used as the total loss function: where N is the number of image blocks.

[0043] The Adam optimizer is used to train the model.

[0044] The attention weight distribution module constructed based on the gating attention mechanism and the classifier composed of only one full connection layer adopt cross entropy as a loss function.

[0045] The model is trained by using an Adam optimizer.

[0046] For each binary classifier in the multi-level classification head, cross entropy is used as a loss function.

[0047] The model is trained by using an Adam optimizer.

[0048] The application further provides an imaging method using the pathological image layer-by-layer classification method for a gastric cancer pathological image.

[0049] S6. The pathological image classification result obtained in step S5 is subjected to heat map imaging on an actual pathological image, and finally a pathological image with a pathological image classification result is output.

[0050] The innovation of the application is that supervised contrast learning across cancer levels is used to reduce the noise influence caused by adjacent levels of cancer. The multi-level classification head is used to realize the correspondence between cancer severity priority and diagnosis priority, to consider cancer image blocks to severe diseases first, and to reduce the missed diagnosis of cancer.

[0051] The working principle of the application is that for cancer pathological tissues, cancer is a gradual deterioration process, so a cancer tissue block of a cancer level is composed of multiple cancer tissue blocks with the current cancer as the main one, and the possibility of deterioration to the current cancer is relatively large. Therefore, although the annotation region of the tissue region is correct after the image block is cropped from the tissue region, there will be noise in the cropped image block. This will affect the training of the model and the discrimination of the classification. Through supervised contrast learning across levels, the influence of image blocks with incorrect labels on model training can be reduced. Through the multi-level classification head, similar image blocks can be preferentially discriminated as more severe diseases to prevent missed diagnosis of diseases caused by noise.

[0052] The pathological image layer-by-layer classification method for a gastric cancer pathological image provided by the application is used for training and actual pathological image classification. Therefore, the application can not only realize the classification of pathological images, but also reduce the influence of noise on model training while ensuring high accuracy and reliability.

[0053] The advantage of the application is that it can more accurately distinguish cancer regions of different levels, and significantly improve the explainability and robustness of the intelligent pathological image diagnosis system in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 It is a method flowchart of the classification method of the application.

[0055] Figure 2 The method flowchart of the imaging method of the present application. DETAILED DESCRIPTION

[0056] The technical solutions of the present application are further described below with reference to the drawings.

[0057] Example 1

[0058] As Figure 1 The flowchart of the classification method of the present application is shown: the lesion hierarchical classification method for the pathological image of gastric cancer provided by the present application includes the following steps:

[0059] S1. Obtain an existing pathological image; specifically including the following contents:

[0060] The existing pathological image is a pathological image containing one or more lesion tissues, and has accurate annotations for the lesion tissues. The entire pathological image needs to be labeled with label information, and the lesion tissues need to be cropped and labeled at the same time. For example: for gastric cancer, a type of disease, the positive pathological image contains low-grade cancer tissue, high-grade cancer tissue, poorly differentiated cancer tissue, and signet ring cell cancer tissue;

[0061] S2. Preprocess the pathological image obtained in step S1 to construct a training data set, specifically including the following steps:

[0062] Image block annotation area range filtering: for each annotated tissue area, perform binary processing, i.e. the annotation area is white and the non-annotation area is black. Crop the pathological image block from this part, and require that the proportion of white pixels in the cropped image block under the condition of binary annotation area is not less than a set value, to complete the reading and filtering of the image block;

[0063] Image block effective information set value filtering: extract each image block; for each image block, analyze the red channel in the image block, count the proportion of pixels with pixel values less than a set value, and identify the background if the pixel proportion is less than the set value, to complete the reading and filtering of the image block.

[0064] Image block edge information set value filtering: extract each image block; for each image block, analyze the edge information in the image block, count the edge information value, and identify the image block as lacking sufficient information if the edge information is less than a set value, to complete the reading and filtering of the image block;

[0065] In specific implementation, a high-level image processing library can be used to accurately extract each image block (such as 224*224) from a large-size pathological picture; the extracted image block is subjected to preliminary filtering to exclude non-labeled areas, background areas or information-lacking areas, and after the labeled area is binarized, the white pixel value is analyzed to identify the labeled area; the pixel value of the red channel (the proportion of pixel values less than 200) is analyzed to identify the block containing a large amount of background; the edge information value of the image block (the edge information value is less than 30) is analyzed to identify the image block lacking information; this filtering operation further optimizes to ensure the relative accuracy of the image block and the sufficient effective information contained in the image block;

[0066] The above preprocessing operation can ensure that the image block of a uniform specification is obtained regardless of the size of the original image, ensure the label accuracy of the image block, and provide an effective image block;

[0067] S3. Constructing a pathological image lesion hierarchical classification network based on a Swin-Transformer network;

[0068] The constructed model includes a feature extraction module, an attention score module and two classification modules; the feature extraction module is used for feature extraction of the input pathological image block and outputs a feature vector of each image block; the attention score module is used for assigning weights to the feature vector of each image block and performing linear weighted summation to aggregate a feature vector of a pathological image, which is used to reduce the influence of non-key image blocks on classification; the two classification modules are respectively used for classification of the pathological image according to the feature vector generated by the attention score module and for classification of the pathological image block according to the feature vector generated by the feature extraction module;

[0069] Specifically comprising the following steps:

[0070] Constructing a feature extraction module based on a Swin-Transformer network; specifically comprising the following steps:

[0071] Rearranging the image block obtained in step S2: in specific implementation, the rerrange function of the einops library is used to realize, which is represented as X windows = rerrange(X unfolded , 'bc(wm1)(hm2)->(bwm)m1m2c', m1=M, m2=M) indicates that the image is divided into multiple M*M small windows, and the windows are rearranged into a new tensor;

[0072] X windows represents the rearranged image block; X unfolded represents the original input image; b is the batch size; c is the number of image channels; w is the width of the image block; h is the height of the image block; m1 and m2 are the row and column numbers in the window; M is the window size;

[0073] Each image block is divided into several non-overlapping small windows according to the set requirements, and each window contains a fixed number of image patches;

[0074] The image block in each window is encoded by using a window attention mechanism;

[0075] The encoded small window is moved in the spatial dimension to form a cross-window interaction mechanism, thereby realizing the context information interaction between image blocks;

[0076] The image features are input into a pre-trained Swin Transformer model for processing, and the Swin Transformer model is composed of several layered Transformer modules, each of which encodes the image block through a window multi-head attention mechanism and a feedforward network;

[0077] The processing process of each layer of the Transformer network includes the following operations: LayerNorm(x+WindowAttention(x,x,x)) and LayerNorm(x+FeedForward(x)), where LayerNorm() is a layer normalization operation, x is an input feature, WindowAttention() is a multi-head attention mechanism based on a local window, and FeedForward() is a fully connected network processing function;

[0078] After processing by each layer of the Transformer module, the feature map is downsampled and the channel number is increased through a Patch Merging operation; the receptive field is gradually expanded and global feature information is obtained through layer-by-layer stacking;

[0079] Finally, the encoded image block features are reshaped into a two-dimensional feature map, and feature aggregation is performed to obtain a high-level semantic representation of each image block;

[0080] An attention allocation module is constructed based on a gated attention mechanism, including the following steps:

[0081] The feature vector of the input image block is processed by a fully connected layer for dimension reduction, and the processing process of the fully connected layer is represented as out=Wx+b, where out is the output of the fully connected layer, x is the input of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer;

[0082] The dimension-reduced feature vector is mapped using two independent fully connected layers, and the processes are a=tanh(W a x+b a ) and b=σ(W bx + b b ), where tanh is the hyperbolic tangent activation function, and is the sigmoid activation function, W a and W b are the weight matrices of the two fully connected layers, respectively, x is the input of the fully connected layer after dimension reduction, b a and b b are the bias vectors of the two fully connected layers, respectively.

[0083] The two mapping results are multiplied element by element to obtain the gated attention representation A = a ® b; then an linear layer is passed through to obtain a = W c A + b c , where W c is the weight matrix.

[0084] The values in the final obtained a represent the attention scores of each image block, and the features of each image block are linearly summed according to the corresponding attention scores, as follows: z = sum (a x x), where z is the feature vector of the entire pathological picture, a is the attention score matrix, and x is the input feature vector matrix.

[0085] The feature vector finally obtained by the attention allocation module can reduce the influence of non-key image blocks on feature information, capture key local feature information in the pathological image, and significantly improve the processing speed and accuracy.

[0086] Two classification modules are constructed based on fully connected layers, and the specific structure is as follows:

[0087] The first classification module is for classifying the feature vector of the entire image, which includes a fully connected layer, and the processing process of the fully connected layer is represented as y = Wx + b, where y is the output of the fully connected layer, x is the input of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer.

[0088] The second classification module is a multi-level classifier composed of multiple single-disease binary classifiers for multi-classification. Each binary classifier has the same structure, which is to classify the feature vector of each image block, including a first full connection layer, a first activation layer, a second full connection layer, a second activation layer, and a third full connection layer. The first full connection layer is used to map the input features to a space with a specified dimension. The first activation layer is used to activate the output of the first full connection layer to eliminate the linear relationship from the first full connection layer to the second full connection layer. The second full connection layer is used to compress the input features to a space with a specified dimension. The second activation layer is used to activate the output of the second full connection layer to eliminate the linear relationship from the second full connection layer to the third full connection layer. The third full connection layer is used to map the input features to a space with a specified dimension. The processing process of the first full connection layer is represented as out1=W1x1+b1, where out1 is the output of the first full connection layer, x1 is the input of the first full connection layer, and b1 is the bias vector of the first full connection layer. The processing process of the second full connection layer is represented as out2=W2x2+b2, where out2 is the output of the second full connection layer, x2 is the input of the second full connection layer, and b2 is the bias vector of the second full connection layer. The processing process of the third full connection layer is represented as out3=W3x3+b3, where out3 is the output of the third full connection layer, x3 is the input of the third full connection layer, and b3 is the bias vector of the third full connection layer. In specific implementation, for the input of the feature vector of each image block, first, the binary classifier of the most serious disease is used for judgment. If the judgment for this disease is negative, the binary classifier of the relatively less serious disease is used for judgment. If it is still negative, the judgment is performed again. If all binary classifiers are negative, the image block is judged as negative. If one of the binary classifiers is positive in the successive judgment, the image block is judged as the disease type. This module can increase the accuracy of the model in recognizing single diseases and make the model pay more attention to serious diseases.

[0089] S4. Training the lesion hierarchical classification model for the gastric cancer pathological graph constructed in step S3 using the training data set constructed in step S2 to obtain the hierarchical classification model;

[0090] In specific implementation, the training specifically includes the following steps:

[0091] The feature extractor constructed based on Swin-Transformer is trained using cross-cancer level supervised contrast learning, and the following formula is used as the loss function of each image block: where i, j, and k represent the indexes of different image blocks, q i represents the feature obtained after the i-th image block is extracted by the network, q krepresents the feature obtained after the kth image block is extracted through the network, q j represents the feature obtained after the jth image block is extracted through the network. S(i) represents a set of indexes of the same category as the ith image block, F(i) represents a set of indexes of the image blocks that need to be compared with the image block with the current index i in terms of features, and τ represents a temperature parameter.

[0092] The following formula is used as the total loss function: Wherein N is the number of image blocks.

[0093] The model is trained by using an Adam optimizer.

[0094] The attention weight distribution module constructed based on the gating attention mechanism and the classifier composed of only one fully connected layer adopt cross-entropy as the loss function.

[0095] The model is trained by using an Adam optimizer.

[0096] For each binary classifier in the above multi-level classification head, cross-entropy is used as the loss function.

[0097] The model is trained by using an Adam optimizer.

[0098] Embodiment 2

[0099] As Figure 2 shown in the method flowchart of the imaging method of the present application: the present embodiment provides an imaging method applying the lesion hierarchical classification method for gastric cancer pathological images of embodiment 1, comprising the following steps:

[0100] S1. Obtain an existing pathological image;

[0101] S2. Preprocess the pathological image obtained in step S1 to construct a training data set;

[0102] S3. Based on the Swin-Transformer network, construct a pathological image hierarchical classification network;

[0103] S4. Using the training data set constructed in step 2, train each module of the pathological image classification constructed in step S3 according to different training strategies to obtain a pathological image classification model;

[0104] S5. Using the pathological image classification model obtained in step S4, classify the actual entire pathological image and pathological image block.

[0105] S6. Perform heat map imaging on the actual pathological image based on the pathological image classification result obtained in step S5, and finally output the pathological image with the pathological image classification result.

[0106] The imaging method disclosed by the application can be directly applied to existing pathological image acquisition devices or computer devices. In specific application, an existing scheme is used to acquire an actual pathological image, the pathological image is input into a device in which the model is deployed, a classification prediction result of the pathological image is obtained, and the classification prediction result is displayed on the pathological image through text and a heat map, so as to help medical staff quickly find a lesion area and promote subsequent work.

Claims

1. A method for hierarchical classification of lesions based on gastric cancer pathology images, comprising the following steps: S1. Obtain an existing dataset; S2. Preprocess the pathological images in step 1 to construct a training dataset; S3. Based on the Swin-Transformer network, a hierarchical classification network for lesions in pathology images was constructed. A feature extractor is built based on Swin-Transformer to extract features from the input pathological image blocks and output the feature vector of each image block; An attention weight allocation module is built based on the gated attention mechanism. The attention weight allocation module is used to assign an attention value to each feature vector and linearly aggregate it into a feature vector for the entire pathological image, reducing the impact of non-critical information on the model. Two classification modules are constructed based on the fully connected layer; the classification module is used to classify the feature vector of the entire pathological image obtained by the gated attention module weight block and the feature vector of the pathological image block obtained by the feature extractor; S4 using the training data set constructed in step 2, the various modules of the pathological image classification constructed in step S3 are trained according to different training strategies to obtain a pathological image classification model; S5. Use the pathological image classification model obtained in step S4 to perform actual classification of the entire pathological image and pathological image blocks.

2. The method for classifying lesions hierarchically based on gastric cancer pathology images according to claim 1, wherein: The existing pathological image described in step S1 specifically includes the following: Existing pathological images contain one or more pathological tissues and have accurate annotations for the diseased tissues; it is necessary to mark the entire pathological image with label information, and at the same time, crop the diseased tissues and mark the labels.

3. The method for hierarchical classification of lesions based on gastric cancer pathology images according to claim 1, characterized in that: The pre-processing described in step S2 specifically includes the following steps: Image block annotated area range filtering: For each annotated tissue area, perform binarization processing, that is, the annotated area is white and the unannotated area is black; the pathology image block is cropped from this part, and the proportion of white pixels in the cropped image block in the binarized annotated area is required to be no less than the set value, completing the image block reading and filtering; Image block valid information set value filtering: extract each image block; analyze the red channel in each image block, count the pixel ratio of pixels with a pixel value less than the set value, and identify the pixels with a pixel ratio less than the set value as background, completing the reading and filtering of the image block; Image block edge information set value filtering: extract each image block; analyze the edge information in each image block, count the edge information value, determine that the image block with edge information less than the set value lacks sufficient information, and complete the reading and filtering of the image block.

4. The method for hierarchical classification of lesions based on gastric cancer pathology images according to claim 1, characterized in that: The feature extraction module is constructed based on the Swin-Transformer network in step S3, which specifically includes the following steps: Rearranging the image blocks obtained in step S2; Divide each image block into several non-overlapping small windows according to the set requirements, and each window contains a fixed number of image patches; The window attention mechanism is used to encode the image blocks within each window; The encoded small window is moved in the spatial dimension to form a cross-window interaction mechanism, thereby realizing the interaction of contextual information between image blocks; The image features are input into the pre-trained Swin Transformer model for processing. The Swin Transformer model consists of several layered Transformer modules, each of which encodes the image block through a multi-head attention mechanism within the window and a feedforward network; The processing of each layer of the Transformer network includes the following operations: LayerNorm(x+WindowAttention(x,x,x)) and LayerNorm(x+FeedForward(x)), where LayerNorm() is the layer normalization operation, x is the input feature, WindowAttention() is the multi-head attention mechanism based on the local window, and FeedForward() is the fully connected network processing function; After each layer of Transformer module processing, the feature map is downsampled through the Patch Merging operation, while the number of channels is increased. The receptive field is gradually expanded and global feature information is obtained by stacking layer by layer. Finally, the encoded image block features are reshaped into a two-dimensional feature map, and feature aggregation is performed to obtain a high-level semantic representation of each image block.

5. The method for hierarchical classification of lesions based on gastric cancer pathology images according to claim 1, characterized in that: The construction of the attention allocation module based on the gated attention mechanism described in step S3 specifically includes the following steps: The feature vector of the incoming image block is processed through a fully connected layer for dimensionality reduction. The processing process of this fully connected layer is expressed as out = Wx + b, where x is the output of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer. For the feature vector after dimensionality reduction, two independent fully connected layers are used to map it respectively. The process is a=tanh(W a x+b a ) and b=σ(W b x+b b ), where tanh is the hyperbolic tangent activation function, σ is the sigmoid activation function, and W a and W b They are the weight matrices in the two fully connected layers, x is the input of the fully connected layer after dimensionality reduction, and b a and b b are the bias vectors in the two fully connected layers respectively; Multiply the above two mapping results element by element to get the gated attention represented as A=a⊙b; then pass it through a linear layer α=W c A+b c , where W c is the weight matrix; The final value of α represents the attention score of each image block. The features of each image block are linearly summed according to the corresponding attention score. The process is as follows: z = sum(α×x), where z is the eigenvector of the entire pathological image, α is the attention score matrix, and x is the input eigenvector matrix.

6. The method for classifying lesions layer by layer based on gastric cancer pathology images according to claim 1, wherein: The two classification modules built based on the fully connected layer in step 3 have the following specific structures: The first classification module classifies the feature vector of the entire image classification and includes a fully connected layer. The processing process of the fully connected layer is expressed as y = Wx + b, where y is the output of the fully connected layer, x is the input of the fully connected layer, W is the weight matrix of the fully connected layer, and b is the bias vector of the fully connected layer. The second classification module is a multi-level classifier, consisting of multiple binary classifiers for single diseases, used to achieve multi-classification. Each binary classifier has the same structure and classifies the feature vector of each image block. It includes the first fully connected layer, the first activation layer, the second fully connected layer, the second activation layer, and the third fully connected layer. The first fully connected layer is used to map the input features to a space of a set dimension. The first activation layer is used to activate the output of the first fully connected layer to eliminate the linear relationship from the first fully connected layer to the second fully connected layer. The second fully connected layer is used to compress the input features to a space of a set dimension. The second activation layer is used to activate the output of the second fully connected layer to eliminate the linear relationship from the second fully connected layer to the third fully connected layer; the third fully connected layer is used to map the input features to the set dimensional space; the processing process of the first fully connected layer is expressed as out1=W1x1+b1, out1 is the output of the first fully connected layer, x1 is the input of the first fully connected layer, and b1 is the bias vector of the first fully connected layer; the processing process of the second fully connected layer is expressed as out2=W2x2+b2, out2 is the output of the second fully connected layer, x2 is the input of the second fully connected layer, and b2 is the bias vector of the second fully connected layer; the processing process of the third fully connected layer is expressed as out3=W3x3+b3, out3 is the output of the third fully connected layer, x3 is the input of the third fully connected layer, and b3 is the bias vector of the third fully connected layer.

7. The method for hierarchical classification of lesions based on gastric cancer pathology images according to claim 1, characterized in that: The training described in step S4 specifically includes the following steps: The feature extractor built on Swin-Transformer is trained using supervised contrastive learning across cancer levels, with the following formula as the loss function for each image patch: Where i, j, k represent the indexes corresponding to different image blocks, q i represents the features of the i-th image block after extraction through the network, q k represents the features obtained after the kth image block is extracted through the network, q j Represents the features obtained after the j-th image block is extracted through the network; S(i) represents the set of indices of the same category as the i-th image block, F(i) represents a set that represents the indices of the image blocks that need to be compared with the image block with the current index i in terms of features, and τ represents the temperature parameter; The following formula is used as the total loss function: Where N is the number of image blocks; The Adam optimizer is used to train the model; An attention weight allocation module built on a gated attention mechanism and a classifier consisting of only one fully connected layer, using cross entropy as the loss function; The Adam optimizer is used to train the model; For each binary classifier in the above multi-level classification head, cross entropy is used as the loss function; The Adam optimizer is used to train the model.

8. An imaging method using the method for hierarchical classification of lesions for gastric cancer pathology images according to claim 1, characterized in that Also includes: S6. Perform thermal imaging on the actual pathological image using the pathological image classification result obtained in step S5, and finally output a pathological image with the pathological image classification result.

Images