Lung cancer pathology image classification system based on multi-scale fusion and attention mechanism

Abstract

The present invention proposes a lung cancer pathology image classification system based on multi-scale fusion and attention mechanism, which relates to the technical field of pathology image processing and solves the problem of low classification efficiency and unfavorable pathological diagnosis in the current cancer pathology image classification method. First, a lung cancer pathology image classification model is constructed. The lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module connected in sequence and a feature aggregation module based on the attention mechanism. The multi-scale fusion method is used to enhance the robustness of the model. The feature aggregation module based on the attention mechanism is further combined to improve the flexibility and adaptability of the model. Then, the constructed lung cancer pathology image classification model is trained, and the trained lung cancer pathology image classification model is used to classify lung cancer pathology images. The structure of the lung cancer pathology image classification model is optimized, the training complexity is reduced, and the training speed is improved, thereby improving the lung cancer diagnosis efficiency and classification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of pathological image processing, and in particular to a lung cancer pathological image classification system based on multi-scale fusion and attention mechanism. Background Art

[0002] Lung cancer is a malignant tumor of the bronchial mucosa or glands of the lungs, posing a huge threat to human life and health. The incidence and mortality rates of lung cancer increase at the fastest rate each year, bringing serious impacts on human economic and social development. Therefore, research on the diagnosis and differentiation of lung cancer pathological types and the formulation of corresponding treatment plans can improve the survival rate of lung cancer patients.

[0003] Currently, the diagnosis of lung cancer pathology primarily relies on the classification of lung cancer pathology images by experienced pathologists observing lung cancer pathology slides under a microscope. However, lung cancer pathology images are diverse, complex, and contain a large amount of information. Manual classification of lung cancer pathology images by pathologists alone is prone to misjudgment and significantly increases the pathologists' workload. Furthermore, there is a severe shortage of experienced lung cancer pathologists in economically underdeveloped areas and grassroots hospitals. Therefore, when manual classification fails to meet practical needs, research on intelligent classification of lung cancer pathology images is of great significance for the diagnosis of lung cancer pathology. The prior art discloses a method for classifying cancer pathology images. First, data acquisition and preprocessing are performed to screen tumor blocks. Then, a multi-scale fusion attention deep neural network model is built to focus on spatial scale information and channel information. The model uses ResNet-50 as the backbone network, introduces the pyramid convolution PyConv module and the channel attention SE module, and focuses on different channels of model training, making it easier for the model to explore feature expressions related to cancer molecular typing, thereby improving the performance of cancer pathology image classification. However, the model adds PyConv modules and SE modules to the backbone network ResNet-50, resulting in a complex structure. Different channels of the model have different training focuses, resulting in high training complexity and slow training speed, which affects the efficiency of cancer pathology image classification. Summary of the Invention

[0004] To address the problem of low classification efficiency and poor pathological diagnosis in current cancer pathology image classification methods, the present invention proposes a lung cancer pathology image classification system based on multi-scale fusion and attention mechanism, which optimizes the structure of the lung cancer pathology image classification model, improves the flexibility and adaptability of the model, reduces training complexity, and increases training speed, thereby improving the diagnostic efficiency and classification accuracy of lung cancer.

[0005] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:

[0006] A lung cancer pathology image classification system based on multi-scale fusion and attention mechanism, the system comprising:

[0007] A pathological image acquisition module, used to acquire lung cancer pathological images to form a first image data set;

[0008] a pathological image processing module, configured to preprocess the lung cancer pathological images in the first image dataset to obtain a second image dataset;

[0009] A lung cancer pathology image classification model construction module is used to construct a lung cancer pathology image classification model, wherein the lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on an attention mechanism connected in sequence;

[0010] The training module uses the second image data set to train the constructed lung cancer pathology image classification model to obtain a trained lung cancer pathology image classification model, and the trained lung cancer pathology image classification model is used for classifying lung cancer pathology images.

[0011] In this technical solution, a lung cancer pathology image classification model is first constructed. The lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on an attention mechanism that are connected in sequence. The multi-scale fusion method is used to enhance the robustness of the lung cancer pathology image classification model. The feature aggregation module based on the attention mechanism is further combined to enable the lung cancer pathology image classification model to find regions of interest. Then, the constructed lung cancer pathology image classification model is trained, and the trained lung cancer pathology image classification model is used to classify lung cancer pathology images to obtain classification results of lung cancer pathology images. The structure of the lung cancer pathology image classification model is optimized, the flexibility and adaptability of the model are improved, the training complexity is reduced, and the training speed is increased, thereby improving the diagnostic efficiency and classification accuracy of lung cancer.

[0012] Preferably, the lung cancer pathology images are obtained from the public image data of the lung adenocarcinoma and lung squamous cell carcinoma projects of The Cancer Genome Atlas, and the diagnosis results provided by the pathologist are used as data labels.

[0013] Preferably, the specific steps of preprocessing the lung cancer pathology images in the first image dataset are:

[0014] S11. Entering the lung cancer pathology images of the first image dataset into the computer in the form of downsampling;

[0015] S12. storing the lung cancer pathology images input into the computer in a unified RGB format;

[0016] S13. Converting the lung cancer pathology image in RGB format to HSV format, performing median filtering on the lung cancer pathology image in HSV format to obtain a denoised lung cancer pathology image;

[0017] S14. Performing threshold segmentation on the denoised lung cancer pathology image to obtain a tissue contour image in the pathology section;

[0018] S15. Patch the tissue contour image to obtain K a patch image;

[0019] S16.Yes K The patch image is downsampled four times to obtain K A patch downsampled image.

[0020] Preferably, in the pathological image processing module, the first image data set is divided into a training set, a validation set and a test set according to a set ratio, and the training set and the validation set are combined to form the second image data set.

[0021] Preferably, the multi-scale feature extraction and fusion module is provided with a first fully connected layer and a feature extraction network, the first fully connected layer is connected to the feature extraction network, the feature extraction network and the first fully connected layer are respectively used to extract and fuse the features of the preprocessed lung cancer pathology image in the first image dataset, and the feature extraction network adopts a simplified ResNet50 network, including a convolutional layer, a batch normalization layer, a ReLu activation function layer, a maximum pooling layer, a first residual convolution block, a second residual convolution block, a third residual convolution block and an adaptive mean spatial pooling layer connected in sequence.

[0022] Preferably, each patch image and the patch downsampled image corresponding to each patch image are input into a multi-scale feature extraction and fusion module. The specific processing process of the multi-scale feature extraction and fusion module on the patch image and the patch downsampled image is as follows:

[0023] S21. Use the feature extraction network to extract 1024-dimensional features of each patch image ; Use the feature extraction network to extract the 1024-dimensional features of the patch downsampled image corresponding to each patch image ;

[0024] S22. and Connect and get the connection features , The specific expression is:

[0025]

[0026] S23. Use the first fully connected layer and activation function to fuse the connected features and output the fused features .

[0027] Preferably, in step S23, the fusion feature The calculation expression is:

[0028]

[0029] in, and Represent the two training parameters of the first fully connected layer, R eLu Represents the activation function, satisfying the following relationship:

[0030]

[0031] Where x represents R eLu Input to the activation function.

[0032] Preferably, the feature aggregation module is provided with an attention submodule, and the attention submodule is provided with a second fully connected layer and a third fully connected layer connected in sequence to fuse the features. Input feature aggregation module, the feature aggregation module fusion feature The specific processing process is:

[0033] S31. Extract fusion features using attention submodule The correlation information between the fusion features Assign the corresponding attention score and use the softmax function to normalize the attention score. The specific calculation expression is:

[0034]

[0035] in, Represents fusion features The attention score, and Represent the two parameters of the second fully connected layer, and Represent the two parameters of the third fully connected layer, tanh represents the activation function, satisfying the following relationship:

[0036] ;

[0037] in, express tanh The input of the activation function;

[0038] S32. Score attention As the weight, weighted average of all , output the aggregated features of the entire lung cancer pathology image , The calculation expression is:

[0039] .

[0040] Preferably, the lung cancer pathology image classification model further includes a classification module connected to the multi-scale feature extraction and fusion module, and the classification module is provided with a fourth fully connected layer to aggregate the features. Input the fourth fully connected layer, which outputs the classification probability of lung cancer pathology images p , the classification probability p The calculation expression is:

[0041]

[0042] in, and Represents the two parameters of the fourth fully connected layer, softmax ()express softmax The activation function satisfies the following relationship:

[0043]

[0044] Among them, N represents the number of predicted categories, and is the input of the softmax function, is the output of the softmax function, and is the neuron index of the softmax output layer, Is the observed sample belonging to the category probability.

[0045] Preferably, S41. uses a cross entropy loss function to calculate the error between the classification result of the lung cancer pathology image classification model and the data label. The specific calculation formula is as follows:

[0046]

[0047] in, L cross-emtropy Represents the error between the classification result of the lung cancer pathology image classification model and the data label, Indicates that the observed sample belongs to k The probability of the categories, Represents the probability that the observed sample belongs to the kth category in the forward propagation output prediction result of the lung cancer pathology image classification model;

[0048] S42. Backpropagate the classification results of the lung cancer pathology image classification model and the error between the data labels to update the network parameters of the lung cancer pathology image classification model until the prediction accuracy of the lung cancer pathology image classification model converges and the training of the lung cancer pathology image classification model is completed.

[0049] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0050] The present invention proposes a lung cancer pathology image classification system based on multi-scale fusion and attention mechanism. First, a lung cancer pathology image classification model is constructed. The lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on the attention mechanism connected in sequence. The multi-scale fusion method is used to enhance the robustness of the lung cancer pathology image classification model. The feature aggregation module based on the attention mechanism is further combined to enable the lung cancer pathology image classification model to find regions of interest. Then, the constructed lung cancer pathology image classification model is trained, and the trained lung cancer pathology image classification model is used to classify lung cancer pathology images to obtain classification results of lung cancer pathology images. The structure of the lung cancer pathology image classification model is optimized, the flexibility and adaptability of the model are improved, the training complexity is reduced, and the training speed is increased, thereby improving the diagnostic efficiency and classification accuracy of lung cancer. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A structural diagram showing a lung cancer pathology image classification system based on multi-scale fusion and attention mechanism proposed in an embodiment of the present invention;

[0052] Figure 2 The figure shows the overall flow chart of lung cancer pathology image processing proposed in the embodiment of the present invention;

[0053] Figure 3 A flowchart showing the preprocessing of lung cancer pathology images proposed in an embodiment of the present invention;

[0054] Figure 4 A structural diagram showing a feature extraction network proposed in an embodiment of the present invention;

[0055] Figure 5 A flowchart showing multi-scale pactch image feature extraction and fusion proposed in an embodiment of the present invention;

[0056] Figure 6 Flowchart showing feature aggregation based on attention mechanism;

[0057] Figure 7 A schematic diagram showing the division of the first image data set;

[0058] Figure 8 A graph showing the classification results of lung cancer pathology images. DETAILED DESCRIPTION

[0059] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0060] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent actual sizes. The description of the directions of parts such as "upper" and "lower" does not limit this patent;

[0061] It is understandable to those skilled in the art that some well-known contents may be omitted in the drawings;

[0062] The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0063] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0064] Example 1

[0065] like Figure 1 As shown, this embodiment proposes a lung cancer pathology image classification system based on multi-scale fusion and attention mechanism, the system comprising:

[0066] A pathological image acquisition module, used to acquire lung cancer pathological images to form a first image data set;

[0067] In the pathology image acquisition module, the lung cancer pathology images are acquired from the public image data of the lung adenocarcinoma and lung squamous cell carcinoma projects of the Cancer Genome Atlas, and the diagnosis results provided by the pathologist are used as data labels.

[0068] a pathological image processing module, configured to preprocess the lung cancer pathological images in the first image dataset to obtain a second image dataset;

[0069] In the pathological image processing module, the first image data set is divided into a training set, a validation set, and a test set according to a set ratio, and the training set and the validation set are combined into a second image data set; see Figure 2 and Figure 3 , the specific steps of preprocessing the lung cancer pathology images in the first image dataset are:

[0070] S11. Entering the lung cancer pathology images of the first image dataset into the computer in the form of downsampling;

[0071] S12. storing the lung cancer pathology images input into the computer in a unified RGB format;

[0072] S13. Converting the lung cancer pathology image in RGB format to HSV format, performing median filtering on the lung cancer pathology image in HSV format to obtain a denoised lung cancer pathology image;

[0073] S14. Performing threshold segmentation on the denoised lung cancer pathology image to obtain a tissue contour image in the pathology section;

[0074] S15. Patch the tissue contour image to obtain K a patch image;

[0075] In step S15, due to the high resolution of lung cancer pathology images, computers cannot process them directly, and thumbnails will lose a lot of details. It is necessary to divide the pathology images into small blocks for processing. Therefore, this step uses patch division to cut the tissue contour image, among which patch images with more than 75% of the space containing tissue are retained.

[0076] S16.Yes K The patch image is downsampled four times to obtain K A patch downsampled image.

[0077] In step S16, the plurality of patch images are downsampled by a factor of four to obtain patch downsampled images having the same receptive field but a smaller resolution.

[0078] A lung cancer pathology image classification model construction module is used to construct a lung cancer pathology image classification model, wherein the lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on an attention mechanism connected in sequence;

[0079] In the lung cancer pathology image classification model construction module, the multi-scale feature extraction and fusion module is provided with a first fully connected layer and a feature extraction network. The first fully connected layer is connected to the feature extraction network. The feature extraction network and the first fully connected layer are respectively used to extract and fuse the features of the pre-processed lung cancer pathology images in the first image dataset; see Figure 4The feature extraction network adopts a simplified ResNet50 network, including a convolutional layer, a batch normalization layer, a ReLu activation function layer, a maximum pooling layer, a first residual convolution block, a second residual convolution block, a third residual convolution block and an adaptive mean spatial pooling layer connected in sequence. The first residual convolution block is set to 256 convolution kernels, and the size of the convolution kernel is 128x128; the second residual convolution block is set to 512 convolution kernels, and the size of the convolution kernel is 64x64; the third residual convolution block is set to 1024 convolution kernels, and the size of the convolution kernel is 32x32; since the patch image is not labeled, the ResNet50 network pre-trained on ImageNet is used to extract the features of the patch image. The existing ResNet50 network includes a convolutional layer, a batch normalization layer, a ReLu activation function layer, a maximum pooling layer, four residual convolution blocks and a global average pooling layer connected in sequence. Unlike the existing ResNet50 network, since the residual block contains many convolution BN activations and pooling layers, the simplified ResNet50 network used in this embodiment uses an adaptive mean spatial pooling layer to replace the fourth residual convolution block of the existing ResNet, which simplifies the structure of the ResNet50 network, reduces semantic noise and speeds up the training speed of the lung cancer pathology image classification model.

[0080] The training module uses the second image data set to train the constructed lung cancer pathology image classification model to obtain a trained lung cancer pathology image classification model, and the trained lung cancer pathology image classification model is used for classifying lung cancer pathology images.

[0081] In this embodiment, a lung cancer pathology image classification model is first constructed. The lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on an attention mechanism connected in sequence. The multi-scale fusion method is used to enhance the robustness of the lung cancer pathology image classification model. The feature aggregation module based on the attention mechanism is further combined to enable the lung cancer pathology image classification model to find regions of interest. Then, the constructed lung cancer pathology image classification model is trained, and the trained lung cancer pathology image classification model is used to classify lung cancer pathology images to obtain classification results of lung cancer pathology images. The structure of the lung cancer pathology image classification model is optimized, the flexibility and adaptability of the model are improved, the training complexity is reduced, and the training speed is increased, thereby improving the efficiency of lung cancer diagnosis and classification accuracy.

[0082] Example 2

[0083] See also Figure 2 and Figure 5, each patch image and the patch down-sampled image corresponding to each patch image are input into the multi-scale feature extraction and fusion module. The specific processing process of the multi-scale feature extraction and fusion module on the patch image and the patch down-sampled image is as follows:

[0084] S21. Use the feature extraction network to extract 1024-dimensional features of each patch image ; Use the feature extraction network to extract the 1024-dimensional features of the patch downsampled image corresponding to each patch image ;

[0085] S22. and Connect and get the connection features , The specific expression is:

[0086]

[0087] S23. Use the first fully connected layer and activation function to fuse the connected features and output the fused features .

[0088] In step S23, the fusion features The calculation expression is:

[0089]

[0090] in, and Represent the two training parameters of the first fully connected layer, R eLu Represents the activation function, satisfying the following relationship:

[0091]

[0092] Where x represents R eLu Input to the activation function.

[0093] The feature aggregation module is provided with an attention submodule, and the attention submodule is provided with a second fully connected layer and a third fully connected layer connected in sequence. Input feature aggregation module, see Figure 6 , the feature aggregation module fuses the features The specific processing process is:

[0094] S31. Extract fusion features using attention submodule The correlation information between the fusion features Assign the corresponding attention score and use the softmax function to normalize the attention score. The specific calculation expression is:

[0095]

[0096] in, Represents fusion features The attention score, and Represent the two parameters of the second fully connected layer, and Represent the two parameters of the third fully connected layer, tanh represents the activation function, satisfying the following relationship:

[0097] ;

[0098] in, express tanh The input of the activation function;

[0099] S32. Score attention As the weight, the weighted average of all , output the aggregated features of the entire lung cancer pathology image , The calculation expression is:

[0100] .

[0101] See also Figure 2 The lung cancer pathology image classification model also includes a classification module connected to the multi-scale feature extraction and fusion module, and the classification module is provided with a fourth fully connected layer to aggregate features. Input the fourth fully connected layer, which outputs the classification probability of lung cancer pathology images p , the classification probability p The calculation expression is:

[0102]

[0103] in, and Represents the two parameters of the fourth fully connected layer, softmax ()express softmax The activation function satisfies the following relationship:

[0104]

[0105] Where N represents the number of predicted categories, and is the input of the softmax function, is the output of the softmax function, and is the neuron index of the softmax output layer, Is the observed sample belonging to the category probability.

[0106] Example 3

[0107] The specific training process of the lung cancer pathology image classification model is as follows:

[0108] S41. Use the cross entropy loss function to calculate the error between the classification results of the lung cancer pathology image classification model and the data label. The specific calculation formula is as follows:

[0109]

[0110] in, L cross-emtropy Represents the error between the classification result of the lung cancer pathology image classification model and the data label, Indicates that the observed sample belongs to k The probability of the categories, Represents the probability that the observed sample belongs to the kth category in the forward propagation output prediction result of the lung cancer pathology image classification model;

[0111] S42. Backpropagate the classification results of the lung cancer pathology image classification model and the error between the data labels to update the network parameters of the lung cancer pathology image classification model until the prediction accuracy of the lung cancer pathology image classification model converges and the training of the lung cancer pathology image classification model is completed.

[0112] When training the lung cancer pathology image classification model, the lung cancer pathology image was magnified 20 times, see Figure 7 , using lung cancer pathology images under 20 times magnification to form a first image dataset, the first image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5, and the training set and validation set are divided into a fixed ratio of 7:1.5 according to the first image dataset. Five groups of training sets and validation sets set in a ratio of 7:1.5 are selected from the second image dataset, and a lung cancer pathology image classification model is trained using each group of training sets and validation sets. Then, the hyperparameters are adjusted according to the performance of the model on the validation set, and the classification accuracy of the lung cancer pathology image classification model trained corresponding to each group of data is tested using the test set, and the network parameters corresponding to the lung cancer pathology image classification model with the highest classification accuracy are selected and saved; as shown in Table 1, the test set selected in this embodiment contains 88 lung cancer pathology images. As can be seen from Table 1, the classification accuracy of lung adenocarcinoma LUAD reaches 0.84, and the classification accuracy of lung squamous cell carcinoma LUSC reaches 0.93.

[0113] Table 1 Lung cancer pathology image classification accuracy

[0114]

[0115] Furthermore, the lung cancer pathology image classification model of this embodiment is robust to the magnification of lung cancer pathology images. The first data set consisting of 20x magnified lung cancer pathology images is replaced with a first data set of mixed 20x and 40x magnified lung cancer pathology images. The lung cancer pathology image classification model is then trained. The trained lung cancer pathology image classification model is tested on a test set containing 160 lung cancer pathology images. The classification results are shown in Table 2. As can be seen from Table 2, the classification accuracy for lung adenocarcinoma (LUAD) reaches 0.89, and the classification accuracy for lung squamous cell carcinoma (LUSC) reaches 0.84.

[0116] Table 2 Robustness test accuracy of lung cancer pathology images

[0117]

[0118] In addition, in order to further assist doctors in diagnosis, Figure 7 This visualization can provide pathologists with information about how the lung cancer pathology image classification model makes decisions internally; for example, see Figure 8 Marking high attention score patch images with green boxes can help pathologists quickly locate the region of interest. At the same time, listing these high attention score patch images makes it easier for doctors to observe cell morphology.

[0119] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A lung cancer pathology image classification system based on multi-scale fusion and attention mechanism, characterized by: The system comprises: A pathological image acquisition module, used to acquire lung cancer pathological images to form a first image data set; a pathological image processing module, configured to preprocess the lung cancer pathological images in the first image dataset to obtain a second image dataset; A lung cancer pathology image classification model construction module is used to construct a lung cancer pathology image classification model, wherein the lung cancer pathology image classification model includes a multi-scale feature extraction and fusion module and a feature aggregation module based on an attention mechanism connected in sequence; a training module, using the second image data set to train the constructed lung cancer pathology image classification model to obtain a trained lung cancer pathology image classification model, wherein the trained lung cancer pathology image classification model is used for classifying lung cancer pathology images; The multi-scale feature extraction and fusion module is provided with a first fully connected layer and a feature extraction network, wherein the first fully connected layer is connected to the feature extraction network, and the feature extraction network and the first fully connected layer are respectively used to extract and fuse the features of the preprocessed lung cancer pathology image in the first image dataset, and the feature extraction network adopts a simplified ResNet50 network, including a convolution layer, a batch normalization layer, a ReLu activation function layer, a maximum pooling layer, a first residual convolution block, a second residual convolution block, a third residual convolution block and an adaptive mean spatial pooling layer connected in sequence; Each patch image and the patch downsampled image corresponding to each patch image are input into a multi-scale feature extraction and fusion module, wherein each patch image and the patch downsampled image corresponding to each patch image are obtained by preprocessing the lung cancer pathology image; the specific processing process of the patch image and the patch downsampled image by the multi-scale feature extraction and fusion module is as follows: S21. Use the feature extraction network to extract 1024-dimensional features of each patch image ; Use the feature extraction network to extract the 1024-dimensional features of the patch downsampled image corresponding to each patch image ; S22. and Connect and get the connection features , The specific expression is: S23. Use the first fully connected layer and activation function to fuse the connected features and output the fused features ; In step S23, the fusion features The calculation expression is: in, and Represent the two training parameters of the first fully connected layer, R eLu Represents the activation function, satisfying the following relationship: Where x represents R eLu The input of the activation function; The feature aggregation module is provided with an attention submodule, and the attention submodule is provided with a second fully connected layer and a third fully connected layer connected in sequence. Input feature aggregation module, the feature aggregation module fusion feature The specific processing process is: S31. Extract fusion features using attention submodule The correlation information between the fusion features Assign the corresponding attention score and use the softmax function to normalize the attention score. The specific calculation expression is: in, Represents fusion features The attention score, and Represent the two parameters of the second fully connected layer, and Represent the two parameters of the third fully connected layer, tanh represents the activation function, satisfying the following relationship: ； in, express tanh The input of the activation function; S32. Score attention As the weight, weighted average of all , output the aggregated features of the entire lung cancer pathology image , The calculation expression is: 。 2. The lung cancer pathology image classification system based on multi-scale fusion and attention mechanism according to claim 1 is characterized in that: The lung cancer pathology images are obtained from the public image data of the lung adenocarcinoma and lung squamous cell carcinoma projects of the Cancer Genome Atlas, and the diagnosis results provided by the pathologists are used as data labels.

3. The lung cancer pathology image classification system based on multi-scale fusion and attention mechanism according to claim 2 is characterized in that: The specific steps of preprocessing the lung cancer pathology images in the first image dataset are: S11. Entering the lung cancer pathology images of the first image dataset into the computer in the form of downsampling; S12. storing the lung cancer pathology images input into the computer in a unified RGB format; S13. Converting the lung cancer pathology image in RGB format to HSV format, performing median filtering on the lung cancer pathology image in HSV format to obtain a denoised lung cancer pathology image; S14. Performing threshold segmentation on the denoised lung cancer pathology image to obtain a tissue contour image in the pathology section; S15. Patch the tissue contour image to obtain K a patch image; S16.Yes K The patch image is downsampled four times to obtain K A patch downsampled image.

4. The lung cancer pathology image classification system based on multi-scale fusion and attention mechanism according to claim 3 is characterized in that: In the pathological image processing module, the first image data set is divided into a training set, a validation set, and a test set according to a set ratio, and the training set and the validation set are combined into a second image data set.

5. The lung cancer pathology image classification system based on multi-scale fusion and attention mechanism according to claim 4 is characterized in that: The lung cancer pathology image classification model also includes a classification module connected to the multi-scale feature extraction and fusion module, and the classification module is provided with a fourth fully connected layer to aggregate features. Input the fourth fully connected layer, which outputs the classification probability of lung cancer pathology images p , the classification probability p The calculation expression is: in, and Represents the two parameters of the fourth fully connected layer, softmax ()express softmax The activation function satisfies the following relationship: Among them, N represents the number of prediction categories, and is the input of the softmax function, is the output of the softmax function, and is the neuron index of the softmax output layer, Is the observed sample belonging to the category probability.

6. The lung cancer pathology image classification system based on multi-scale fusion and attention mechanism according to claim 5, characterized in that: The specific training process of the lung cancer pathology image classification model is as follows: S41. Use the cross entropy loss function to calculate the error between the classification results of the lung cancer pathology image classification model and the data label. The specific calculation formula is as follows: in, L cross-emtropy Represents the error between the classification result of the lung cancer pathology image classification model and the data label, Indicates that the observed sample belongs to k The probability of the categories, Represents the probability that the observed sample belongs to the kth category in the forward propagation output prediction result of the lung cancer pathology image classification model; S42. Backpropagate the classification results of the lung cancer pathology image classification model and the error between the data labels to update the network parameters of the lung cancer pathology image classification model until the prediction accuracy of the lung cancer pathology image classification model converges and the training of the lung cancer pathology image classification model is completed.

Images