Medical image analysis method and system of residual MLP network based on sparse attention mechanism

Abstract

The invention discloses a medical image classification auxiliary diagnosis system which is based on a deep learning model and interpretability design and aims to improve the accuracy and efficiency of medical image classification. For the significant difference between a medical image and a natural image in the aspects of imaging principle data quality and the like, the generalization ability of a traditional model is limited. The invention designs a residual multi-layer perceptron network based on a sparse attention mechanism, and aims to mine a key focus area in a medical image and enhance the interpretability of a model. Finally, an auxiliary diagnosis system comprising a data enhancement and image classification module is constructed, convenient diagnosis support is provided for medical staff, and the misdiagnosis rate and the missed diagnosis rate are reduced. The system has a good application prospect, the medical image classification performance can be effectively improved, and the development of intelligent medical treatment is promoted.

Description

Technical Field

[0001] The present invention relates to deep learning and medical image processing technology, in particular to a medical image analysis method combining a sparse attention mechanism with a residual multilayer perceptron (MLP) network, and belongs to the application field of artificial intelligence in medical image-assisted diagnosis and treatment. Background Art

[0002] Image classification tasks are at the core of the computer vision field. From traditional machine learning to current deep learning, mainstream deep learning models are mainly studied on natural images. However, medical images and natural images are very different in imaging principles, data volume, data quality, feature expression, etc. If deep learning models can be effectively used to classify medical images, learn and identify complex features in medical images, then it can help doctors improve the accuracy and timeliness of diagnosis, accelerate the diagnosis and treatment of diseases, reduce medical costs, and promote the further development of smart healthcare.

[0003] Medical image analysis is an important means of clinical diagnosis. Traditional image classification and diagnosis rely on the doctor's experience, which has the disadvantages of strong subjectivity and unstable accuracy. In recent years, deep learning has made significant progress in the field of medical image classification, but due to the difference in information features between medical images and natural images, the existing models have poor performance on medical images. In particular, models based on multi-layer perceptrons and convolutional neural networks, although they show good classification results, still face challenges in the interpretability of medical images and the extraction of key features.

[0004] With the increasing application of deep learning in medical image processing, the performance of existing models in image classification tasks has made significant progress. However, medical images (such as CT, MRI, etc.) have the characteristics of small lesion areas and high noise ratio, which makes it difficult for existing models to identify key lesion areas. At the same time, deep learning models usually lack interpretability and cannot provide the basis behind classification decisions. To this end, the present invention aims to improve the classification performance of the model and enhance its interpretability by embedding a sparse attention mechanism and a residual connection structure. Summary of the invention

[0005] The present invention proposes a residual MLP network based on a sparse attention mechanism for the classification and interpretability analysis of medical images. Through the sparse attention mechanism, the model can adaptively select the most relevant areas in the medical image for the classification task, and combine the loss function to guide the optimization, thereby reducing the inconsistency between the attention mechanism and the classification task. At the same time, the present invention provides a medical image-assisted diagnosis system to help doctors perform efficient image diagnosis and decision support.

[0006] The present invention has the following beneficial effects:

[0007] 1) Improve the accuracy of medical image classification: By introducing a sparse attention mechanism, the present invention can focus on the key lesion area in the medical image and ignore irrelevant noise and background information, thereby greatly improving the classification model's recognition and classification accuracy of medical images. In particular, when the medical image is highly complex and has sparse features, the attention mechanism can dynamically adjust the focus of the model and improve the reliability of diagnosis;

[0008] 2) Improve the interpretability of the model: Traditional deep learning models often cannot explain the basis of classification decisions, but the present invention uses a sparse attention mechanism to clearly indicate which parts of the image features play a key role in the decision, helping doctors understand the decision-making process of the model. This is particularly important for medical diagnosis, helping doctors better explain the diagnosis results and support clinical decision-making;

[0009] 3) Adapt to small sample data sets and enhance generalization ability: Medical image data usually has a small sample size. Traditional deep learning models rely heavily on large-scale data and are difficult to perform well on small samples. The present invention uses a sparse attention mechanism to more effectively extract key information in small sample conditions and improve the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 The schematic diagram of the structure of the sparse attention mechanism embedded in the residual MLP network in the present invention is shown;

[0011] Figure 2 Shows an example of a heat map of important image regions filtered out by the attention mechanism;

[0012] Figure 3 The functional module diagram of the medical image-assisted diagnosis system is shown. DETAILED DESCRIPTION

[0013] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0014] Example

[0015] Figure 1 It is a structural diagram of the sparse attention mechanism embedded in the residual MLP network in the present invention.

[0016] like Figure 1 As shown, the specific steps of embedding the sparse attention mechanism into the residual MLP network in the present invention include:

[0017] S101: Image preprocessing:

[0018] The original medical image I is preprocessed, including denoising, enhancement and standardization. The processed image is divided into image blocks X of size n×n.

[0019] S102: Feature extraction and initial representation:

[0020] The input image block X is subjected to feature extraction, and the input image block is preliminarily processed through the residual multi-layer perceptron (MLP) structure to obtain the initial feature representation Z. The calculation of each layer of the network is as follows:

[0021] Z (l+1) =σ(W (l) Z (l) +b (l) )

[0022] Among them, Z (l) is the output of layer l, W (l) is the weight matrix, b (l) is the bias and σ is the activation function.

[0023] S103: Sparse Attention Mechanism Embedding:

[0024] The network uses the attention mechanism to assign corresponding weights to important patches, and then implements information interaction between patches through a linear layer. The attention mechanism is used again to weight these patches, enhancing the model's ability to focus on core features and reducing reliance on redundant and irrelevant information.

[0025] Assuming the input is X, which represents the feature map of a medical image, the sparse attention mechanism controls which areas (patches) need more attention through the attention weight matrix A:

[0026] Calculate the attention weights: For each patch x of the input image i , the attention weight a is calculated through the sparse attention mechanism j .

[0027] Assume that the input image X is divided into M patches, and H = {h m} M 1 ∈R C×M is the output of the L-1th layer, and θ L ∈R C×K represents the parameters of the Lth layer, h m ∈R C represents the feature representation of the mth patch of image X, and C is the number of channels. m} M1 is the output of the image block at layer L, where p m ∈R K It is the logit (class vector) of the mth patch, and its calculation method is p m =h m θ L . And [a i1 ,…,a iM ] T They represent patch logits[p i1 ,…,p iM ] weight. Then the proposed attention mechanism can be expressed as:

[0028]

[0029] h j ←a j h j

[0030]

[0031] For sample data X, a j is the attention weight of the jth patch, ξ∈[0,1] is the threshold used to remove insignificant patches, z∈R K is the output of X at layer L. It is worth noting that the formula uses L 2,1 Norm such as To make P∈R M×K It has row sparsity, which increases the weight of important patches and reduces the weight of trivial image blocks. In addition, this chapter empirically sets the maximum value of ξ to 1 during training, because when ξ>1, the weights of all patches during training may be zero.

[0032] S104: Residual Connection:

[0033] In order to avoid the gradient vanishing or gradient exploding problem, a residual connection is used to pass the input directly to the output, as shown in the following formula:

[0034] Z (l+1) =Z (l) +F(Z (l) ,W (l) )

[0035] Where F is the residual function calculated by the MLP sublayer. Through the residual connection, the input is directly added to the output, allowing the network to learn deeper feature representations.

[0036] S105: Loss-guided Optimization:

[0037] The input is the feature Z after sparse attention weighting and residual connection residual , the output is the optimized network weight W, and the loss function consists of the cross entropy loss and the regularization term of the sparse attention mechanism. The cross entropy loss is used for classification tasks and is defined as:

[0038]

[0039] Among them, y i is the true label, is the predicted probability. The total loss function is:

[0040] L=L classification +λL attention

[0041] Among them, λ is the weight of the regularization term, which is used to balance the relationship between the classification task and attention sparsification.

[0042] S106: Output and Interpretable Analysis:

[0043] The input is the optimized feature matrix Z, and the output is the classification result and Heat map.

[0044] Figure 2 The present invention Figure 2 Shows an example of a heat map of important image regions filtered out by the attention mechanism. Figure 2 As shown, an example of a heat map of important image regions filtered out by the attention mechanism.

[0045] Figure 3 The functional module diagram of the medical image-assisted diagnosis system is shown. The system mainly consists of three core modules: image preprocessing module, classification detection module and result visualization module. The image preprocessing module is responsible for the standardization, denoising and data enhancement of medical images; the classification detection module classifies medical images based on the sparse attention mechanism and residual MLP network; the result visualization module generates classification results and corresponding attention heat maps to help doctors intuitively analyze and understand the decision basis and provide auxiliary diagnosis support.

[0046] In order to better illustrate the technical effect of the present invention, the present invention is experimentally verified by using specific examples. Experiments are conducted on both natural image datasets and medical image datasets. The natural image datasets are CIFAR-10 and Tiny-ImageNet datasets, and the medical image datasets are NIH Chest X-ray, PCam32, PCam96, ChinaSet ChestX-ray, and BreakHis datasets.

[0047] Table 1 Experimental results of medical image dataset

[0048]

[0049] Table 1

[0050] As shown in Table 1, our SaResMLP method performs well on multiple medical image datasets, especially on the PCam32 and PCam96 datasets, with classification accuracy rates of 86.62% and 86.35%, significantly better than the residual MLP model without attention mechanism (81.77%). This shows that the introduction of sparse attention mechanism effectively helps the model focus on key image areas and improves classification performance.

[0051] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A medical image classification method based on sparse attention mechanism, characterized in that: The following steps are involved: S1: Data preprocessing: normalize, denoise and enhance the input medical image data, including image rotation, scaling, flipping, etc.; S2: Low-level feature extraction: Extract the low-level features of medical images through convolutional neural networks (CNN) conv , where the convolution operation is: F conv =f(X*W+b) Where X is the input image, W is the convolution kernel, b is the bias term, * is the convolution operation, and f is the nonlinear activation function; S3: Sparse attention mechanism embedding: The sparse attention mechanism is introduced in the residual multilayer perceptron (ResMLP) to generate the attention weight A, which can be expressed as: Where Q is the query matrix, K is the key matrix, and d k is the characteristic dimension; S4: Use a multiple instance learning framework to classify images and generate interpretable results through an attention mechanism. S5: Classification result output: The output of the sparse attention mechanism is fused with the original features to generate the prediction results of medical image classification The formula is as follows: S6: Loss function optimization: Use a comprehensive loss function that includes classification loss and attention consistency loss to optimize the model. The loss function L is: L=L classification +λL attention Among them, L classification is the classification error, L attention is the difference between the attention weight and the true lesion area, and λ is the balance coefficient.

2. The medical image classification method according to claim 1, characterized in that: The sparse attention mechanism limits the non-zero elements in the attention matrix A, making the model more sparse when detecting key lesion areas, thereby reducing the impact of redundant features.

3. The medical image classification method according to claim 1 or 2, characterized in that: Using transfer learning technology, the convolutional network is first pre-trained on a natural image dataset and then fine-tuned on a medical image dataset to improve the generalization ability of the model.

4. A medical image classification system based on a sparse attention mechanism, comprising a medical image preprocessing module; a classification module based on a residual MLP network, which identifies key feature areas in medical images through a sparse attention mechanism and outputs classification results.

Images