A classification method and system for gastrointestinal diseases based on convolution and state-space models

Abstract

This invention discloses a classification method and system for gastrointestinal diseases based on convolution and state-space models. The classification method includes extracting texture features from medical images of gastrointestinal diseases, determining lesion regions in the medical images based on the texture features, adaptively enhancing the contrast of the lesion regions, performing preprocessing, and inputting the preprocessed features into the patch embedding module of a trained deep learning model to segment the medical image into multiple patches. The patches are then sequentially input into two fusion modules, where the convolutional units and state-space units of each fusion module extract local and global features from the patches, respectively, and then fuse the local and global features. A patch merging module is used to merge the fused features of multiple patches, and the merged features are input into the two fusion modules of the deep learning model to generate fused features of the merged features. The previous step is repeated a preset number of times to obtain the final feature map, which is then input into the classifier of the deep learning model to obtain the classification result.

Description

Technical Field

[0001] This invention relates to image classification technology, specifically to a method and system for classifying gastrointestinal diseases based on convolution and state space models. Background Technology

[0002] In recent years, with the rise of deep learning technology, especially the application of convolutional neural networks (CNNs) in computer vision, significant progress has been made in medical image classification tasks. CNNs can automatically extract features from images and perform multi-level feature learning, making the classification of medical images more accurate. However, CNNs still have shortcomings in capturing global information, handling long-range dependencies, and robustness. Specifically, when processing medical images of gastrointestinal diseases (such as ulcers, polyps, and tumors), although CNNs can identify local lesions in the image, they often ignore the influence of the global context, which may lead to misdiagnosis of some minor lesions.

[0003] The Transformer model, initially applied in natural language processing, is gaining widespread attention in image analysis as well. Transformers excel at certain tasks by modeling long-distance dependencies between features globally through self-attention. In processing medical images, the Transformer architecture offers superior capabilities compared to CNNs, particularly in modeling global information. However, the computational complexity of Transformers increases quadratically with increasing input data (O(N)). 2 The quadratic computational complexity, where N represents the dimension of the input data or the size of the image, creates enormous computational pressure and resource consumption when processing large-scale images, especially in medical image analysis, thus limiting its practical application.

[0004] To address these issues, State-Space Models (SSMs) have been introduced into the field of deep learning in recent years. SSMs can effectively handle long-range dependencies in images, improving classification accuracy by dynamically modeling global information within the image. Research combining SSMs with deep neural networks (such as CNNs) has gained momentum, forming a new generation of image analysis methods. These methods significantly improve the accuracy and robustness of image classification by fusing the local feature extraction capabilities of CNNs with the global information modeling capabilities of SSMs.

[0005] Although some research has made progress in this area, existing fusion methods still suffer from high computational complexity and low efficiency. Therefore, designing an efficient image classification method that can capture global information while maintaining low computational complexity remains an important research topic in the field of medical image analysis. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, the present invention provides a gastrointestinal disease classification method and system based on convolution and state space models, which solves the problem of low processing efficiency caused by high computational complexity in existing methods when classifying medical images.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0008] Firstly, a method for classifying gastrointestinal diseases based on convolution and state-space models is provided, which includes the following steps:

[0009] S1. Extract texture features from medical images of gastrointestinal diseases, determine the lesion areas in the medical images based on the texture features, and perform adaptive contrast enhancement on the lesion areas.

[0010] S2. The medical image after contrast enhancement of the lesion area is preprocessed and then input into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches.

[0011] S3. Input the patch into the two fusion modules of the deep learning model in sequence. The convolutional unit and state space unit of each fusion module extract local and global features from the patch respectively. Then, the local and global features of the patch are fused to obtain the fused features.

[0012] S4. The patch merging module of the deep learning model merges the fusion features of multiple patches, and inputs the merged features into the two fusion modules of the deep learning model to generate the fusion features of the merged features.

[0013] S5. Repeat step S4 a preset number of times to obtain the final feature map. Then, input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result.

[0014] Furthermore, step S1 further includes:

[0015] S11. Extract texture features from medical images of gastrointestinal diseases using a Gabor filter:

[0016]

[0017] Where G(x,y) is the texture feature of the pixel with spatial coordinates (x,y); exp is the exponential function; σ is the scale parameter of the Gabor filter; f is the frequency of the Gabor filter; and θ is the orientation angle of the Gabor filter.

[0018] S12. Determine whether the texture feature G(x,y) of the pixel with spatial coordinates (x,y) is greater than the preset threshold. If it is, mark it as a lesion area; otherwise, mark it as a normal area.

[0019] S13. Calculate the fitness threshold of the pixels corresponding to the lesion region:

[0020] threshold(x,y)=μ(x,y)+λ·σ(x,y)

[0021] Where threshold(x,y) is the fitness threshold of the pixel with spatial coordinates (x,y); μ(x,y) and σ(x,y) are the mean and standard deviation of the gray values ​​of all pixels in the neighborhood centered at (x,y) in the medical image, respectively; and λ is the sensitivity adjustment parameter.

[0022] S14. Determine whether the gray value of the pixel with spatial coordinates (x,y) is greater than its corresponding fitness threshold. If yes, proceed to step S15; otherwise, do not perform contrast enhancement on the pixel.

[0023] S15. Enhance the contrast of pixels:

[0024]

[0025] Where I(x,y) and I′(x,y) are the gray values ​​of the pixel with spatial coordinates (x,y) before and after enhancement; α controls the intensity of contrast enhancement.

[0026] Furthermore, methods for segmenting medical images into multiple patches include:

[0027] The medical image is divided into patches of size P×P, each patch representing a local region in the medical image;

[0028] Each patch is transformed into a vector of a set dimension using convolution operations. The resulting embedding is represented as a tensor containing the features of all patches. This tensor X patch_embedding The shape is:

[0029]

[0030] Where B is the batch size. denoted as the number of patches in a medical image; D as the dimension of each patch mapped to the feature space; H as the height of the medical image; W as the medical image dimension; and P as the side length of each patch.

[0031] Furthermore, step S3 further includes:

[0032] S31. The patch is input into the fusion module. The convolutional units of the fusion module use multiple convolution operations to gradually extract the spatial features of the image, and obtain local features through batch normalization and non-linear activation functions:

[0033] Y = X p +ReLU(Batch Norm(ReLU(Batch Norm(X p *K1 (1) +b1 (1) ))*K2 (2) +b2 (2) )*K3 (3) +b3 (2) ))

[0034] Where Y represents the local features of the patch; ReLU() is the activation function; and Batch Norm() is the batch normalization operation. For the first 1×1 convolution kernel, b1 (1) For K1 (1) The bias term; C represents the number of channels after dimensionality reduction. in The number of channels in the input tensor; It is a 2×2 convolution kernel, b2 (2) For K2 (2) The bias term, C out The number of channels for the output tensor; For the second 1×1 convolution kernel, b3 (3) For K3 (3) The bias term; * indicates multiplication; X p This is the patch obtained after transposing;

[0035] S32. The state space unit of the fusion module extracts global features from the input patch, and then concatenates the global and local features corresponding to the same patch to obtain the concatenated features.

[0036] S33. The channel attention module of the fusion module is used to calculate the channel attention of the spliced ​​features to obtain the channel attention map:

[0037] M c (F)=σ′(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0038] Where F represents the splicing feature; M c (F) is the channel attention map of the concatenated feature F; σ′() is the Sigmoid activation function; Avg Pool() is the average pooling operation; MLP() is the multilayer perceptron; Max Pool() is the max pooling operation.

[0039] S34. Multiply the stitched features element-wise with their corresponding channel attention maps, then input them into the spatial attention module of the fusion module for spatial attention calculation to obtain the spatial attention map:

[0040] M s (F′)=σ′(f 7×7 ([Avg Pool(F′);Max Pool(F′)]))

[0041] Where F′ is the feature map obtained by element-wise multiplication; M s (F′) is the spatial attention map corresponding to feature map F′; f 7 ×7 This is a 7×7 convolution operation;

[0042] S35. Multiply the spatial attention map and the spliced ​​features element-wise to obtain the first fused feature;

[0043] S36. Input the first fusion feature into another fusion module, and repeat steps S31 to S35 to obtain the final fusion feature of each patch.

[0044] Furthermore, methods for merging the fusion features of multiple patches include:

[0045] The four segmented regions are merged according to the channel dimension by concatenating the features of the four regions together; the feature dimension of each patch is increased by four times; then, each new feature map obtained after merging is normalized, and then a linear transformation is used to reduce the dimensionality of the normalized features to obtain the merged features.

[0046] Furthermore, the expression for the loss function of the deep learning model is:

[0047]

[0048] in, Standard cross-entropy loss; Loss of focus; For structural similarity loss; For boundary loss; λ1, λ2, λ3, and λ4 are respectively and The weighting coefficients.

[0049] Furthermore, and The expressions are as follows:

[0050]

[0051] Where α′ is a hyperparameter that adjusts the class balance; pt γ represents the prediction probability of the deep learning model for the correct category; γ is a parameter for adjusting the difficulty of samples; SIMM is the structural similarity index; I and These are the original image during training and the predicted image from the deep learning model, respectively. and Let I be the k-th original image during training. k and the corresponding deep learning model's predicted image The boundary gradient, 1≤k≤U, where U is the total number of medical images in the dataset during training; |·| is the absolute value sign.

[0052] Secondly, a gastrointestinal disease classification system based on convolution and state-space models is provided, which includes:

[0053] The image filtering and enhancement module is used to extract texture features from medical images of gastrointestinal diseases, determine the lesion areas in the medical images based on the texture features, and perform adaptive contrast enhancement on the lesion areas.

[0054] The preprocessing module is used to preprocess medical images after contrast enhancement of the lesion area;

[0055] The patch generation module is used to input the preprocessed image into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches;

[0056] The first fusion feature generation module is used to input the patch into the two fusion modules of the deep learning model in sequence. The convolutional unit and state space unit of each fusion module extract local and global features from the patch respectively. Then, the local and global features of the patch are fused to obtain the fusion feature.

[0057] The feature merging module is used to merge the fused features of multiple patches using the patch merging module of the deep learning model to obtain merged features;

[0058] The second fusion feature generation module is used to generate fusion features from the two fusion modules of the deep learning model by inputting the merged features;

[0059] The judgment module is used to determine whether the executed feature merging module and the second fusion feature generation module are the last group. If so, the output of the last second fusion feature generation module is output as the final feature map; otherwise, the next group of feature merging modules and the second fusion feature generation module are entered.

[0060] The classification module is used to input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result.

[0061] The beneficial effects of this invention are as follows: This scheme can identify lesion areas through filtering, thereby enhancing the model's sensitivity and classification accuracy to lesion areas; the fusion module can fuse features from different modules to enhance the performance of image classification tasks, thus helping the model understand a wider range of image information; patch merging is a key step in this scheme for integrating patches that have undergone feature fusion processing. By merging the features of adjacent patches, it reduces the spatial resolution of the feature map while increasing the feature dimension, thereby providing a richer and more compact feature representation for subsequent network processing. The patch merging operation ensures that the multi-scale features of the image can be effectively fused while maintaining computational efficiency. Attached Figure Description

[0062] Figure 1 This is a flowchart of a gastrointestinal disease classification method based on convolution and state space models.

[0063] Figure 2 This is a partial block diagram of a gastrointestinal disease classification method based on convolution and state space models.

[0064] Figure 3 This is a block diagram illustrating the principle of a convolutional unit.

[0065] Figure 4 This is a block diagram of the state-space unit principle.

[0066] Figure 5 This is a schematic diagram of the fusion module. Detailed Implementation

[0067] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0068] refer to Figure 1 , Figure 1 A flowchart of a gastrointestinal disease classification method based on convolution and state-space models is shown, such as... Figure 1 and Figure 2 As shown, the method S includes steps S1 to S5.

[0069] In step S1, texture features of medical images of gastrointestinal diseases are extracted, lesion areas in the medical images are determined based on the texture features, and adaptive contrast enhancement is performed on the lesion areas.

[0070] In one embodiment of the present invention, step S1 further includes:

[0071] S11. Extract texture features from medical images of gastrointestinal diseases using a Gabor filter:

[0072]

[0073] Wherein, G(x,y) is the texture feature of the pixel with spatial coordinates (x,y); exp is the exponential function; σ is the scale parameter of the Gabor filter; f is the frequency of the Gabor filter, which is used to control the frequency features of the texture and can capture the frequency information of detailed changes; θ is the orientation angle of the Gabor filter, which is used to control the orientation information of the texture and help identify lesion features in different directions in gastrointestinal images.

[0074] S12. Determine whether the texture feature G(x,y) of the pixel with spatial coordinates (x,y) is greater than the preset threshold. If it is, mark it as a lesion area; otherwise, mark it as a normal area.

[0075] Since lesion areas typically have different texture patterns, these patterns can be highlighted through the output of the Gabor filter, thereby helping the model to identify lesion areas and highlight the differences between lesion areas and normal tissue.

[0076] S13. Calculate the fitness threshold of the pixels corresponding to the lesion region:

[0077] threshold(x,y)=μ(x,y)+λ·σ(x,y)

[0078] Where threshold(x, y) is the fitness threshold of the pixel with spatial coordinates (x, y); μ(x, y) and σ(x, y) are the mean and standard deviation of the gray values ​​of all pixels in the neighborhood centered at (x, y) in the medical image, respectively; λ is the sensitivity adjustment parameter.

[0079] S14. Determine whether the gray value of the pixel with spatial coordinates (x, y) is greater than its corresponding fitness threshold. If yes, proceed to step S15; otherwise, do not perform contrast enhancement on the pixel.

[0080] The fitness threshold of this scheme is calculated based on the mean and standard deviation of the local region, which has good adaptability. It can specifically enhance the contrast of pixels in the lesion area, thereby making it easier for the classifier to identify and distinguish the lesion area.

[0081] S15. Enhance the contrast of pixels:

[0082]

[0083] Where I(x, y) and I′(x, y) are the gray values ​​of the pixel with spatial coordinates (x, y) before and after enhancement; α controls the intensity of contrast enhancement.

[0084] Enhancing contrast can make lesions (such as tumors, ulcers, polyps, etc.) more prominent in images, enabling subsequent classification algorithms to identify these areas more accurately.

[0085] In step S2, the contrast-enhanced medical image of the lesion area is preprocessed and then input into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches. The preprocessing here is to adjust the size of the contrast-enhanced medical image so that the input image matches the size requirements of the deep learning model.

[0086] In one embodiment of the present invention, a method for segmenting a medical image into multiple patches includes:

[0087] The medical image is divided into small patches of size (P×P). Each patch represents a local region in the image, and the size of the patch is controlled by the parameter patch_size. For example, when the patch size is set to (P=4), each patch is a small region of (4×4).

[0088] Next, a convolution operation is used to transform each patch into a fixed-size feature vector. This convolution operation maps each (P×P) region in the image to a new feature space, and the output feature vector can be used to describe the content of that region.

[0089] Specifically, the convolution operation scans the entire image, dividing it into non-overlapping patches according to a set stride (i.e., the distance moved each time). Each patch is mapped to a new feature vector, the dimension of which is determined by the settings of the convolution operation.

[0090] After the convolution operation, each patch in the image is represented as a fixed-dimensional vector, and the final embedding is represented as a tensor containing the features of all patches. This tensor X patch_embedding The shape is:

[0091]

[0092] Where B is the batch size. denoted as the number of patches in a medical image; D as the dimension of each patch mapped to the feature space; H as the height of the medical image; W as the medical image dimension; and P as the side length of each patch.

[0093] In step S3, the patch is sequentially input into two fusion modules of the deep learning model. The convolutional unit and state space unit of each fusion module extract local and global features from the patch, respectively. Then, the local and global features of the patch are fused to obtain fused features.

[0094] In one embodiment of the present invention, step S3 further includes:

[0095] S31. Input the patch into the fusion module. The convolutional unit of the fusion module (the principle block diagram of the convolutional unit can be found in [reference]). Figure 3 The spatial features of the image are extracted stepwise using multiple convolution operations, and local features are obtained through batch normalization and non-linear activation functions.

[0096] Y = X p +ReLU(Batch Norm(ReLU(Batch Norm(X p *K1 (1) +b1 (1) ))*K2 (2) +b2 (2) )*K3 (3) +b3 (2) ))

[0097] Where Y represents the local features of the patch; ReLU() is the activation function; and Batch Norm() is the batch normalization operation. For the first 1×1 convolution kernel, b1 (1) For K1 (1) The bias term; C represents the number of channels after dimensionality reduction. in The number of channels in the input tensor; It is a 2×2 convolution kernel, b2 (2) For K2 (2) The bias term, C out The number of channels for the output tensor; For the second 1×1 convolution kernel, b3 (3) For K3 (3) The bias term; * indicates multiplication; X p This is the patch obtained after transposing;

[0098] S32. The state space unit of the fusion module extracts global features from the input patch, and then concatenates the global and local features corresponding to the same patch to obtain the concatenated features.

[0099] The State Space Model Unit (SSM) is the fundamental module in this scheme for extracting global features from the input image. It aims to extract global features of the image and enhance the representational power of these features through a dynamic state transition model. In this example, State Space Models (SSM) are an efficient model for processing sequential data, based on continuous linear ordinary differential equations (ODEs). An ODE describes the data from a one-dimensional input function or sequence. To the intermediate hidden state Then output The mapping.

[0100] The state matrices A and B are transformed into discrete parameters using the zero-order preserved discretization rule. and

[0101]

[0102] in, It is a state matrix; and All are projection parameters; t is time; exp is the exponential function used to calculate the transformation from continuous to discrete; I′ is the identity matrix.

[0103] After discretization, global features are obtained by linear recursion or global convolution. The expression for linear recursion is:

[0104]

[0105] y(t)=(Ch'(t)

[0106] Where h′(t) is X p (t) represents the hidden state; h′(t) is the intermediate parameter; y(t) is the value of X. p (t) corresponds to the global feature;

[0107] The expression for global convolution calculation is:

[0108]

[0109] in, L is the structured convolution kernel; L is the X p Length; It is a structured convolution kernel.

[0110] refer to Figure 4 The extraction of global features by the state space unit includes the following steps:

[0111] 1. First, layer normalization is performed on the input features to ensure input stability. 2. Then, the input features are divided into two branches. The first branch is processed through linear layers and activation functions, while the second branch is processed through depthwise separable convolution and then activated before entering the 2D selective scan (SS2D) module. 3. In the SS2D module, the input image is unfolded along four different directions, generating multiple sequences. These sequences are processed by the S6 block to restore the output image to the same size as the input. 4. Finally, the features are blended through linear layers and element-wise multiplied with the output of the first branch, ultimately fusing the features from both branches.

[0112] S33. The channel attention module of the fusion module is used to calculate the channel attention of the spliced ​​features to obtain the channel attention map:

[0113] M c (F)=σ′(MLP(Avg Pool(F))+MLP(Max Pool(F)))

[0114] Where F represents the splicing feature; M c (F) is the channel attention map of the concatenated feature F; σ′() is the Sigmoid activation function; Avg Pool() is the average pooling operation; MLP() is the multilayer perceptron; Max Pool() is the max pooling operation.

[0115] S34. Multiply the stitched features element-wise with their corresponding channel attention maps, then input them into the spatial attention module of the fusion module for spatial attention calculation to obtain the spatial attention map:

[0116] M s (F′)=σ′(f 7×7 ([Avg Pool(F′);Max Pool(F′)]))

[0117] Where F′ is the feature map obtained by element-wise multiplication; M s (F′) is the spatial attention map corresponding to feature map F′; f 7 ×7 This is a 7×7 convolution operation;

[0118] S35. Multiply the spatial attention map and the stitched features element-wise to obtain the first fused feature; the principle block diagram of the fusion module processing procedure corresponding to steps S31 to S35 can be referred to Figure 5 .

[0119] S36. Input the first fusion feature into another fusion module, and repeat steps S31 to S35 to obtain the final fusion feature of each patch.

[0120] In step S4, the patch merging module of the deep learning model is used to merge the fusion features of multiple patches, and the merged features are input into two fusion modules of the deep learning model to generate the fusion features of the merged features.

[0121] In implementation, this solution preferably employs the following methods for merging the fusion features of multiple patches:

[0122] The four segmented regions are merged along the channel dimension by concatenating their features. This merging enhances the features of each region, increasing the size of the merged feature map in each patch dimension. The merged feature map now contains information from all four original patches, with each patch's feature dimension increasing fourfold. Each new merged feature map is then normalized, and a linear transformation is used to reduce the dimensionality of the normalized features, resulting in the merged feature map.

[0123] S5. Repeat step S4 a preset number of times to obtain the final feature map. Then, input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result.

[0124] Y = Softmax(W′F final +b′)

[0125] in, The output of the classifier; W′ and b′ are the weight parameters and bias term of the classifier, respectively; F final This is the final feature map; N′ is the number of categories;

[0126] In image classification of gastrointestinal diseases, lesion regions often possess subtle and complex textures, making it difficult for traditional loss functions to fully capture the key features of these regions. Therefore, to address this issue, this proposal combines the standard cross-entropy loss function with focal loss, structural similarity loss (SSIMLoss), and boundary loss through a weighted combination to optimize classification accuracy, particularly in the identification of lesion regions.

[0127] In implementation, the preferred expression for the loss function of the deep learning model in this scheme is:

[0128]

[0129] in, Standard cross-entropy loss; Focus loss is used to mitigate class imbalance.

[0130] The problem is that, in particular, by focusing attention on areas that are difficult to classify, the model's ability to identify rare lesions can be improved. Structural similarity loss is used to optimize the structural learning of lesion regions and enhance the structural similarity of images, especially in complex gastrointestinal images, to help the model accurately model the minute structures of lesion regions; Boundary loss is specifically designed to improve the accuracy of lesion region boundaries, reduce misjudgments of edges, and optimize the segmentation accuracy of lesion regions; λ1, λ2, λ3, and λ4 are respectively... and The weighting coefficients.

[0131] in, and The expressions are as follows:

[0132]

[0133] Where α′ is a hyperparameter that adjusts the class balance; p t γ represents the prediction probability of the deep learning model for the correct category; γ is a parameter for adjusting the difficulty of samples; SIMM is the structural similarity index; I and These are the original image during training and the predicted image from the deep learning model, respectively. and Let I be the k-th original image during training. k and the corresponding deep learning model's predicted image The boundary gradient, 1≤k≤U, where U is the total number of medical images in the dataset during training; |·| is the absolute value sign.

[0134] The loss function provided by this solution has the following advantages:

[0135] Addressing class imbalance: Focal Loss enhances the focus on rare lesion regions, avoids the dominance of common classes in the training process, and improves the classification accuracy of rare lesion regions.

[0136] Detailed structural learning: Through SSIM Loss, the model can learn the complex structure of lesion areas in gastrointestinal disease images, especially in small lesion areas, ensuring that lesion details are not ignored.

[0137] Accurate boundary recognition: By using boundary loss, the recognition of lesion area boundaries is optimized, reducing the model's ambiguous judgment of lesion areas and improving the segmentation accuracy of lesion areas.

[0138] Through the above loss function design, this scheme can significantly improve the model's performance in the image classification task of gastrointestinal diseases, especially in the identification of subtle lesions and boundary segmentation, ensuring that the model can more accurately identify and classify gastrointestinal diseases.

[0139] This solution also provides a gastrointestinal disease classification system based on convolution and state-space models, which includes:

[0140] The image filtering and enhancement module is used to extract texture features from medical images of gastrointestinal diseases, determine the lesion areas in the medical images based on the texture features, and perform adaptive contrast enhancement on the lesion areas.

[0141] The preprocessing module is used to preprocess medical images after contrast enhancement of the lesion area;

[0142] The patch generation module is used to input the preprocessed image into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches;

[0143] The first fusion feature generation module is used to input the patch into the two fusion modules of the deep learning model in sequence. The convolutional unit and state space unit of each fusion module extract local and global features from the patch respectively. Then, the local and global features of the patch are fused to obtain the fusion feature.

[0144] The feature merging module is used to merge the fused features of multiple patches using the patch merging module of the deep learning model to obtain merged features;

[0145] The second fusion feature generation module is used to generate fusion features from the two fusion modules of the deep learning model by inputting the merged features;

[0146] The judgment module is used to determine whether the executed feature merging module and the second fusion feature generation module are the last group. If so, the output of the last second fusion feature generation module is output as the final feature map; otherwise, the next group of feature merging modules and the second fusion feature generation module are entered.

[0147] The classification module is used to input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result.

[0148] In summary, the gastrointestinal disease classification method proposed in this scheme can capture global information from medical images while achieving efficient image classification with low computational complexity.

Claims

1. A gastrointestinal disease classification method based on convolution and state-space models, characterized in that, Including the following steps: S1. Extract texture features from medical images of gastrointestinal diseases, determine the lesion areas in the medical images based on the texture features, and perform adaptive contrast enhancement on the lesion areas. S2. The medical image after contrast enhancement of the lesion area is preprocessed and then input into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches. S3. Input the patch into the two fusion modules of the deep learning model in sequence. The convolutional unit and state space unit of each fusion module extract local and global features from the patch respectively. Then, the local and global features of the patch are fused to obtain the fused features. S4. The patch merging module of the deep learning model merges the fusion features of multiple patches, and inputs the merged features into the two fusion modules of the deep learning model to generate the fusion features of the merged features. S5. Repeat step S4 a preset number of times to obtain the final feature map. Then, input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result. Step S3 further includes: S31. The patch is input into the fusion module. The convolutional units of the fusion module use multiple convolution operations to gradually extract the spatial features of the image, and obtain local features through batch normalization and non-linear activation functions: Where Y represents the local features of the patch; ReLU() is the activation function; and Batch Norm() is the batch normalization operation. For the first 1×1 convolution kernel, for The bias term; This represents the number of channels after dimensionality reduction. The number of channels in the input tensor; It is a 2×2 convolution kernel. for The bias term, The number of channels for the output tensor; For the second 1×1 convolution kernel, for The bias term; * indicates multiplication; X p This is the patch obtained after transposing; S32. The state space unit of the fusion module extracts global features from the input patch, and then concatenates the global and local features corresponding to the same patch to obtain the concatenated features. S33. The channel attention module of the fusion module is used to calculate the channel attention of the spliced ​​features to obtain the channel attention map: Where F represents the splicing feature; M c (F) is the channel attention map of the concatenated feature F; σ′() is the Sigmoid activation function; Avg Pool() is the average pooling operation; MLP() is the multilayer perceptron; Max Pool() is the max pooling operation. S34. Multiply the stitched features element-wise with their corresponding channel attention maps, then input them into the spatial attention module of the fusion module for spatial attention calculation to obtain the spatial attention map: Where F′ is the feature map obtained by element-wise multiplication; M s (F′) is the spatial attention map corresponding to feature map F′; f 7×7 This is a 7 × 7 convolution operation; S35. Multiply the spatial attention map and the spliced ​​features element-wise to obtain the first fused feature; S36. Input the first fusion feature into another fusion module, and repeat steps S31 to S35 to obtain the final fusion feature of each patch.

2. The gastrointestinal disease classification method based on convolution and state-space model according to claim 1, characterized in that, Step S1 further includes: S11. Extract texture features from medical images of gastrointestinal diseases using a Gabor filter: Where G(x, y) is the texture feature of the pixel with spatial coordinates (x, y); exp is the exponential function; σ is the scale parameter of the Gabor filter; f is the frequency of the Gabor filter; and θ is the orientation angle of the Gabor filter. S12. Determine whether the texture feature G(x, y) of the pixel with spatial coordinates (x, y) is greater than a preset threshold. If it is, mark it as a lesion area; otherwise, mark it as a normal area. S13. Calculate the fitness threshold of the pixels corresponding to the lesion region: Where threshold(x, y) is the fitness threshold for the pixel with spatial coordinates (x, y); µ(x, y) and σ(x, y) are the mean and standard deviation of the gray values ​​of all pixels in the neighborhood centered at (x, y) in the medical image, respectively; λ is the sensitivity adjustment parameter. S14. Determine whether the gray value of the pixel with spatial coordinates (x, y) is greater than its corresponding fitness threshold. If yes, proceed to step S15; otherwise, do not perform contrast enhancement on the pixel. S15. Enhance the contrast of pixels: Where I(x, y) and I′(x, y) are the gray values ​​of the pixel with spatial coordinates (x, y) before and after enhancement; α controls the intensity of contrast enhancement.

3. The gastrointestinal disease classification method based on convolution and state-space model according to claim 1, characterized in that, Methods for segmenting medical images into multiple patches include: Divide medical images into sizes of The patches, each representing a local region in a medical image; Each patch is transformed into a vector of a set dimension using convolution operations. The resulting embedding is represented as a tensor containing the features of all patches. The shape is: Where B is the batch size. denoted as the number of patches in a medical image; D as the dimension of each patch mapped to the feature space; H as the height of the medical image; W as the medical image dimension; and P as the side length of each patch.

4. The gastrointestinal disease classification method based on convolution and state-space model according to claim 1, characterized in that, Methods for merging the fusion features of multiple patches include: The four segmented regions are merged according to the channel dimension by concatenating the features of the four regions together, which increases the feature dimension of each patch by four times. Then, each new feature map obtained after merging is normalized, and then a linear transformation is used to reduce the dimensionality of the normalized features to obtain the merged features.

5. The gastrointestinal disease classification method based on convolution and state-space models according to any one of claims 1-4, characterized in that, The expression for the loss function of the deep learning model is: in, Standard cross-entropy loss; Loss of focus; For structural similarity loss; For boundary loss; λ1, λ2, λ3, and λ4 are respectively , , and The weighting coefficients.

6. The gastrointestinal disease classification method based on convolution and state-space model according to claim 5, characterized in that, , and The expressions are as follows: ， in, To adjust the hyperparameters for class balance; p t γ represents the probability of the deep learning model predicting the correct category; γ is a parameter for adjusting the difficulty of samples; SIMM is the structural similarity index; I and These are the original image during training and the predicted image from the deep learning model, respectively. and The k-th original image during training. and the corresponding deep learning model's predicted image The boundary gradient, 1≤k≤U, where U is the total number of medical images in the dataset during training; It is the absolute value symbol.

7. A gastrointestinal disease classification system based on convolution and state-space models, characterized in that, A gastrointestinal disease classification system is applied to the gastrointestinal disease classification method based on convolution and state-space models as described in claim 1, comprising: The image filtering and enhancement module is used to extract texture features from medical images of gastrointestinal diseases, determine the lesion areas in the medical images based on the texture features, and perform adaptive contrast enhancement on the lesion areas. The preprocessing module is used to preprocess medical images after contrast enhancement of the lesion area; The patch generation module is used to input the preprocessed image into the patch embedding module of the trained deep learning model to segment the medical image into multiple patches; The first fusion feature generation module is used to input the patch into the two fusion modules of the deep learning model in sequence. The convolutional unit and state space unit of each fusion module extract local and global features from the patch respectively. Then, the local and global features of the patch are fused to obtain the fusion feature. The feature merging module is used to merge the fused features of multiple patches using the patch merging module of the deep learning model to obtain merged features; The second fusion feature generation module is used to generate fusion features from the two fusion modules of the deep learning model by inputting the merged features; The judgment module is used to determine whether the executed feature merging module and the second fusion feature generation module are the last group. If so, the output of the last second fusion feature generation module is output as the final feature map; otherwise, the next group of feature merging modules and the second fusion feature generation module are entered. The classification module is used to input the final feature map of the medical image into the classifier of the deep learning model to obtain the classification result.

Images