Deep learning-based inflammatory bowel disease classification method and system

Abstract

The invention discloses an inflammatory bowel disease classification method and system based on deep learning. The method comprises the following steps: acquiring an original RGB image of an enteroscope; constructing a classification model based on a convolutional neural network; inputting an original colonoscope RGB image into a backbone network of the classification model, generating a space attention graph through multi-stage cascade processing of a convolution unit and a double-branch statistics-receptive field attention module, and inputting the space attention graph into a cross-stage attention fusion module; an original colonoscope RGB image is input into an endoscope depth-of-field feature extractor, and the endoscope depth-of-field feature extractor extracts multiple pieces of depth-of-field information of the colonoscope and inputs the depth-of-field information into a multi-scale visual Transform encoder; the cross-stage attention fusion module utilizes a dynamic feature fusion mechanism, combines a multi-scale visual Transform encoder, and dynamically weights and fuses the space attention map and Transform features to generate global features. According to the technical scheme, the visual features and the anatomical structure features of the enteroscopy image are fused, and the recognition precision is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical equipment and relates to an inflammatory bowel disease classification method and system based on deep learning. BACKGROUND

[0002] Inflammatory bowel disease (IBD) is a recurrent and non-specific inflammatory disease, including Crohn's disease (CD) and ulcerative colitis (UC). IBD is still a rare disease in China, but the number of patients has increased rapidly in the past 20 years.

[0003] The clinical manifestations of CD are diverse, and have many common features with ITB, and each feature can lead to misdiagnosis of the other and have the risk of causing harmful consequences. Immunosuppressants are usually used to treat celiac disease, but they can cause complications in ITB patients. When the diagnosis is uncertain, clinicians often empirically use starting anti-tuberculosis treatment (ATT) and evaluate the response to treatment.

[0004] And the existing detection technology usually only uses the image itself, and it is difficult to effectively process the barrel distortion and depth gradient effect unique to the colonoscopy image, resulting in insufficient recognition rate of the distal tissue. SUMMARY

[0005] The application aims to provide an inflammatory bowel disease classification method and system based on deep learning, which fuses the visual features and anatomical structure features of the colonoscopy image to improve the recognition accuracy.

[0006] In order to achieve the above-mentioned purpose, the basic scheme of the application is as follows: an inflammatory bowel disease classification method based on deep learning, comprising the following steps:

[0007] Collecting a colonoscopy original RGB image;

[0008] Based on a convolutional neural network, a classification model is constructed, which includes a double-branch statistical-receptive field attention module, a cross-stage attention fusion module and an endoscope depth feature extractor;

[0009] The colonoscopy original RGB image is input into the backbone network of the classification model, processed by the convolution unit and the double-branch statistical-receptive field attention module in multiple levels, to generate a spatial attention map and input it into the cross-stage attention fusion module; the colonoscopy original RGB image is input into the endoscope depth feature extractor, which extracts multiple depth information of the colonoscopy and inputs it into the multi-scale visual Transformer encoder;

[0010] The cross-stage attention fusion module utilizes a dynamic feature fusion mechanism, combines a multi-scale visual Transformer encoder, dynamically weights and fuses spatial attention maps and Transformer features, generates global features, and after global pooling, obtains each class probability (the number of classes is set according to the task) through a fully connected layer FC. The output class probability obtains a probability result of 1 for each class through an external softmax function. For example, now for the task of judging ulcerative colitis and Crohn's disease, the output is 【0.2, 0.8】 (the former represents the probability of ulcerative colitis, and the latter represents the probability of Crohn's disease), and after post-processing, the model result is 1 (class number: 0 represents ulcerative colitis, and 1 represents Crohn's disease). The classification result of the model is Crohn's disease.

[0011] The working principle and beneficial effects of the basic scheme are that the technical scheme fuses the visual features and anatomical structure features of the enteroscopy image, and is used for three classification of ulcerative colitis (UC), Crohn's disease (CD) and normal tissue in the enteroscopy image.

[0012] The convolutional neural network (CNN) is improved by introducing a double-branch statistical-receptive field attention mechanism (DB-SRFA), so that the network can learn more comprehensive features. The advantages of CNN and multi-scale visual Transformer (MViT) are fused, and a dynamic feature fusion mechanism is used to realize cross-stage feature interaction through a cross-stage attention fusion module (CSAF).

[0013] When learning global features using a multi-scale visual Transformer, the endoscope depth feature extractor (EDEF) fuses the depth information specific to enteroscopy, so that the model can learn the structure-specific features of the endoscopy image and improve the recognition accuracy.

[0014] Further, the enteroscopy original RGB image is input into the backbone network of the classification model for five-stage convolution feature extraction.

[0015] At the beginning of the fourth and fifth stages, the anatomical structure features extracted by the endoscope depth feature extractor (ring branch) are spliced with the features;

[0016] The ring branch processes the ring mask and the original image through the endoscope depth feature extractor to generate depth feature vectors aligned with the image feature space, and reduces the dimension to 96 through linear transformation.

[0017] The multi-scale features output by the backbone network of the classification model are spliced with the shallow features after upsampling to form local-global hybrid features with a channel number of 128, which are converted to sequence features with a dimension of 96 through a convolutional feed-forward network (CFFN) module.

[0018] The sequence feature and the ring feature vector are spliced and input into a multi-scale visual Transformer encoder, and a two-stage multi-scale attention mechanism is used to generate global representation, stage 4: 56x56→56x56, stage 5: 28x28→28x28;

[0019] The high-level semantic features output by the multi-scale visual Transformer encoder and the high-level features of the backbone network of the classification model are dynamically weighted and fused in the CSAF module, and the joint features are used for multi-task prediction after global average pooling.

[0020] Based on the classification model, data analysis and processing are realized, which is beneficial to use.

[0021] Further, the original RGB image of the enteroscope is input into the backbone network of the classification model, and is processed by a multi-level cascade of convolution units and double-branch statistical-receptive field attention modules to generate a spatial attention map.

[0022] The input enteroscope original RGB image X∈R C×H×W is compressed in channel dimension by a shared dimension reduction layer:

[0023] X red =ReLU(BN(Conv 1x1 (X)))

[0024] Where X red is the dimension reduction feature, ReLU is a nonlinear activation function, BN is batch normalization, and Conv 1x1 is a convolution kernel with a size of 1x1.

[0025] The output channel number is C / r, and the feature is extracted by a double-branch, specifically:

[0026] The multi-scale receptive field branch adopts double-expansion convolution to capture different scale features:

[0027]

[0028] F dil =ReLU(BN(Conv 1x1 ([F dil1 ;F dil2 ])))

[0029] Where F dil1 is the convolution result with an expansion rate of 1, F dil2 is the convolution result with an expansion rate of 2, d is the expansion rate of the expansion convolution, and F dil is the multi-scale receptive field branch feature; Conv represents convolution operation.

[0030] Statistical multi-scale receptive field branch, contains two complementary statistics:

[0031] Compute standard deviation feature F std , representing local fluctuations:

[0032] F std = Conv 1x1 (Std(X red ))

[0033] Where Std(·) is the channel dimension standard deviation calculation:

[0034]

[0035] Where Std(X) h,w is the cross-channel standard deviation of input data x at spatial position (h, w), C is the number of channels, X c,h,w is the specific value at channel c, height h, width w, μ h,w is the mean of all channels at spatial position (h, w);

[0036] Compute global statistical feature F global , representing the overall distribution:

[0037]

[0038] Where H is the height of the feature map, W is the width of the feature map, is the specific value at channel c, height h, width w;

[0039] Get the fusion feature F stats is:

[0040] F stats = ReLU(BN(Conv3x3([F std ; F global ])));

[0041] Generate spatial attention map A through large kernel convolution:

[0042] A = σ(Conv 5x5 ([F dil ; F stats ]))

[0043] Where σ(·) is the Sigmoid function, output value range [0, 1] attention weight map;

[0044] Finally, the feature is reweighted using the gated residual structure:

[0045] X 双支 = X ⊙ (1 + A)

[0046] wherein X 双支 is the final output feature, and is the dot product.

[0047] The double-branch statistical-receptive field attention mechanism combines the local standard deviation feature and the multi-scale dilated convolution to construct a double-branch attention mechanism, thereby enhancing the recognition ability of villus morphology and microvessel patterns.

[0048] Further, multiple depth-of-field information of the enteroscope is extracted by the endoscope depth-of-field feature extractor EDEF, specifically:

[0049] The original ring-shaped mask is down-sampled to align the mask with the feature space:

[0050] M down =Resize (bilinear) (M R ,(H,W))

[0051] wherein M R is the original ring-shaped mask, R is the number of mask rings; M down is the ring-shaped mask after size adjustment, H is the height of the target size, W is the width of the target size, and Resize (bilinear) is an operation of size adjustment using the bilinear interpolation method;

[0052] The mask is changed in the channel dimension:

[0053] F mask =ReLU(BN(Conv 1x1 (M down )))

[0054] wherein F mask is the mask feature after spatial alignment;

[0055] At this time, the mask has been processed, and the processed mask is spatially weighted with the input original feature X 原始 :

[0056] F masked =X 原始 ⊙F mask

[0057] wherein F masked is the weighted feature map, and X 原始 is the original feature

[0058] The weighted feature is used to generate depth-of-field attention:

[0059] A d =σ(Conv 3x3 (ReLU(BN(Conv 3x3 (F masked )))))

[0060] where σ is a Sigmoid function; A d is the depth attention map;

[0061] The extracted depth features are combined with the input original features X 原始 to enhance and fuse the features:

[0062] F att = X 原始 ⊙ A d

[0063] F res = X 原始 + F att F out = Conv 1x1 (F res )

[0064] where F att is the depth attention enhanced feature, F res is the residual connection feature, and F out is the final output feature.

[0065] EDFE introduces a ring partition, which significantly improves the adaptability to the geometric characteristics of the intestinal cavity by combining convolution features with predefined ring regions. It effectively solves the physical constraint problem of wide-angle lens imaging.

[0066] Further, the method of dynamically weighting and fusing the spatial attention map and the Transformer feature to generate global features is as follows:

[0067] Upsample the Transformer feature to match the size of the spatial attention map and fuse along the channel dimension:

[0068]

[0069] X 全局拼接 = [feat1; feat2'] ∈ R B×(C1+C2)×H×W

[0070] where feat2' is the Transformer feature map after matching the size of the spatial attention map, s is the step size of the transpose convolution, feat2 is the input Transformer feature, feat1 is the backbone feature after the 3rd stage, B is the data batch size, C1 is the number of feat1 channels, C2 is the number of feat2 channels, H is the feature map height, W is the feature map width, and X 全局拼接 is the feature map after concatenating the two features;

[0071] Feature extraction is performed through initial channel attention:

[0072] Z = GAP(X) 全局拼接 )

[0073]

[0074] Where Z represents the feature after global average pooling, A1c represents the channel attention weight, and X represents the feature value after global average pooling. 1 W1 is the feature map after channel attention weighting, W2 and W1 are learnable weight matrices; GAP(·) is global average pooling, δ is the ReLU6 activation function, σ is the Sigmoid function, w is the learnable weight matrix, and ⊙ is element-wise dot product;

[0075] Fusion by enhancing channel attention:

[0076] Z avg =GAP(X) 1 );Z max =GMP(X) 1 )

[0077]

[0078] Among them, Z avg Z is the spatial average feature extracted through global average pooling. max To extract the spatially largest feature through global max pooling, To combine the channel attention weights from average and max pooling features, W4 and W3 are learnable weight matrices, and X... 2 The feature map is weighted by the second-stage channel attention; GMP(·) is global max pooling;

[0079] Spatial feature compression is achieved through spatial attention:

[0080]

[0081] A s =σ(Conv kxk ([M avg M max ]))

[0082] X 3 =X 2 ⊙A s

[0083] Among them, M avg C represents the global average pooling feature in the spatial dimension, and C represents the input feature map X. 2 The number of channels, For X 2 The two-dimensional feature map M on channel c max For the global max-pooling feature in the spatial dimension, A sFor spatial attention weight, k is the size of the convolution kernel, X 3 is the feature map after spatial attention weighting;

[0084] Finally, the original feature information is retained through the residual connection:

[0085] X 融合 = X 3 + X 全局拼接

[0086] Wherein, X 融合 is the global feature after dynamically weighting the fusion of spatial attention map and Transformer feature.

[0087] By adaptive pooling to capture the complementary relationship between global context and local details, combined with dynamic spatial attention weighting, the feature fusion efficiency is improved, and the model convergence speed is accelerated. The integrity of the wrinkle continuity feature is maintained, providing more reliable feature representation for clinical diagnosis.

[0088] Further, task one: ulcerative colitis and Crohn's disease detection:

[0089] Two categories are classified, and two category numbers are preset: 0 represents ulcerative colitis, and 1 represents Crohn's disease;

[0090] The output category probability is obtained by the external softmax function, and the probability result is summed to 1 for each category. The result is compared with the two category numbers, if it is 0, it is judged that the image corresponds to ulcerative colitis, if it is 1, it is judged that the image corresponds to Crohn's disease;

[0091] Task two: ulcerative colitis severity classification:

[0092] Four categories are classified, and four severity levels of 0, 1, 2 and 3 are divided, and the corresponding prediction probability is set. The obtained probability result is compared with the prediction probability, and the original RGB image of the colonoscope is judged as the corresponding severity of ulcerative colitis.

[0093] Based on the output result, the inflammatory bowel disease category is judged, which is convenient to operate.

[0094] The application also provides an inflammatory bowel disease classification system based on deep learning, comprising a data acquisition unit and a processing unit, the data acquisition unit is used for acquiring the original RGB image of the colonoscope, and transmitting to the processing unit;

[0095] The processing unit executes the method of the application, and outputs the inflammatory bowel disease classification result.

[0096] The system is based on deep learning, and the visual features and anatomical structure features of the colonoscopy image are fused, so that more accurate inflammatory bowel disease recognition is realized.

[0097] Further, the processing unit is internally provided with a classification model, which includes a double-branch statistical-receptive field attention module, a cross-stage attention fusion module and an endoscope depth-of-field feature extractor;

[0098] The double-branch statistical-receptive field attention module is used to receive the original RGB image of the enteroscope, and generate a spatial attention map;

[0099] The cross-stage attention fusion module is used to generate a Transformer feature, and fuse the spatial attention map and the Transformer feature to generate a joint feature;

[0100] The endoscope depth-of-field feature extractor is used to extract the depth-of-field information of the enteroscope and input it into the cross-stage attention fusion module.

[0101] The processing unit is internally provided with a classification model, which is used to complete the identification of inflammatory bowel disease with higher accuracy.

[0102] Further, the double-branch statistical-receptive field attention module includes a shared 1x1 convolution layer, a batch normalization layer BN, a ReLU activation function layer, a multi-scale receptive field branch, a statistical feature branch, a 5x5 convolution layer and a Sigmoid function;

[0103] The double-branch statistical-receptive field attention module uses the shared 1x1 convolution layer to perform channel compression on the input feature map, and completes dimension reduction processing in cooperation with the batch normalization BN and the ReLU activation function to generate a low-dimensional feature representation;

[0104] The low-dimensional feature representation is input into two complementary branches in parallel: the multi-scale receptive field branch captures local and extended spatial information through two 3x3 dilated convolution layers with different dilation rates (d=1 and d=2), respectively, and then splices and fuses the multi-scale features through a 3x3 convolution layer after BN and ReLU;

[0105] The statistical feature branch calculates the channel dimension standard deviation (representing local pixel fluctuations) and global average pooling (extracting overall distribution features) in parallel, respectively, and then splices and further refines the statistical information through a 3x3 convolution layer after feature abstraction through a 1x1 convolution layer and processing through BN and ReLU.

[0106] The output features of the two branches are input into a 5x5 convolution layer after splicing, and a spatial attention weight map is generated by combining a Sigmoid function, the weight map is point multiplied with the reduced features to realize feature weighting, and finally the feature enhancement output is completed by adding the original reduced features through a gated residual structure (1+attention map).

[0107] The double-branch statistical-receptive field attention module structure is easy to use and obtain the required parameters.

[0108] Further, the cross-stage attention fusion module comprises a 3x3 transpose convolution layer, a 1x1 convolution layer, a 5x5 convolution layer, a global maximum pooling layer, a global average pooling layer, an average pooling layer, a maximum pooling layer, a full connection layer and a ReLU6 activation function;

[0109] The cross-stage attention fusion module up-samples the Transformer feature to match the size of the CNN feature, and completes feature alignment through a 3x3 transpose convolution, and generates a hybrid feature representation by channel dimension splicing and fusion;

[0110] The hybrid feature representation is sequentially processed by a triple attention mechanism:

[0111] The initial channel attention compresses spatial information through global average pooling, and generates a channel weight map through two full connection layers (including ReLU6 activation) to preliminarily weight the hybrid feature;

[0112] The enhanced channel attention combines the complementary statistics of global average pooling and global maximum pooling, extracts more robust channel correlation through a shared full connection layer, and generates a second weighted feature;

[0113] The spatial attention compresses the channel weighted feature in the spatial dimension, extracts the spatial key area through the parallel operation of average pooling and maximum pooling, and generates a spatial attention map after 5x5 convolution layer fusion to calibrate the feature in the spatial dimension;

[0114] Finally, the calibrated feature is added to the original hybrid feature through residual connection to retain multi-scale information while strengthening the cross-modal feature expression, realizing efficient cross-stage feature fusion and enhancement.

[0115] The cross-stage attention fusion module realizes feature fusion, which is beneficial to use.

[0116] Further, the endoscope depth of field feature extractor comprises a bilinear interpolation down-sampling layer, a 1x1 convolution layer, a 3x3 convolution layer, a batch normalization layer BN, a ReLU activation function layer and a Sigmoid function;

[0117] The endoscope depth of field feature extractor performs bilinear interpolation down-sampling on the original ring mask, adjusts the resolution to match the size of the input feature map, and generates a spatially aligned mask representation;

[0118] The mask is adjusted in the channel dimension through a 1x1 convolution layer, and the mask feature preprocessing is completed through batch normalization (BN) and ReLU activation function;

[0119] The processed mask and the current input feature are subjected to spatial weighting operation (element-by-element point multiplication), to strengthen the area in the feature map that conforms to the depth of field distribution pattern;

[0120] The weighted features sequentially pass through two 3*3 convolution layers (including BN and ReLU) to extract depth-related features, and generate a spatial attention weight map through a Sigmoid function.

[0121] Finally, the attention map is multiplied with the original feature to realize feature weighting, the residual connection is combined to retain the original information without weighting, and then 1*1 convolution layer is used to complete cross-channel feature fusion, and enhanced feature representation with depth perception ability is output.

[0122] The module structure is simple and convenient to use. BRIEF DESCRIPTION OF DRAWINGS

[0123] Figure 1 is a flowchart of the inflammatory bowel disease classification method based on deep learning of the present application;

[0124] Figure 2 is a structural diagram of the double-branch statistical-receptive field attention module of the inflammatory bowel disease classification system based on deep learning of the present application;

[0125] Figure 3 is a structural diagram of the cross-stage attention fusion module of the inflammatory bowel disease classification system based on deep learning of the present application;

[0126] Figure 4 is a structural diagram of the endoscope depth feature extractor of the inflammatory bowel disease classification system based on deep learning of the present application. DETAILED DESCRIPTION

[0127] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary, only for explaining the present application, and cannot be understood as a limitation of the present application.

[0128] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0129] In the description of the present application, unless otherwise specified and limited, it is necessary to explain that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication inside two elements, it can be direct connection or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0130] The application discloses a kind of inflammatory bowel disease classification method based on deep learning, fusion intestinal mirror image visual feature and anatomical structure feature, for ulcerative colitis (UC), Crohn's disease (CD) and three classification of normal tissue in intestinal mirror image and the Mayo score classification of UC. As shown in Figure 1 The application discloses a kind of inflammatory bowel disease classification method based on deep learning, fusion intestinal mirror image visual feature and anatomical structure feature, for ulcerative colitis (UC), Crohn's disease (CD) and three classification of normal tissue in intestinal mirror image and the Mayo score classification of UC. As shown in

[0131] Collecting intestinal mirror original RGB image;

[0132] Based on convolutional neural network, build classification model, the classification model includes double-branch statistical-receptive field attention module, cross-stage attention fusion module and endoscope depth feature extractor;

[0133] Intestinal mirror original RGB image is input into the backbone network of classification model, is processed by convolution unit and double-branch statistical-receptive field attention module multistage cascade, generates spatial attention map and is input into cross-stage attention fusion module;Intestinal mirror original RGB image is input into endoscope depth feature extractor, the endoscope depth feature extractor extracts multiple depth information of intestinal mirror, and input into multi-scale visual Transformer encoder;

[0134] Cross-stage attention fusion module utilizes dynamic feature fusion mechanism, combines multi-scale visual Transformer encoder, dynamically weighted fusion spatial attention map and Transformer feature, generates global feature and is after global pooling, through fully connected layer FC obtains each class probability (class number is according to task setting), the probability result of each class is obtained by the external softmax function. For example, now for judging ulcerative colitis and Crohn's disease task, output is 【0.2, 0.8】 (the former represents ulcerative colitis probability, the latter represents Crohn's disease probability), the model result after post-processing is 1 (class number: 0 represents ulcerative colitis, 1 represents Crohn's disease), that is, the classification result of model is Crohn's disease.

[0135] In a preferred scheme of the application, intestinal mirror original RGB image is input into the backbone network of classification model to extract five-stage convolution features, and the channel number is 64→1024 or 64→2048, etc.

[0136] At the beginning of the fourth stage (output size 56x56) and the fifth stage (output size 28x28), feature splicing is performed on the anatomical structure features extracted by the endoscope depth feature extractor (ring branch);

[0137] The ring branch processes the annular mask (which is a fixed input picture) and the original image through the endoscope depth feature extractor to generate a depth feature vector aligned with the image feature space, and the depth feature vector is dimensionally reduced to 96 dimensions through linear transformation;

[0138] The multi-scale features output by the backbone network of the classification model (i.e., the first three stages) are spliced with the shallow features after being upsampled, forming local-global hybrid features with a channel number of 128, which are converted into sequence features with 96 dimensions through a CFFN module (convolutional feed-forward network, CFFN, which is an existing module);

[0139] The sequence features are spliced with the ring feature vectors and then input into a multi-scale visual Transformer (MViT) encoder to generate global representations through a two-stage multi-scale attention mechanism: stage 4: 56x56→56x56, stage 5: 28x28→28x28;

[0140] The high-level semantic features (192 dimensions) output by the multi-scale visual Transformer encoder are dynamically weighted and fused with the high-level features (1024 dimensions) of the backbone network of the classification model in a CSAF module, and the output joint features are used for multi-task prediction after global average pooling.

[0141] In a preferred scheme of the present application, the endoscopic original RGB image is input into the backbone network of the classification model, and a method for generating a spatial attention map through multi-level cascading processing of convolution units and a dual branch statistical-receptive field attention module (DB-SRFA) is as follows:

[0142] The input endoscopic original RGB image X∈R C×H×W is compressed in channel dimension through a shared dimension reduction layer:

[0143] X red = ReLU(BN(Conv 1x1 (X)))

[0144] wherein X red is the dimension-reduced feature, ReLU is a nonlinear activation function, BN is batch normalization, and Conv 1x1 is a convolution with a convolution kernel size of 1x1;

[0145] The output channel number is C / r, and feature extraction is performed through a double branch, specifically:

[0146] The multi-scale receptive field branch adopts double dilation convolution to capture different scale features:

[0147]

[0148] F dil = ReLU(BN(Conv 1x1 ([F dil1 ; F dil2 ])))

[0149] Where F dil1 is the convolution result with dilation rate = 1, F dil2 is the convolution result with dilation rate = 2, d is the dilation rate of the dilation convolution, and F dil is the multi-scale receptive field branch feature; Conv represents convolution operation;

[0150] Statistical multi-scale receptive field branch, including two complementary statistics:

[0151] Calculate the standard deviation feature F std , representing local fluctuations:

[0152] F std = Conv 1x1 (Std(X red ))

[0153] Where Std(·) is the channel dimension standard deviation calculation:

[0154]

[0155] Where Std(X) h,w is the cross-channel standard deviation of the input data x at the spatial position (h, w), C is the channel number, X c，h，w is the specific value at channel c, height h, and width w, and μ h，w is the mean of all channels at the spatial position (h, w);

[0156] Calculate the global statistical feature F global , representing the overall distribution:

[0157]

[0158] Where H is the height of the feature map, W is the width of the feature map, is the specific value at channel c, height h, and width w;

[0159] The fusion feature F stats is obtained as:

[0160] F stats = ReLU(BN(Conv3x3([F std ; F global ])))

[0161] Generate spatial attention map A by large kernel convolution:

[0162] A = sigma(Conv 5x5 ([F dil ; F stats ]))

[0163] Where sigma(·) is the Sigmoid function, and the output is an attention weight map with a value range of [0, 1];

[0164] Finally, the feature is reweighted using the gated residual structure:

[0165] X 双支 = X o (1 + A)

[0166] Where X 双支 is the final output feature, and o is the dot product.

[0167] The double-branch statistical-receptive field attention mechanism (DB-SRFA) is introduced to improve the convolutional neural network (CNN), enabling the network to learn more comprehensive features. The DB-SRFA module surpasses traditional attention methods by fusing statistical features and multi-scale receptive fields. Existing techniques such as SE and CBAM rely solely on global statistical information, making it difficult to capture local microstructure changes and multi-scale features of lesion areas in colonoscopy images.

[0168] DB-SRFA innovatively combines local standard deviation features with multi-scale dilated convolution to construct a double-branch attention mechanism, enhancing the recognition ability of villous morphology and microvascular patterns.

[0169] In a preferred embodiment of the present application, multiple depth information of the colonoscope is extracted by an endoscopic depth feature extractor (EDFE), specifically:

[0170] Downsample the original ring mask to align the mask with the feature space:

[0171] M down = Resize (bilinear) (M R ,(H,W))

[0172] Where M R is the original ring mask, R is the number of mask rings (default is 6); M downFor the resized annular mask, H is the height of the target size, W is the width of the target size, Resize (bilinear) For the operation of size adjustment using the bilinear interpolation method;

[0173] Channel dimension change is performed on the mask:

[0174] F mask = ReLU(BN(Conv 1x1 (M down )))

[0175] Where F mask is the spatially aligned mask feature;

[0176] At this time, the mask has been processed, and the processed mask is combined with the input original feature X 原始 to perform spatial weighting:

[0177] F masked = X 原始 ⊙F mask

[0178] Where F masked is the weighted feature map, and X 原始 is the original feature;

[0179] The weighted feature is used to generate a depth attention:

[0180] A d = σ(Conv 3x3 (ReLU(BN(Conv 3x3 (F masked )))))

[0181] Where σ is the Sigmoid function; A d is the depth attention map;

[0182] The extracted depth feature is combined with the input original feature X 原始 to perform feature enhancement and fusion:

[0183] F att = X 原始 ⊙A d

[0184] F res = X 原始 +F att

[0185] F out = Conv 1x1 (F res )

[0186] Where F attFor depth attention enhanced features, F res For residual connection features, F out For final output features.

[0187] When learning global features by using a multi-scale visual Transformer, the endoscopic depth feature extractor (EDEF) fuses depth information specific to a colonoscope, so that the model can learn structure-specific features of an endoscopic image.

[0188] In a preferred scheme of the present application, the method for generating global features by dynamically weighting the spatial attention map and the Transformer features is as follows:

[0189] Upsample the Transformer feature to match the size of the spatial attention map, and fuse along the channel dimension:

[0190]

[0191] X 全局拼接 =[feat1;feat2′]∈R B×(C1+C2)×H×W

[0192] wherein feat2' is the Transformer feature map after matching the size of the spatial attention map, s is the step size of the transpose convolution, feat2 is the input Transformer feature, feat1 is the backbone feature after the 3rd stage, B is the data batch size, C1 is the number of feat1 channels, C2 is the number of feat2 channels, H is the feature map height, W is the feature map width, and X 全局拼接 is the feature map after splicing the two features;

[0193] Feature extraction is performed by initial channel attention:

[0194] Z=GAP(X 全局拼接 )

[0195]

[0196] wherein Z is the feature after global average pooling, is the channel attention weight, X 1 is the feature map after channel attention weighting, W2 and W1 are learnable weight matrices; GAP(·) is global average pooling, δ is the ReLU6 activation function, σ is the Sigmoid function, w is the learnable weight matrix, and is the element-wise point multiplication;

[0197] Fusion is performed by enhancing channel attention:

[0198] Z avg =GAP(X 1 );Z max=GMP(X) 1 )

[0199]

[0200] Among them, Z avg Z is the spatial average feature extracted through global average pooling. max To extract the spatially largest feature through global max pooling, To combine the channel attention weights from average and max pooling features, W4 and W3 are learnable weight matrices, and X... 2 The feature map is weighted by the second-stage channel attention; GMP(·) is global max pooling;

[0201] Spatial feature compression is achieved through spatial attention:

[0202]

[0203] A s =σ(Conv kxk ([M avg M max ]))

[0204] X 3 =X 2 ⊙A s

[0205] Among them, M avg C represents the global average pooling feature in the spatial dimension, and C represents the input feature map X. 2 The number of channels, For X 2 The two-dimensional feature map M on channel c max For the global max-pooling feature in the spatial dimension, A s Here, k represents the spatial attention weights, k is the kernel size, and X is the spatial attention weight. 3 The feature map is spatially attention-weighted;

[0206] Finally, the original feature information is preserved through residual connections:

[0207] X 融合 =X 3 +X 全局拼接

[0208] Among them, X 融合 This is the global feature obtained by dynamically weighting and fusing the spatial attention map and Transformer features.

[0209] By combining the advantages of CNN and Multi-Scale Visual Transformer (MViT), and utilizing a dynamic feature fusion mechanism, cross-stage feature interaction is achieved through a cross-stage attention fusion module (CSAF).

[0210] In a preferred scheme of the present application, task one: ulcerative colitis and Crohn's disease detection:

[0211] Binary classification is performed, and two class numbers are preset: 0 represents ulcerative colitis, and 1 represents Crohn's disease.

[0212] The output class probability is obtained by an external softmax function, and the probability result is summed up to 1 for each class. The result is compared with the two class numbers. If it is 0, it is judged that the image corresponds to ulcerative colitis, and if it is 1, it is judged that the image corresponds to Crohn's disease. For example, the output result is two classes 【0.2, 0.8】, the former represents the probability of ulcerative colitis as 0.2, and the latter represents the probability of Crohn's disease as 0.8. After comparison and value taking, the final model result is 1 (that is, the second class), and it is considered that the image is Crohn's disease.

[0213] Task two: ulcerative colitis severity classification:

[0214] Four classifications are performed, and four severity levels (0, 1, 2, and 3) are divided (the severity mayo score can be referred to: Mayo endoscopic score | Baidu health · medical popular science), and the corresponding prediction probability is set. The probability result is compared with the prediction probability to determine that the colonoscopy original RGB image is the corresponding severity of ulcerative colitis. For example, 【0.1, 0.1, 0.2, 0.6】 respectively represent the prediction probability of 0, 1, 2, and 3 four severity levels. After comparison and value taking, the final model result is 3, and it is considered that the image is ulcerative colitis with a severity of 3.

[0215] The standard cross-entropy loss (Cross-Entropy Loss) is a function for measuring the difference between two probability distributions, which is used to optimize the performance of the model. Its formula is Where Loss CE is the calculated loss, n is the number of classification categories, y i is the true label, and is the probability distribution predicted by the model.

[0216] The present application also provides an inflammatory bowel disease classification system based on deep learning, which comprises a data acquisition unit and a processing unit. The data acquisition unit is used to acquire colonoscopy original RGB images and transmit them to the processing unit. The processing unit executes the method described in the present application and outputs the inflammatory bowel disease classification result.

[0217] In a preferred scheme of the present application, the processing unit is provided with a classification model, which comprises a double-branch statistical-receptive field attention module, a cross-stage attention fusion module, and an endoscope depth feature extractor.

[0218] The double-branch statistical-receptive field attention module is used for receiving a colonoscope original RGB image and generating a spatial attention map.

[0219] The cross-stage attention fusion module (CSAF) is used for generating a Transformer feature and fusing the spatial attention map and the Transformer feature to generate a joint feature.

[0220] The complementary relationship between global context and local details is captured through adaptive pooling, and dynamic spatial attention weighting is combined, so that the feature fusion efficiency is improved, and the model convergence speed is accelerated.

[0221] The endoscope depth feature extractor is used for extracting the depth information of the colonoscope and inputting the cross-stage attention fusion module. The prior art usually only uses the image itself, and it is difficult to effectively process the barrel distortion and depth gradient effect unique to the colonoscope image, resulting in insufficient recognition rate of the distal tissue. The endoscope depth feature extractor EDFE innovatively introduces a ring partition, combines the convolution features with the predefined ring area, and significantly improves the adaptability to the intestinal cavity geometric characteristics.

[0222] In a preferred scheme of the present application, as shown in Figure 2 The double-branch statistical-receptive field attention module includes a shared 1×1 convolution layer, a batch normalization layer BN, a ReLU activation function layer, a multi-scale receptive field branch, a statistical feature branch, a 5×5 convolution layer and a Sigmoid function.

[0223] The double-branch statistical-receptive field attention module uses a shared 1×1 convolution layer to compress the channel of the input feature map, and completes the dimension reduction processing with the batch normalization BN and the ReLU activation function to generate a low-dimensional feature representation.

[0224] The low-dimensional feature representation is input into two complementary branches in parallel: the multi-scale receptive field branch captures local and extended spatial information through two 3×3 dilated convolution layers with different dilation rates (d=1 and d=2), and then splices and fuses the multi-scale features through a 3×3 convolution layer after BN and ReLU.

[0225] The statistical feature branch calculates the channel dimension standard deviation (representing local pixel fluctuation) and global average pooling (extracting overall distribution features) in parallel, respectively abstracts the features through a 1x1 convolution layer, splices the features after BN and ReLU processing, and further refines the statistical information through a 3x3 convolution layer;

[0226] The output features of the two branches are spliced and input into a 5x5 convolution layer to generate a spatial attention weight map in combination with a Sigmoid function. The weight map is point multiplied with the reduced features to realize feature weighting. Finally, the features are added to the original reduced features through a gated residual structure (1+attention map) to complete feature enhancement output.

[0227] As shown in Figure 3 , the cross-stage attention fusion module includes a 3x3 transpose convolution layer, a 1x1 convolution layer, a 5x5 convolution layer, a global maximum pooling layer, a global average pooling layer, an average pooling layer, a maximum pooling layer, a fully connected layer, and a ReLU6 activation function.

[0228] The cross-stage attention fusion module up-samples the Transformer features to match the CNN feature size and completes feature alignment through a 3x3 transpose convolution to generate a hybrid feature representation by splicing and fusing along the channel dimension.

[0229] The hybrid feature representation is processed in turn by a triple attention mechanism:

[0230] The initial channel attention compresses spatial information through global average pooling and generates a channel weight map through two fully connected layers (including ReLU6 activation) to preliminarily weight the hybrid features.

[0231] The enhanced channel attention extracts more robust channel correlations through a shared fully connected layer in combination with the complementary statistics of global average pooling and global maximum pooling to generate a second weighted feature.

[0232] The spatial attention performs spatial dimension compression on the channel-weighted features, extracts spatial key regions through parallel operations of average pooling and maximum pooling, generates a spatial attention map after fusion by a 5x5 convolution layer, and performs spatial dimension calibration on the features.

[0233] Finally, the calibrated features are added to the original hybrid features through a residual connection to retain multi-scale information while strengthening cross-modal feature expression, realizing efficient cross-stage feature fusion and enhancement.

[0234] As shown in Figure 4 , the endoscope depth of field feature extractor includes a bilinear interpolation down-sampling layer, a 1x1 convolution layer, a 3x3 convolution layer, a batch normalization layer BN, a ReLU activation function layer, and a Sigmoid function.

[0235] The endoscope depth-of-field feature extractor performs bilinear interpolation down-sampling on the original annular mask, adjusts its resolution to match the input feature map size, and generates a spatially aligned mask representation;

[0236] The mask is adjusted in channel dimension through a 1x1 convolutional layer, and the mask feature preprocessing is completed by batch normalization (BN) and ReLU activation function;

[0237] The processed mask and the current input feature are subjected to spatial weighting operation (element-wise point multiplication), which strengthens the area in the feature map that conforms to the depth-of-field distribution pattern;

[0238] The weighted feature is sequentially subjected to two 3x3 convolutional layers (including BN and ReLU) to extract depth-related features, and a spatial attention weight map is generated through a Sigmoid function;

[0239] Finally, the attention map is multiplied by the original feature to realize feature weighting, the unweighted original information is retained through residual connection, and cross-channel feature fusion is completed through a 1x1 convolutional layer to output an enhanced feature representation with depth perception capability.

[0240] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0241] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A deep learning-based inflammatory bowel disease classification method, characterized by, The method comprises the following steps: Collecting an enteroscope original RGB image; Based on a convolutional neural network, a classification model is constructed, which comprises a double-branch statistical-receptive field attention module, a cross-stage attention fusion module, and an endoscope depth feature extractor; The enteroscope original RGB image is input into the backbone network of the classification model, and is processed by a convolution unit and a double-branch statistical-receptive field attention module in multiple cascades to generate a spatial attention map and input the spatial attention map into the cross-stage attention fusion module; The enteroscope original RGB image is input into the endoscope depth feature extractor, and the endoscope depth feature extractor extracts multiple depth information of the enteroscope and inputs the multiple depth information into a multi-scale visual Transformer encoder; The cross-stage attention fusion module combines the multi-scale visual Transformer encoder by using a dynamic feature fusion mechanism, dynamically weights and fuses the spatial attention map and the Transformer feature, generates global features, and after global pooling, obtains a class probability through a fully connected layer FC, and outputs a class probability result through an external softmax function. 2.The deep learning-based inflammatory bowel disease classification method of claim 1, wherein, The enteroscope original RGB image is input into the backbone network of the classification model for five-stage convolution feature extraction; At the beginning of the fourth stage and the fifth stage, the anatomical structure features extracted by the endoscope depth feature extractor are spliced with the features; The annular mask is processed by the endoscope depth feature extractor together with the original image to generate a depth feature vector aligned with the image feature space, and the depth feature vector is dimensionally reduced through linear transformation; The multi-scale features output by the backbone network of the classification model are spliced with the shallow features after upsampling to form local-global hybrid features, which are converted into sequence features through a convolution feedforward neural network CFFN module; the sequence features are spliced with the annular feature vector and input into the multi-scale visual Transformer encoder to generate global representations through a two-stage multi-scale attention mechanism; The high-level semantic features output by the multi-scale visual Transformer encoder are dynamically weighted and fused with the high-level features of the classification model backbone network in the CSAF module, and the output joint features are used for multi-task prediction after global average pooling. 3.The deep learning-based inflammatory bowel disease classification method of claim 1, wherein, The method for generating a spatial attention map by inputting the enteroscope original RGB image into the backbone network of the classification model and processing the enteroscope original RGB image by a convolution unit and a double-branch statistical-receptive field attention module in multiple cascades comprises the following steps: Input endoscopic raw RGB image X ∈ R C×H×W By sharing the dimension reduction layer to compress the channel dimension: X red = ReLU(BN(Conv 1x1 (X))) wherein X red is a reduced dimension feature, ReLU is a nonlinear activation function, BN is batch normalization, Conv 1x1 is a convolution with a kernel size of 1x1. The output channel dimension is C / r, r is a compression ratio, and the default value is 16. The features are extracted through a double-branch path. Specifically, the double-branch path comprises: The multi-scale receptive field branch adopts a double-expansion convolution to capture different scale features: F dil = ReLU(BN(Conv 1x1 ([F dil1 ; F dil2 ]))) wherein F dil1 is the convolution result with dilation rate = 1, F dil2 is the convolution result with dilation rate = 2, d is the dilation rate of the dilation convolution, F dil is the multi-scale receptive field branch feature; Conv represents a convolution operation; The statistical multi-scale receptive field branch comprises two complementary statistics: Computing the standard deviation feature F std characterizes local fluctuations: F std = Conv 1x1 (Std(X red )) Wherein, Std(·) is a channel dimension standard deviation calculation: Where, Std(X) h,w Let C be the cross-channel standard deviation of input data x at spatial location (h, w), and let C be the number of channels. c,h,w To represent the specific values ​​at channel c, height h, and width w, μ h,w Let be the mean of all channels at spatial location (h, w); Computing global statistical features F global characterizing the overall distribution: where H is the height of the feature map and W is the width of the feature map, is the specific value at channel c, height h, and width w. Obtaining the fusion feature F stats is: F stats = ReLU(BN(Conv 3x3 ([F std ; F global ]))) ; A spatial attention map A is generated through a large kernel convolution: A = σ(Conv 5x5 ([F dil ; F stats ])) Wherein, σ(·) is a Sigmoid function, and the output value range is [0, 1]. Finally, a gated residual structure is adopted to realize feature reweighting: X 双支 = X O (1 + A) where X 双支 is the final output feature, and is the dot product. 4.The deep learning-based inflammatory bowel disease classification method of claim 1, wherein, The endoscope depth feature extractor EDEF extracts multiple depth information of the enteroscope, and the specific steps are as follows: The original annular mask is down-sampled to align the mask with the feature space: M down = Resize (bilinear) (M R , (H, W)) wherein M R is the original annular mask, R is the number of annular masks; M down is the resized annular mask, H is the height of the target size, W is the width of the target size, Resize (bilinear) is the operation of resizing using the bilinear interpolation method; The mask is changed in channel dimension: F mask = ReLU(BN(Conv 1x1 (M down ))) where F mask is the spatially aligned mask feature; At this time, the mask has been processed, and the processed mask is combined with the input original feature X 原始 Spatial weighting is performed: F masked = X 原始 O F mask wherein F masked is the weighted feature map, X 原始 is the original feature; The weighted features are used to generate depth attention: A d = σ(Conv 3x3 (ReLU(BN(Conv 3x3 (F masked ))))) wherein σ is a sigmoid function; A d is a depth attention map; The extracted depth features are combined with the input raw features X 原始 Feature enhancement and fusion are performed: F att = X 原始 O A d F res = X 原始 + F att F out = Conv 1x1 (F res ) where F att is the depth attention enhanced feature, F res is the residual connection feature, F out is the final output feature.

5. The deep learning-based inflammatory bowel disease classification method of claim 1, wherein, The dynamic weighted fusion of spatial attention map and Transformer feature is used to generate global feature as follows: The up-sampling Transformer feature matches the size of spatial attention map and is fused along the channel dimension: X 全局拼接 = [feat1; feat2'] E R B×(C1+C2)×H×W wherein feat2' is the matched spatial attention map size of the Transformer feature map, s is the step of the transposed convolution, feat2 is the input Transformer feature, feat1 is the backbone feature after the 3rd stage, B is the data batch size, C1 is the channel number of feat1, C2 is the channel number of feat2, H is the height of the feature map, W is the width of the feature map, X 全局拼接 is the feature map after the two features are spliced; Feature extraction is performed through initial channel attention: Z = GAP(X 全局拼接 ) wherein Z is the feature after global average pooling, is the channel attention weight, X 1 is the feature map after channel attention weighting, W2 and W1 are learnable weight matrices; GAP(·) is global average pooling, δ is ReLU6 activation function, σ is Sigmoid function, w is learnable weight matrix, and is element-wise point multiplication; Fusion is performed through enhanced channel attention: Z avg = GAP(X 1 ); Z max = GMP(X 1 ) wherein Z avg is a spatial average feature extracted by global average pooling, max is a spatial maximum feature extracted by global maximum pooling, is a channel attention weight combining average and maximum pooling features, W4, W3 are learnable weight matrices, X 2 is a feature map weighted by the second stage channel attention; GMP(·) is global maximum pooling. Spatial feature compression is completed through spatial attention: A s = σ(Conv kxk ([M avg ; M max ])) X 3 = X 2 O A s Among them, M avg C represents the global average pooling feature in the spatial dimension, and C represents the input feature map X. 2 The number of channels, For X 2 The two-dimensional feature map M on channel c max For the global max-pooling feature in the spatial dimension, A s Here, k represents the spatial attention weights, k is the kernel size, and X is the spatial attention weight. 3 The feature map is spatially attention-weighted; Finally, the original feature information is preserved through residual connection: X 融合 = X 3 + X 全局拼接 wherein X 融合 is the global feature after dynamically weighting the spatial attention map and the Transformer feature.

6. The deep learning-based inflammatory bowel disease classification method of claim 1, wherein, Task 1: Ulcerative colitis and Crohn's disease detection: Two categories are classified, with two category numbers preset: 0 represents ulcerative colitis, and 1 represents Crohn's disease. The output category probability is obtained by an external softmax function, which gives a probability result of 1 for each category. Compare this result with the two category numbers. If it is 0, it is judged that the image corresponds to ulcerative colitis. If it is 1, it is judged that the image corresponds to Crohn's disease. Task 2: Ulcerative colitis severity classification: Four categories are classified, with four severity levels of 0, 1, 2, and 3, and the corresponding prediction probabilities are set. Compare the obtained probability results with the prediction probabilities to determine the severity of the original RGB image of the colonoscope as the corresponding ulcerative colitis.

7. A deep learning-based inflammatory bowel disease classification system, characterized by, It includes a data acquisition unit and a processing unit, the data acquisition unit is used to acquire the original RGB image of the colonoscope and transmit it to the processing unit; The processing unit executes the method of any one of claims 1-6, and outputs the inflammatory bowel disease classification result.

8. The deep learning-based inflammatory bowel disease classification system of claim 7, wherein, The processing unit is provided with a classification model, which includes a double-branch statistical-receptive field attention module, a cross-stage attention fusion module, and an endoscope depth feature extractor; The double-branch statistical-receptive field attention module is used to receive the original RGB image of the colonoscope and generate a spatial attention map; The cross-stage attention fusion module is used to generate Transformer features and fuse the spatial attention map and the Transformer features to generate joint features; The endoscope depth feature extractor is used to extract the depth information of the colonoscope and input it into the cross-stage attention fusion module.

9. The deep learning-based inflammatory bowel disease classification system of claim 8, wherein, The double-branch statistical-receptive field attention module includes a shared 1×1 convolution layer, a batch normalization layer BN, a ReLU activation function layer, a multi-scale receptive field branch, a statistical feature branch, a 5×5 convolution layer, and a Sigmoid function; The double-branch statistical-receptive field attention module uses a shared 1×1 convolution layer to compress the channel of the input feature map, and completes the dimension reduction processing with batch normalization BN and ReLU activation function to generate a low-dimensional feature representation; This low-dimensional feature representation is input into two complementary branches in parallel: the multi-scale receptive field branch captures local and extended spatial information through two 3×3 dilated convolution layers with different dilation rates, concatenates after BN and ReLU, and then fuses the multi-scale features through a 3×3 convolution layer; The statistical feature branch calculates the channel dimension standard deviation and global average pooling in parallel, abstracts the features through a 1×1 convolution layer, concatenates after BN and ReLU processing, and extracts statistical information through a 3×3 convolution layer; The two branch output features are spliced and input into a 5*5 convolution layer to generate a spatial attention weight map combined with a Sigmoid function. The weight map is point multiplied with the reduced dimension features to realize feature weighting. Finally, the weighted features are added to the original reduced dimension features through a gated residual structure, The feature enhancement output is completed.

10. The deep learning-based inflammatory bowel disease classification system of claim 8, wherein, The cross-stage attention fusion module includes a 3*3 transpose convolution layer, a 1*1 convolution layer, a 5*5 convolution layer, a global max pooling layer, a global average pooling layer, an average pooling layer, a max pooling layer, a fully connected layer, and a ReLU6 activation function. The cross-stage attention fusion module up-samples the Transformer features to match the size of the CNN features, aligns the features through a 3*3 transpose convolution, and generates a hybrid feature representation by splicing and fusing along the channel dimension. The hybrid feature representation is processed by a triple attention mechanism in turn: The initial channel attention compresses spatial information through global average pooling, generates a channel weight map through two fully connected layers, and preliminarily weights the hybrid features. The enhanced channel attention extracts more robust channel correlations through a shared fully connected layer based on the complementary statistics of global average pooling and global max pooling, and generates a second weighted feature. The spatial attention compresses the channel-weighted features in the spatial dimension, extracts spatial key regions through parallel operations of average pooling and max pooling, generates a spatial attention map after fusion through a 5*5 convolution layer, and calibrates the features in the spatial dimension. Finally, the calibrated features are added to the original hybrid features through a residual connection to retain multi-scale information while strengthening the cross-modal feature representation.

11. The deep learning-based inflammatory bowel disease classification system of claim 8, wherein, The endoscope depth of field feature extractor includes a bilinear interpolation down-sampling layer, a 1*1 convolution layer, a 3*3 convolution layer, a batch normalization layer BN, a ReLU activation function layer, and a Sigmoid function. The endoscope depth of field feature extractor bilinearly interpolates and down-samples the original ring mask to adjust its resolution to match the size of the input feature map, generating a spatially aligned mask representation. The mask is adjusted in the channel dimension through a 1*1 convolution layer, and the mask feature is preprocessed through batch normalization and ReLU activation function. The processed mask is spatially weighted with the current input feature to strengthen the areas in the feature map that conform to the depth of field distribution pattern. The weighted feature is sequentially processed by two 3*3 convolution layers to extract depth-related features, and a spatial attention weight map is generated through a Sigmoid function. Finally, the attention map is point multiplied with the original feature to realize feature weighting, combined with a residual connection to retain the original information that is not weighted, and a 1*1 convolution layer is used to complete cross-channel feature fusion to output an enhanced feature representation with depth perception capability.

Images