Gastrointestinal medical image feature classification system based on feature fusion and attention mechanism

Abstract

This invention discloses a gastrointestinal medical image feature classification system based on feature fusion and attention mechanisms, comprising at least an image acquisition module, a model building module, and a classification module. The image acquisition module acquires gastrointestinal medical image data, performs preprocessing, and constructs a labeled dataset. The model building module constructs a deep convolutional neural network (DCNN) model for classifying gastrointestinal medical image features and trains the DCNN model based on the dataset. The DCNN model includes a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer, and a fully connected layer. The classification module performs gastrointestinal medical image feature classification based on the trained DCNN model. This invention offers advantages such as high classification accuracy, low computational requirements, and robustness in gastrointestinal medical image classification.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision, and specifically to a gastrointestinal medical image feature classification system based on feature fusion and attention mechanisms. Background Technology

[0002] Gastrointestinal (GI) diseases are a global health challenge whose severity cannot be ignored. Malignant tumors such as stomach cancer and colorectal cancer not only pose significant life risks to patients but also place a heavy burden on healthcare systems. Early detection and treatment of these diseases are crucial for improving patient survival rates and treatment outcomes. However, traditional diagnostic methods, such as endoscopy, CT scans, and MRI, while helpful to doctors in detecting diseases to some extent, still have limitations in detecting early abnormalities. These limitations may stem from the resolution limitations of the technology itself, differences in operator experience, or the fact that subtle changes in the early stages of disease are difficult to detect with the naked eye.

[0003] For this reason, artificial intelligence (AI) and machine learning (ML) technologies, especially deep learning (DL), are gradually emerging in the field of gastrointestinal medical image classification. Deep learning algorithms, particularly convolutional neural networks (CNNs), with their powerful feature extraction capabilities, can automatically identify and extract subtle patterns crucial for disease diagnosis from medical images. This ability has enabled DL models to demonstrate outstanding performance in GI image classification tasks, and they hold promise for providing strong support for early disease detection.

[0004] Transfer learning, as an important technique in deep learning, has further improved the performance of deep learning models in GI tasks. By utilizing models pre-trained on large datasets and fine-tuning them to adapt to specific GI tasks, transfer learning not only reduces the time and resources required for model training but also improves the model's generalization ability. This allows for the training of high-performance models even with relatively limited data. Furthermore, the introduction of feature fusion methods has brought new breakthroughs to GI image classification. By combining endoscopic images with other clinical data (such as patient history, laboratory test results, etc.), feature fusion methods can provide more comprehensive and accurate disease information, thereby further improving classification accuracy. This multidisciplinary approach helps doctors gain a more comprehensive understanding of patients' conditions and develop more personalized treatment plans.

[0005] Despite the remarkable achievements of deep learning (DL) and feature fusion techniques in gastrointestinal (GI) image classification, several technical limitations remain. Among these, data heterogeneity is a major challenge, significantly impacting the generalization performance of models. Because different medical institutions may employ significantly different equipment, imaging techniques, and even operational procedures, the acquired GI image data often varies greatly in quality, resolution, and color reproduction. This data variation not only increases the difficulty of model training but can also lead to models performing exceptionally well on certain datasets while performing poorly on others, severely limiting the widespread application of these models in clinical practice.

[0006] Furthermore, high computational demands are a major bottleneck restricting the widespread application of deep learning (DL) models in clinical settings. The training and inference processes of DL models typically consume significant computational resources, which is undoubtedly a heavy burden for resource-constrained medical institutions. This is especially true in applications such as real-time diagnosis or telemedicine, where the requirements for computational efficiency and response speed are even more stringent, and existing DL models often struggle to meet these demands. Summary of the Invention

[0007] To address the issues of data generalization ability and high computational requirements of deep learning and feature fusion technologies in the field of gastrointestinal medical image classification, this invention proposes a gastrointestinal medical image feature classification system based on feature fusion and attention mechanisms. This system has the advantages of high classification accuracy, low computational requirements, and good robustness in gastrointestinal medical image classification.

[0008] The gastrointestinal medical image feature classification system disclosed in this invention, which utilizes feature fusion and attention mechanisms, includes at least:

[0009] An image acquisition module is used to acquire gastrointestinal medical image data, perform preprocessing, and construct a labeled dataset.

[0010] The model building module is used to build a deep convolutional neural network model for classifying gastrointestinal medical image features, and to train the deep convolutional neural network model based on the dataset.

[0011] The deep convolutional neural network model includes, in sequence:

[0012] The feature extraction layer includes a first network obtained by lightweighting the DenseNet network and a second network obtained by lightweighting the MobileNet network; the first network and the second network are connected in parallel and are used to extract features from the image data respectively.

[0013] The feature fusion layer is used to fuse the extraction results of the first network and the second network to form a feature set;

[0014] A multi-level residual module is provided, wherein the input of the multi-level residual module is connected to the output of the feature fusion layer. The multi-level residual module includes several parallel split convolutional layers for multi-scale feature extraction of the feature set. The extraction results of the split convolutional layers are concatenated and then residual connections are established with the feature set to form multi-scale features.

[0015] A convolutional block attention module, the input of which is connected to the output of the multi-level residual module, is used to identify feature regions of the multi-scale features through an attention mechanism;

[0016] A global average pooling layer, the input of which is connected to the output of the convolutional block attention module, is used to perform global average pooling on the recognition results of the convolutional block attention module to form a feature map;

[0017] A fully connected layer, the input of which is connected to the output of the global average pooling layer, is used to map the feature map to a class probability distribution to obtain a classification result.

[0018] The classification module is used to classify gastrointestinal medical image features based on the trained deep convolutional neural network model.

[0019] In a preferred embodiment, the process of lightweighting the DenseNet network to obtain the first network specifically includes the following steps:

[0020] The DenseNet121 network in the DenseNet network was selected as the lightweight object;

[0021] The first network is formed by retaining the first and second dense blocks in the DenseNet121 network, as well as the transition layer between the first and second dense blocks.

[0022] In a preferred embodiment, the process of lightweighting the MobileNet network to obtain the second network specifically includes the following steps:

[0023] The MobileNet-V1 network in the MobileNet network is selected as the lightweight object;

[0024] The first five depthwise separable convolutional layers of the MobileNet-V1 network are retained to form the second network.

[0025] In a preferred embodiment, the deep convolutional neural network model further includes an integrated auxiliary layer located between the feature extraction layer and the feature fusion layer. The integrated auxiliary layer includes a convolutional layer, a batch normalization layer, a max pooling layer, and a dropout layer, which are used to adjust the dimensions of the extraction results of the first network and the second network.

[0026] In a preferred embodiment, the integrated auxiliary layer includes a first integrated auxiliary layer connected to the output of the first network and a second integrated auxiliary layer connected to the output of the second network. The dropout layer of the first integrated auxiliary layer uses a 1x1 convolutional kernel and the dropout rate is set to 0.2. The dropout layer of the second integrated auxiliary layer uses a 2x2 convolutional kernel and the dropout rate is set to 0.2.

[0027] In a preferred embodiment, the feature fusion layer uses an element-wise add method or a concat method to fuse the feature extraction results of the first network and the second network to form a feature set.

[0028] In a preferred embodiment, the split convolutional layer includes a 3x3 split convolutional layer, a 5x5 split convolutional layer, and a 7x7 split convolutional layer, which are respectively connected to the output of the feature fusion layer.

[0029] In a preferred embodiment, the convolutional block attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module is used to capture the inter-channel dependencies of the multi-scale features and input the results into the spatial attention module. The spatial attention module is used to capture the spatial dependencies of the multi-scale features and input the results into the global average pooling layer.

[0030] In a preferred embodiment, the preprocessing of the image data in the dataset specifically includes the following steps:

[0031] Image data cleaning involves deleting damaged or irrelevant image data from the dataset and adjusting the image data to a size that satisfies the calculation requirements of the deep convolutional neural network model.

[0032] Normalization converts the pixel values ​​of image data into a range of 0-1 or -1-1.

[0033] Data augmentation involves rotating, flipping, scaling, and moving image data.

[0034] The dataset is divided into a training set and a validation set, with a ratio of 4:1 between the training set and the validation set.

[0035] In a preferred embodiment, the batch size, learning rate, and training epochs of the deep convolutional neural network model are set to 64.

[0036] Compared with existing technologies, the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism disclosed in this invention has the following advantages:

[0037] (1) The deep convolutional neural network model established by the model building module disclosed in this invention includes a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer, and a fully connected layer. The feature extraction layer includes a first network obtained by lightweighting the DenseNet network. Through the dense connections of the DenseNet network, extensive feature reuse is promoted, and the information flow within the network is enhanced, thereby improving the deep convolutional neural network model's ability to classify gastrointestinal medical images. The feature extraction layer also includes a second network obtained by lightweighting the MobileNet network. Through the depthwise separable convolution of the MobileNet network, the parameters are minimized, enabling the deep convolutional neural network model to improve overall computational efficiency in resource-constrained environments without affecting classification accuracy. The feature fusion layer fuses the extraction results of the first and second networks, expanding the feature representation in the network, enhancing the network's expressiveness, and effectively capturing complex patterns in endoscopic image classification. The multi-level residual module includes several parallel separable convolutional layers for multi-scale feature extraction from the feature set. By concatenating the extraction results of the separable convolutional layers and establishing residual connections with the feature set, multi-scale features are formed. On one hand, multi-scale feature extraction and concatenation of the input feature set allows the network to simultaneously capture detailed information and global contextual information in the image. By concatenating features at different scales, the multi-level residual module can generate richer and more comprehensive feature representations. These feature representations not only contain basic information from the original image but also incorporate spatial structure and texture information at different scales, thereby enhancing the network's ability to recognize complex patterns. On the other hand, establishing residual connections with the input feature set ensures direct information transfer. This design helps alleviate the gradient vanishing problem in deep networks, enabling more stable training and easier convergence to a better solution. The convolutional block attention module uses an attention mechanism to identify feature regions of the multi-scale features, allowing the network to focus on key information in the image, suppressing unnecessary features, and thus generating more accurate and robust feature representations.

[0038] (2) The first network of this invention, formed by retaining the first and second dense blocks of the DenseNet121 network, as well as the transition layer between the first and second dense blocks, ensures, on the one hand, that the first network implements a feature reuse mechanism, effectively extracting low-level spatial features and edge information; on the other hand, through lightweight processing, it significantly reduces the computational load while ensuring feature extraction capabilities, thus reducing the demand for computing resources. The second network, formed by retaining the first 5 depthwise separable convolutional layers of the MobileNet-V1 network, effectively captures low-level and mid-level spatial features while ensuring computational efficiency. The efficient operation of the network is achieved through lightweight processing and efficient computational strategies, enabling the network to operate stably in various resource-constrained environments and providing real-time responses to users. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the technical implementation of the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the deep convolutional neural network model in an embodiment of the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention.

[0041] Figure 3 This is a schematic diagram of the multi-level residual module in an embodiment of the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention.

[0042] Figure 4 This is a schematic diagram of the classification results of an embodiment of the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0046] Furthermore, in this invention, the use of terms such as "first" and "second" is for descriptive purposes only and does not specifically refer to any order or sequence, nor is it intended to limit the invention. They are merely used to distinguish components or operations described using the same technical terms, and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but only if they are feasible for those skilled in the art. If a combination of technical solutions is contradictory or impossible to implement, such a combination should be considered nonexistent and not within the scope of protection claimed by this invention.

[0047] The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism in this embodiment includes at least an image acquisition module, a model construction module, and a classification module.

[0048] The image acquisition module is used to acquire gastrointestinal medical image data, construct a labeled dataset, perform preprocessing, and build a labeled dataset.

[0049] In this embodiment, the gastrointestinal medical image data obtained are endoscopic images of the gastrointestinal tract. Regions of interest are captured from various gastrointestinal image data to classify the regions of interest.

[0050] In this embodiment, the image data in the dataset is preprocessed, specifically including the following steps:

[0051] Image data cleaning removes damaged or irrelevant images from the dataset, ensuring that the data in the training set is high-quality and task-relevant. This helps deep convolutional neural network models learn more effective features and resizes the image data to meet the computational requirements of deep convolutional neural network models.

[0052] Normalization converts image data pixel values ​​to a range of 0-1 or -1-1. On one hand, normalizing pixel values ​​to this range can accelerate the convergence speed of deep convolutional neural network models. On the other hand, normalized data has a smaller numerical range, making gradient descent algorithms more efficient, reducing the impact of extreme values ​​in the data on model training, and improving model stability.

[0053] Data augmentation involves rotating, flipping, scaling, and moving image data. On one hand, it can generate more variations from the original image, thereby increasing the diversity of the training data. This helps deep convolutional neural network models learn more robust features and improves their generalization ability. On the other hand, increasing the diversity of training data reduces the risk of overfitting of deep convolutional neural network models to the training set.

[0054] In this embodiment, the dataset is divided into a training set and a validation set, with a ratio of 4:1. This ensures that the training set has enough data to train the model, while also ensuring that the validation set has enough data to evaluate the performance of the deep convolutional neural network model. This partitioning method helps to make more effective use of limited data resources.

[0055] The model building module is used to preprocess the image data in the dataset, build a deep convolutional neural network model for classifying gastrointestinal medical image features, and train the deep convolutional neural network model based on the dataset.

[0056] like Figure 1 and Figure 2 As shown, the deep convolutional neural network model includes a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer, and a fully connected layer.

[0057] The feature extraction layer comprises a first network obtained by lightweighting the DenseNet network, and a second network obtained by lightweighting the MobileNet network. The first and second networks are configured in parallel for feature extraction from image data. By removing the head and layers, and freezing certain layers to reduce model size and parameter complexity, the DenseNet and MobileNet networks are lightweighted while retaining their core architectural features. The dense connections of the DenseNet network facilitate extensive feature reuse and enhance information flow within the network, thereby improving the deep convolutional neural network model's ability to classify gastrointestinal medical images. The depthwise separable convolutions of the MobileNet network minimize parameters, enabling the deep convolutional neural network model to improve overall computational efficiency in resource-constrained environments without compromising classification accuracy.

[0058] In this embodiment, the DenseNet121 network within the DenseNet network is selected as the lightweight object. The first and second dense blocks, as well as the transition layer between them, are retained to form the first network. The computational parameters are reduced from 7.17M to 218K. This ensures that the first network implements a feature reuse mechanism, effectively extracting low-level spatial features and edge information. Furthermore, the lightweighting process significantly reduces the computational load while maintaining feature extraction capabilities, thus reducing the demand for computational resources.

[0059] In this embodiment, the MobileNet-V1 network within the MobileNet network is selected as the lightweight target. The first five depthwise separable convolutional layers of the MobileNet-V1 network are retained to form a second network. The computational parameters are reduced from the original 3.36M to 137K, effectively capturing low- and mid-level spatial features while ensuring computational efficiency. Through lightweight processing and efficient computational strategies, the network achieves high-efficiency operation, enabling it to run stably in various resource-constrained environments and provide real-time responses to users.

[0060] While individual networks exhibit impressive feature extraction capabilities, relying on them alone can limit their effectiveness in complex medical image classification tasks. Therefore, the embodiments shown in this invention fuse these networks to create diverse feature sets, which is particularly valuable when dealing with limited datasets.

[0061] The deep convolutional neural network model employs lightweight DenseNet121 and MobileNet-V1 networks for initial feature extraction from image data. While the lightweight DenseNet121 and MobileNet-V1 networks reduce network weights and complexity, their incompatible output shapes hinder layer-by-layer fusion. To enhance fusion, the deep convolutional neural network model also includes an integration auxiliary layer located between the feature extraction layer and the feature fusion layer. This integration auxiliary layer comprises convolutional layers, batch normalization layers, max pooling layers, and dropout layers, used to adjust the dimensionality of the extraction results from the first and second networks. The convolutional layers extract high-level features from the outputs of the first and second networks and pass the results to the batch normalization layer. The batch normalization layer normalizes the results, promoting stable training. The results are then passed to the max pooling layer, which reduces spatial dimensionality while preserving key information. Finally, the results are passed to the dropout layer, which introduces regularization to prevent overfitting.

[0062] In this embodiment, the integrated auxiliary layer includes a first integrated auxiliary layer connected to the output of the first network and a second integrated auxiliary layer connected to the output of the second network. The dropout layer of the first integrated auxiliary layer uses a 1x1 convolutional kernel with a dropout rate of 0.2, and the dropout layer of the second integrated auxiliary layer uses a 2x2 convolutional kernel with a dropout rate of 0.2. The first integrated auxiliary layer normalizes the output of the first network to 56×56×192. The second integrated auxiliary layer normalizes the output of the second network to 28×28×128, improving fusion efficiency and ensuring compatibility.

[0063] The feature fusion layer is used to fuse the extraction results of the first and second networks to form a feature set. The feature fusion layer uses either element-wise add or concat methods to fuse the features extracted by the first and second networks. Element-wise add performs addition operations on corresponding elements of two or more feature maps (or tensors) of the same shape. Each element is added to the elements at the same position in other feature maps, generating a new feature map where each element is the result of adding the original elements. Concat concatenates two or more feature maps (or tensors) along a certain dimension (usually the channel dimension). This results in a new feature map containing all the elements of the original feature maps, but their size increases along the concatenation dimension. When concatenating two feature maps along the channel dimension, the new feature map will have the sum of the number of channels of the two original feature maps.

[0064] In this embodiment, the element-wise add method is used to fuse the feature extraction results of the first network and the second network to form a feature set.

[0065] like Figure 3As shown, the input of the multi-level residual module is connected to the output of the feature fusion layer. The multi-level residual module includes several parallel separable convolutional layers for multi-scale feature extraction from the feature set. By concatenating the extraction results of the separable convolutional layers and establishing residual connections with the feature set, multi-scale features are formed. The multi-level residual module refines and integrates the fused feature set, improving feature richness. On one hand, multi-scale feature extraction and concatenation of the input feature set enables the network to simultaneously capture detailed information and global contextual information in the image. By concatenating features at different scales, the multi-level residual module can generate richer and more comprehensive feature representations. These feature representations not only contain basic information from the original image but also incorporate spatial structure and texture information at different scales, thereby enhancing the network's ability to recognize complex patterns. On the other hand, establishing residual connections with the input feature set ensures direct information transfer, helping to alleviate the gradient vanishing problem in deep networks, enabling the network to train more stably and converge to a better solution more easily.

[0066] In this embodiment, as Figure 3 As shown in 'a', the separable convolutional layers include 3x3, 5x5, and 7x7 separable convolutional layers, each connected to the output of the feature fusion layer. These three layers are tightly connected to the output of the feature fusion layer, ensuring that feature information from the previous layer is comprehensively and effectively received and processed. Features are captured at multiple scales through these 3x3, 5x5, and 7x7 separable convolutional layers. This multi-scale processing greatly enriches the deep convolutional network's ability to understand features, enabling it to more comprehensively grasp the details and global information in the image.

[0067] In this embodiment, as Figure 3As shown in Figure b, the multi-level residual module, through its unique residual connection design, effectively fuses the input feature set with features processed by three separable convolutional layers of different sizes (3×3, 5×5, and 7×7). In this process, the 3×3 convolutional kernel focuses on extracting local detail features, the 5×5 convolutional kernel captures medium-range contextual information, and the 7×7 convolutional kernel extracts broader global image features. The role of residual connections is to directly add the output of each convolutional layer to the original input features. This design not only promotes the transmission and fusion of feature information at different scales within the network but also significantly improves model performance. By directly passing input features to the output layer, residual connections effectively alleviate the gradient vanishing problem, making the training of deep networks more stable. Furthermore, residual connections significantly enhance the information transmission and fusion between features at different resolutions, enabling the model to more accurately capture detailed information and global context in the image. This design not only improves the training efficiency of the model but also ensures that the model can fully utilize multi-scale feature information during training, thereby further enhancing model performance. Ultimately, because residual connections can maintain the flow of information, the model can converge to the optimal solution more quickly during training, thus showing better performance in practical applications.

[0068] In this embodiment, the multi-stage residual module also employs the ReLU (Rectified Linear Unit) activation function to introduce nonlinear characteristics. The mathematical expression of the ReLU function is:

[0069] f(x) = max(0,x)

[0070] When the input x is greater than 0, the output of the ReLU function is x itself; when the input x is less than or equal to 0, the output of the ReLU function is 0. This nonlinear transformation not only increases the expressive power of the model, but also enables deep convolutional neural network models to better fit complex nonlinear relationships.

[0071] The input of the convolutional block attention module is connected to the output of the multi-level residual module. It is used to identify feature regions of multi-scale features through the attention mechanism, so that the deep convolutional neural network model can focus on the key information in the image, suppress unnecessary features, and thus generate more accurate and robust feature representations.

[0072] In this embodiment, the convolutional block attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module captures the inter-channel dependencies of multi-scale features and inputs the results into the spatial attention module. The spatial attention module captures the spatial dependencies of multi-scale features and inputs the results into a global average pooling layer. By effectively capturing the inter-channel and spatial dependencies in multi-scale features and dynamically adjusting the focus on the most relevant channels and spatial regions, the convolutional block attention module improves accuracy and reduces interference by filtering noise and prioritizing key information. It also significantly enhances the neural network's processing and representation capabilities for images or data.

[0073] The input of the global average pooling layer is connected to the output of the convolutional block attention module. It is used to perform global average pooling on the recognition results of the convolutional block attention module to form a feature map. This not only reduces the dimensionality of the features, but also helps to reduce the risk of overfitting.

[0074] The fully connected layer connects to the output of the global average pooling layer, and is used to map the feature map to the class probability distribution to obtain the classification result.

[0075] In this embodiment, the batch size in the hyperparameters of the deep convolutional neural network model is set to 64. In each iteration, the deep convolutional neural network model processes 64 samples for gradient calculation and subsequent weight updates. This setting aims to balance training speed with the stability of gradient estimation, fully utilizing the parallel computing power of the GPU without causing excessive memory consumption. The learning rate is set to 0.001 to control the step size of weight updates, which helps avoid drastic fluctuations and potential instability during training, thereby promoting more stable convergence of the network to a better solution. The number of training epochs is set to 20. Through 20 training epochs, the network will have sufficient opportunities to learn from the data and extract useful features, while also having enough time to adjust its parameters to minimize the loss function. The Adam optimizer is chosen, combining the core ideas of momentum and RMSprop algorithms. It dynamically adjusts the learning rate of each parameter by calculating the first and second moment estimates of the gradient, improving its effectiveness in handling complex architectures. In this embodiment, the classification cross-entropy loss is used. The error of the model is quantified by calculating the average of the logarithmic difference between the predicted probability of each class and the true label, thereby evaluating the multi-class classification ability of the deep convolutional neural network model.

[0076] Category modules, such as Figure 4 As shown, it is used to classify gastrointestinal medical image features based on a trained deep convolutional neural network model.

[0077] In the comparative evaluation, the deep convolutional neural network model trained in this embodiment demonstrated superior performance, with its advantages being obvious compared to MFuRe-CNN. Specifically, the deep convolutional neural network model achieved an impressive accuracy of 99.37%, while maintaining a high level of precision and recall of 99.38%, and an F1 score of 99.37%, comprehensively surpassing the corresponding metrics of MFuRe-CNN, which achieved 97.75% in these key performance indicators.

[0078] Furthermore, from the perspective of model complexity, the deep convolutional neural network model also demonstrates higher efficiency. It only requires 0.47 million parameters to achieve the aforementioned superior performance, while MFuRe-CNN requires as many as 4.8 million parameters. This not only means that the deep convolutional neural network model in this embodiment is more economical in terms of storage and computing resources, but is also easier to deploy and optimize in practical applications.

[0079] Furthermore, the deep convolutional neural network model in this embodiment also demonstrates a significant advantage in terms of test time. Its test time is only 8.99 seconds, far lower than MFuRe-CNN's 12.40 seconds. This result shows that the deep convolutional neural network model in this embodiment is faster in processing speed and response time, providing users with a smoother and more efficient user experience.

[0080] In summary, the deep convolutional neural network model trained in this embodiment demonstrates significant advantages in multiple aspects, including accuracy, precision, recall, F1 score, model complexity, and testing time, fully proving its potential and value in practical applications.

[0081] In summary, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A gastrointestinal medical image feature classification system based on feature fusion and attention mechanism, characterized in that, At least comprising: An image acquisition module, which is used to acquire gastrointestinal medical image data, and to preprocess and construct a labeled data set; A model construction module, which is used to construct a deep convolutional neural network model for classifying gastrointestinal medical image features, and to train the deep convolutional neural network model based on the data set; The deep convolutional neural network model sequentially comprises: A feature extraction layer, which comprises a first network obtained by lightweight processing of a DenseNet network, and a second network obtained by lightweight processing of a MobileNet network; the first network and the second network are arranged in parallel and are respectively used for feature extraction of image data; A feature fusion layer, which is used to fuse the extraction results of the first network and the second network to form a feature set; A multi-level residual module, an input end of the multi-level residual module being connected with an output end of the feature fusion layer, the multi-level residual module comprising a plurality of parallel separation convolution layers used for multi-scale feature extraction of the feature set, and the multi-scale features being formed by concatenating the extraction results of the separation convolution layers and establishing a residual connection with the feature set; A convolution block attention module, an input end of the convolution block attention module being connected with an output end of the multi-level residual module, and being used for identifying feature regions of the multi-scale features through an attention mechanism; A global average pooling layer, an input end of the global average pooling layer being connected with an output end of the convolution block attention module, and being used for global average pooling of the identification results of the convolution block attention module to form a feature map; A full connection layer, an input end of the full connection layer being connected with an output end of the global average pooling layer, and being used for mapping the feature map to a category probability distribution to obtain a classification result; An integrated auxiliary layer located between the feature extraction layer and the feature fusion layer, the integrated auxiliary layer comprising a convolution layer, a batch normalization layer, a max pooling layer and a dropout layer, and being used for adjusting the dimensions of the extraction results of the first network and the second network; A classification module, which is used to classify features of an input gastrointestinal medical image based on the trained deep convolutional neural network model.

2. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 1, characterized in that, The first network obtained by lightweight processing of the DenseNet network comprises the following steps: Selecting a DenseNet121 network in the DenseNet network as a lightweight object; Reserving a first dense block and a second dense block in the DenseNet121 network and a transition layer between the first dense block and the second dense block to form the first network. 3.The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 1 or 2, characterized in that, The second network obtained by lightweight processing of the MobileNet network comprises the following steps: Selecting a MobileNet-V1 network in the MobileNet network as a lightweight object; Reserving the first 5 deep separable convolution layers of the MobileNet-V1 network to form the second network.

4. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 3, wherein: The integrated auxiliary layer includes a first integrated auxiliary layer connected with the output end of the first network, and a second integrated auxiliary layer connected with the output end of the second network, a dropout layer of the first integrated auxiliary layer adopts a 1x1 convolution kernel, and a Dropout rate is set to 0.2, and a dropout layer of the second integrated auxiliary layer adopts a 2x2 convolution kernel, and a Dropout rate is set to 0.

2.

5. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2 or 4, characterized in that: The feature fusion layer adopts an element-wise add manner or a concat manner to perform feature fusion on the first network and the second network to form a feature set.

6. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2 or 4, characterized in that: The separation convolution layer includes a 3x3 separation convolution layer, a 5x5 separation convolution layer and a 7x7 separation convolution layer connected with the output end of the feature fusion layer respectively.

7. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 6, characterized in that: The convolution block attention module includes a channel attention module and a spatial attention module connected in sequence, the channel attention module is used to capture the inter-channel dependency of the multi-scale feature, and the result is input into the spatial attention module, the spatial attention module is used to capture the spatial dependency of the multi-scale feature, and the result is input into the global average pooling layer.

8. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2, 4 or 7, characterized in that, The gastrointestinal medical image data preprocessing specifically includes the following steps: Image data cleaning, deleting damaged or irrelevant image data in the data set, and adjusting the image data to meet the size of the deep convolutional neural network model calculation; Normalization processing, converting the pixel value of the image data to 0-1 or -1-1 range; Data enhancement, rotating, flipping, scaling and moving the image data; The data set is divided into a training set and a validation set, and the proportion of the training set and the validation set is 4:

1.

9. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 8, characterized in that: The batch size in the deep convolutional neural network model hyperparameter is set to 64, the learning rate is set to 0.001, and the training round is set to 20.

Images