Intestine and stomach medical image feature classification system based on feature fusion and attention mechanism

Abstract

The gastrointestinal medical image feature classification system based on feature fusion and an attention mechanism at least comprises an image acquisition module, a model construction module and a classification module. The image acquisition module is used for acquiring gastrointestinal medical image data, preprocessing the gastrointestinal medical image data and constructing a data set with labels; the model construction module is used for constructing a deep convolutional neural network model for classifying gastrointestinal medical image features, and training the deep convolutional neural network model based on the data set, the deep convolutional neural network model comprises a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer and a full connection layer; and the classification module is used for realizing gastrointestinal medical image feature classification based on the trained deep convolutional neural network model. The method has the advantages of being high in classification precision, low in calculation requirement and good in gastrointestinal medical image classification robustness.

Description

Technical Field

[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism. Background Art

[0002] Gastrointestinal (GI) diseases are a global health challenge whose severity cannot be ignored. Malignant tumors such as gastric cancer and colorectal cancer not only pose great risks to patients, but also impose a heavy burden on the medical system. Early detection and treatment of these diseases are of vital importance to improving patients' survival rates and treatment outcomes. However, traditional diagnostic methods, such as endoscopy, CT scans, and MRI, although they can help doctors detect the disease to a certain extent, still have certain limitations in early abnormality detection. These limitations may be due to the resolution limitations of the technology itself, differences in operator experience, or subtle changes in the disease in the early stages that are difficult to capture with the naked eye.

[0003] For this reason, artificial intelligence (AI) and machine learning (ML) technologies, especially deep learning (DL), have gradually emerged in the field of gastrointestinal medical image classification. Deep learning algorithms, especially convolutional neural networks (CNNs), can automatically identify and extract subtle patterns from medical images that are critical for disease diagnosis due to their powerful feature extraction capabilities. This capability enables DL models to demonstrate excellent performance in GI image classification tasks, and is expected to provide strong support for early disease detection.

[0004] As an important technology in the field of DL, transfer learning has further improved the performance of DL models in GI tasks. By leveraging models pre-trained on large datasets and fine-tuning them to adapt to specific GI tasks, transfer learning not only reduces the time and resources required for model training, but also improves the generalization ability of the model. This makes it possible to train high-performance models even when the amount of data is relatively limited. In addition, the introduction of feature fusion methods has also brought new breakthroughs in GI image classification. By combining endoscopic images with other clinical data (such as patient medical history, laboratory test results, etc.), feature fusion methods can provide more comprehensive and accurate disease information, thereby further improving the accuracy of classification. This multidisciplinary fusion approach helps doctors understand the patient's condition more comprehensively and develop more personalized treatment plans.

[0005] Although deep learning (DL) and feature fusion techniques have achieved remarkable results in the field of gastrointestinal (GI) image classification, there are still a series of technical defects. Among them, data heterogeneity, as a core challenge, significantly affects the generalization performance of the model. Since the equipment, imaging technology and even operating specifications used by different medical institutions may vary significantly, the collected GI image data often vary greatly in quality, resolution and color performance. This difference in data not only increases the difficulty of model training, but may also cause the model to perform well on specific datasets but mediocre on other datasets, which seriously restricts the widespread application of the model in clinical practice.

[0006] In addition, high computing requirements are also a major bottleneck that restricts the popularization and application of DL models in clinical environments. The training and reasoning process of DL models usually consumes a lot of computing resources, which is undoubtedly a heavy burden for medical institutions with limited resources. Especially in application scenarios such as real-time diagnosis or telemedicine, the requirements for computing efficiency and response speed are more stringent, and existing DL models often cannot meet these requirements. Summary of the invention

[0007] In response to the problems of data generalization ability and high computing demand of deep learning and feature fusion technology in the field of gastrointestinal medical image classification, the present invention proposes a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism, which has the advantages of high classification accuracy, low computing demand and good robustness in gastrointestinal medical image classification.

[0008] The gastrointestinal medical image feature classification system with feature fusion and attention mechanism disclosed in the present invention at least includes:

[0009] An image acquisition module, which is used to acquire gastrointestinal medical image data, perform preprocessing, and construct a labeled data set;

[0010] A model building module, used to build a deep convolutional neural network model for classifying gastrointestinal medical image features, and train the deep convolutional neural network model based on the data set;

[0011] The deep convolutional neural network model includes:

[0012] A feature extraction layer, wherein the feature extraction layer includes a first network obtained by lightweight processing of a DenseNet network and a second network obtained by lightweight processing of a MobileNet network; the first network and the second network are arranged in parallel and are respectively used for extracting features from image data;

[0013] A feature fusion layer, used to fuse the extraction results of the first network and the second network to form a feature set;

[0014] A multi-level residual module, wherein the input end of the multi-level residual module is connected to the output end of the feature fusion layer, and the multi-level residual module comprises a plurality of separation convolution layers connected in parallel for performing multi-scale feature extraction on the feature set, and the multi-scale feature is formed by splicing the extraction results of the separation convolution layers and establishing a residual connection with the feature set;

[0015] A convolutional block attention module, the input end of which is connected to the output end of the multi-level residual module, and is used to identify feature areas of the multi-scale features through an attention mechanism;

[0016] A global average pooling layer, the input end of the global average pooling layer is connected to the output end of the convolutional block attention module, and is used to perform global average pooling on the recognition result of the convolutional block attention module to form a feature map;

[0017] A fully connected layer, wherein the input end of the fully connected layer is connected to the output end of the global average pooling layer, and is used to map the feature map to a category probability distribution to obtain a classification result.

[0018] A classification module is used to implement gastrointestinal medical image feature classification based on the trained deep convolutional neural network model.

[0019] In a preferred embodiment, the step of obtaining the first network by performing lightweight processing on the DenseNet network specifically includes the following steps:

[0020] Select the DenseNet121 network in the DenseNet network as a lightweight object;

[0021] The first dense block and the second dense block in the DenseNet121 network, as well as the transition layer between the first dense block and the second dense block are retained to form the first network.

[0022] In a preferred embodiment, the step of obtaining the second network by performing lightweight processing on the MobileNet network specifically includes the following steps:

[0023] Select the MobileNet-V1 network in the MobileNet network as a lightweight object;

[0024] The first five depth-separable convolutional layers of the MobileNet-V1 network are retained to form the second network.

[0025] In a preferred embodiment, the deep convolutional neural network model also includes an integrated auxiliary layer located between the feature extraction layer and the feature fusion layer, and the integrated auxiliary layer includes a convolution layer, a batch normalization layer, a maximum pooling layer and a discard layer, which are used to adjust the dimensions of the extraction results of the first network and the second network.

[0026] In a preferred embodiment, the integrated auxiliary layer includes a first integrated auxiliary layer connected to the output end of the first network, and a second integrated auxiliary layer connected to the output end of the second network, the discard layer of the first integrated auxiliary layer adopts a 1x1 convolution kernel, and the Dropout rate is set to 0.2, and the discard layer of the second integrated auxiliary layer adopts a 2x2 convolution kernel, and the Dropout rate is set to 0.2.

[0027] In a preferred embodiment, the feature fusion layer uses an element-wise add method or a concat method to fuse the features extracted by the first network and the second network to form a feature set.

[0028] In a preferred embodiment, the separation convolution layer includes a 3x3 separation convolution layer, a 5x5 separation convolution layer and a 7x7 separation convolution layer respectively connected to the output end of the feature fusion layer.

[0029] In a preferred embodiment, the convolution block attention module includes a channel attention module and a spatial attention module connected in sequence, the channel attention module is used to capture the inter-channel dependencies of the multi-scale features and input the results into the spatial attention module, the spatial attention module is used to capture the spatial dependencies of the multi-scale features and input the results into the global average pooling layer.

[0030] In a preferred embodiment, the preprocessing of the image data in the data set specifically comprises the following steps:

[0031] Image data cleaning, deleting damaged or irrelevant image data in the data set, and adjusting the image data to a size that satisfies the calculation of the deep convolutional neural network model;

[0032] Normalization processing converts the pixel values ​​of image data into the range of 0-1 or -1-1;

[0033] Data augmentation: rotating, flipping, scaling, and moving image data;

[0034] The data set is divided into a training set and a validation set, and the ratio of the training set to the validation set is 4:1.

[0035] In a preferred embodiment, the batch size in the deep convolutional neural network model hyperparameters is set to 64, the learning rate is set to 0.001, and the training rounds are set to 20.

[0036] Compared with the prior art, the gastrointestinal medical image feature classification system based on feature fusion and attention mechanism disclosed in the present invention has the following beneficial effects:

[0037] (1) The deep convolutional neural network model established by the model building module disclosed in the present invention includes a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer and a fully connected layer. The feature extraction layer includes a first network obtained by lightweight processing of the DenseNet network. The dense connection of the DenseNet network promotes extensive feature reuse and enhances the information flow within the network, thereby improving the deep convolutional neural network model's ability to classify gastrointestinal medical images. The feature extraction layer also includes a second network obtained by lightweight processing of the MobileNet network. The parameters are minimized by the deep separable convolution of the MobileNet network, so that the deep convolutional neural network model improves the overall computational efficiency in a resource-limited environment without affecting the classification accuracy. The feature fusion layer fuses the extraction results of the first network and the second network, expands the feature representation in the network, enhances the network's expressiveness, and effectively captures the complex patterns in the classification of endoscopic images. The multi-level residual module includes several separate convolutional layers connected in parallel for multi-scale feature extraction of feature sets. Multi-scale features are formed by splicing the extraction results of the separate convolutional layers and establishing residual connections with the feature sets. On the one hand, multi-scale feature extraction and splicing are performed on the input feature set, so that the network can simultaneously capture the detail information and global context information in the image. By splicing features of different scales, the multi-level residual module can generate richer and more comprehensive feature representations. These feature representations not only contain the basic information in the original image, but also incorporate spatial structure and texture information at different scales, thereby enhancing the network's ability to recognize complex patterns; on the other hand, residual connections are established with the input feature set to ensure direct information transmission. This design helps alleviate the gradient vanishing problem in deep networks, allowing the network to be trained more stably and more easily converge to a better solution. The convolutional block attention module identifies the feature area of ​​the multi-scale features through the attention mechanism, so that the network focuses on the key information in the image, suppresses unnecessary features, and generates more accurate and robust feature representations.

[0038] (2) The first network of the present invention is formed by retaining the first dense block and the second dense block in the DenseNet121 network, as well as the transition layer between the first dense block and the second dense block. On the one hand, it ensures that the first network implements a feature reuse mechanism and can effectively extract low-level spatial features and edge information; on the other hand, through lightweight processing, the amount of calculation is significantly reduced while ensuring the feature extraction capability, thereby reducing the demand for computing resources. The second network is formed by retaining the first 5 depth-separable convolutional layers of the MobileNet-V1 network. While ensuring computing efficiency, it can effectively capture low-level and mid-level spatial features. The efficient operation of the network is achieved through lightweight processing and efficient computing strategies, which enables the network to operate stably in various resource-constrained environments and provide real-time responses to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a technical implementation flow chart of gastrointestinal medical image feature classification in an embodiment of a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention;

[0040] Figure 2 It is a structural schematic diagram of a deep convolutional neural network model of an embodiment of a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention;

[0041] Figure 3 It is a schematic diagram of a multi-level residual module of an embodiment of a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention;

[0042] Figure 4 Schematic diagram of classification results of an embodiment of a gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "or / and" used herein includes any and all combinations of one or more related listed items.

[0046] In addition, in the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes, and do not specifically refer to the order or sequence, nor are they used to limit the present invention. They are only used to distinguish components or operations described with the same technical terms, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0047] The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism of this embodiment includes at least an image acquisition module, a model building module and a classification module.

[0048] The image acquisition module is used to acquire gastrointestinal medical image data to construct a labeled data set, perform preprocessing, and construct a labeled data set.

[0049] The gastrointestinal medical image data acquired in this embodiment is an endoscopic image of the gastrointestinal tract, and regions of interest in various gastrointestinal image data are captured to achieve classification of the regions of interest.

[0050] In this embodiment, the image data in the data set is preprocessed, specifically including the following steps:

[0051] Image data cleaning, deleting damaged or irrelevant image data in the dataset, can ensure that the data in the training set is high-quality and task-related, which helps the deep convolutional neural network model learn more effective features and adjust the image data to a size that meets the calculation requirements of the deep convolutional neural network model.

[0052] Normalization converts the pixel values ​​of image data into the range of 0-1 or -1-1. On the one hand, normalizing the pixel values ​​to the range of 0-1 or -1-1 can speed up the convergence of the deep convolutional neural network model. On the other hand, the normalized data has a smaller numerical range, making the gradient descent algorithm more efficient, reducing the impact of extreme values ​​in the data on model training, and improving the stability of the model.

[0053] Data augmentation involves rotating, flipping, scaling, and moving image data. On the one hand, more variants can be generated from the original image, thereby increasing the diversity of training data, which helps the deep convolutional neural network model learn more robust features and improve generalization capabilities; on the other hand, by increasing the diversity of training data, the risk of overfitting the deep convolutional neural network model to the training set is reduced.

[0054] In this embodiment, the data set is divided into a training set and a validation set, and the ratio of the training set to the validation set is 4:1. On the one hand, it ensures that the training set has enough data to train the model, and on the other hand, it ensures that the validation set has enough data to evaluate the performance of the deep convolutional neural network model. This division method helps to make more effective use of limited data resources.

[0055] The model building module is used to preprocess the image data in the dataset, build a deep convolutional neural network model for classifying gastrointestinal medical image features, and train the deep convolutional neural network model based on the dataset.

[0056] like Figure 1 and Figure 2 As shown in Figure 1, the deep convolutional neural network model includes a feature extraction layer, a feature fusion layer, a multi-level residual module, a convolutional block attention module, a global average pooling layer and a fully connected layer.

[0057] The feature extraction layer includes a first network obtained by lightweight processing of the DenseNet network and a second network obtained by lightweight processing of the MobileNet network; the first network and the second network are set in parallel and are used to extract features from image data respectively. The DenseNet network and the MobileNet network are lightweighted by removing the head and layer and freezing some layers to reduce the model size and parameter complexity while retaining the core architecture features. The dense connection of the DenseNet network promotes extensive feature reuse and enhances the information flow within the network, thereby improving the classification ability of the deep convolutional neural network model for gastrointestinal medical images. The parameters are minimized by the deep separable convolution of the MobileNet network, so that the overall computational efficiency of the deep convolutional neural network model is improved in a resource-limited environment without affecting the classification accuracy.

[0058] In this embodiment, the DenseNet121 network in the DenseNet network is selected as the lightweight object, and the first dense block and the second dense block in the DenseNet121 network, as well as the transition layer between the first dense block and the second dense block, are retained to form the first network. The calculation parameters are reduced from the original 7.17M to 218K. On the one hand, it ensures that the first network implements the feature reuse mechanism and can effectively extract low-level spatial features and edge information; on the other hand, through lightweight processing, the amount of calculation is significantly reduced while ensuring the feature extraction capability, reducing the demand for computing resources.

[0059] In this embodiment, the MobileNet-V1 network in the MobileNet network is selected as the lightweight object, and the first 5 depth-separable convolutional layers of the MobileNet-V1 network are retained to form the second network. The calculation parameters are reduced from the original 3.36M to 137K, and low-level and mid-level spatial features can be effectively captured while ensuring the calculation efficiency. The efficient operation of the network is achieved through lightweight processing and efficient computing strategies, which enables the network to operate stably in various resource-constrained environments and provide real-time responses to users.

[0060] Although individual networks demonstrate impressive feature extraction capabilities, relying on them alone may limit their effectiveness in complex medical image classification tasks. Therefore, the embodiments shown in the present invention fuse these networks to create a diverse feature set, which is particularly valuable when dealing with limited data sets.

[0061] The deep convolutional neural network model uses lightweight DenseNet121 network and MobileNet-V1 network to perform preliminary feature extraction on image data. Although the lightweight DenseNet121 network and MobileNet-V1 network reduce the weight and complexity of the network, the incompatible output shapes hinder layer-by-layer fusion. To enhance fusion, the deep convolutional neural network model also includes an integrated auxiliary layer located between the feature extraction layer and the feature fusion layer. The integrated auxiliary layer includes a convolution layer, a batch normalization layer, a maximum pooling layer, and a dropout layer, which is used to adjust the dimensions of the extraction results of the first network and the second network. The convolution layer is used to extract high-level features from the output results of the first network and the second network, and transmit the results to the batch normalization layer, which normalizes the results to promote stable training, and then transmits the results to the maximum pooling layer, which reduces the spatial dimension while retaining key information. Finally, the results are transmitted to the dropout layer, and regularization is introduced to prevent overfitting.

[0062] In this embodiment, the integrated auxiliary layer includes a first integrated auxiliary layer connected to the output end of the first network, and a second integrated auxiliary layer connected to the output end of the second network. The drop layer of the first integrated auxiliary layer uses a 1x1 convolution kernel, and the Dropout rate is set to 0.2, and the drop layer of the second integrated auxiliary layer uses a 2x2 convolution kernel, and the Dropout rate is set to 0.2. The output result of the first network is standardized to 56×56×192 through the first integrated auxiliary layer. The output result of the second network is standardized to 28×28×128 through the second integrated auxiliary layer, which improves the fusion efficiency and ensures compatibility.

[0063] The feature fusion layer is used to fuse the extraction results of the first network and the second network to form a feature set. The feature fusion layer uses the element-wise add method or the concat method to fuse the features of the first network and the second network extraction results to form a feature set. The element-wise add method adds the corresponding elements of two or more feature maps (or tensors) of the same shape. The elements at each position are added to the elements at the same position in other feature maps to generate a new feature map, in which each element is the result of the addition of the original elements. The concat method concatenates two or more feature maps (or tensors) along a certain dimension (usually the channel dimension). In this way, the new feature map will contain the elements of all the original feature maps, but their size in the concatenated dimension will increase. When concatenating two feature maps in the channel dimension, the new feature map will have the sum of the number of channels of the two original feature maps.

[0064] In this embodiment, an element-wise add method is used to fuse the features extracted by the first network and the second network to form a feature set.

[0065] like Figure 3As shown in FIG. 1 , the input end of the multi-level residual module is connected to the output end of the feature fusion layer. The multi-level residual module includes several separate convolutional layers connected in parallel for extracting multi-scale features from the feature set. The multi-scale features are formed by splicing the extraction results of the separate convolutional layers and establishing residual connections with the feature set. The multi-level residual module refines and integrates the fused feature set to improve the richness of the features. On the one hand, multi-scale feature extraction and splicing are performed on the input feature set, so that the network can capture the detail information and global context information in the image at the same time. By splicing features of different scales, the multi-level residual module can generate richer and more comprehensive feature representations. These feature representations not only contain the basic information in the original image, but also incorporate the spatial structure and texture information at different scales, thereby enhancing the network's ability to recognize complex patterns; on the other hand, the residual connection is established with the input feature set to ensure the direct transmission of information, which helps to alleviate the gradient vanishing problem in the deep network, so that the network can be trained more stably and more easily converge to a better solution.

[0066] In this embodiment, Figure 3 As shown in a in the figure, the separation convolution layer includes a 3x3 separation convolution layer, a 5x5 separation convolution layer, and a 7x7 separation convolution layer, which are connected to the output of the feature fusion layer respectively. These three convolution layers are closely connected to the output of the feature fusion layer, ensuring that the feature information from the previous level can be fully and effectively received and processed. Features are captured at multiple scales through 3x3 separation convolution layers, 5x5 separation convolution layers, and 7x7 separation convolution layers. This multi-scale processing method greatly enriches the deep convolution network's ability to understand features, enabling it to more comprehensively grasp the details and global information in the image.

[0067] In this embodiment, Figure 3As shown in Figure b, the multi-level residual module achieves effective fusion of the input feature set and the features processed by three separable convolutional layers of different sizes (3×3, 5×5, and 7×7) through its unique residual connection design. In this process, the 3×3 convolution kernel focuses on extracting local detail features, the 5×5 convolution kernel can capture medium-range context information, and the 7×7 convolution kernel is used to extract a wider range of global features of the image. The role of the residual connection is that it directly adds the output of each convolutional layer to the original input features. This design not only promotes the transmission and fusion of feature information of different scales in the network, but also significantly improves the performance of the model. By directly transmitting the input features to the output layer, the residual connection effectively alleviates the gradient vanishing problem, making the training of deep networks more stable. In addition, the residual connection also significantly enhances the information transmission and fusion between features of different resolutions, allowing the model to more accurately capture the detail information and global context in the image. This design not only improves the training efficiency of the model, but also ensures that the model can fully utilize multi-scale feature information during training, thereby further improving the performance of the model. Ultimately, because residual connections can maintain the fluidity of information, the model can converge to the optimal solution faster during training, thereby showing better performance in practical applications.

[0068] In this embodiment, the multi-level residual module also uses a ReLU (Rectified Linear Unit) activation function to introduce nonlinear characteristics. The mathematical expression of the ReLU function is:

[0069] f(x)=max(0,x)

[0070] When the input x is greater than 0, the output of the ReLU function is x itself; when the input x is less than or equal to 0, the output of the ReLU function is 0. This nonlinear transformation not only increases the expressive power of the model, but also enables the deep convolutional neural network model to better fit complex nonlinear relationships.

[0071] The input of the convolutional block attention module is connected to the output of the multi-level residual module, which is used to identify feature areas of multi-scale features through the attention mechanism, so that the deep convolutional neural network model focuses on the key information in the image, suppresses unnecessary features, and generates more accurate and robust feature representation.

[0072] In this embodiment, the convolutional block attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module is used to capture the inter-channel dependencies of multi-scale features and input the results into the spatial attention module. The spatial attention module is used to capture the inter-spatial dependencies of multi-scale features and input the results into the global average pooling layer. The convolutional block attention module not only effectively captures the inter-channel dependencies and inter-spatial dependencies in multi-scale features, dynamically adjusts the attention to the most relevant channels and spatial regions, but also improves accuracy and reduces interference by filtering noise and prioritizing key information. It also significantly improves the processing and representation capabilities of neural networks for images or data.

[0073] The input end of the global average pooling layer is connected to the output end of the convolutional block attention module, which is used to perform global average pooling on the recognition results of the convolutional block attention module to form a feature map, which not only reduces the dimension of the feature but also helps to reduce the risk of overfitting.

[0074] Fully connected layer, the input of the fully connected layer is connected to the output of the global average pooling layer to map the feature map to the category probability distribution to obtain the classification result.

[0075] In this embodiment, the batch size in the deep convolutional neural network model hyperparameter is set to 64. In each iteration, the deep convolutional neural network model will process 64 samples for calculating gradients and subsequent weight updates. Such a setting is intended to balance the training speed and the stability of the gradient estimation, which can fully utilize the parallel computing power of the GPU without causing excessive memory usage. The learning rate is set to 0.001 to control the step size of the weight update, which helps to avoid drastic fluctuations and potential instability during training, thereby promoting the network to converge to a better solution more stably. The training rounds are set to 20. Through 20 rounds of training, the network will have enough opportunities to learn from the data and extract useful features, and also have enough time to adjust its parameters to minimize the loss function. The Adam optimizer is selected, which combines the core ideas of the momentum and RMSprop algorithms, and dynamically adjusts the learning rate of each parameter by calculating the first-order moment estimate and second-order moment estimate of the gradient, thereby improving the effectiveness of processing complex architectures. In this embodiment, the classification cross entropy loss is used to quantify the model error by calculating the average of the logarithmic difference between the predicted probability and the true label of each category, thereby evaluating the multi-class classification ability of the deep convolutional neural network model.

[0076] Classification modules, such as Figure 4 As shown, it is used to realize gastrointestinal medical image feature classification based on the trained deep convolutional neural network model.

[0077] In the comparative evaluation, the deep convolutional neural network model trained in this embodiment showed excellent performance, and its advantages over MFuRe-CNN were obvious. Specifically, the deep convolutional neural network model achieved an impressive 99.37% accuracy, while the precision and recall rates also maintained a high level of 99.38%, and the F1 score also reached 99.37%, which fully surpassed the corresponding indicators of MFuRe-CNN, which was 97.75% in these key performance indicators.

[0078] Furthermore, from the perspective of model complexity, the deep convolutional neural network model also shows higher efficiency. It only requires 0.47 million parameters to achieve the above excellent performance, while MFuRe-CNN requires up to 4.8 million parameters. This not only means that the deep convolutional neural network model of this embodiment is more economical in storage and computing resources, but also easier to deploy and optimize in practical applications.

[0079] In addition, in terms of test time, the deep convolutional neural network model of this embodiment also showed significant advantages. Its test time was only 8.99 seconds, much lower than the 12.40 seconds of MFuRe-CNN. This result shows that the deep convolutional neural network model of this embodiment is faster in processing speed and response time, and can bring users a smoother and more efficient user experience.

[0080] In summary, the deep convolutional neural network model after training in this embodiment has shown significant advantages in many aspects such as accuracy, precision, recall, F1 score, model complexity and test time, which fully demonstrates its potential and value in practical applications.

[0081] In short, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A gastrointestinal medical image feature classification system based on feature fusion and attention mechanism, characterized by: At least: An image acquisition module, which is used to acquire gastrointestinal medical image data, perform preprocessing, and construct a labeled data set; A model building module, used to build a deep convolutional neural network model for classifying gastrointestinal medical image features, and train the deep convolutional neural network model based on the data set; The deep convolutional neural network model includes: A feature extraction layer, wherein the feature extraction layer includes a first network obtained by lightweight processing of a DenseNet network and a second network obtained by lightweight processing of a MobileNet network; the first network and the second network are arranged in parallel and are respectively used for extracting features from image data; A feature fusion layer, used to fuse the extraction results of the first network and the second network to form a feature set; A multi-level residual module, wherein the input end of the multi-level residual module is connected to the output end of the feature fusion layer, and the multi-level residual module comprises a plurality of separation convolution layers connected in parallel for performing multi-scale feature extraction on the feature set, and the multi-scale feature is formed by splicing the extraction results of the separation convolution layers and establishing a residual connection with the feature set; A convolutional block attention module, the input end of which is connected to the output end of the multi-level residual module, and is used to identify feature areas of the multi-scale features through an attention mechanism; A global average pooling layer, the input end of the global average pooling layer is connected to the output end of the convolutional block attention module, and is used to perform global average pooling on the recognition result of the convolutional block attention module to form a feature map; A fully connected layer, wherein an input end of the fully connected layer is connected to an output end of the global average pooling layer, and is used to map the feature map to a category probability distribution to obtain a classification result; A classification module is used to perform feature classification on the input gastrointestinal medical image based on the trained deep convolutional neural network model.

2. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 1, characterized in that: The first network is obtained by performing lightweight processing on the DenseNet network, which specifically includes the following steps: Select the DenseNet121 network in the DenseNet network as a lightweight object; The first dense block and the second dense block in the DenseNet121 network, as well as the transition layer between the first dense block and the second dense block are retained to form the first network.

3. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 1 or 2, characterized in that: The second network is obtained by performing lightweight processing on the MobileNet network, which specifically includes the following steps: Select the MobileNet-V1 network in the MobileNet network as a lightweight object; The first five depth-separable convolutional layers of the MobileNet-V1 network are retained to form the second network.

4. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 3, characterized in that: The deep convolutional neural network model also includes an integrated auxiliary layer located between the feature extraction layer and the feature fusion layer, and the integrated auxiliary layer includes a convolution layer, a batch normalization layer, a maximum pooling layer and a discard layer, which are used to adjust the dimensions of the extraction results of the first network and the second network.

5. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 4, characterized in that: The integrated auxiliary layer includes a first integrated auxiliary layer connected to the output end of the first network, and a second integrated auxiliary layer connected to the output end of the second network, the discard layer of the first integrated auxiliary layer adopts a 1x1 convolution kernel, and the Dropout rate is set to 0.2, and the discard layer of the second integrated auxiliary layer adopts a 2x2 convolution kernel, and the Dropout rate is set to 0.

2.

6. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2 or 4-5, characterized in that: The feature fusion layer uses an element-wise add method or a concat method to fuse the features extracted by the first network and the second network to form a feature set.

7. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2 or 4-5, characterized in that: The separation convolution layer includes a 3x3 separation convolution layer, a 5x5 separation convolution layer and a 7x7 separation convolution layer respectively connected to the output end of the feature fusion layer.

8. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 7, characterized in that: The convolutional block attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module is used to capture the inter-channel dependencies of the multi-scale features and input the results into the spatial attention module. The spatial attention module is used to capture the inter-spatial dependencies of the multi-scale features and input the results into the global average pooling layer.

9. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to any one of claims 1-2, 4-5 or 8, characterized in that: The gastrointestinal medical image data preprocessing specifically includes the following steps: Image data cleaning, deleting damaged or irrelevant image data in the data set, and adjusting the image data to a size that satisfies the calculation of the deep convolutional neural network model; Normalization processing converts the pixel values ​​of image data into the range of 0-1 or -1-1; Data augmentation: rotating, flipping, scaling, and moving image data; The data set is divided into a training set and a validation set, and the ratio of the training set to the validation set is 4:

1.

10. The gastrointestinal medical image feature classification system based on feature fusion and attention mechanism according to claim 9, characterized in that: The batch size, learning rate, and training rounds in the deep convolutional neural network model hyperparameters are set to 64, 0.001, and 20.

Images