Colorectal lesion typing method based on KAN network and variable attention mechanism

Abstract

The invention relates to a colorectal lesion typing method based on a KAN network and a variable attention mechanism, and belongs to the technical field of medical image computer processing. According to the method, colorectal lesions are classified; a KAN-ResNet backbone network typing model is established and initialized; setting an optimizer and a loss function; model training; storing and outputting the model; and parting the model. According to the method, a ResNet backbone network and a KAN mechanism are combined to capture complex modes and interdependence relations in colorectal images. The KAN-ResNet model aims at improving the classification performance of a fine-grained data set, so that the accuracy of distinguishing subtle changes of colorectal lesions is improved. Layered features are effectively captured by using a deep residual learning framework of a residual network ResNet, a variable attention mechanism and complexity of adapting to lesions are combined, so that the model can focus on related areas, accurate features can still be captured although deformation exists, and adaptability and accuracy in the aspect of colorectal lesion classification can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image computer processing, and in particular to a colorectal lesion classification method based on a KAN network and a variable attention mechanism. Background Art

[0002] Colorectal cancer (CRC) is the second leading cause of cancer-related death worldwide. In 2023, it is expected that there will be more than 1.53 million new cases of colorectal cancer and more than 52,550 deaths from colorectal cancer worldwide. At the same time, the incidence rate in young patients (less than 50 years old) is increasing year by year, and the proportion of late diagnosis is higher. Early screening and treatment of common colorectal diseases such as ulcerative colitis, polyps, intestinal bleeding and stained polyps are crucial to reduce the incidence and mortality of colorectal cancer. In clinical practice, endoscopy is used to directly observe the gastrointestinal mucosa, which can provide clearer and more detailed images, making it an important tool for diagnosing colorectal lesions (CRL). Doctors use these images not only to detect colorectal polyps and ulcerative colorectal lesions early, but also to directly remove lesions during the examination, thereby preventing the development of colorectal cancer. However, experienced doctors need to classify a large number of colorectal images (>10,000), including identifying the type of lesion, determining its presence or absence, and assessing its severity. This results in excessive time and labor costs and produces subjective evaluation results.

[0003] Colorectal lesion image classification is a key area in medical image research, aiming to help early diagnosis and treatment through accurate detection and classification, thereby reducing the incidence of colorectal cancer. Traditional methods usually rely on manually extracted features and classical machine learning algorithms. Some people use a method combining concurrent matrices with two-dimensional discrete wavelet transform for image feature representation and use neural networks to classify endoscopic images. Some people propose an advanced fuzzy inference neural network to classify endoscopic images. Some people extract geometric features and texture features to detect polyps. Given that polyps are usually characterized by protrusions, some people analyze the geometric information and texture content of the frame to determine the presence of polyps. Some people propose a classification method that improves LBP by assigning an adaptive neighborhood radius to each pixel, and a spatial neighboring histogram strategy to encode microstructures. Similarly, a new method for polyp recognition in wireless capsule images that combines the advantages of LBP and wavelet transform has been proposed. Some people use texture and local shape features to classify polyps.

[0004] Computer-aided diagnosis (CAD) methods can help doctors automatically and accurately classify endoscopic images, thereby improving diagnostic efficiency and accuracy and further reducing the morbidity and mortality of colorectal cancer. However, CAD systems still face challenges in automatically classifying colorectal lesion images. Due to the characteristics of colorectal diseases, the lesion area in the endoscopic image may only occupy a small area or be distributed along a specific direction. Therefore, the lesion area shows diverse changes in illumination, shape, color, and texture, resulting in a wide distribution of features within the same category. This makes it difficult for the model to capture all feature changes with a limited amount of data, which may lead to overfitting. Early CAD models mainly used traditional machine learning methods to classify colorectal lesions. Most of these studies relied on feature extraction and selection to train classification models, focusing on statistical, shape, and texture features. Support vector machines (SVMs) were used to identify polyps based on texture features, while random forest methods based on shape features were used to classify adenoma-related and non-adenoma-related lesions. However, these classification models are highly dependent on manually extracted features, and their performance and generalization capabilities are limited in complex environments.

[0005] In order to improve the versatility of classification models in complex environments, deep learning models have made significant progress in the field of medical images in recent years. A method called Triplet Attention and Dual-pool Contrastive Learning (TA-DCL) has been proposed for clinically driven multi-label medical image classification. This method mines effective information from medical images by combining class attention, self-attention, and cross-attention to learn high-quality label embeddings for all disease labels. In addition, by designing flexible label representations and optimizing the clustering centers of label embeddings, TA-DCL improves the accuracy of classification while also improving the ability to detect unseen diseases. A sample-specific fine-grained prototype learning (SFPL) method has been proposed to handle imbalanced data in medical image classification. This method is highly adaptable to individual features by building multiple prototypes for the majority class and learning a specific cosine classifier for each sample. Experiments show that SFPL has advantages in multiple different tasks. In the classification of colorectal lesions, these models are able to identify and distinguish different types of colorectal lesions, such as polyps, tumors, and inflammation. A transfer learning method using CNNs was proposed to distinguish hyperplastic polyps from adenomatous polyps. A novel self-peripheral-attention (SPA) mechanism was proposed, which mimics human peripheral visual processing and significantly improves the accuracy and efficiency of endoscopic image classification and segmentation tasks by combining fine-grained features in the central region and coarse-grained features in the peripheral region. A CNN architecture based on end-to-end learning was proposed for classifying colorectal histological images into cancerous and normal colorectal images. A comprehensive analysis of classification methods in gastrointestinal endoscopic imaging was conducted. By participating in MediaEval 2017, the Medico task in 2018, and the BioMediaACM MM Grand Challenge in 2019, a variety of computer vision methods for multi-organ monitoring were demonstrated, aiming to establish a benchmark for different computer vision methods for multi-class endoscopic image applications and promote the construction of new methods that can be reliably used in clinical practice. A patch-based framework was proposed and a ResNet architecture was deployed to classify colorectal polyps using whole slide images. A model based on deep convolutional neural networks was proposed for real-time assessment of polyp subtypes in colorectal endoscopy videos and further classification of wireless capsule endoscopy images into three categories: gastric, small intestine, and large intestine. A deep learning-based end-to-end fully automated system for the classification of ulcerative colitis in colorectal endoscopy videos was proposed.

[0006] So far, deep learning models have shown significant advantages in the automatic classification of colorectal lesions (CRLs). However, existing deep learning methods related to medical imaging still face several challenges, including the following aspects:

[0007] (1) In the CRL image dataset, the number of samples of certain types of CRL is insufficient, while the number of samples of common lesions is relatively large, resulting in a long-tail distribution of the data. This imbalance in class distribution increases the difficulty of accurately classifying rare CRLs.

[0008] (2) CRL images are usually collected in complex environments using dedicated endoscopic imaging equipment. These images usually have low resolution and insufficient contrast, which reduces the identifiable detail features and poses greater challenges to classification models.

[0009] (3) Due to the high similarity between CRLs of different categories and the significant intra-class variability, many models rely on relatively simple feature extraction methods and may not be able to fully capture the complex structural feature changes and subtle differences in CRL images.

[0010] Although KAN has shown potential in different fields in multiple fields, its application in colorectal lesion classification is relatively limited. Due to the limitation of its receptive field, deep learning models have difficulty capturing long-range dependencies in images. This limitation hinders the accurate extraction of colorectal image features and makes it difficult to effectively classify images with subtle differences. Summary of the invention

[0011] In view of the above-mentioned problems and shortcomings of the prior art, the problem to be solved by the present invention is to propose a colorectal lesion classification method based on the KAN network and the variable attention mechanism. The method adopts the deep learning framework Kolmogorov-Arnold residual variable attention network (KAR-DANet) to represent the complex spatial relationships in the data to achieve the classification of colorectal lesions, and improves the adaptability and accuracy of the method in classifying colorectal lesions.

[0012] In order to solve the above problems, the present invention adopts the following technical solutions:

[0013] The present invention is a colorectal lesion classification method based on a KAN network and a variable attention mechanism, which is characterized in that the steps are as follows:

[0014] (1) Classify the original data of colorectal lesions;

[0015] (2) Establish and initialize the KAN-ResNet backbone network classification model;

[0016] Initialize the model parameters of the KAN-ResNet backbone network according to the requirements of each classification task, including the basic convolutional layer, pooling layer, and built-in KAN-based network module;

[0017] Initialize the parameters of the Deformable Transformer network to ensure that it can effectively process the feature information extracted by KAN-ResNet;

[0018] Finally, the classification module is initialized to generate prediction results for multi-classification tasks;

[0019] The KAN-ResNet backbone network consists of ResNet and KAN-based network modules; the KAN module can weight features through the kernelized attention mechanism;

[0020] The Deformable Transformer network adapts to changes in different feature scales by introducing deformable convolution and deformable attention module to achieve accurate positioning and classification of complex image scenes. The Deformable Transformer network flexibly captures image features of different scales and deformations by adaptively adjusting sampling positions and attention weights, thereby improving the classification performance of the model.

[0021] (3) Set the optimizer and loss function;

[0022] 3.1. Adopt AdamW optimization strategy and combine it with exponential decay learning rate scheduler LearningRateScheduler to optimize model parameters; the learning rate adjustment formula is as follows:

[0023] η t =η0·exp(-λ·t)

[0024] Among them, η t is the learning rate of the tth round, η0 is the initial learning rate, and λ is the decay rate hyperparameter;

[0025] 3.2. According to the characteristics of multi-classification tasks, the cross entropy loss and the inter-class Gaussian loss are introduced for comparison; the cross entropy loss function is defined as follows:

[0026]

[0027] Among them, y c and are the true label and predicted probability, respectively, and C is the total number of categories; the inter-category Gaussian loss aims to improve the separability of the feature distribution of each category, thereby effectively dealing with the category imbalance problem in the data set and ensuring accurate detection of minority classes in medical data;

[0028] (4) Model training;

[0029] The images in the training data set are classified according to the lesion category and annotation, and the features of each type of image are extracted based on the KAN-ResNet backbone network; the Deformable Transformer network is used to process the features extracted by KAN-ResNet, and the feature distribution is further optimized through the variable attention mechanism, and the key areas are enhanced; mixed precision training is introduced in the model training process to improve the model calculation efficiency and reduce the memory usage; the variable attention module adaptively adjusts the attention weight according to the distribution of local and global features in the image, and uses the Deformable Convolution operation to realize the positioning and perception of the key areas in the feature map, thereby enhancing the discrimination ability of complex images;

[0030] (5) Model saving and output

[0031] If the validation set performance of the current training round is better than the best result of the previous round, the current model parameters are saved and the relevant performance indicators are recorded to ensure that the final optimal model parameters can be retained; after the training is completed, the final parameters of the model and the best accuracy on the validation set are output;

[0032] (6) Model classification

[0033] For the colorectal lesion images that need to be identified, the model predicts the classification of each patient's colon lesion images; the calculated classification results are output and displayed on the terminal interface of the system.

[0034] The colorectal lesion classification method based on the KAN network and the variable attention mechanism described in the present invention has a further preferred technical solution: in step (1), the colon lesions are divided into three types: non-tumor polyps, inflammatory lesions, and tumor lesions; wherein, non-tumor polyps include hyperplastic polyps and inflammatory polyps; inflammatory lesions include ulcerative colitis, which is specifically graded as 0-1, grade 1, grade 1-2, grade 2, grade 2-3, and grade 3; and tumor lesions include adenomatous polyps.

[0035] The colorectal lesion classification method based on the KAN network and the variable attention mechanism described in the present invention has a further preferred technical solution: it is divided into binary classification and multi-classification according to the classification task requirements;

[0036] Binary classification task 1: Classify lesions into non-neoplastic polyps and inflammatory lesions;

[0037] Binary classification task 2: tumor lesions and inflammatory lesions;

[0038] Multi-classification task: subdivided into polyps, ulcerative colitis grade 0-1, ulcerative colitis grade 1, ulcerative colitis grade 1-2, ulcerative colitis grade 2, ulcerative colitis grade 2-3, ulcerative colitis grade 3.

[0039] In binary classification task 1: if the output result predicted by the model is greater than or equal to the threshold value 0.5, the output label is determined to be 1, and the lesion is classified as an inflammatory lesion; if the output result predicted by the model is less than the threshold value 0.5, the output label is determined to be 0, and the lesion is classified as a non-neoplastic polyp;

[0040] In binary classification task 2: if the output result predicted by the model is greater than or equal to the threshold value 0.5, the output label is determined to be 1, and the lesion is classified as an inflammatory lesion; if the output result predicted by the model is less than the threshold value 0.5, the output label is determined to be 0, and the lesion is classified as a tumor lesion;

[0041] In the multi-classification task: if the output label result predicted by the model is 0, the lesion is judged to be a polyp; if the output label result predicted by the model is 1, the lesion is judged to be ulcerative colitis grade 0-1; if the output label result predicted by the model is 2, the lesion is judged to be ulcerative colitis grade 1; if the output label result predicted by the model is 3, the lesion is judged to be ulcerative colitis grade 1-2; if the output label result predicted by the model is 4, the lesion is judged to be ulcerative colitis grade 2; if the output label result predicted by the model is 5, the lesion is judged to be ulcerative colitis grade 2-3; if the output label result predicted by the model is 6, the lesion is judged to be ulcerative colitis grade 3.

[0042] The colorectal lesion classification method based on the KAN network and the variable attention mechanism described in the present invention has a further preferred technical solution: the model training includes the following contents:

[0043] 4.1 Data Preprocessing

[0044] The training data set and the validation data set were divided into a set ratio of 8:2. The input image data of the training set was preprocessed by random horizontal flipping, random vertical flipping and standardized data enhancement methods to improve the generalization ability of the model and prevent overfitting. For the colorectal lesion image data, image smoothing, denoising and morphological transformation data enhancement operations were further introduced.

[0045] 4.2 Forward Propagation

[0046] 4.2.1. Use the KAN-ResNet backbone network to extract features from the input image and obtain the primary feature map. The KAN module improves the recognition of the feature map through the kernelized attention mechanism.

[0047] 4.2.2. Input the primary feature map into the Deformable Transformer network for feature processing: i. Map and normalize the feature map through the MLP module including the multi-layer perceptron;

[0048] ii. Feature adjustment is performed through the Deformable Attention Module, which can adaptively adjust the attention weight based on the geometric shape of the feature map, focusing on the key feature areas in the image and performing weighted processing on them. In this step, the deformable convolution operation is used to perform feature sampling on multi-scale features to improve the model's ability to adapt to irregular changes in image space.

[0049] 4.2.3. Input the processed high-level features into the classification module, and finally output the prediction results of the multi-classification task;

[0050] 4.3. Calculating Loss and Back Propagation

[0051] i. Calculate the loss value based on the model's predicted output and the true label, and introduce auxiliary loss to improve the model's discriminative ability;

[0052] ii. Use the back-propagation algorithm to calculate the gradient and update the gradient of each layer parameters according to the loss;

[0053] 4.4 Model parameter update

[0054] i. Update the model parameters according to the optimizer strategy, and introduce the learning rate scheduler to dynamically adjust the learning rate to ensure that the model can stably converge to the global optimal solution;

[0055] ii. Use mixed precision training to reduce computing resource consumption;

[0056] 4.5 Model Validation

[0057] After each training round, the model is verified using the validation dataset; the accuracy, recall, and F1 score are calculated to comprehensively evaluate the generalization ability of the model in the current round.

[0058] Compared with the prior art, the method of the present invention has the following beneficial effects:

[0059] The method of the present invention proposes a new colorectal lesion CRL typing network framework, Kolmogorov-Arnold residual variable attention network KAR-DANet, which improves the adaptability and accuracy in classifying colorectal lesions by combining the Kolmogorov-Arnold network KAN with a variable attention mechanism.

[0060] The method of the present invention adopts the KAN-ResNet model, combining the ResNet backbone network with the KAN mechanism to capture complex patterns and interdependencies in colorectal images. Taking advantage of this feature, the KAN-ResNet model aims to improve the classification performance of fine-grained datasets, thereby improving the accuracy of distinguishing subtle changes in colorectal lesions. The method proposed by the inventors effectively captures hierarchical features using the deep residual learning framework of the residual network ResNet. A variable attention mechanism is combined to adapt to the complexity of the lesions, allowing the model to focus on relevant areas and capture precise features despite deformations.

[0061] At the same time, the integration of KAN enhances the nonlinear feature representation between lesions and improves the model's ability to handle complex lesion features. Extensive experimental results show that KAR-DANet significantly outperforms state-of-the-art methods, achieving higher classification accuracy and better generalization ability. This work highlights the potential of advanced deep learning techniques in transforming CRC diagnosis, ultimately aiming to reduce its morbidity and mortality. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is the overall framework diagram of the KAR-DANet model;

[0063] Figure 2 A normalized confusion matrix diagram of the inventor's method and other methods on the Kvasir dataset;

[0064] Figure 3 A normalized confusion matrix diagram of the inventor's method and other methods on the HyperKvasir dataset;

[0065] Figure 4 T-SNE visualization of KAR-DANet on Kvasir and HyperKvasir datasets;

[0066] Figure 5 T-SNE visualization of ResNet in Kvasir and HyperKvasir datasets;

[0067] Figure 6 The confusion matrix and ROC result graph of the ResNet model on the Kvasir dataset and HyperKvasir dataset;

[0068] Figure 7 This is a visualization diagram of the Grad-CAM of the method of the present invention and ResNet. DETAILED DESCRIPTION

[0069] The specific technical solutions of the present invention are further described below to help those skilled in the art further understand the present invention without limiting the rights thereof.

[0070] Example 1, colorectal lesion classification method based on KAN network and variable attention mechanism:

[0071] The system implementing the method is composed of a KAN-based network module, a variable attention module and a classification module. 1.1. KAN-based Encoder Module (KARNet)

[0072] like Figure 1 As shown, the KAN-ResNet network is a complex fusion of ResNets and KAN, which aims to enhance feature extraction and representation learning in deep neural networks. The inventors analyzed the theoretical basis of the KAR-DANet network, the integration of residual learning and attention mechanisms, and verified its impact on high-level feature representation tasks.

[0073] 1.1.1. Residual Learning Network

[0074] A deep neural network is a stack of convolutional layers. The output of the residual block can be expressed as:

[0075]

[0076] Among them, x represents the input of the residual module. (x,{W i})+x represents the residual function, and the parameter is the weight {W i}. y is the output of the residual module.

[0077] The residual function {W i It usually consists of a series of convolutional layers, batch normalization, and activation functions. Adding input to the residual function is conducive to learning the residual mapping, enabling the network to alleviate the gradient vanishing problem and promote more efficient training of deep models.

[0078] 1.1.2. Kolmogorov's representation theorem

[0079] Kolmogorov's theorem, and in particular his work on function representation, states that any continuous function can be approximated by a finite combination of continuous functions. This theoretical result is the basis of the KARNet architecture, which uses deep residual mappings to approximate complex functions through a series of combinations.

[0080] Formally, given a continuous function Kolmogorov's theorem states:

[0081] f(x≈φ1(φ2(…φ k (x))…) (2)

[0082] Among them, φ i is a continuous function. This idea is realized through the iterative combination of residual blocks in KARNet, which enables the network to model complex mapping relationships. Introducing Arnold's dynamical system theory, KARNet ensures stability and effective learning through carefully designed residual connections. Residual connections alleviate the problems of gradient vanishing and gradient exploding by allowing gradients to flow directly through jump connections. This design ensures that the network remains stable in multi-layer structures and can learn effectively.

[0083] Theoretically, the stability of KARNet can be analyzed by considering the Jacobian matrix of the residual mapping.

[0084] The Jacobian matrix J of the residual block can be expressed as:

[0085]

[0086] Where I is the same matrix. Adding J can ensure that the Jacobian matrix is ​​well conditioned, thereby improving the stability of the network training dynamics.

[0087] KARNet is consistent with Kolmogorov's theorem by using residual blocks, which enables the network to approximate complex functions through a series of composite operations. This architecture facilitates the learning of complex feature representations by decomposing complex mappings into simpler, composable functions.

[0088] Leveraging insights from dynamical systems theory, KARNet improves the stability of deep networks. Residual connections ensure that gradients can flow efficiently through the network, alleviating the vanishing and exploding gradient problems common in deep architectures and promoting more stable learning dynamics.

[0089] The Kolmogorov-Arnold ResNet network (KARNet) has made significant progress in neural network architecture by combining Kolmogorov's theory of function representation with Arnold's theory of dynamical systems. This theoretical foundation supports the network's ability to approximate complex functions during training and ensures stability during training.

[0090] 1.2. Variable Attention Module

[0091] The attention mechanism revolutionized neural network architectures by enabling models to dynamically focus on relevant parts of input data. The basic premise of attention is to weight different input features according to their importance for a specific task, thereby improving the model's ability to capture key information.

[0092] like Figure 1 As shown in Figure 2, the variable attention module extends the traditional attention mechanism by introducing spatial variability, allowing the model to adaptively adjust the attention area. This extension addresses the limitations of the traditional attention mechanism, especially its fixed and uniform attention window.

[0093] 1.2.1. Variable Attention Mechanism

[0094] In the variable attention module (DAM), the attention mechanism is not restricted to a fixed receptive field, but can adaptively deform its attention field to capture different spatial contexts. This variability is achieved by introducing a learnable offset that adjusts the attention area for each input feature.

[0095] Mathematically, the variable attention mechanism can be expressed as

[0096] y i =∑ j α ij ·x j +Δ ij (4)

[0097] Among them, x j represents the feature at position j, Δ ij represents the variable offset applicable to position j for the i-th attention query and yi is the output feature of the i-th query.

[0098] 1.2.2. Offset Generation

[0099] Offset Δ ij are learned parameters that dynamically adapt to changes in the attention field. They are calculated from the input features and used to sample from non-uniform locations in the feature map, allowing the attention mechanism to flexibly adjust to the spatial distribution of relevant features.

[0100] The learning of the offset is usually implemented via a separate convolutional network, often called an offset network, which predicts the offset based on the input feature map x:

[0101] Δ ij =Conv offect (x) (5)

[0102] Among them, Conv offectRepresents the convolution operation used to generate the offset.

[0103] The Deformable Attention Module (DAM) introduces significant flexibility by allowing the attention field to adapt to the spatial distribution of relevant features. This adaptability enables the model to better capture complex patterns and dependencies in the input data, information that may be overlooked by traditional fixed attention mechanisms. By deforming the attention field, DAM improves the model's ability to represent complex spatial relationships in the data. This enhancement is particularly beneficial in tasks involving different spatial environments, such as object detection and segmentation. Although variable attention increases the complexity of the model, DAM improves computational efficiency by focusing attention on relevant parts of the input, reducing the need to exhaustively consider all possible spatial regions.

[0104] 1.3 Classification Module

[0105] The variable attention module is combined with a classification layer specially designed for multi-class classification. The classification layer includes a fully connected layer and a softmax layer to generate predicted probabilities reflecting the likelihood of each patient suffering from different types of diseases.

[0106] 1.4 Loss Function

[0107] 1.4.1 Softmax and Cross Entropy Loss

[0108] The Softmax function is often used in multi-category classification tasks to convert the original output (logits) of the model into a probability distribution over the categories. It is defined as follows:

[0109]

[0110] Among them, z i is the logit value of class i, and K is the total number of classes. The cross entropy loss function is used to measure the difference between the predicted probability distribution and the true distribution. For class label y and predicted probability p of class y y , the cross entropy loss is defined as follows:

[0111]

[0112] This approach works well in many scenarios, but it assumes a deterministic assignment of classes and does not account for the inherent uncertainty between classes.

[0113] 1.4.2 Gaussian loss function

[0114] In practical applications, there is usually uncertainty in the boundaries between categories. Standard loss functions may not be able to adequately handle this uncertainty. In order to better deal with this situation, it is necessary to introduce a loss function that can reflect the uncertainty between categories. The Inter-Class Gaussian Loss Function compensates for the limitations of traditional loss functions by explicitly modeling the uncertainty between category boundaries. This method uses Gaussian distribution to represent the inherent uncertainty and variability in category assignments, thereby providing a more nuanced approach to classification problems.

[0115] The inter-class Gaussian loss function aims to penalize the difference between the predicted probability and the Gaussian distribution target value. The key to this method is to integrate the intra-class uncertainty and the inter-class uncertainty into the loss calculation.

[0116] Suppose for some data point x n , let p i,n Represents the predicted probability of category i. Each category i corresponds to a Gaussian distribution with a mean of μ i , the variance is The goal is to transform the predicted probability p i,n Align with these Gaussian distribution targets.

[0117] The Gaussian distribution of class i is defined as follows:

[0118]

[0119] in, Indicates that the mean is μ i , the variance is Gaussian distribution.

[0120] To quantify the difference between the predicted probability and the Gaussian distribution target, the inventors use the negative log-likelihood of the predicted probability under a given Gaussian distribution. For a dataset with N data points and K categories, the inter-category Gaussian loss function is defined as follows:

[0121]

[0122] in, Penalizes the squared deviation of the predicted probability from the class mean, and this value is normalized by the variance of the Gaussian distribution The loss is normalized according to the variance of the Gaussian distribution.

[0123] The inter-class Gaussian loss function is particularly effective in scenarios where class boundaries are uncertain or overlapping. The model is able to account for uncertainty in the confidence level of the classification predictions. Larger variance Indicates that there is greater uncertainty in class membership, resulting in a wider Gaussian distribution. This allows the model to be more flexible in predicting class probabilities. Smaller variance Represents sharper class boundaries, leading to a more concentrated Gaussian distribution, which encourages the model to make more confident predictions.

[0124] 1.5.Optimize strategy settings

[0125] The AdamW optimization strategy is used, and the exponential decay learning rate scheduler is combined to optimize the model parameters; the learning rate adjustment formula is as follows:

[0126] η t =η0·exp(-λ·t) (10)

[0127] Among them, η t is the learning rate of the tth round, η0 is the initial learning rate, and λ is the decay rate hyperparameter;

[0128] The steps of the KAR-DANet network training algorithm are as follows: first, initialize the model parameters, optimizer, and loss function, and randomly shuffle the data set in each training round. The model mainly consists of a KAN-based network module, a deformable attention module, and a classification module. During the training process, the feature map is extracted through the KAN-based network module, and then the feature map is input into the deformable attention module for feature transformation and fusion. Finally, the classification module outputs the multi-category prediction results. Then, the prediction value and loss are calculated through forward propagation, and the gradient is calculated and the parameters of all modules are updated through back propagation. At the end of each training round, the validation set is used to evaluate the model performance, and the current model parameters are saved when the validation accuracy is improved. Finally, the trained model parameters and validation accuracy are output.

[0129] Example 2: Experiment and results of colorectal lesion classification method based on KAN network and variable attention mechanism

[0130] 2.1. Dataset Introduction

[0131] The inventors' model was comprehensively evaluated on two important medical imaging datasets, Kvasir and Hyper-Kvasir, to verify its adaptability and performance in different scenarios. The Kvasir dataset consists of representative frames extracted from endoscopic videos, with image resolutions ranging from 720×576 pixels to 1920×1072 pixels. These images are carefully annotated by experienced endoscopists to ensure high-quality annotations. This dataset is mainly used to evaluate the effectiveness of various computer-assisted image classification and recognition methods. In this experiment, the inventors paid special attention to two categories related to colorectal lesions: polyps and ulcerative colitis, each containing 500 images.

[0132] Automatic classification of gastrointestinal endoscopic images faces significant challenges due to the complex pathological features and interference from external factors such as bubbles and food residues. In addition, during the endoscopic operation, angle changes and the natural peristalsis of the gastrointestinal tract can cause image deformation, such as Figure 1 As shown in Figure 2, this further increases the complexity of the classification task. Therefore, the proposed algorithm needs to be able to effectively identify lesion features in these noises and irregular changes, highlighting its strong adaptability and robustness.

[0133] Table 1 Detailed distribution of the Kvasir dataset

[0134] Type Train Vlalidation Total polyps 400 100 500 ulcerative-colitis-grade 400 100 500 total 800 200 1000

[0135] Table 2 Detailed distribution of HyperKvasir dataset

[0136] Type Train Vlalidation Total polyps 823 205 1028 ulcerative-colitis-grade-0-1 28 7 35 ulcerative-colitis-grade-1 161 40 201 ulcerative-colitis-grade-1-2 9 2 11 ulcerative-colitis-grade-2 355 88 443 ulcerative-colitis-grade-2-3 23 5 28 ulcerative-colitis-grade-3 107 26 133 total 1056 373 1429

[0137] In order to further verify the generalization ability of the inventor's model and its robustness to different lesions, the inventor conducted experiments on the more complex Hyper-Kvasir dataset. The Hyper-Kvasir dataset was released by the Kvasir team and is one of the largest gastrointestinal endoscopy visual datasets. This dataset covers a wider range of image categories, and the category distribution is seriously unbalanced, which poses a greater challenge to the classification algorithm. In this dataset, ulcerative colitis is further subdivided into multiple categories according to the severity of the lesion, which increases the complexity of classification and places higher requirements on the algorithm's ability to distinguish subtle features. In the experiment, the inventor randomly selected 800 images (400 per class) from the Kvasir dataset as a training set and 200 images (100 per class) as a validation set, as shown in Table 1. For the Hyper-Kvasir dataset, the inventor randomly selected 1,056 images (80% per class) as a training set and 373 images (20% per class) as a validation set, as shown in Table 2 for details.

[0138] Experimental setup

[0139] In order to improve the generalization ability and classification performance of the model, the inventors standardized the images in the Kvasir and HyperKvasir datasets. First, the size of all images was uniformly adjusted to 512×512 to ensure the consistency of the input. Then, data augmentation techniques, such as random horizontal and vertical flipping, were applied to increase data diversity and prevent model overfitting. Subsequently, the images were converted to tensor format and normalized so that the mean of each pixel value was 0.5 and the standard deviation was 0.5. These preprocessing steps effectively enhanced the stability of the model and improved its robustness in different environments.

[0140] In the experiment, the hyperparameters were carefully configured. The ResNet part adopted a 34-layer architecture, using BasicBlock as the basic module, and the number of layers was set to 3, 4, 6, and 3 respectively. In order to prevent overfitting, Dropout layers were added to the fully connected layer and KAN module. The optimizer used AdamW, the learning rate was set to 1e-3, and the ExponentialLR learning rate scheduler was used with a decay rate of 0.8. All experiments were performed on an NVIDIA GeForce RTX4060 Laptop GPU.

[0141] 2.3. Introduction to experimental indicators

[0142] Four main indicators, namely accuracy, precision, recall and F1-score, are used to quantitatively evaluate the proposed model. The definitions of each indicator are as follows:

[0143]

[0144]

[0145]

[0146]

[0147] Among them, TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively.

[0148] 2.4. Comparative Experiment

[0149] To demonstrate the superiority of the proposed model, the inventors conducted comparative experiments with several mainstream deep learning models, including representative classification networks such as AlexNet, VGGNet, GoogLeNet, MobileNet, Swin-Transformer, MobileViT, and ResNet. The inventors obtained the code from the open source code repository and retrained the model on the inventors' dataset to generate classification results. To ensure a fair comparison, the inventors used two different training and test sets that were similar to the inventors' models. In addition, the same hyperparameters were used during training to maintain consistency in the experiments.

[0150] The experimental results are shown in Tables 3 and 4, indicating that the inventor's KAR-DANet method outperforms all the comparison models on both the Kvasir and HyperKvasir datasets, demonstrating its generalization ability. In the binary classification task in Table 3, KAR-DANet achieved 89.3% accuracy, 88.6% precision, 88.3% recall, and 88.2% F1-score, surpassing all other models in terms of various indicators. For example, ResNet's accuracy is 88.2%, close to the inventor's model, but its accuracy and F1-score are 0.5% and 0.4% lower, respectively. In contrast, AlexNet performed the worst, with only 73.8% accuracy. In the seven-classification task in Table 4, KAR-DANet achieved 81.4% accuracy, 76.9% precision, 78.8% recall, and 76.8% F1-score. The model outperformed most competitors, such as Densenet and EfficientNet, demonstrating its effectiveness in complex classification tasks. Overall, the comparison results highlight the superiority of KAR-DANet in feature extraction and classification accuracy, leading to better detection of lesions in colorectal images. To further verify the efficiency of the model, the inventors measured the throughput of their model and ResNet on a GPU. The results showed that while significantly improving the classification accuracy, the inference throughput of the inventors' model was 51.7 images / second.

[0151] like Figure 2 and Figure 3 As shown, the inventors plotted the confusion matrix of each model and compared it with the classic model. These results show that the inventors' model performs better in the classification tasks of the Kvasir dataset and the HyperKvasir dataset, especially on some confusing categories, the misclassification rate is significantly reduced. As can be seen from the confusion matrix, the model's predictions in all categories are more accurate, the number of correct classifications has increased significantly, and the number of misclassifications has decreased significantly. This shows that the model has stronger discrimination ability when dealing with complex categories, and can distinguish similar categories more effectively, thereby improving the overall classification accuracy. Through this result, the inventor's model not only performs well on a single category, but also shows comprehensive advantages in multi-category classification tasks, proving its robustness and reliability in dealing with fine-grained classification problems.

[0152] Table 3 Comparison of the proposed model with other models on the Kvasir dataset

[0153] Method Accuracy Precision Recall F1-score AlexNet 0.738 0.792 0.320 0.31 VggNet 0.858 0.87 0.856 0.855 Googlenet 0.794 0.820 0.810 0.791 Mobilenet 0.864 0.948 0.772 0.850 Swim-Transformer 0.755 0.732 0.852 0.775 Resnet_MLP 0.882 0.881 0.882 0.877 ResNet 0.892 0.838 0.922 0.872 KAR-DANet 0.893 0.886 0.883 0.882

[0154] Table 4 Comparison of the proposed model with other models on the HyperKvasir dataset

[0155] Method Accurcay Precision Recall F1-score AlexNet 0.742 0.793 0.332 0.321 VggNet 0.751 0.333 0.335 0.322 ResNet 0.790 0.754 0.767 0.770 Googlenet 0.714 0.788 0.298 0.286 Mobilenet 0.769 0.793 0.349 0.338 Densenet 0.827 0.459 0.437 0.440 Efficientnet 0.811 0.436 0.416 0.411 Swim-Transformer 0.714 0.822 0.297 0.277 MobileViT 0.814 0.438 0.432 0.421 Resnet_MLP 0.788 0.752 0.788 0.761 KAR-DANet 0.814 0.763 0.788 0.768

[0156] 2.5. Ablation Experiment Analysis

[0157] In order to fully evaluate the changes in model performance, the inventors made important modifications to its architecture, especially deleting the original KAN module. Subsequently, the inventors conducted experiments on the Kvasir and HyperKvasir datasets to verify the effectiveness of this structural adjustment. In order to intuitively show the impact of these changes on model performance, the inventors used t-SNE for feature visualization and used ROC curves and confusion matrices to verify the classification performance of the model. All experiments used the same training configuration as described above. The ablation experiment results on the test set show that integrating the KAN module can significantly enhance the feature separation effect.

[0158] 4.5.1 T-SNE Visualization Analysis

[0159] like Figure 4 and Figure 5 As shown in the figure, the inventors performed T-SNE visualization analysis on the ablation experiment. It can be noted that after adding the KAN module, the classification performance on the Kvasir and HyperKvasir datasets is significantly improved: the clustering of similar feature points becomes more compact, and the classification boundaries become more obvious. This enhancement of feature clustering not only enhances the model's ability to capture disease-specific features, but also greatly improves the accuracy and reliability of classification. In addition, the previously vague inter-class relationships become clearer, fully demonstrating the advantages of the KAN module in guiding the network to focus on key disease-related features in CRL images.

[0160] 4.5.2 Quantitative results analysis

[0161] To demonstrate the performance of KAN in the proposed KAR-DANet method, the inventors further compared the ROC curves, area under the curve (AUC) values, and confusion matrices of all models in the Kvasir and HyperKvasir datasets on each category. This helps to comprehensively evaluate the classification performance of different models. Figure 6 and Figure 7As shown in the figure, the proposed KAR-DANet model outperforms other models on the ROC curve, and its curve is closer to the upper left corner, indicating higher classification accuracy. Compared with ResNet, the micro-averaged AUC value of KAR-DANet reaches 0.97, while ResNet is only 0.91, highlighting the significant overall improvement in classification performance. Specifically, for the polyp category, the AUC value of KAR-DANet is 0.98, which is significantly higher than ResNet's 0.84, indicating that it has stronger discrimination ability in identifying more complex categories. In addition, the results of the confusion matrix further support this conclusion, showing that the KAR-DANet model performs well in reducing misclassifications and increasing the number of correct classifications, further verifying the superiority of KAN over the inventor's model.

[0162] 4.6. Visualization analysis of colorectal lesions and its medical significance

[0163] In order to verify the feature extraction ability of the proposed method, the inventors used Grad-CAM visualization technology to demonstrate the advantages of the designed architecture in focusing on relevant features. By using the Grad-CAM visualization tool, the inventors successfully highlighted the diseased areas in colorectal images, which are particularly able to generalize in diverse and complex colorectal imaging data, with obvious advantages over ResNet and other methods.

[0164] The clarity and accuracy of lesion visualization suggest that our method is a valuable addition to the clinical workflow, potentially providing diagnostic assistance to radiologists and gastroenterologists. In summary, Grad-CAM-based visualization of lesion regions in colorectal images demonstrates the significant clinical benefits of our proposed KAR-DANet model, especially in terms of improved diagnostic accuracy, interpretability, and robustness. These features highlight the superiority of our method in both research and real-world clinical applications.

[0165] In summary, the proposed method combines a KAN-based network, a variable attention module (DAM), and a classification module, and shows significant improvements in the detection and classification of colorectal lesions. The KAN-based network takes advantage of the deep feature extraction capabilities of ResNet and the nonlinear representation capabilities of KAN to better capture the complex structural features in colorectal images. In addition, the variable attention mechanism enables the model to flexibly adapt to spatial deformations and more accurately focus on relevant lesion areas, thereby improving classification accuracy and generalization ability.

Claims

1. A colorectal lesion classification method based on KAN network and variable attention mechanism, characterized in that: The steps are as follows: (1) Classify the original data of colorectal lesions; (2) Establish and initialize the KAN-ResNet backbone network classification model; Initialize the model parameters of the KAN-ResNet backbone network according to the requirements of each classification task, including the basic convolutional layer, pooling layer, and built-in KAN-based network module; Initialize the parameters of the Deformable Transformer network to ensure that it can effectively process the feature information extracted by KAN-ResNet; Finally, the classification module is initialized to generate prediction results for multi-classification tasks; The KAN-ResNet backbone network consists of ResNet and KAN-based network modules; the KAN module can weight features through the kernelized attention mechanism; The Deformable Transformer network adapts to changes in different feature scales by introducing deformable convolution and deformable attention module to achieve accurate positioning and classification of complex image scenes. The Deformable Transformer network flexibly captures image features of different scales and deformations by adaptively adjusting sampling positions and attention weights, thereby improving the classification performance of the model. (3) Set the optimizer and loss function; 3.

1. Adopt the AdamW optimization strategy and combine it with the exponential decay learning rate scheduler to optimize the model parameters; the learning rate adjustment formula is as follows: or t =η0·exp(-λ·t) Among them, η t is the learning rate of the tth round, η0 is the initial learning rate, and λ is the decay rate hyperparameter; 3.

2. According to the characteristics of multi-classification tasks, the cross entropy loss and the inter-class Gaussian loss are introduced for comparison; the cross entropy loss function is defined as follows: Among them, y c and are the true label and predicted probability, respectively, and C is the total number of categories. The inter-category Gaussian loss aims to improve the separability of the feature distribution of each category, thereby effectively dealing with the category imbalance problem in the data set and ensuring accurate detection of minority classes in medical data. (4) Model training; The images in the training data set are classified according to the lesion category and annotation, and the features of each type of image are extracted based on the KAN-ResNet backbone network; the Deformable Transformer network is used to process the features extracted by KAN-ResNet, and the feature distribution is further optimized through the variable attention mechanism, and the key areas are enhanced; mixed precision training is introduced in the model training process to improve the model calculation efficiency and reduce the memory usage; the variable attention module adaptively adjusts the attention weight according to the distribution of local and global features in the image, and uses the Deformable Convolution operation to realize the positioning and perception of the key areas in the feature map, thereby enhancing the discrimination ability of complex images; (5) Model saving and output If the validation set performance of the current training round is better than the best result of the previous round, the current model parameters are saved and the relevant performance indicators are recorded to ensure that the final optimal model parameters can be retained; after the training is completed, the final parameters of the model and the best accuracy on the validation set are output; (6) Model classification For the colorectal lesion images that need to be identified, the model predicts the classification of each patient's colon lesion images; the calculated classification results are output and displayed on the terminal interface of the system.

2. The method according to claim 1, characterized in that: In step (1), colon lesions are divided into three types: non-neoplastic polyps, inflammatory lesions, and neoplastic lesions; wherein non-neoplastic polyps include hyperplastic polyps and inflammatory polyps; inflammatory lesions include ulcerative colitis, which is specifically graded as 0-1, grade 1, grade 1-2, grade 2, grade 2-3, and grade 3; and neoplastic lesions include adenomatous polyps.

3. The method according to claim 2, characterized in that: According to the classification task requirements, it is divided into binary classification and multi-classification; Binary classification task 1: Classify lesions into non-neoplastic polyps and inflammatory lesions; Binary classification task 2: tumor lesions and inflammatory lesions; Multi-classification task: subdivided into polyps, ulcerative colitis grade 0-1, ulcerative colitis grade 1, ulcerative colitis grade 1-2, ulcerative colitis grade 2, ulcerative colitis grade 2-3, ulcerative colitis grade 3. In binary classification task 1: if the output result predicted by the model is greater than or equal to the threshold value 0.5, the output label is determined to be 1, and the lesion is classified as an inflammatory lesion; if the output result predicted by the model is less than the threshold value 0.5, the output label is determined to be 0, and the lesion is classified as a non-neoplastic polyp; In binary classification task 2: if the output result predicted by the model is greater than or equal to the threshold value 0.5, the output label is determined to be 1, and the lesion is classified as an inflammatory lesion; if the output result predicted by the model is less than the threshold value 0.5, the output label is determined to be 0, and the lesion is classified as a tumor lesion; In the multi-classification task: if the output label result predicted by the model is 0, the lesion is judged to be a polyp; if the output label result predicted by the model is 1, the lesion is judged to be ulcerative colitis grade 0-1; if the output label result predicted by the model is 2, the lesion is judged to be ulcerative colitis grade 1; if the output label result predicted by the model is 3, the lesion is judged to be ulcerative colitis grade 1-2; if the output label result predicted by the model is 4, the lesion is judged to be ulcerative colitis grade 2; if the output label result predicted by the model is 5, the lesion is judged to be ulcerative colitis grade 2-3; if the output label result predicted by the model is 6, the lesion is judged to be ulcerative colitis grade 3.

4. The method according to claim 1, characterized in that the model training comprises the following contents: 4.1 Data Preprocessing The training data set and the validation data set were divided into a set ratio of 8:

2. The input image data of the training set was preprocessed by random horizontal flipping, random vertical flipping and standardized data enhancement methods to improve the generalization ability of the model and prevent overfitting. For the colorectal lesion image data, image smoothing, denoising and morphological transformation data enhancement operations were further introduced. 4.2 Forward Propagation 4.2.

1. Use the KAN-ResNet backbone network to extract features from the input image and obtain the primary feature map; the KAN module improves the recognition of the feature map through the kernelized attention mechanism; 4.2.

2. Input the primary feature map into the Deformable Transformer network for feature processing: i. Map and normalize the feature map through the MLP module including the multi-layer perceptron; ii. Feature adjustment is performed through the Deformable Attention Module, which can adaptively adjust the attention weight based on the geometric shape of the feature map, focusing on the key feature areas in the image and performing weighted processing on them. In this step, the deformable convolution operation is used to perform feature sampling on multi-scale features to improve the model's ability to adapt to irregular changes in image space. 4.2.

3. Input the processed high-level features into the classification module, and finally output the prediction results of the multi-classification task; 4.

3. Calculating Loss and Back Propagation i. Calculate the loss value based on the model's predicted output and the true label, and introduce auxiliary loss to improve the model's discriminative ability; ii. Use the back-propagation algorithm to calculate the gradient and update the gradient of each layer parameters according to the loss; 4.4 Model parameter update i. Update the model parameters according to the optimizer strategy, and introduce the learning rate scheduler to dynamically adjust the learning rate to ensure that the model can stably converge to the global optimal solution; ii. Use mixed precision training to reduce computing resource consumption; 4.5 Model Validation After each training round, the model is verified using the validation dataset; the accuracy, recall, and F1 score are calculated to comprehensively evaluate the generalization ability of the model in the current round.

Images