Neuroblastoma pathological image classification method based on CMSwinKAN neural network

Abstract

The invention discloses a neuroblastoma pathological image classification method based on a CMSwinKAN neural network, and the method comprises the steps: firstly building a neuroblastoma pathological image data set, and dividing the data set into a training set and a test set; a CMSwinKAN neural network model is constructed, wherein the CMSwinKAN neural network model comprises a SwinKANsform feature extraction module, a comparison driving multi-scale aggregation module and a kernel activation network classification head; performing feature classification on the slices through an SVM classifier to obtain classification confidence; dynamically weighting the output of the CMSwinKAN neural network model according to the classification confidence, and outputting a final classification result; and taking the training set as input, training the CMSwinKAN model and the SVM classifier, and adjusting and optimizing parameters of the CMSwinKAN neural network model and the SVM classifier.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision deep learning and medical image processing, and specifically to a method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network. Background Art

[0002] Peripheral neuroblastic tumors (pNTs) are malignant tumors originating from the developing sympathetic nervous system and are the most common extracranial solid tumors in children, accounting for 15% of the total deaths related to childhood cancers. Due to the existence of multiple subtypes of peripheral neuroblastic tumors, there are significant differences in the pathological characteristics and corresponding treatment regimens of different subtypes. Accurate subtype classification is of great significance for the stratification of disease risk and the formulation of personalized treatment strategies. Currently, pathological diagnosis is the gold standard for clinical diagnosis of pNTs. However, due to the subtle pathological differences between different subtypes, the current clinical diagnosis mainly relies on the method of manual film reading by pathologists, which has obvious subjectivity and limitations and is difficult to observe the entire WSI image under the full field of view. Therefore, it is urgent to develop an automated method to accurately identify the cyto-pathological morphology by scanning the entire WSI image and achieve accurate subtype diagnosis.

[0003] Currently, some studies have attempted to use automated methods to assist in the diagnosis of WSI images. Although these methods have made some progress, they still face some key issues that cannot be ignored. First, the existing methods still lack the ability in feature expression. Especially, most traditional machine learning methods rely on manually designed features, which usually cannot comprehensively and effectively capture the rich lesion information and subtle structural differences in tissue pathological images, resulting in limited feature expression and restricting the performance ceiling of the model. Second, although deep learning methods have overcome the deficiencies of manual feature extraction to a certain extent, their non-linear modeling ability still has limitations. Especially when dealing with high-heterogeneity and high-complexity image data such as tissue pathological images, the conventional convolutional neural network structure is difficult to efficiently capture the long-range dependence relationship and multi-scale features of the spatial context in the image, which may further affect the robustness of the model in actual clinical applications. In addition, the lack of interpretability is also one of the important factors hindering the clinical implementation of existing models. The current deep learning models are often regarded as black boxes, lacking clear decision explanations. This not only increases the risk of clinical application of the model but also affects the trust of pathologists in the model results, thus restricting the popularization and application of automated auxiliary diagnosis methods in the actual medical environment.

[0004] In addition, the data scale of WSI images is extremely large, and a single image often reaches the GB level, far exceeding the computing power range of current mainstream neural network models. Therefore, the main solution at present is to divide the entire WSI image into multiple independent patches for classification. However, this patch-level processing method has serious problems of information fragmentation and false association. On the one hand, there is a lack of effective spatial constraints between patches, resulting in the possibility that the results of individual classification may ignore the overall spatial structure information of the image. On the other hand, some key lesion areas may be divided into different patches and processed in isolation, which will affect the overall understanding of the lesion and cause the incompleteness and inaccuracy of classification information. These factors will greatly reduce the overall diagnostic accuracy and generalization ability of the model. Therefore, there is an urgent need for a WSI image classification model that can better balance local details and overall context relationships, has stronger feature expression ability and non-linear modeling ability, and at the same time has higher interpretability, so as to truly promote the effective application of automated pathological diagnosis technology in clinical practice. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and propose a method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network to solve the complexity problem of classifying patch-level H&E stained images of neuroblastoma pathological sections. At the same time, a soft voting mechanism based on prior knowledge of tissue component analysis is proposed to achieve accurate WSI image classification.

[0006] In order to achieve the above purpose, the specific technical solutions adopted by the present invention are as follows:

[0007] A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network, comprising the following steps:

[0008] Step 1: Establish a neuroblastoma pathological image dataset and divide it into a training set and a test set;

[0009] Step 2: Construct a CMSwinKAN neural network model, and the CMSwinKAN neural network model includes a SwinKANsformer feature extraction module, a contrast-driven multi-scale aggregation module, and a kernel activation network classification head;

[0010] The Swin KANsformer feature extraction module is composed of four cascaded levels. Each level contains several Swin KANsformer modules, and an image aggregation layer is set between adjacent levels. Inside each Swin KANsformer module, window multi-head self-attention layers and shifted window multi-head self-attention layers are arranged alternately. The contrast-driven multi-scale aggregation module includes a feature dynamic weighting unit and a cross-scale attention fusion unit. The feature dynamic weighting unit adaptively adjusts features of different scales through a learnable scalar weight matrix. The cross-scale attention fusion unit uses bilinear interpolation to unify the resolution of feature maps and align channels. Finally, all output feature maps are fused.

[0011] Step 3: Standardize and stack the neuroblastoma pathological images to generate a slice sequence, and then classify the features of the slices through an SVM classifier to obtain classification confidence.

[0012] Step 4: Dynamically weight the output of the CMSwinKAN neural network model according to the classification confidence to output the final classification result.

[0013] Step 5: Use the training set as input to train the CMSwinKAN model and the SVM classifier, and optimize the parameters of the CMSwinKAN neural network model and the SVM classifier.

[0014] Step 6: Use the test set to test and evaluate the obtained network model for the final classification of neuroblastoma pathological images.

[0015] Preferably, the Swin KANsformer module is based on the Swin Transfomer as the base network, and all multi-layer perceptron layers in the Swin Transfomer are replaced with kernel activation network layers.

[0016] Preferably, the CMSwinKAN neural network model further includes a block segmentation module and a linear embedding layer. First, the input neuroblastoma pathological image is segmented into non-overlapping 4×4 image blocks through the block segmentation module. The feature dimension of each image block is 4×4×3 = 48, and then a linear embedding layer is used to project the features of each image block into any dimension to obtain image features adapted to the size of the Swin KANsformer feature extraction module.

[0017] Preferably, each level of the Swin KANsformer feature extraction module includes d ii Swin KANsformer modules, where i = 1, 2, 3, 4 represents the hierarchical sequence, and the number of Swin KANsformer modules in different hierarchies [d1, d2, d3, d4] = [2, 2, 8, 2].

[0018] Preferably, the first three hierarchies of the Swin KANsformer feature extraction module further include a block merging layer, through which the image features gradually decrease in size after the first three stages. The image features of the first three stages are saved to a feature matrix, and the image features of the first three hierarchies are respectively multiplied by a learnable weight [λ1, λ2, λ3], and then the image features of the second and third hierarchies are upsampled to the size of the first hierarchy through bilinear interpolation. The number of feature channels of the first three hierarchies is [2C, 4C, 8C], and the number of image feature channels is uniformly adjusted to 2 through a 1×1 convolutional block s C to obtain the input of the contrast-driven multi-scale aggregation module.

[0019] Preferably, the method for implementing feature aggregation through the contrast-driven multi-scale aggregation module is as follows:

[0020] First, the image features F1 and F2 output by the first and second hierarchies are respectively calculated through two linear layers to obtain attention weights A1 and A2, and then for each spatial position (x n , y n ) in the image features, the attention weights A1 and A2 are reshaped into and both of which are activated by Softmax;

[0021] Then, the image feature F3 output in the third stage is preliminarily fused through two CBR modules to obtain a feature vector with two channels;

[0022] Subsequently, the feature vector with two channels is mapped into a value vector V3 through a linear layer, and the local window feature of K×K is extracted from the value vector V3

[0023] Through the reshaped attention weights the local window features are weighted twice:

[0024]

[0025] where represents matrix multiplication;

[0026] The weighted features are aggregated according to the spatial position (i, j) to obtain the aggregated feature The expression is as follows:

[0027]

[0028] Preferably, the CBR module includes a convolutional layer, a batch normalization layer, and a ReLU activation function.

[0029] Preferably, in step 3, the method for obtaining the classification confidence by the SVM classifier is as follows: The SVM classifier receives the input feature x of each patch and outputs the corresponding class probability distribution P SVM :

[0030] P SVM = softmax([w0·φ(x)+b0, w1·φ(x)+b1, w2·φ(x)+b2])

[0031] where the softmax function normalizes the output values of the SVM classifier into a probability distribution, and P SVM = [P0, P1, P2] respectively represent the probabilities that the patch belongs to other tissues, neuropil, and Schwann matrix.

[0032] According to the prediction result of the SVM classifier, determine the voting weight w of the patch patch :

[0033] w patch = α·P1 + β·P2

[0034] P1 and P2 are the probabilities of neuropil and Schwann matrix respectively, and α and β are weighting coefficients used to control the importance of different tissue components. Among them, α>β emphasizes the importance of neuropil. If the SVM classifier determines that the patch is other tissue, then w patch = γ, where γ is a lower baseline weight.

[0035] Preferably, in step 4, the dynamic weighting method is as follows: The SVM classifier is used to assist in discriminating the tissue type of the pathological image slice, and according to the output result of the SVM classifier, a dynamic weight is assigned to the classification result of each slice of the CMSwinKAN model. Finally, the classification result of the entire WSI image is obtained by the soft voting method. The expression is as follows:

[0036] P∈RN×C, P∈R N×C , W∈RN×C. W∈R N×C .

[0037]

[0038]

[0039] where Pi,C is the classification category and probability of the C-th patch, w i,C is the corresponding voting weight.

[0040] The present invention has the following characteristics and beneficial effects:

[0041] The present invention proposes a contrast-driven multi-scale feature fusion framework (CMSA). Through the contrast and fusion between multi-scale features, and the adaptive adjustment of the contribution degrees of different-scale features, it captures the long-range dependence relationships and local detail features in pathological slice images, avoids the negative impact of the low-density information scale on the overall feature representation, and solves the problem of insufficient interaction between multi-scale features. The present invention proposes a multi-head self-attention module (Swin KANsformer Block) that combines conventional windows and shifted window configurations, and introduces KAN as the classification head, significantly improving the feature expression ability and non-linear modeling performance of the model. At the same time, based on feature analysis, the model parameters are optimized, the embedding dimension and the number of neurons in the KAN hidden layer are reasonably selected, effectively reducing the number of model parameters while maintaining the fine-grained classification accuracy of the model, and realizing lightweight design. The present invention proposes a WSI classification method based on clinical prior knowledge and a soft voting mechanism. By training a support vector machine (SVM) model, it realizes fine-grained recognition of tissue components at the slice level, and combines a dynamic weight allocation strategy to effectively integrate local slice features and global features, improving the accuracy and robustness of classification.

[0042] 1. Lightweight multi-scale feature extraction

[0043] The Swin KANsformer feature extraction module is adopted to replace the MLP layer of the original Transformer with a learnable Kernel Activation Network (KAN) layer, and a hierarchical window attention mechanism is constructed through alternately arranged window multi-head self-attention (W-MSA) and shifted window multi-head self-attention (SW-MSA), significantly reducing the model parameters and computational amount, while maintaining the efficient capture of global and local features of high-resolution pathological images, providing the possibility for clinical edge device deployment.

[0044] 2. Contrast-driven multi-scale feature aggregation

[0045] The Contrastive-Driven Multi-Scale Aggregation (CMSA) module is introduced. It adaptively weights multi-scale features from different levels through learnable scalar weights [λ1, λ2, λ3], bilinearly interpolates them to the same spatial resolution in the cross-scale attention fusion unit first, then aligns the channels through 1×1 convolution, and completes the fusion and enhancement of depth information based on cross-scale contrastive attention within the CDFA sub-module, significantly improving the discriminative ability and robustness of feature representation.

[0046] 3. KAN classification head with strong interpretability

[0047] Based on the multi-scale aggregated features, a KAN classification head constructed by a learnable activation function based on B-spline is adopted. Each activation function is an adjustable piecewise polynomial. While significantly improving the network's non-linear fitting ability, it can intuitively explain the model's decision-making process through visualizing the basis functions and control points, enhancing the trust of clinicians.

[0048] 4. Dynamic weighted soft voting based on clinical prior

[0049] Combined with the confidence output of the SVM for two key tissue components, neuropil and Schwannian stroma, the slice weights are dynamically assigned, and the classification results of each slice are integrated through weighted soft voting at the WSI level, significantly improving the classification accuracy of the whole slice; the ACC reaches 100% on the PpNTs test set.

[0050] 5. Excellent generalization ability and clinical application value

[0051] It shows excellent classification performance on both the private PpNTs dataset and the public BreakHis dataset (the highest AUC reaches 99.53%), verifying the stability and reliability of the model under different sample distributions, providing strong technical support for clinical AI-assisted diagnosis. Description of the Drawings

[0052] Figure 1 It is the structure diagram of the CMSwinKAN neural network model of the present invention;

[0053] Figure 2 It is the structure diagram of the Swin KANsformer module of the present invention;

[0054] Figure 3 It is the structure diagram of the CMSA module network model of the present invention;

[0055] Figure 4 It is the schematic diagram of the WSI voting mechanism architecture of the present invention. Detailed Embodiment

[0056] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0057] A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network, as Figure 1 shown, includes the following steps:

[0058] Step 1: Establish a neuroblastoma pathological image dataset and divide it into a training set and a test set.

[0059] It should be noted that due to the complexity of data collection and annotation in the academic community, there is a shortage of detailed datasets for neuroblastoma (NB). To solve this problem, this embodiment cooperates with the Pediatric Oncology Department of Zhejiang University Children's Hospital to create a private dataset called Pediatric Peripheral Neuroblastoma (PpNTs). This dataset includes 102 children diagnosed with NB at Zhejiang University Children's Hospital, and a total of 304 whole-slide images (WSIs) are included. Digitalization is performed at a magnification of 20 times using a Pannoramic SCAN II digital scanner.

[0060] 1) Sliding window segmentation: Use a sliding window algorithm with a non-overlapping strategy, starting from the upper left corner (0,0) of the WSI. The window slides horizontally with a step size of 512 pixels, and the right and bottom regions that cannot be completely covered by a 512×512 pixel window are discarded to avoid invalid data.

[0061] 2) Data quality improvement: In this embodiment, a two-stage filtering rule is adopted: Blocks with more than 50% white pixels (RGB≥245) are discarded because they may represent a blank background or staining defects. Blocks with more than 50% red pixels are classified as dense red blood cell regions and are also discarded because they may mask important pathological features.

[0062] Step 2: Construct a CMSwinKAN neural network model, and the CMSwinKAN neural network model includes a SwinKANsformer feature extraction module, a contrast-driven multi-scale aggregation module, and a kernel activation network classification head;

[0063] The Swin KANsformer feature extraction module is composed of four cascaded levels. Each level contains several Swin KANsformer modules, and an image aggregation layer is set between adjacent levels. Inside each Swin KANsformer module, window multi-head self-attention layers and shifted window multi-head self-attention layers are arranged alternately. The contrast-driven multi-scale aggregation module includes a feature dynamic weighting unit and a cross-scale attention fusion unit. The feature dynamic weighting unit adaptively adjusts features of different scales through a learnable scalar weight matrix. The cross-scale attention fusion unit uses bilinear interpolation to unify the resolution of the feature maps and aligns the channels. Finally, all the output feature maps are fused.

[0064] Among them, each level of the Swin KANsformer feature extraction module includes d i Swin KANsformer modules, where i = 1, 2, 3, 4 represents the level sequence, and the number of Swin KANsformer modules in different levels [d1, d2, d3, d4] = [2, 2, 8, 2].

[0065] The first three levels of the Swin KANsformer feature extraction module also include a block merging layer. Through the block merging layer, the image features gradually decrease in size after the first three stages. The image features of the first three stages are saved to a feature matrix. The image features of the first three levels are respectively multiplied by a learnable weight [λ1, λ2, λ3], and then the image features of the second and third levels are upsampled to the size of the first level through bilinear interpolation. The number of feature channels of the first three levels is [2C, 4C, 8C]. The number of image feature channels is uniformly adjusted to 2 s C through a 1×1 convolution block to obtain the input of the contrast-driven multi-scale aggregation module.

[0066] The Swin KANsformer module replaces the traditional MLP layer with a KAN layer in the Swin Transformer module. Inherited from the Swin Transformer module, the multi-head self-attention layer in its structure is a window multi-head self-attention (W-MSA) layer and a shifted window multi-head self-attention layer (SW-MSA) respectively in two alternating Swin KANsformer Blocks. In the whole model, first, the input RGB H&E stained pathological image is segmented into non-overlapping 4×4 image patches through the Patch Partition module. The feature dimension of each patch is 4×4×3 = 48, and then it is projected to an arbitrary dimension (denoted as C) using a linear embedding layer.

[0067] To facilitate the preservation of multi-scale features, we re-divided the stages of the model. Except for the last stage which only contains d3 Swin KANsformer modules, each stage includes d i Swin KANsformer modules and a Patchmerging layer. In our base model, the number of Swin KANsformer Blocks in different stages [d1, d2, d3, d4] = [2, 2, 8, 2]. Due to the Patch merging layer, the size of the image features gradually decreases after the first three stages

[0068] Furthermore, the method for feature aggregation implemented by the contrast-driven multi-scale aggregation module is as follows:

[0069] First, the image features F1 and F2 output by the first and second levels are respectively passed through two linear layers to calculate the attention weights A1 and A2. Then, for each spatial position (x n , y n ) in the image features, the attention weights A1 and A2 are reshaped into and where both are activated by Softmax;

[0070] Specifically, in this embodiment, the contrast-driven multi-scale feature fusion module (CMSA) introduces the CDFA module to address the two major challenges of "soft boundaries" and co-occurrence phenomena in medical images through contrast-driven feature enhancement. In our model, the CDFA module is used as the fusion module for multi-scale features from different layers of Swin KANsformer.

[0071] The CDFA module calculates the attention weights within the K×K window centered at (i, j) by combining the details of the shallow and deep features at each spatial position (i, j). Specifically, the processed shallow features (features retained in the first and second stages) F1 and F2 are respectively passed through two different linear layers to calculate the attention weights The calculation of the attention weights can be expressed as:

[0072] A1 = W1·F1, A2 = W2·F2

[0073] where are the linear transformation weight layers of the multi-scale feature maps in the first and second stages respectively. Subsequently, the attention weights at each spatial position (i, j) in the two feature maps are reshaped into and both of which are activated by Softmax.

[0074] Then, the image features output in the third stage are initially fused through two CBR modules to obtain feature vectors of two channels.

[0075] Specifically, the processed deep features (features retained in the third stage) are initially fused through 2 CBR blocks. The CBR module combines a convolution layer, a batch normalization layer, and a ReLU activation function to simplify the network construction process and help improve the training efficiency and performance of the network.

[0076] Subsequently, the feature vectors of two channels (2C-dimensional feature vectors) are mapped into a value vector through a linear layer The local window features of K×K are extracted from the value vector V3

[0077]

[0078] Through the reshaped attention weights and the local window features are weighted twice:

[0079]

[0080] where represents matrix multiplication;

[0081] The weighted features are aggregated according to the spatial positions (i, j) to obtain the aggregated features The expression is as follows:

[0082]

[0083] Furthermore, in this embodiment, each activation function of the KAN classification head is a learnable spline function, and each activation function can be expressed as:

[0084] φ(x) = w b b(x) + w s spline(x)

[0085] where b(x) is a fixed basis function (such as SiLU), and spline(x) is a learnable spline function.

[0086] KAN uses B-spline basis functions to construct piecewise polynomials, which have local support (i.e., each basis function is non-zero only in a finite interval), enabling the model to efficiently process high-dimensional data.

[0087] B-spline basis functions are the core part used in KAN to construct piecewise polynomials. For the input data x and the given grid points t = {t0, t1, …, t n}, the calculation method of the B-spline basis function N i,k (x) is as follows:

[0088] N i,k (x) = w i,k (x) · N i,k-1 (x) + (1 - w i+1,k (x)) · N i+1,k-1 (x)

[0089] where,

[0090]

[0091] Step 3: Standardize and stack the neuroblastoma pathological images to generate a slice sequence, and then classify the features of the slices through an SVM classifier to obtain the classification confidence.

[0092] Specifically, the SVM model receives the input feature x of each patch and outputs the corresponding class probability distribution P SVM :

[0093] P SVM = softmax([w0·φ(x) + b0, w1·φ(x) + b1, w2·φ(x) + b2])

[0094] where the softmax function normalizes the output values of the SVM into a probability distribution. P SVM = [P0, P1, P2] respectively represent the probabilities that the patch belongs to other tissues, neuropil, and Schwann stroma.

[0095] According to the prediction results of the SVM, determine the voting weight w patch :

[0096] w patch = α·P1 + β·P2

[0097] P1 and P2 are the probabilities of neuropil and Schwann stroma respectively. α and β are weighting coefficients used to control the importance of different tissue components. Among them, α > β emphasizes the importance of neuropil. If the SVM determines that the patch is other tissue, then w patch = γ, where γ is a lower reference weight.

[0098] Step 4: Dynamically weight the output of the CMSwinKAN neural network model based on the classification confidence, and output the final classification result;

[0099] Step 5: Use the training set as input to train the CMSwinKAN model and the SVM classifier, and tune the parameters of the CMSwinKAN neural network model and the SVM classifier.

[0100] Specifically, this experiment was carried out on the NVIDIA Tesla V100-SXM2 GPU (32GB video memory) hardware platform, and the software environment was built based on Python 3.8.20 and PyTorch 2.4.1 (CUDA 12.1 acceleration library). The training process lasted for 30 epochs on our private dataset, with a batch size of 64, taking approximately 75 hours.

[0101] For the multi-classification task of pathological images, the label was standardized using the class encoding mapping {'PD':0, 'D':1, 'GN':2, 'GNB':3, 'UD':4}. For the training data, the input images were resized to 224×224 pixels, and data augmentation techniques such as random horizontal flipping were applied to enhance robustness. For the test data, the images were directly resized to 224×224 pixels. The model was not pre-trained, and the model parameters were randomly initialized using the He normal distribution. The AdamW optimizer was used, combined with the cosine annealing algorithm. The cosine annealing algorithm was used during the training process, and gradient accumulation (number of steps = 4) was used to expand the equivalent global batch size to 256, combined with amp mixed-precision training to improve computational efficiency and numerical stability. The model used cross-entropy loss as the optimization objective, with a fixed batch size of 64, and 20% structured pruning was applied to the convolutional layer to reduce computational complexity.

[0102] For the WSI soft voting strategy, the weighted coefficients of the key tissue components were set as α = 1 (neuropil), β = 8 (Schwann stroma), and the reference weight of the non-key tissue area was γ = 1. The diagnostic specificity was improved through weighted fusion.

[0103] Step 6: Use the test set to test and evaluate the obtained network model for the final classification of neuroblastoma pathological images.

[0104] In this embodiment, in order to comprehensively evaluate the performance of different models on different datasets, the following standard evaluation metrics were used in this embodiment: Accuracy, Balanced Accuracy (BACC), Weighted F1, Cohen's kappa coefficient, and Area Under The Curve (AUC). When calculating BACC, Recall and Precision need to be calculated first.

[0105]

[0106] In the above formula, TP (True Positive) and TN (True Negative) represent the samples correctly predicted as true and false, while FP (False Positive) and FN (False Negative) represent the samples incorrectly predicted as true and false, respectively. P is the number of positive samples, and N is the number of negative samples. p i is the prediction score for positive samples, that is, the probability of predicting positive samples as positive examples; n j is the prediction score for negative samples, that is, the probability of predicting negative samples as positive examples.

[0107] To further verify the effectiveness of the technical solution of the present invention, a comparative experiment was conducted on the PpNTs dataset in this embodiment. The model of this embodiment was compared with other typical classification models, general large models in the field of pathology, and advanced pathological image classification models, and the effectiveness of the model was verified from two dimensions: patch classification and WSI classification. Compared with other typical models, general large models in the field of pathology, or advanced pathological image classification models, CMSwinKAN shows significant advantages in processing microscopic image classification tasks.

[0108] TABLE VI: Ablation studies of CMSwinKAN on our private PpNTs and public BreakHis datasets.

[0109]

[0110] Table 1 shows the comparison results between the embodiments of the present invention and other models.

[0111] To further verify the generalization ability of the proposed CMSwinKAN model, it was compared with other typical classification models, general large models in the field of pathology, and advanced pathological image classification models on the publicly available breast cancer histopathology image classification dataset, the BreakHis dataset. The results show that CMSwinKAN also shows strong generalization ability on unbalanced datasets and achieves excellent results in all metrics.

[0112] In the ablation experiment part, the influence of the KAN network, CDFA component, and contrast-driven multi-scale feature fusion module (CMSA) on the classification performance of these two datasets was further analyzed. In this embodiment, the performance of the CMSwinKAN model after restoring the KAN network, replacing the CDFA component with a linear layer, and removing the CMSA module was compared with the performance of the complete CMSwinKAN to evaluate its performance. As shown in Table 1, the results show that these three modules have a positive impact on the classification performance of the model.

[0113] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and the descriptions in the specification are only preferred examples of the present invention, and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A method for classifying pathological images of neuroblastoma based on the CMSwinKAN neural network, characterized in that, It includes the following steps: Step 1: Establish a neuroblastoma pathological image dataset and divide it into a training set and a test set; Step 2: Construct a CMSwinKAN neural network model, and the CMSwinKAN neural network model includes a SwinKANsformer feature extraction module, a contrast-driven multi-scale aggregation module, and a kernel activation network classification head; The Swin KANsformer feature extraction module is cascaded by four levels. Each level contains several SwinKANsformer modules, and an image aggregation layer is set between adjacent levels; within each Swin KANsformer module, window multi-head self-attention layers and shifted window multi-head self-attention layers are alternately arranged; the contrast-driven multi-scale aggregation module includes a feature dynamic weighting unit and a cross-scale attention fusion unit. The feature dynamic weighting unit adaptively adjusts features of different scales through a learnable scalar weight matrix. The cross-scale attention fusion unit uses bilinear interpolation to unify the resolution of feature maps, performs channel alignment, and finally fuses all output feature maps; Step 3: Generate a slice sequence by normalizing, cutting, and stacking neuroblastoma pathological images, and then use an SVM classifier to classify the features of the slices to obtain a classification confidence; Step 4: Dynamically weight the output of the CMSwinKAN neural network model according to the classification confidence and output the final classification result; Step 5: Use the training set as input to train the CMSwinKAN model and the SVM classifier, and optimize the parameters of the CMSwinKAN neural network model and the SVM classifier; Step 6: Use the test set to test and evaluate the obtained network model for final neuroblastoma pathological image classification.

2. The method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 1, wherein, The Swin KANsformer module is based on the Swin Transfomer as the basic network, and all multi-layer perceptron layers in the SwinTransfomer are replaced with kernel activation network layers.

3. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 1, characterized in that, The CMSwinKAN neural network model also includes a patch segmentation module and a linear embedding layer. First, the input neuroblastoma pathological image is segmented into non-overlapping 4×4 image patches by the patch segmentation module. The feature dimension of each image patch is 4×4×3 = 48, and then a linear embedding layer is used to project the features of each image patch into any dimension to obtain image features adapted to the size of the Swin KANsformer feature extraction module.

4. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 3, characterized in that, Each level of the Swin KANsformer feature extraction module includes d i SwinKANsformer modules, where i = 1, 2, 3, 4 represents the level sequence, and the number of Swin KANsformer modules in different levels is [d1, d2, d3, d4] = [2, 2, 8, 2].

5. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 4, characterized in that, The first three levels of the Swin KANsformer feature extraction module also include block merging layers, through which the image features gradually decrease in size after the first three stages. The image features of the first three stages are saved to a feature matrix, and the image features of the first three levels are respectively multiplied by a learnable weight [λ1, λ2, λ3], and then the image features of the second and third levels Are upsampled to the size of the first level through bilinear interpolation. The number of feature channels of the first three levels is [2C, 4C, 8C], and the number of image feature channels is uniformly adjusted to 2 s C, to obtain the input of the contrast-driven multi-scale aggregation module.

6. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 5, characterized in that, The method for realizing feature aggregation through the contrast-driven multi-scale aggregation module is as follows: First, the image features F1 and F2 output by the first and second levels are respectively calculated through two linear layers to obtain attention weights A1 and A2. Then, for each spatial position (x n , y n ) in the image features, the attention weights A1 and A2 are reshaped into and where both are activated by Softmax; Then, the image features F3 output in the third stage are preliminarily fused through two CBR modules to obtain feature vectors of two channels; Subsequently, the feature vectors of the two channels are mapped into a value vector V3 through a linear layer, and local window features of K×K are extracted from the value vector V3 Through the reshaped attention weights For the local window features Perform two weightings: wherein represents matrix multiplication; Aggregate the weighted features according to the spatial positions (i, j) to obtain the aggregated features The expression is as follows:

7. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 6, characterized in that, The CBR module includes a convolutional layer, a batch normalization layer, and a ReLU activation function.

8. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 1, characterized in that, In the said step 3, the method for obtaining the classification confidence by the SVM classifier is as follows: the SVM classifier receives the input feature x of each patch and outputs the corresponding class probability distribution P SVM : P SVM = softmax([w0·φ(x)+b0, w1·φ(x)+b1, w2·φ(x)+b2]) Among them, the softmax function normalizes the output value of the SVM classifier into a probability distribution, and P SVM = [P0, P1, P2] respectively represent the probabilities that the patch belongs to other tissues, neuropil, and Schwann matrix Determine the voting weight w of the patch according to the prediction result of the SVM classifier patch : w patch = α·P1 + β·P2 P1 and P2 are the probabilities of neuropil and Schwann matrix respectively, and α and β are weighting coefficients used to control the importance of different tissue components. Among them, α>β emphasizes the importance of neuropil. If the SVM classifier determines that the patch is other tissue, then w patch =γ, where γ is a lower reference weight.

9. A method for classifying neuroblastoma pathological images based on the CMSwinKAN neural network according to claim 8, characterized in that, In the above step 4, the dynamic weighting method is as follows: the SVM classifier is used to assist in discriminating the tissue types of pathological image slices, and dynamic weights are assigned to the classification results of each slice of the CMSwinKAN model according to the output results of the SVM classifier. Finally, the classification result of the entire WSI image is obtained by the soft voting method, and the expression is as follows: P ∈ ℝⁿ×ᶜ, P ∈ ℝⁿ ×C , W ∈ ℝⁿ×ᶜ. W ∈ ℝⁿ ×C . where P i,C is the classification category and probability of the C-th patch, and w i,C is the corresponding voting weight.

Images