A Medical Image Classification Method and System Based on Graph Fusion Spatial State Model

Abstract

This invention discloses a medical image classification method and system based on a graph fusion spatial state model, relating to the fields of medical image processing and deep learning artificial intelligence. The method includes: acquiring multimodal medical images and preprocessing them; inputting the preprocessed multimodal medical images into an overlapping stem embedding layer, generating initial features through convolutional embedding operations; inputting the initial features into a hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, with each adjacent MambaGNN block connected by a downsampling layer, and the final enhanced features output by the last MambaGNN block; inputting the final enhanced features into a classification head, outputting the predicted category probability of the multimodal medical image. This invention addresses the shortcomings of existing technologies in terms of local-global modeling balance, computational complexity control, and topological relationship capture.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence deep learning technology, and more specifically to a medical image classification method and system based on a graph fusion spatial state model. Background Technology

[0002] Medical image classification is a core technology supporting clinical disease diagnosis and screening. In recent years, deep learning-based methods have been widely used in multimodal medical image analysis. Current mainstream technical frameworks mainly include three categories: Convolutional Neural Networks (CNNs), Visual Transformers, and State-Space Models (SSMs). CNNs excel in extracting anatomical details due to their local convolutional operations; Transformers achieve global dependency capture through self-attention mechanisms; and SSMs have become a research hotspot due to their linear time complexity sequence modeling capabilities, with typical examples including SpatialMamba (Xiao et al., 2024) and MedMamba. Some schemes introduce Structure-Aware State Fusion (SASF) mechanisms to enhance spatial modeling capabilities. These technologies have been successfully applied to scenarios such as skin lesion detection, chest disease recognition, and fetal ultrasound planar classification. However, the complex anatomical structures, multi-scale lesion distribution, and modal differences unique to medical images still pose challenges to the overall performance of these models.

[0003] Current research mainly focuses on the following types of methods:

[0004] (1) Convolutional Neural Network (CNN) series models: ResNet (He et al., CVPR 2016) and EfficientNet (Tan & Le, ICML 2019) extract local features through hierarchical convolution and residual connections, and perform stably in local anatomical detail recognition tasks.

[0005] However, this type of model relies on a fixed receptive field, making it difficult to capture long-range anatomical relationships across regions and having a weak ability to model global topological relationships.

[0006] (2) Visual Transformer and its improved models: For example, the Swin Transformer (Liu et al., ICCV 2021) employs a hierarchical self-attention mechanism, which improves the efficiency of global feature modeling. However, this type of model suffers from a large number of parameters and high floating-point operations (FLOPs), with computational complexity increasing quadratically with input resolution, making it difficult to deploy in clinical resource-constrained scenarios.

[0007] (3) State-space models (SSMs) and related improvements: SpatialMamba (Xiao et al., 2024) introduced a spatial fusion module into SSM, while MedMamba optimized the sequence modeling process for medical images.

[0008] However, traditional SSMs require image patches to be flattened into a one-dimensional sequence for processing. The unidirectional scanning mechanism limits the explicit modeling of spatial topological relationships, has insufficient ability to represent complex anatomical structures, and is difficult to adapt to the diverse topological features of multimodal medical images.

[0009] Based on the above research status, existing medical image classification technologies still have the following shortcomings: (1) Imbalance in modeling: CNNs focus on local features but neglect long-range dependencies, while Transformers and SSMs are difficult to accurately model both local details and global topology.

[0010] (2) Low computational efficiency: The Transformer class model has too many parameters and computational overhead, which cannot meet the deployment requirements of real-time clinical diagnosis.

[0011] (3) Lack of topological modeling: The one-dimensional sequence processing method of SSMs cannot explicitly capture the spatial topological relationship of anatomical structures, making it difficult to adapt to the diverse topological features of multimodal medical images and resulting in insufficient accuracy in classifying complex lesions.

[0012] (4) Weak multimodal adaptability: The topological features of different modal medical images (such as dermoscopy, X-ray, ultrasound) differ significantly, and existing models lack a unified modeling framework.

[0013] (5) Insufficient feature fusion: Existing models have a simple way of fusing local anatomical details and global topological features, and have not formed a collaborative enhancement mechanism.

[0014] These problems are particularly prominent in the practical application of multimodal medical imaging intelligent diagnostic systems, severely limiting the clinical applicability and promotional value of the models.

[0015] Therefore, how to address the shortcomings of existing technologies in terms of local-global modeling balance, computational complexity control, and topological relationship capture is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0016] In view of the above problems, the present invention proposes a medical image classification method and system based on a graph fusion spatial state model to overcome or at least partially solve the above problems.

[0017] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a medical image classification method based on the graph fusion spatial state model (Mamba), comprising: Acquire multimodal medical images and perform preprocessing; The preprocessed multimodal medical images are input into an overlapping stem embedding layer, and initial features are generated through convolutional embedding operations. The initial features are input into a hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, each pair of adjacent MambaGNN blocks is connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block; The final enhanced features are input into the classification head, and the predicted category probability of the multimodal medical image is output.

[0018] Furthermore, the multimodal medical images include at least an RGB three-channel dermoscopy image, a grayscale X-ray image, and an ultrasound image.

[0019] Furthermore, each MambaGNN block integrates a structure-aware state-space model module, a graph convolutional network, and a feedforward network; within each MambaGNN block: The MambaGNN block input features are subjected to layer normalization to obtain normalized features; The normalized features are input into the structure-aware state space model module, and multi-scale local state features are aggregated through multi-dilation rate convolution. The multi-scale local state features are residually connected to the MambaGNN block input features to obtain the first residual connection features; The first residual connection feature is input into the graph convolutional network, and the convolutional feature is output. The convolutional feature is residually concatenated with the first residual connection feature to obtain the second residual connection feature; After performing layer normalization on the second residual connection feature, it is input into the feedforward network, and the output of the feedforward network is residually connected with the second residual connection feature to output the enhanced feature of the current MambaGNN block.

[0020] Furthermore, the multi-scale local state features are represented as follows:

[0021] in, Indicates the current time Multi-scale local state features; This represents the set of domains participating in multi-scale aggregation; Representing the neighborhood The learnable fusion weights corresponding to the kth spatial location; This represents the eigenvalue of the normalized feature at the spatial mapping location.

[0022] Furthermore, the graph convolutional network is a global topological network constructed based on an 8-nearest neighbor graph; the 8-nearest neighbor graph uses the spatial location of the feature grid as nodes, covering the top, bottom, left, right and 45° diagonal associations.

[0023] Furthermore, the convolutional features are represented as:

[0024] in, The graph convolutional network represents the first... The node feature matrix of the layer; Indicates the first The convolutional features output by the layer; express The corresponding trainable weight matrix; Indicates the activation function; This represents the adjacency matrix after adding self-connections; express The corresponding degree matrix.

[0025] Furthermore, the downsampling layer employs a convolution operation with a stride of 2, reducing the spatial dimension of the feature map to half and expanding the channel dimension to twice the size.

[0026] Furthermore, the classification head includes a global average pooling layer and a fully connected network; The final enhanced features are converted into feature vectors of a preset dimension through the global average pooling layer. The preset dimension feature vector is converted into category probabilities through the fully connected network, and the classification result is output.

[0027] Secondly, the present invention provides a medical image classification system based on a graph fusion spatial state model, which, when applied using the above-mentioned method, includes: The data acquisition and processing unit is used to acquire multimodal medical images and perform preprocessing. The initial feature generation unit is used to input the preprocessed multimodal medical image into the overlapping stem embedding layer and generate initial features through convolutional embedding operations; The feature enhancement unit is used to input the initial features into a hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, each pair of adjacent MambaGNN blocks is connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block; The prediction result generation unit is used to input the final enhanced features into the classification head and output the category probability prediction result corresponding to the multimodal medical image.

[0028] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a medical image classification method and system based on a graph fusion spatial state model, which has the following beneficial effects: (1) The present invention adopts an overlapping stem embedding and adaptive downsampling design to improve feature representation capability while controlling the number of parameters and floating-point operations (FLOPs); (2) This invention integrates the structure-aware state space model module (SSM) and graph convolutional network through the MambaGNN block to achieve collaborative learning of local anatomical details and global topological relationships, ensuring accurate capture of dual-dimensional features; (3) Traditional SSMs lack explicit topological modeling. This invention adapts the topological features of different modal medical images through an 8-nearest neighbor graph convolution module. (4) Existing models are mostly optimized for a single modality, while this invention performs well on multiple datasets and has stronger adaptability; (5) In the MambaGNN block, the output of the structure-aware state space model, the output of the graph convolutional network and the output of the feedforward network are respectively residually connected to the corresponding inputs, so that the local spatial state features and the global topological features are transmitted and fused step by step, reducing the information decay in deep network training, improving the feature fusion efficiency and suppressing noise interference. (6) MambaGNN-T has only 27.8M parameters and 4.7G FLOPs, and its model parameter count and computational cost are far lower than those of Transformer models such as Swin-T. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the medical image classification method based on a graph fusion spatial state model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the framework of the medical image classification method based on graph fusion spatial state model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the visualization effect provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This invention discloses a medical image classification method based on a graph fusion spatial state model, such as... Figure 1 and Figure 2 As shown, it includes the following steps: S1. Acquire multimodal medical images and perform preprocessing; S2. Input the preprocessed multimodal medical image into the overlapping stem embedding layer, and generate initial features through convolutional embedding operation; S3. Input the initial features into the hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, each adjacent MambaGNN block is connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block; S4. Input the final enhanced features into the classification head and output the category probability prediction results corresponding to the multimodal medical image.

[0033] This method effectively addresses the problems of local-global modeling imbalance, low computational efficiency, and poor multimodal adaptability in existing models, providing an efficient and reliable technical solution for intelligent classification of medical images. It is applicable to intelligent classification and assisted diagnosis of multimodal medical images such as skin lesions, local X-rays, endoscopic images, and neck ultrasound. The following sections will provide a detailed explanation of each step.

[0034] In step S1 above, multimodal medical images are acquired, such as RGB three-channel dermoscopy images, grayscale X-ray images, and ultrasound images. The multimodal medical images are then preprocessed, including size adjustment, normalization, and data augmentation, to improve the robustness of the model. In step S2 above, the preprocessed multimodal medical image is denoted as... ,and ,in Represent real numbers, H Indicates altitude, W Indicates width; Multimodal medical images The input is fed into an overlapping stem embedding layer, where convolutional embedding operations generate a low-dimensional feature representation. This avoids information loss while reducing subsequent computational complexity, providing a foundation for hierarchical feature extraction. The low-dimensional feature representation output from the overlapping stem embedding layer is denoted as the initial feature. ,and , where C represents the number of channels; The aforementioned overlapping stem embedding layer is connected to the feature extraction network to process the initial features. The features are passed to the first layer of the feature extraction network, the MambaGNN block, to achieve initial encoding and unified mapping of the features.

[0035] In step S3 above, the initial features are input into a hierarchical feature extraction network for hierarchical feature extraction. This hierarchical feature extraction network includes multiple MambaGNN blocks (i.e., MambaGNN neural network blocks), with each pair of adjacent MambaGNN blocks connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block. Wherein: (1) MambaGNN block: The MambaGNN blocks mentioned above include two variants (Tiny / Small type), which can be configured with different drop path rates (0.2 / 0.3) and block numbers to adapt to different accuracy requirements.

[0036] Each MambaGNN block integrates a structure-aware state-space model module (SSM), a graph convolutional network (GNN), and a feedforward network (FFN); for example Figure 2 As shown, SSM, GNN, and FFN are connected sequentially according to the feature propagation order, and residual connections are used to preserve input information and enhance the stability of feature propagation. The following operations are performed in each MambaGNN block: 1) Perform layer normalization on the input features of the MambaGNN block to obtain normalized features; 2) The normalized features are input into the Structure-Aware State Space Model (SSM) module, which aggregates multi-scale local state features through multi-dilation rate convolution. Specifically, this SSM module, based on the SASF mechanism, selects convolution kernels with dilation rates of 1, 3, and 5 to aggregate neighborhood states, adapting to multi-scale anatomical local features in medical images. The multi-scale local state features are represented as follows:

[0037] in, Indicates the current time Multi-scale local state features; This represents the neighborhood set that participates in multi-scale aggregation; Representing the neighborhood The Middle Learnable fusion weights corresponding to each spatial location, and It is negatively correlated with the gray-level variance of neighboring pixels, thus suppressing noise interference; This represents the eigenvalue of the normalized feature at the spatial mapping location.

[0038] 3) Perform residual connections between the multi-scale local state features and the MambaGNN block input features to obtain the first residual connection features. , represented as:

[0039] in, Indicates the output mapping weights; Indicates the characteristics of the current state; This represents the input features of the MambaGNN block; Indicating residual jump terms The weights are determined. Linear transformation and residual completion are achieved through residual connections, enhancing the ability to capture local features.

[0040] 4) Input the first residual connection feature into the graph convolutional network and output the convolutional feature; specifically, the graph convolutional network is a global topological network constructed based on the 8 nearest neighbor graph; wherein the 8 nearest neighbor graph uses the spatial position of the feature grid as the node, covering the top, bottom, left, right and 45° diagonal association.

[0041] The above convolutional features are represented as follows:

[0042] in, The graph convolutional network represents the first... The node feature matrix of the layer, Indicates the first The convolutional features output by the layer; express The corresponding trainable weight matrix; Indicates the activation function; This represents the adjacency matrix after adding self-connections; Let A represent the degree matrix corresponding to A. This formula updates node features, explicitly models topological relationships, and adapts to the anatomical characteristics of different modalities of medical images.

[0043] 5) Perform a residual connection between the convolutional features and the first residual connection features to obtain the second residual connection features; 6) After performing layer normalization on the second residual connection feature, input it into the feedforward network, and perform residual connection between the output of the feedforward network and the second residual connection feature to output the enhanced feature of the current MambaGNN block.

[0044] By using the above-mentioned stepwise residual fusion method, it is possible to achieve the coordinated expression of local spatial state features and global topological features while preserving the original input information, thereby alleviating the gradient vanishing problem in the training process of deep networks and improving the discriminative ability of medical image classification features.

[0045] The augmented features output by the last MambaGNN block are denoted as the final augmented features, and this last MambaGNN block is connected to the classification head to input the final augmented features into the classification head.

[0046] (2) Downsampling layer: The downsampling layer is located between consecutive MambaGNN blocks. It is used to reduce the spatial dimension of the feature map and expand the channel dimension to build a hierarchical feature pyramid, balancing computational cost and feature richness, and adapting to the needs of lesion recognition at different scales.

[0047] This downsampling layer uses a convolution operation with a stride of 2 to reduce the spatial dimension of the feature map to half and expand the channel dimension to twice the size, forming a hierarchical feature extraction process that outputs feature representations at different scales in sequence.

[0048] In step S4 above, the final enhanced features are input into the classification head, and the predicted category probability results corresponding to the multimodal medical image are output. This classification head includes a global average pooling layer and a fully connected network. The global average pooling layer converts the final enhanced features into feature vectors of a preset dimension. The fully connected network then converts these feature vectors into class probabilities, outputting the classification result. The maximum value is taken as the final prediction result. This enables accurate classification of diseases or anatomical structures, providing decision support for clinical diagnosis.

[0049] In this embodiment of the invention, other graph neural network variants such as GAT can be used to replace the 8 nearest neighbor graph convolution module, which can also achieve topological relationship modeling and adapt to the accuracy and efficiency requirements of different scenarios.

[0050] In this embodiment of the invention, stride convolution or pooling operations can be used to replace the downsampling layer, flexibly adapting to different hardware environments while ensuring the dimensionality adjustment effect.

[0051] In this embodiment of the invention, a learnable channel transformation matrix can be used to replace convolutional embedding, thereby achieving adaptive adaptation to different modal inputs and improving model flexibility.

[0052] Based on the same inventive concept, embodiments of the present invention also provide a medical image classification system based on a graph fusion spatial state model, comprising: The data acquisition and processing unit is used to acquire multimodal medical images and perform preprocessing. The initial feature generation unit is used to input the preprocessed multimodal medical image into the overlapping stem embedding layer and generate initial features through convolutional embedding operations; The feature enhancement unit is used to input the initial features into the hierarchical feature extraction network for hierarchical feature extraction. The hierarchical feature extraction network includes multiple MambaGNN blocks, which are connected to each other through a downsampling layer, and the final enhanced features are output by the last MambaGNN block. The prediction result generation unit is used to input the final enhanced features into the classification head and output the category probability prediction results corresponding to the multimodal medical image.

[0053] Since the principle behind the problem solved by this system is similar to that of the aforementioned medical image classification method based on graph fusion spatial state model, the implementation of this system can be found in the implementation of the aforementioned method, and the repetitions will not be repeated.

[0054] The system is implemented in the form of a deep learning model, mainly consisting of trainable network parameters, modular functions, and data processing flow. Its core lies in the collaborative design of each module and the feature fusion mechanism.

[0055] In this embodiment of the invention, the model uses the AdamW optimizer during training and optimization, with a learning rate of 1e-4 and a weight decay of 0.05. The training strategy is a maximum of 12 epochs, a linear warm-up for the first 1000 steps, and a learning rate of 1 / 10 at 8 and 11 epochs. The model is jointly trained on multiple datasets to evaluate its classification accuracy and generalization performance.

[0056] This invention can be implemented on general-purpose GPUs, CPUs, or AI acceleration chips, without relying on specific types of integrated circuits; if deployed in an AI chip, inference acceleration can be achieved through CUDA, TensorRT, or ONNX Runtime to meet the needs of real-time clinical diagnosis.

[0057] The system of this invention implements modular loading and forward inference through a deep learning framework (such as PyTorch 2.0.1); the operation logic is as follows: input loading → data preprocessing → stem embedding encoding → hierarchical MambaGNN feature extraction → downsampling → global pooling → classification output.

[0058] The following examples will further illustrate the medical image classification method and system based on graph fusion spatial state model provided by this invention.

[0059] Example 1: System Overall Structure and Implementation Process: like Figure 2 As shown, the medical image classification system based on the graph fusion spatial Mamba provided by this invention includes an overlapping stem embedding layer, a hierarchical Mamba GNN block, a downsampling layer, and a classification head.

[0060] Overlapping stem embedding layers receive multimodal medical images and generate initial features; The hierarchical MambaGNN block extracts local and global features through the collaborative extraction of structure-aware SSM and graph convolution modules, which are then enhanced by a fusion mechanism. The downsampling layer adjusts the feature dimensions and constructs a feature pyramid; The classification head outputs the predicted category probability.

[0061] The system is implemented in a GPU environment using the PyTorch 2.0.1 framework. The model parameters are optimized through joint training on multiple datasets, making it suitable for various medical image classification tasks.

[0062] Example 2: Application in multimodal medical image classification tasks: To verify the classification performance and generalization ability of the present invention, the following were selected: Figure 3 A comparative experiment was conducted using the publicly available datasets PAD-UFES-20 (2298 skin lesion images), CPN X-ray (5228 chest X-ray images), and Fetal-Planes-DB (ultrasound images).

[0063] The experiment used a unified hardware environment (NVIDIA RTX 4090 GPU, Intel Xeon CPU, 64GB RAM) and software environment (PyTorch 2.0.1+cu118, Python 3.10, OpenCV 4.12.0), with the training and test sets divided in an 8:2 ratio.

[0064] (1) Experimental comparison setup: The following mainstream models were selected as comparison methods: ResNet-50: A classic convolutional neural network model; Swin-T: A hierarchical visual Transformer model; MedMamba-T: An SSM model optimized for medical imaging; The MambaGNN-T proposed in this invention is an improved structure that integrates structure-aware SSM and graph convolution modules, with a droppath rate of 0.2 and the number of blocks in each stage being 2, 4, 8, and 4.

[0065] (2) Performance comparison results: Experimental results show that the MambaGNN-T proposed in this invention achieves superior performance on all three types of datasets mentioned above: PAD-UFES-20 and Fetal-Planes-DB datasets: The model is able to highlight lesions or key anatomical regions in skin lesion images and fetal ultrasound images. The classification results are consistent with the heatmap localization results, demonstrating good cross-modal generalization ability. CPN X-ray dataset: AUC reaches 99.8%, significantly outperforming the comparison model; In terms of parameter count and floating-point operations (FLOPs), MambaGNN-T has only 27.8M parameters and 4.7G FLOPs, with a much lower parameter count and computational cost than Transformer models such as Swin-T.

[0066] (3) Results analysis: The results show that the structural design of this invention has significant advantages: the structure-aware SSM module effectively captures multi-scale local anatomical features, the graph convolution module accurately models global topological relationships, and the synergistic fusion of the two enhances the comprehensive representation capability of the model; the multi-level residual connection mechanism enhances feature transfer efficiency and suppresses noise interference; the low parameter count and low FLOPs design ensure the deployment flexibility of the model and meet the needs of real-time clinical diagnosis.

[0067] Therefore, the solution of this invention can maintain high accuracy, high efficiency and strong generalization in multimodal medical image classification tasks, and has good prospects for clinical application.

[0068] Visualization effect diagrams as follows Figure 3 As shown in the figure, this diagram demonstrates the model's ability to locate lesions on different medical images by comparing the results.

[0069] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0070] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image classification method based on a graph fusion spatial state model, characterized in that, include: Acquire multimodal medical images and perform preprocessing; The preprocessed multimodal medical images are input into an overlapping stem embedding layer, and initial features are generated through convolutional embedding operations. The initial features are input into a hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, each pair of adjacent MambaGNN blocks is connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block; The final enhanced features are input into the classification head, and the predicted category probability of the multimodal medical image is output.

2. The medical image classification method based on graph fusion spatial state model as described in claim 1, characterized in that, The multimodal medical images include at least RGB three-channel dermoscopy images, grayscale X-ray images, and ultrasound images.

3. The medical image classification method based on a graph fusion spatial state model as described in claim 1, characterized in that, Each MambaGNN block integrates a structure-aware state-space model module, a graph convolutional network, and a feedforward network; within each MambaGNN block: The MambaGNN block input features are subjected to layer normalization to obtain normalized features; The normalized features are input into the structure-aware state space model module, and multi-scale local state features are aggregated through multi-dilation rate convolution. The multi-scale local state features are residually connected to the MambaGNN block input features to obtain the first residual connection features; The first residual connection feature is input into the graph convolutional network, and the convolutional feature is output. The convolutional feature is residually concatenated with the first residual connection feature to obtain the second residual connection feature; After performing layer normalization on the second residual connection feature, it is input into the feedforward network, and the output of the feedforward network is residually connected with the second residual connection feature to output the enhanced feature of the current MambaGNN block.

4. The medical image classification method based on graph fusion spatial state model as described in claim 3, characterized in that, The multi-scale local state features are represented as follows: in, Indicates the current time Multi-scale local state features; This represents the set of domains participating in multi-scale aggregation; Representing the neighborhood The learnable fusion weights corresponding to the kth spatial location; This represents the eigenvalue of the normalized feature at the spatial mapping location.

5. The medical image classification method based on graph fusion spatial state model as described in claim 3, characterized in that, The graph convolutional network is a global topology network constructed based on an 8-nearest neighbor graph; the 8-nearest neighbor graph uses the spatial location of the feature grid as nodes, covering the top, bottom, left, right and 45° diagonal associations.

6. The medical image classification method based on a graph fusion spatial state model as described in claim 3, characterized in that, The convolutional features are represented as follows: in, This represents the graph convolutional network. The node feature matrix of the layer; Indicates the first The convolutional features output by the layer; express The corresponding trainable weight matrix; Indicates the activation function; This represents the adjacency matrix after adding self-connections; express The corresponding degree matrix.

7. The medical image classification method based on a graph fusion spatial state model as described in claim 1, characterized in that, The downsampling layer employs a convolution operation with a stride of 2, reducing the spatial dimension of the feature map to half and expanding the channel dimension to twice the size.

8. The medical image classification method based on a graph fusion spatial state model as described in claim 1, characterized in that, The classification head includes a global average pooling layer and a fully connected network; The final enhanced features are converted into feature vectors of a preset dimension through the global average pooling layer. The preset dimension feature vector is converted into category probabilities through the fully connected network, and the classification result is output.

9. A medical image classification system based on a graph fusion spatial state model, characterized in that, The method described by any one of claims 1-8 comprises: The data acquisition and processing unit is used to acquire multimodal medical images and perform preprocessing. The initial feature generation unit is used to input the preprocessed multimodal medical image into the overlapping stem embedding layer and generate initial features through convolutional embedding operations; The feature enhancement unit is used to input the initial features into a hierarchical feature extraction network for hierarchical feature extraction; the hierarchical feature extraction network includes multiple MambaGNN blocks, each pair of adjacent MambaGNN blocks is connected by a downsampling layer, and the final enhanced features are output by the last MambaGNN block; The prediction result generation unit is used to input the final enhanced features into the classification head and output the category probability prediction result corresponding to the multimodal medical image.

Images