Medical image classification unsupervised domain self-adaption and multi-granularity alignment system and method

Abstract

The invention provides a medical image classification unsupervised domain self-adaption and multi-granularity alignment system and a medical image classification unsupervised domain self-adaption and multi-granularity alignment method. According to the method, a coarse-to-fine progressive feature optimization strategy is provided, that is, a fine-grained feature encoder is initialized by using the training weight of a coarse-grained feature encoder, and alignment optimization of fine-grained features is guided while original semantic information is reserved. Specifically, a staged and layer-by-layer unfreezing fine adjustment method is adopted, and the adaptive capacity of fine-grained features is enhanced on the premise that the invariance of a coarse-grained domain is not damaged. According to the strategy, the generalization and convergence efficiency of the model on target domain data is remarkably improved. And experimental verification is carried out on a plurality of cross-domain medical image classification data sets. Results show that no matter on a small-scale data set or a large-scale data set, the method provided by the invention can obtain performance obviously superior to that of an existing method, and generalization ability and robustness are stronger.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an unsupervised domain adaptation and multi-granularity alignment system and method for medical image classification. Background Art

[0002] Unsupervised domain adaptation (UDA) is a potential solution to the domain shift problem, which uses labeled data from the source domain and unlabeled data from the target domain to train the model. The goal of the technique is to reduce the distribution difference between the labeled source data and the unlabeled target data, which is a promising approach to solve the domain shift problem.

[0003] In the field of medical image classification, most existing UDA methods mainly rely on image-level transformation or feature-level alignment to alleviate the problem of coarse-grained domain differences between different medical centers, while ignoring the extraction of fine-grained domain-invariant representations. In 2014, Tzeng et al. proposed the marginal distribution difference (MDD) method to solve the domain shift problem by reducing the feature distribution difference between the source domain and the target domain. In 2015, Ganin and Lempitsky proposed the domain adversarial neural network (DANN), which uses an adversarial learning framework for domain adaptation. In the same year, Long et al. proposed the deep adaptation network (DAN), which is also used to reduce the feature distribution difference between domains. In 2017, Tzeng et al. proposed the adversarial discriminative domain adaptation method (ADDA). In 2018, Long et al. proposed the conditional adversarial domain adaptation method (CDAN) based on DANN by considering the conditional distribution, and extended it by introducing the entropy minimization strategy (CDAN+E). In 2019, Chang et al. proposed the domain-specific batch normalization method (DSBN), which extracts domain-invariant features by providing independent batch normalization for different domains. In 2021, Wei et al. combined meta-strategy with unsupervised domain adaptation and proposed a task-oriented alignment method (ToAlign) to enhance the classification ability of the model. In addition, in terms of domain generalization, in 2018, Li et al. proposed a domain generalization method (MLDG) based on meta-learning, which improved the generalization ability of the model through meta-learning strategies. In 2019, Li et al. proposed a domain generalization method based on scenario training (Epi-FCR), and Carlucci et al. proposed a method to achieve domain generalization by solving jigsaw problems (JiGen). In 2021, Zhou et al. proposed the MixStyle method to enhance the generalization of the model by mixing style information from different domains.

[0004] The above-mentioned existing unsupervised domain adaptation methods mainly guide the alignment of source and target domain data from two aspects: 1) image-level alignment, which retains the content features of the source domain image while transforming its appearance to match the style of the target domain; 2) feature-level alignment, which projects both the source and target domains into a common latent space by learning cross-domain invariant features. For image-level alignment, researchers mainly use deep learning-based CycleGAN or statistical Fourier transform technology to achieve image appearance transformation. However, CycleGAN-based methods may cause interruptions to the image structure, while Fourier transform cannot capture high-level semantic information in medical images, which may seriously affect the performance of the model. In order to circumvent the substantial challenges associated with image-level alignment, some studies have used feature-level alignment methods using statistical distribution distance metrics or adversarial learning.

[0005] Although the methods mentioned above are effective in aligning overall features in a common domain, they often rely too much on a single coarse-grained alignment space and ignore the information existing at different granularities, such as image texture and spatial relationships, which are crucial for accurate medical diagnosis. Therefore, in the process, they may fail to align key fine-grained features, resulting in poor model performance in more complex medical imaging cases. Summary of the invention

[0006] The purpose of the present invention is to provide a medical image classification unsupervised domain adaptation and multi-granularity alignment system and method to improve the classification generalization performance of the model on medical images in different fields.

[0007] The present invention is achieved in that:

[0008] In view of the fact that existing methods are difficult to effectively solve the problem of differences in distribution and feature representation of medical images in different domains, the present invention proposes an unsupervised domain adaptation and multi-granularity alignment system and method for medical image classification. After extracting the source domain and target domain features and transmitting the weight information, the dual-feature decoupling module is used to extract the coarse-grained domain invariant features and task discriminant features of the image, which preliminarily reduces the distribution difference between the source domain and the target domain; then a fine-grained feature alignment module is designed to further extract fine-grained structural semantic features through local graph convolution to make up for the shortcomings of coarse-grained global features; finally, a dynamic multi-granularity feature fusion mechanism is introduced into the classification model to adaptively combine the coarse and fine-grained complementary feature representations to achieve end-to-end optimization of domain adaptive classification tasks.

[0009] The medical image classification unsupervised domain adaptation and multi-granularity feature alignment system provided by the present invention comprises:

[0010] An image collection module is used to collect medical image data and form a source domain and a target domain, where the source domain data has classification labels and the target domain data is unlabeled;

[0011] The data preprocessing module is used to perform preprocessing operations such as normalization, random cropping, and data enhancement on medical images in the source and target domains;

[0012] A coarse-grained feature extraction module, including a coarse-grained feature encoder and a dual-feature decoupling learning module; the coarse-grained feature encoder is used to encode the input image features using a pre-trained convolutional neural network and extract high-level semantic features; the dual-feature decoupling learning module is used to extract domain-invariant features and discriminant features;

[0013] The fine-grained feature extraction module is used to further mine the fine-grained texture, structure and other features of the image based on the coarse-grained features to make up for the shortcomings of the global features;

[0014] The fine-grained feature extraction module includes a fine-grained feature encoder and a fine-grained feature alignment module;

[0015] The weight transfer module is used to transfer the training weights of the coarse-grained feature encoder to the fine-grained feature encoder as its initialization parameters to inherit the learned semantic knowledge;

[0016] The fine-grained feature alignment module is used to cut the image into local patches, and then use the graph convolutional neural network to model the patch nodes and refine the regional semantic features through neighborhood information propagation and aggregation.

[0017] In the above scheme, preferably, the fine-grained feature alignment module adopts a layer-by-layer unfreezing fine-tuning strategy to smoothly and efficiently achieve the coordinated alignment of coarse and fine-grained features.

[0018] In the above scheme, preferably, the coarse-grained feature encoder adopts mainstream network structures such as Kan and Mamba to map the input image to a high-dimensional feature space.

[0019] In the above scheme, preferably, the dual-feature decoupling learning module extracts domain-invariant features by minimizing the feature distance of samples of the same category and maximizing the feature distance of samples of different categories, and uses the self-attention mechanism to adaptively refine the discriminative features most relevant to downstream tasks.

[0020] In the above scheme, preferably, the dual-feature decoupling learning module specifically includes:

[0021] The domain-invariant feature extraction submodule based on contrastive learning obtains domain-invariant representations that are semantically consistent but have large appearance differences by minimizing the feature distance of samples in the same category and maximizing the feature distance of samples in different categories.

[0022] The discriminative feature extraction submodule based on meta-learning uses the self-attention mechanism to adaptively extract the discriminative features most relevant to downstream tasks from the domain-invariant representation.

[0023] In the above solution, preferably, the fine-grained feature alignment module specifically includes:

[0024] The slicing layer is used to divide the image into local patches of fixed size (such as 3×3, 16×16, etc.);

[0025] The graph convolution layer is used to model local patches using a graph convolutional neural network and to extract the structural semantic features of local areas through the propagation and aggregation of neighborhood information;

[0026] The pooling layer is used to globally pool the output feature map of the graph convolution to obtain a fine-grained feature representation that is consistent with the scale of the coarse-grained features.

[0027] In the above scheme, preferably, the specific implementation of the weight transfer module is:

[0028] First, a coarse-grained feature encoder is pre-trained on the source domain annotated data to learn preliminary domain-invariant features and discriminative features;

[0029] Then, the weight parameters of the trained coarse-grained feature encoder are passed to the fine-grained feature encoder as its initialization parameters.

[0030] In the above scheme, preferably, in the fine-grained feature extraction stage, a step-by-step, layer-by-layer unfreezing fine-tuning strategy is adopted: first fix the coarse-grained encoder parameters, and focus on optimizing the graph convolution part to learn complementary fine-grained features; then gradually unfreeze the parameters of each layer of the coarse-grained encoder, and perform joint fine-tuning to improve the adaptability of the overall features without destroying the original domain invariance.

[0031] Furthermore, the specific implementation of the dual-feature decoupling learning module is as follows:

[0032] A contrastive learning method based on cross-entropy loss is used to extract domain-invariant components from encoded features by minimizing the feature distance of samples in the same category and maximizing the feature distance of samples in different categories.

[0033] A meta-learning paradigm based on the self-attention mechanism is adopted to adaptively extract the discriminative components most relevant to downstream tasks from the domain-invariant representation, accelerating the model's adaptation to new domain data.

[0034] The present invention solves the domain adaptation problem in medical image classification by using a multi-granularity feature alignment method, and realizes feature alignment in both coarse-grained feature space and fine-grained feature space. The constructed dual-feature decoupling learning module effectively extracts domain-invariant and discriminative features, and the fine-grained feature alignment module fully mines fine-grained information such as texture and edge. The present invention can significantly improve the classification performance of the model on target domain data, and is expected to be applied to medical scenarios such as auxiliary disease diagnosis.

[0035] The present invention also provides the following technical solution:

[0036] A method for unsupervised domain adaptation and multi-granularity alignment of medical image classification, comprising the following steps:

[0037] S1. Build the above-mentioned unsupervised domain adaptation and multi-granularity alignment system for medical image classification;

[0038] S2. Collect medical image data to form a source domain and a target domain, where the source domain data has classification labels and the target domain data is not labeled; perform preprocessing operations on the original images, including normalization, random cropping, data enhancement, etc., to improve data quality and enrich sample diversity;

[0039] S3. Use the coarse-grained feature extraction module to encode the preprocessed images: use contrastive learning to extract domain-invariant features, minimize the feature distance of samples of the same category, and maximize the feature distance of samples of different categories; use meta-learning to adaptively refine the most relevant discriminative features for downstream tasks;

[0040] S4. passing the weights of the trained coarse-grained feature encoder to the fine-grained feature extraction module as its initialization parameters;

[0041] S5. Extract fine-grained features using the fine-grained feature alignment module: Divide the image into local patches, then use a graph convolutional neural network to model the topological relationship between patches, and extract regional semantic features through neighborhood information propagation and aggregation;

[0042] S6. Use a staged, layer-by-layer unfreezing fine-tuning strategy to train the fine-grained feature extraction module: first fix the coarse-grained feature encoder parameters and focus on optimizing the graph convolution part to learn complementary fine-grained features; then gradually unfreeze the coarse-grained encoder parameters for joint fine-tuning to improve the domain adaptability of the overall feature while preserving the semantic information;

[0043] S7. The coarse-grained features and fine-grained features are fused and input into the classifier. Through the joint optimization of multiple adversarial loss functions, the distribution difference between the source domain and the target domain is further reduced, thereby improving the generalization performance of the classification model.

[0044] S8. Evaluate the classification performance of the model on the target domain samples, calculate indicators such as classification accuracy and AUC value, and compare them with existing methods to verify the superiority of the method of the present invention.

[0045] Compared with the prior art, the advantages of the present invention are as follows:

[0046] 1. This paper innovatively proposes a multi-granularity domain adaptation framework, which comprehensively reduces the inter-domain differences in terms of semantic level and spatial dimension by collaboratively aligning coarse-grained global features and fine-grained local features, thereby enhancing the domain adaptability and robustness of the model.

[0047] 2. The present invention uniquely designs a coarse-to-fine progressive feature optimization strategy, which fully utilizes the semantic knowledge learned by the coarse-grained feature encoder to guide the fine-grained feature extraction, and smoothly and efficiently realizes the coordinated alignment of coarse-grained and fine-grained features through weight transfer and layer-by-layer fine-tuning, effectively overcoming the limitations of separate training of coarse-grained and fine-grained features, and improving the quality and efficiency of feature alignment.

[0048] 3. The present invention cleverly improves the network structure and learning paradigm of each key module, such as using contrastive learning to obtain domain-invariant representation, meta-learning to refine discriminative components, graph convolution to capture structural information, and dynamic adjustment of multiple adversarial losses. It strengthens the domain invariance, discriminability of features and generalization of the model in multiple dimensions, providing new ideas and solutions for cross-domain medical image classification tasks.

[0049] The present invention integrates innovative points such as coarse-grained and fine-grained complementary perspectives, progressive optimization alignment strategy, and graph convolutional structure modeling to form a comprehensive and efficient multi-granularity domain adaptive classification framework for medical images. It has achieved a performance improvement of 5%-10% significantly better than existing methods on multiple cross-domain benchmark datasets, providing a new technical path for the cross-scenario promotion and application of models in the field of medical AI, and has broad application prospects and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the overall flow chart of the method of the present invention.

[0051] Figure 2 Schematic diagram of the structure of the dual-feature decoupling learning module.

[0052] Figure 3 Schematic diagram of the structure of the fine-grained feature alignment module.

[0053] Figure 4 It is a schematic diagram of the experimental results of Example 1 of the present invention on abdominal CT images. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below with reference to specific embodiments and drawings.

[0055] Figure 1 It is the overall flow chart of the present invention, which shows in detail the process from feature extraction to weight transfer, to dual-feature decoupling learning module and fine-grained feature alignment module, and finally to output results.

[0056] Example 1

[0057] An unsupervised domain adaptation and multi-granularity alignment system for medical image classification, comprising the following steps:

[0058] Step 1: Image collection. Two datasets were constructed, which came from hospitals A, B, and C. Specifically, the GEDiscovery HD750 64-slice CT scanner of Hospital A contained 1938 CT slices of 101 patients with adrenal adenomas and 92 patients with adrenal metastases, and 3771 slices of 50 patients with non-injurious pneumonia and 35 patients with injurious pneumonia; the Philips 256-slice Brilliance iCT scanner of Hospital B contained 1052 CT slices of 31 patients with adrenal adenomas and 40 patients with adrenal metastases; the Siemens SOMATOM Perspective scanner of Hospital C contained 4664 slices of 74 patients with non-injurious pneumonia and 37 patients with injurious pneumonia. The data of the same disease from different hospitals are each other's source domain and target domain. The hospitals use different CT scanners and imaging protocols, and there is an obvious domain shift problem.

[0059] Step 2: Image annotation. Using the image annotation module, three experienced radiologists finely delineated the tumor area of ​​the source domain CT image to obtain pixel-level segmentation labels. The majority voting method was used to integrate the annotation results of the three doctors to guide the training of the domain adaptation model.

[0060] Step 3: Data preprocessing. The data preprocessing module is used to perform preprocessing operations such as normalization, random cropping, and data enhancement on the source and target domain images. Normalization scales the pixel values ​​to the range [0, 1], the cropping size is set to 224×224, and data enhancement includes random flipping, rotation of ±10°, and Gaussian noise (σ=0.01). After enhancement, the number of source and target domain samples is expanded to 5000 respectively.

[0061] Step 4: Coarse-grained feature extraction.

[0062] The coarse-grained feature extraction module includes a coarse-grained feature encoder and a dual-feature decoupling learning module.

[0063] Use the pre-trained Kan as the coarse-grained feature encoder (i.e. Figure 1 Encoder 1 in , extracts high-level semantic features from the preprocessed source and target domain images.

[0064] Then use Figure 2 The dual-feature disentangled learning module shown further extracts domain-invariant features and discriminative features:

[0065] (1) Contrastive learning submodule: Figure 2The left part shows that based on the encoded features, the domain-invariant representation with the same semantics but different appearances is obtained by minimizing the feature distance of samples of the same category (i.e., positive sample pairs) and maximizing the feature distance of samples of different categories (i.e., negative sample pairs).

[0066] (2) Meta-learning submodule: Figure 2 The right part shows that the self-attention mechanism and MAML paradigm are used to quickly and adaptively extract the discriminative components most relevant to the tumor classification task from the domain-invariant representation.

[0067] Step 5: Weight transfer.

[0068] like Figure 1 As shown between encoder 1 and encoder 2, the weight parameters of the trained coarse-grained feature encoder Kan are passed to the fine-grained feature extraction module as its initialization parameters to inherit the semantic features learned by the coarse-grained network.

[0069] The fine-grained feature extraction module includes a fine-grained feature encoder (ie, encoder 2) and a fine-grained feature alignment module.

[0070] Step 6: Fine-grained feature extraction.

[0071] Combination Figure 1 and Figure 3 , the preprocessed source domain and target domain images are cut into pieces, and then the fine-grained feature encoder (i.e. encoder 2) initializes the parameters of the cut images. Then the fine-grained feature alignment module is used to fully mine the fine-grained information of the image, such as texture and edge.

[0072] like Figure 3 As shown in the figure, based on the coarse-grained image, a 16×16 image patch is obtained by local cutting, and a topological map between patches is constructed. The graph convolutional neural network is used to capture fine-grained features such as texture and edges in the local area. The graph convolution layer aggregates the information of neighboring nodes layer by layer to extract high-order semantics. The pooling layer integrates the graph convolution output to obtain a fine-grained representation with the same scale as the coarse-grained features.

[0073] Step 7: Progressive feature optimization.

[0074] We adopt a fine-tuning strategy of staged and layer-by-layer unfreezing. We first freeze the coarse-grained feature encoder parameters and train only the graph convolution module for 4 epochs, so that it can focus on learning complementary fine-grained features under fixed coarse-grained features. Then, we unfreeze Kan's parameters layer by layer (such as the last 1-2 blocks) and fine-tune them jointly for 8 epochs with a smaller learning rate, so as to further improve the adaptability of the overall features in the target domain without destroying the original semantic features.

[0075] Step 8: Multi-granularity feature fusion and adversarial training.

[0076] The fine-tuned coarse-grained and fine-grained features are cascaded and input into the classifier. The classifier uses weighted cross entropy loss to learn tumor discrimination ability under the supervision of source domain labels. At the same time, the attention mechanism is introduced to dynamically adjust the weights of coarse-grained and fine-grained features, so that the classifier focuses on the most discriminative semantic components. Through domain adversarial training for 20 epochs, the distribution difference between the source domain and the target domain is further narrowed, and the generalization performance of the classification model is improved.

[0077] Step 9: Model evaluation.

[0078] The model performance was evaluated on the target domain clinical data. The fine-tuned model was used for tumor classification prediction of the target domain images. A 5-fold cross validation was used to calculate the average classification accuracy (ACC), precision (PRE), sensitivity (SEN), specificity (SPE), area under the curve (AUC) and other indicators. The results were compared with mainstream domain adaptation methods such as traditional transfer learning, adversarial learning, and meta-learning. The results are shown in Tables 1 and 2. The baseline method in the table refers to the method with only ordinary encoders and decoders, and does not distinguish between coarse-grained and fine-grained. In each table, the left side of the arrow is the source domain, and the right side is the target domain.

[0079] Table 1 Comparison of the results of classification and prediction of adrenal adenoma metastasis data using various methods

[0080]

[0081]

[0082] As can be seen from Table 1, the multi-granularity domain adaptation method of the present invention improves the classification accuracy, precision, sensitivity, specificity, and AUC value of the adrenal gland dataset, respectively, which confirms that multi-granularity feature alignment significantly improves the cross-domain generalization ability of the model.

[0083] Table 2 Comparison of the results of classification prediction of pneumonia data using various methods

[0084]

[0085] As can be seen from Table 2, the multi-granularity domain adaptation method of the present invention improves the classification accuracy, precision, sensitivity, specificity, and AUC value of the pneumonia dataset, respectively, which confirms that multi-granularity feature alignment significantly improves the cross-domain generalization ability of the model.

[0086] This embodiment also tests the ablation effects of different modules in the invention strategy, as shown in Table 3:

[0087] Table 3

[0088]

[0089]

[0090] Figure 4 This is the result diagram of the abdominal CT image from the source domain to the target domain in Example 1. The accuracy of the classification result of the method of the present invention can be seen from the figure.

[0091] As can be seen from Example 1, the unsupervised classification migration of abdominal and lung CT images between different hospitals is realized by using the method of the present invention, which provides an effective solution for the cross-domain deployment of medical image intelligent diagnosis systems. The present invention can be applied to scenarios such as remote consultation and disease screening between medical institutions, and has broad application prospects.

[0092] Example 2

[0093] This example applies the present invention to cross-device classification and segmentation diagnosis of adrenal MRI images to demonstrate the generalizability and practicality of the method.

[0094] Step 1: Data collection and preprocessing.

[0095] The GE Discovery HD750 64-slice CT scanner of Hospital A scanned 1938 CT slices of 101 patients with adrenal adenomas and 92 patients with adrenal metastases; the Philips 256-slice Brilliance iCT scanner of Hospital B scanned 1052 CT slices of 31 patients with adrenal adenomas and 40 patients with adrenal metastases. The data of the two hospitals were mixed together and divided into three data sets of U, V, and A according to different CT phases (non-enhanced phase, venous phase, and arterial phase). This embodiment implements adaptive generalization experiments in different domains, and tests the generalization of the two data sets on the third one. The adrenal glands preprocessed the original images with N4 bias field correction, standardization, image registration, etc. Considering that the proportion of the adrenal region in the entire pelvic MRI is relatively small, the ROI detection method based on deep learning is used to automatically extract the adrenal region and resize it to a fixed size of 256×256. On this basis, the sample sizes of the source and target domains are expanded respectively through data enhancement methods such as elastic deformation, Fourier transform, gamma correction, and Gaussian noise. At the same time, the segmentation annotations of the source domain are aligned with the enhanced image as pixel-level supervision information.

[0096] Step 2: Coarse-grained feature extraction.

[0097] The Mamba model pre-trained on a large-scale adrenal multi-parametric CT dataset is used as a coarse-grained feature encoder (i.e. Figure 1The encoder (a) extracts high-level semantic features of the source and target domain images. The mamba model uses the U-Net architecture and embeds self-attention and multi-scale fusion mechanisms in the encoder part, which can model the anatomical structure and texture features of the adrenal region from local to global perspectives.

[0098] On this basis, the dual-feature decoupling learning module is used to further extract domain-invariant and discriminative components:

[0099] (1) Contrastive learning submodule: A supervised contrastive learning paradigm is adopted, with pixels of the same lesion category embedded as positive sample pairs and pixels of different categories embedded as negative sample pairs. By minimizing the distance of positive sample pairs and maximizing the distance of negative sample pairs, a domain-independent pixel-level representation is obtained.

[0100] (2) Meta-learning submodule: It adopts a metric-based meta-learning method to model the intra-class similarity and inter-class difference of pixel embeddings through matching networks and relation networks, respectively, to quickly adapt to the target domain data and extract the specific representations that are most relevant to the classification and segmentation tasks.

[0101] Step 3: Weight transfer.

[0102] The weights of the mamba feature encoder obtained by training on the source domain for 30 epochs are passed to the fine-grained feature extraction module (including encoder 2 and fine-grained feature alignment module) as initialization parameters. Considering the anatomical characteristics of the adrenal region, in addition to passing all the weights of the encoder, some weights corresponding to the adrenal region in the decoder (such as the central cropped area) are also passed in a refined manner to inherit the coarse-grained semantic information more specifically.

[0103] Step 4: Fine-grained feature extraction.

[0104] First, the source domain and target domain images are cut into pieces, and then the cut images are input into the fine-grained feature encoder (i.e. Figure 1 In encoder 2), the fine-grained feature encoder is used to initialize the parameters of the sliced ​​image, and then outputs it to the fine-grained feature alignment module. The fine-grained feature alignment module further mines the fine-grained texture, structure and other features of the image based on the coarse-grained features to make up for the shortcomings of the global features.

[0105] The fine-grained feature alignment module uses pyramid pooling to extract local area blocks with different receptive fields on the multi-scale feature map output by encoder 2, and constructs a spatial topology map between area blocks. The low-dimensional latent variable representation of the area block is obtained through graph convolution and downsampling, and the original area block is reconstructed through upsampling and deconvolution to achieve unsupervised modeling and learning of the local structural texture of the adrenal gland. At the same time, contrast loss is introduced to promote the clustering and separation of different lesion areas in the embedding space. Finally, the multi-scale fine-grained features are integrated through the graph pooling layer, and a feature vector of the same length as the coarse-grained feature encoder is output.

[0106] Step 5: Progressive feature optimization.

[0107] First, all parameters of the mamba encoder are frozen and only fine-tuned for 6 epochs to focus on mining fine-grained information not covered by coarse-grained features. Then, a progressive unfreezing strategy is adopted to unfreeze the parameters of the mamba encoder layer by layer every 2 epochs (first unfreeze the shallow layer, then unfreeze the deep layer), and the learning rate of fine-grained optimization is reduced to guide the coordinated adaptation of coarse-grained and fine-grained features. The entire fine-tuning process lasts for 15 epochs, achieving cross-domain alignment of multi-grained features in a smooth and meticulous manner.

[0108] Step 6: Multi-granularity feature fusion and adversarial training.

[0109] The mamba decoder is used as the segmentation branch, and the fine-tuned coarse-grained and fine-grained features are fused and input into the classifier to form an end-to-end multi-task domain adaptation network. The classifier combines cross entropy loss and cluster regularization loss to learn the ability to discriminate lesions under the supervision of source domain labels, while minimizing the intra-lesion variance under the clustering assumption and enhancing the compactness of the classification boundary. The segmentation branch uses Tversky loss and intersection-over-union loss to balance the over-detection and under-detection of small lesions. Through joint multi-task adversarial training for 20 epochs, the classification and segmentation performance are guaranteed while minimizing the domain difference.

[0110] Step 7: Model evaluation.

[0111] The model performance was evaluated on the unit test sets of the source domain and the target domain, and the average classification accuracy, precision, sensitivity, specificity, and AUC values ​​of all cases were calculated, as shown in Table 4 below.

[0112] Table 4

[0113]

[0114] The results of the adrenal gland experiment show that the accuracy, sensitivity, specificity and other indicators of the method of the present invention on different target domain data are relatively good, and it has obvious advancedness. The present invention has shown excellent performance and application potential in adrenal gland image analysis, and provides a powerful tool for cross-device and cross-center intelligent diagnosis of tumors. Compared with the existing technology, the main advantages are:

[0115] (1) Aiming at the anatomical characteristics of adrenal CT images, the advanced Mamba model is flexibly used to fully exploit the fine-grained structural texture features while inheriting the coarse-grained semantic information, thus achieving a comprehensive enhancement of feature representation.

[0116] (2) The progressive parameter transfer and unfreezing fine-tuning strategy carefully considers the differences between coarse-grained and fine-grained features, optimizes smoothly based on orderly inheritance, and maximizes the effectiveness of complementary multi-granularity learning.

[0117] (3) Innovatively introduced contrastive meta-learning, clustering regularization, Tversky loss and other mechanisms, strengthened the focus on difficult-to-distinguish samples and inter-class discrimination, and further enhanced the joint training effect of domain adaptation and classification segmentation.

[0118] (4) The method has been comprehensively expanded in terms of its applicability to multiple downstream tasks, such as adaptation in different modality domains, opening up a new paradigm for the radiomics analysis of adrenal diseases and facilitating personalized and precise diagnosis and treatment based on radiomics.

Claims

1. An unsupervised domain adaptation and multi-granularity alignment system for medical image classification, characterized by: include: An image collection module is used to collect medical image data and form a source domain and a target domain, where the source domain data has classification labels and the target domain data is unlabeled; The data preprocessing module is used to perform normalization, random cropping, and data enhancement preprocessing operations on medical images in the source domain and the target domain; A coarse-grained feature extraction module, including a coarse-grained feature encoder and a dual-feature decoupling learning module; The coarse-grained feature encoder is used to encode the input image features using a pre-trained convolutional neural network and extract high-level semantic features; the dual-feature decoupling learning module is used to extract domain-invariant features and discriminative features; Fine-grained feature extraction module, used to mine fine-grained texture and structural features of images; The weight transfer module is used to transfer the training weights of the coarse-grained feature encoder to the fine-grained feature extraction module as its initialization parameters.

2. The unsupervised domain adaptation and multi-granularity alignment system for medical image classification according to claim 1, characterized in that: The fine-grained feature extraction module includes a fine-grained feature encoder and a fine-grained feature alignment module; the fine-grained feature encoder is used to receive the training weights of the coarse-grained feature encoder transmitted by the weight transfer module and initialize relevant parameters; the fine-grained feature alignment module is used to cut the image into local patches, and then use the graph convolutional neural network to model the patch nodes, and refine the regional semantic features through neighborhood information propagation and aggregation.

3. The unsupervised domain adaptation and multi-granularity alignment system for medical image classification according to claim 2, characterized in that: The fine-grained feature alignment module includes: The slicing layer is used to divide the image into local patches of fixed size; The graph convolution layer is used to model local patches using a graph convolutional neural network and to extract the structural semantic features of local areas through the propagation and aggregation of neighborhood information; The pooling layer is used to globally pool the output feature map of the graph convolution to obtain a fine-grained feature representation that is consistent with the scale of the coarse-grained features.

4. The medical image classification unsupervised domain adaptation and multi-granularity alignment system according to claim 1, characterized in that: The dual-feature decoupling learning module specifically includes: The domain-invariant feature extraction submodule based on contrastive learning obtains domain-invariant representations that are semantically consistent but have large appearance differences by minimizing the feature distance of samples in the same category and maximizing the feature distance of samples in different categories. The discriminative feature extraction submodule based on meta-learning uses the self-attention mechanism to adaptively extract the discriminative features most relevant to downstream tasks from the domain-invariant representation.

5. The medical image classification unsupervised domain adaptation and multi-granularity alignment system according to claim 1, characterized in that: The coarse-grained feature encoder adopts a Kan or mamba network structure.

6. The medical image classification unsupervised domain adaptation and multi-granularity alignment system according to claim 2, characterized in that: The fine-grained feature alignment module adopts a layer-by-layer unfreezing fine-tuning strategy to smoothly and efficiently achieve the coordinated alignment of coarse and fine-grained features.

7. An unsupervised domain adaptation and multi-granularity alignment method for medical image classification, characterized in that: The steps include: S1. Constructing the medical image classification unsupervised domain adaptation and multi-granularity alignment system according to any one of claims 1 to 6; S2. Collect medical image data to form a source domain and a target domain, where the source domain data has classification labels and the target domain data is not labeled; perform preprocessing operations on the original image, including normalization, random cropping, and data enhancement; S3. Use the coarse-grained feature extraction module to encode the preprocessed images: use contrastive learning to extract domain-invariant features, minimize the feature distance of samples of the same category, and maximize the feature distance of samples of different categories; use meta-learning to adaptively refine the most relevant discriminative features for downstream tasks; S4. passing the weights of the trained coarse-grained feature encoder to the fine-grained feature extraction module as its initialization parameters; S5. Extract fine-grained features using the fine-grained feature alignment module: Divide the image into local patches, then use a graph convolutional neural network to model the topological relationship between patches, and extract regional semantic features through neighborhood information propagation and aggregation; S6. Fusion of coarse-grained features and fine-grained features is input into the classifier to optimize the loss function, reduce the distribution difference between the source domain and the target domain, and improve the generalization performance of the classifier; S7. Evaluate the classification performance of the classifier on target domain samples.

8. The unsupervised domain adaptation and multi-granularity alignment method for medical image classification according to claim 7, characterized in that: In the process of fine-grained feature extraction, a staged, layer-by-layer unfreezing fine-tuning strategy is adopted to train the fine-grained feature extraction module: first, the coarse-grained feature encoder parameters are fixed, and the graph convolution part is optimized to learn complementary fine-grained features; then the coarse-grained encoder parameters are gradually unfrozen for joint fine-tuning to improve the domain adaptability of the overall features while retaining the semantic information.

Images