A lung infection image classification method based on adaptive neural architecture search

Abstract

This invention discloses a lung infection image classification method based on adaptive neural architecture search. The method first acquires multi-source chest CT image data, performs preprocessing and feature extraction to obtain a fused feature vector F. Next, it constructs a neural architecture search space based on temperature-adaptive Gumbel Softmax, generating a candidate architecture pool. Then, using the fused feature vector F as input data, a multi-constraint, two-layer optimization strategy is employed to select the best-performing neural architecture search space from the architecture pool through Pareto optimal solutions. Finally, based on the best-performing neural architecture, training and testing are performed using a dataset to achieve lung infection image classification. This invention can automatically discover deep learning architectures optimized for multi-classification tasks of lung infections, achieving high-precision automatic classification of various lung infection images.

Description

Technical Field

[0001] This invention belongs to the fields of medical image processing and artificial intelligence technology, and specifically relates to a lung infection image classification method based on adaptive neural architecture search, which can identify images of various lung infections. Background Technology

[0002] Lung infections are a serious threat to human health worldwide, including pneumonia caused by various pathogens such as COVID-19, bacterial pneumonia, viral pneumonia, and tuberculosis. Accurate and rapid classification of lung infections is crucial for developing targeted treatment plans and controlling disease transmission.

[0003] Currently, the diagnosis of lung infections mainly relies on chest CT imaging analysis and laboratory tests. However, different types of lung infections have some overlap in their imaging manifestations, making manual image interpretation inefficient and difficult to meet clinical needs in situations such as large-scale epidemics or when medical resources are strained.

[0004] Existing deep learning-based methods for detecting lung infections suffer from significant technical limitations. First, they lack the ability to classify multiple diseases; existing methods often employ binary classification strategies, failing to accurately distinguish between various types of lung infections. Second, their static architecture design is limited; traditional methods rely on manually designed, fixed network architectures, unable to adaptively adjust the network structure based on the pathological characteristics of different lung infections. Furthermore, they suffer from a lack of feature representation, primarily relying on single-modality CT image features, failing to fully utilize the multidimensional pathological information of lung infections. Finally, they lack clinical adaptability, lacking optimization for different hospitals, equipment, and scanning parameters, often resulting in a significant performance degradation in real-world clinical applications.

[0005] While neural architecture search technology offers a new approach to automatically discovering high-performance network architectures, the application of existing methods in the field of medical imaging still faces challenges such as inappropriate search space design, difficulties in multi-objective optimization, and insufficient consideration of clinical constraints. Summary of the Invention

[0006] The purpose of this invention is to provide a lung infection image classification method based on adaptive neural architecture search. This method, through key technological innovations such as multimodal feature fusion, temperature-adaptive Gumbel Softmax (a differentiable discrete sampling method) sampling, and multi-constraint neural architecture search, can automatically discover deep learning architectures optimized for multi-classification tasks of lung infection images, and achieve high-precision automatic classification of various lung infection images.

[0007] To achieve the above objectives, this invention provides a lung infection image classification method based on adaptive neural architecture search, comprising the following steps:

[0008] Step 1: Multimodal medical image data preprocessing and feature extraction. Obtain multi-source chest CT image data, perform preprocessing and feature extraction, and obtain the fused feature vector F.

[0009] The system receives multi-source chest CT image data, performs standardized preprocessing operations, extracts multi-dimensional feature vectors related to lung infection, and forms a unified feature representation space.

[0010] The multimodal data preprocessing first standardizes the image format, supporting automatic recognition and conversion of formats such as DICOM, NIfTI, PNG, and TIFF, converting all image data into a standard three-dimensional tensor format. Then, a U-Net++ network is used to segment the coarse lung parenchyma region, followed by density feature constraints and morphological post-processing to obtain a refined lung parenchyma region for subsequent analysis.

[0011] The feature extraction process extracts three types of feature vectors from the refined lung parenchyma region. The texture feature vector T uses a gray-level co-occurrence matrix to calculate features such as contrast and correlation, combined with rotation-invariant features of local binary patterns, to describe the microstructural features of lung tissue. The morphological feature vector M extracts geometric features such as lesion area, perimeter, and eccentricity, as well as morphological information such as edge roughness and spatial distribution, through connected component analysis. The density distribution feature vector D divides the HU (Huntersfield unit) value range into 64 intervals, statistically analyzing the voxel distribution proportion in each interval to reflect the density variation pattern of lung tissue. The three types of features are standardized by Z-scores and then concatenated to form a fused feature vector F = [T, M, D], which serves as input data for subsequent adaptive neural architecture search.

[0012] Step 2: Construct a neural architecture search space based on temperature-adaptive Gumbel Softmax and generate a pool of candidate architectures.

[0013] Using the fusion feature vector F extracted in step one as input, a neural architecture search space optimized for the characteristics of the feature vector is constructed. Temperature-adaptive Gumbel Softmax technology is used to achieve differentiable architecture parameter optimization, generating a diverse pool of candidate neural architectures, which is a set of candidate neural network architectures.

[0014] The neural architecture search space uses the fused feature vector F as a unified input to all candidate network architectures. The search objective is to construct a diverse set of neural architecture candidates capable of effectively processing this fused feature vector F. The neural architecture search space design comprises four layers: micro-operation layers, computational block structure layers, cellular structure layers, and macro-architecture layers. The micro-operation layer defines m candidate operations, including 1×1×1 3D convolution, 3×3×3 depthwise separable 3D convolution, residual connections, batch normalization, ReLU activation, and global average pooling. The computational block structure layer consists of several candidate operations; each cell in the cellular structure layer contains multiple computational blocks, with connections between blocks controlled by the architecture parameter α. Each block receives and processes the output features of the backbone network or information from previous blocks. The backbone network receives the fused feature vector F, and its output features enter the macro-architecture layer. The overall network of the macro-architecture layer contains several cascaded cellular structure layers, which, after global average pooling and linear mapping, output image classification probabilities.

[0015] The temperature-adaptive Gumbel Softmax sampling mechanism optimizes the search strategy by dynamically adjusting temperature parameters. For the i-th architecture parameter vector α... i First, Gumbel noise G is generated. i Then calculate the temperature adaptive parameters:

[0016] ( / )^(t / T)

[0017] in , These represent the maximum and minimum values ​​of the temperature adaptive parameters, respectively. t is the current training round, and T is the total number of training rounds. Differentiable operation selection probabilities are generated. :

[0018] / )

[0019] During forward propagation, a single operation is performed via argmax, resulting in a one-hot vector. During backpropagation, gradient continuity is maintained. Compared to the hybrid operation strategy of traditional DARTS (Differentiable Architecture Search) methods, this mechanism reduces the computational complexity of each layer from O(K×C) to O(C), where K is the number of candidate operations and C is the number of feature channels, significantly improving search efficiency. This step outputs an architecture pool containing multiple candidate neural architectures, providing ample architectural diversity for subsequent optimization.

[0020] Step 3: Using the fused feature vector F as input data, a multi-constraint two-layer optimization strategy is adopted to select the best-performing neural architecture search space from the architecture pool through the Pareto optimal solution.

[0021] Based on the candidate architecture pool generated in step two, the fusion feature vector F extracted in step one is used as training data. A multi-constraint two-layer optimization strategy is adopted, taking into account multiple objectives such as classification accuracy, computational efficiency, memory usage, and clinical deployment constraints. The single network architecture with the best performance on feature vector F is obtained by Pareto optimal solution selection.

[0022] In the multi-constraint optimization process, the accuracy loss L acc The accuracy of five-class classification based on feature vector F on the validation set is calculated by inputting F into the candidate architecture to obtain the predicted result and comparing it with the true label. The computational constraint L... flops The number of floating-point operations required to process the fused feature vector F must not exceed a set threshold. Memory constraint L. memory The GPU memory usage during feature vector F processing is limited to a set memory threshold. Delay constraint L latency The inference time from the input feature vector F to the output classification result must not exceed a set inference time threshold. Robustness loss L robustness Evaluate the performance stability of the same architecture on the fusion feature vector F extracted from different hospitals.

[0023] The two-layer optimization is implemented as an alternating iterative process of outer-layer and inner-layer optimization, following a two-stage workflow of search and evaluation. The search stage includes six steps: defining the neural architecture search space, constructing the neural architecture search space, Gumbel Softmax sampling, updating neural architecture parameters, updating network weights, and generating an architecture pool. In outer-layer optimization, the network weights ω are fixed, and the architecture parameters α are updated using the fused feature vector F from the validation set. Inner-layer optimization, the architecture parameters α are fixed, and the network weights ω are updated using the fused feature vector F from the training set. Multiple inner-layer optimizations and one outer-layer optimization are performed per epoch. The evaluation stage includes four steps: candidate architecture selection, complete training, performance evaluation, and outputting the optimal model. Architecture evaluation employs an efficient evaluation strategy based on a surrogate model, quickly predicting architecture performance through training with a small number of feature vectors F, avoiding the need for complete training of every candidate architecture.

[0024] This step outputs a fully validated optimal neural architecture from the architecture pool, serving as the basis for subsequent training and deployment.

[0025] Step 4: Based on the best-performing neural architecture, use the dataset for training and testing to achieve classification of lung infection images.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention can automatically discover deep learning architectures optimized for multi-classification tasks of lung infection images, achieving high-precision automatic classification of various lung infection images. Through multimodal feature fusion technology, this invention integrates texture, morphology, and density features, improving image classification accuracy by approximately 8-12% compared to single-feature methods. Temperature-adaptive GumbelSoftmax technology reduces search time from several days to several hours, significantly improving practicality. A multi-constraint optimization strategy achieves an optimal balance between accuracy and efficiency, reaching a five-class classification accuracy of 94.2% in multi-center validation. The analysis time for a single CT image is reduced from 5-10 minutes of manual image reading to within 3 seconds, significantly improving work efficiency. Attached Figure Description

[0028] Figure 1 This is a flowchart of the data processing process;

[0029] Figure 2 For the search space structure diagram;

[0030] Figure 3 Comparison chart of temperature-adaptive Gumbel Softmax sampling mechanisms;

[0031] Figure 4 This is a flowchart of the neural architecture search process. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] Example 1: Specific Implementation of Multimodal Feature Extraction

[0034] Combination Figure 1 This embodiment describes in detail the technical implementation of the multimodal feature extraction step.

[0035] During the image data standardization process, the input chest CT images are first checked for format compatibility. DICOM format data is parsed using a dedicated library to extract the pixel matrix and scan parameters, NIfTI format data retains spatial orientation information, and PNG / TIFF format data is used for depth reconstruction. All data is uniformly converted to a standard format, with the data type being floating-point and the size standardized to 512×512×number of slices.

[0036] Automatic lung region segmentation was performed using a pre-trained U-Net++ model. The network input was a single-layer CT slice, and the output was a lung mask. The entire CT volume was processed layer by layer, and then the segmentation result was smoothed through 3D morphological operations. The final lung mask was multiplied by the original CT to extract the purified lung region for feature calculation.

[0037] Texture feature extraction is based on segmented lung regions, calculating gray-level co-occurrence matrix features, local binary pattern features, wavelet texture features, and Gabor filter features, totaling 256 dimensions. Morphological feature extraction targets the geometric features, edge features, distribution features, and shape moment features of lung lesions, totaling 128 dimensions. Density distribution feature analysis analyzes the HU value distribution histogram, density gradient features, and regional connectivity features of different density regions in the lung, totaling 64 dimensions.

[0038] Feature fusion combines the three types of features into a 448-dimensional fusion feature vector F. Z-score normalization is applied to each feature dimension, and the normalized feature vector serves as input for subsequent neural architecture search.

[0039] Example 2: Implementation of the Temperature-Adaptive Gumbel Softmax Search Mechanism

[0040] Combination Figure 2 and Figure 3 This embodiment describes the detailed implementation of the search space construction and sampling mechanism.

[0041] The hierarchical search space design is optimized for the characteristics of the 448-dimensional fused feature vector F. The micro-operation candidate set includes 12 operations such as 1×1×1 3D convolution, 3×3×3 depthwise separable 3D convolution, residual connections, batch normalization, ReLU activation, and global average pooling. Each operation is designed to receive the 448-dimensional feature vector F or its transformed result as input. Within the cell structure, each cell contains multiple computational blocks (blocks 1 to 6). The first cell receives the feature vector F from step one, intermediate cells perform feature transformations, and finally, the output is sent to the classification layer through global average pooling and linear mapping. The macro-architecture layer uses a 6-cell cascaded structure to construct a complete network architecture from the 448-dimensional input feature vector F to the 5-dimensional classification output.

[0042] The architecture parameterization uses continuous architecture parameters α to represent discrete operation selections. For each operation node processing feature vector F, an operation selection weight vector is defined, where each element represents the weight of selecting the corresponding operation. The temperature parameter is dynamically adjusted according to an exponential decay strategy to ensure sufficient exploration in the early stages of the search and accurate convergence in the later stages when processing feature vector F.

[0043] In the Gumbel Softmax sampling implementation, the same feature vector F is used for forward propagation of each candidate architecture, and a single operation is selected for execution via argmax. During backpropagation, gradients are calculated using a continuous distribution. The core objective of architecture search is to find the network structure that can most effectively process the 448-dimensional feature vector F and produce accurate classification results.

[0044] Example 3: Implementation of Multi-Constraint Optimization

[0045] Combination Figure 4 This embodiment details the specific implementation of the multi-objective optimization strategy.

[0046] The multi-objective loss function design comprehensively considers five dimensions: classification accuracy, computational constraints, memory constraints, latency constraints, and robustness loss. The accuracy loss is calculated based on the five-class classification accuracy on the validation set. The computational constraints impose a linear penalty on FLOPs (floating-point operations) exceeding a preset threshold. The memory constraints limit GPU memory usage. The latency constraints require inference time to meet clinical needs. The robustness loss evaluates the standard deviation of accuracy across different hospital datasets.

[0047] The two-layer optimization algorithm implements an iterative process consisting of inner and outer layer optimizations. The inner layer updates network weights while fixing the architecture parameters, while the outer layer updates architecture parameters while fixing the network weights. The optimization process employs gradient pruning to prevent gradient explosion and a learning rate scheduling strategy to improve convergence performance.

[0048] The Pareto optimal solution selection employs the NSGA-II algorithm for multi-objective optimization. Through steps such as population initialization, fast non-dominated sorting, crowding distance calculation, selection, crossover and mutation, and environment selection, excellent solutions on the Pareto front are preserved.

[0049] Example 4: Five-category network training for lung infection

[0050] This embodiment details the training process for the optimal architecture.

[0051] The dataset configuration includes training, validation, and test sets, covering five categories: pneumonia caused by COVID-19, bacterial pneumonia, viral pneumonia, tuberculosis, and normal lung tissue. The data comes from multiple top-tier hospitals, encompassing CT equipment from different manufacturers. The optimal architecture obtained through the search comprises a backbone network, six cascaded cells, each cell containing several computational blocks, each computational block containing several candidate operations, a global average pooling layer, and a linear mapping layer, achieving the mapping from a 448-dimensional feature vector to a 5-dimensional classification output.

[0052] Training hyperparameters were configured using the AdamW optimizer and a cosine annealing learning rate scheduling strategy. A multi-objective joint loss function was used. Data augmentation techniques included random rotation, scaling, elastic deformation, and intensity perturbation. During training, performance was evaluated on the validation set every 10 epochs, recording classification accuracy and precision / recall for each category. Early stopping was employed to prevent overfitting.

[0053] Based on the optimal Pareto architecture obtained in step three, formal training was performed using the complete dataset to achieve five-class classification outputs for five types of lung images: pneumonia caused by COVID-19 infection, bacterial pneumonia, viral pneumonia, tuberculosis, and normal lungs.

[0054] The training process uses the 448-dimensional fused feature vector F extracted in step one as input data and employs the optimal network architecture obtained in step three. This architecture includes a backbone network, six cell structures, a global average pooling layer, and a linear mapping layer, achieving end-to-end learning from feature vectors to lung infection image classification. A multi-objective joint loss function is used. The optimization strategy employs cosine annealing learning rate scheduling, training for 300 epochs. Data augmentation is performed at the feature level, including feature perturbation and random dropout. Regularization employs weight decay and early stopping strategies to prevent overfitting.

[0055] The output includes confidence scores for five categories, the final image results, and an entropy-based uncertainty measure.

[0056] Example 5: System Performance Verification Experiment

[0057] This invention was systematically validated on a multi-center dataset. The training dataset consisted of 10,000 chest CT images, covering five disease types. The validation dataset, derived from independent datasets from five tertiary hospitals, was used to evaluate the model's generalization performance. The test dataset comprised data from 10 hospitals in different regions, covering various CT equipment. The comparative experimental design included multiple baseline methods, including six existing image classification methods: ResNet-50, DenseNet-121, EfficientNet-B3, 3D ResNet, DARTS, and PC-DARTS (a differentiable architecture search method with partial channel gradient updates). Performance evaluation metrics covered three dimensions: classification performance, computational efficiency, and robustness.

[0058] Table 1 Comparison of Five-Class Classification Performance

[0059]

[0060] Table 2. Detailed classification performance of each image category

[0061]

[0062] Table 3 Comparison of computational efficiency

[0063]

[0064] The performance comparison results of the five-class classification are shown in Table 1, the detailed classification performance of each category of images is shown in Table 2, and the computational efficiency comparison is shown in Table 3. Experimental results show that the method of this invention achieves an accuracy of 94.2%, precision of 93.7%, recall of 94.0%, F1 score of 93.8%, and AUC of 0.971 on the five-class classification task, significantly outperforming all baseline methods. In terms of computational efficiency, the number of floating-point operations (FLOPs) is 1.9G, the number of parameters is 8.7M, the inference time is 2.8 seconds, and the GPU memory usage is 4.2GB, meeting the requirements for clinical deployment. Cross-hospital robustness analysis shows an average accuracy of 94.2% ± 0.6% and a standard deviation of only 0.6%, demonstrating excellent generalization performance.

[0065] Through systematic experimental verification, the technological advancement, clinical applicability, and deployment feasibility of this invention in the task of classifying images of lung infections have been fully demonstrated.

Claims

1. A lung infection image classification method based on adaptive neural architecture search, characterized in that, Includes the following steps: Step 1: Acquire multi-source chest CT image data, perform preprocessing and feature extraction, and obtain the fused feature vector F; Step 2: Construct a neural architecture search space based on temperature-adaptive Gumbel Softmax and generate a candidate architecture pool; Step 3: Using the fused feature vector F as input data, a multi-constraint two-layer optimization strategy is adopted to select the best-performing neural architecture search space from the architecture pool through the Pareto optimal solution; The multiple constraints include: Accuracy loss is calculated based on the class accuracy of the feature vector F on the validation set, that is, F is input into the candidate architecture to obtain the prediction result and compared with the true label. Computational constraints: The number of floating-point operations required to process the fused feature vector F must not exceed a set threshold. Memory constraints limit the GPU memory usage when processing the fused feature vector F to no more than a set memory threshold. The delay constraint requires that the inference time from the input fused feature vector F to the output classification result does not exceed a set inference time threshold. Robust loss is used to evaluate the performance stability of the same architecture on the fused feature vector F extracted from different hospitals. Step 4: Based on the best-performing neural architecture, use the dataset for training and testing to achieve classification of lung infection images.

2. The lung infection image classification method based on adaptive neural architecture search according to claim 1, characterized in that, The specific process of feature extraction in step one is as follows: First, all image data are converted into a standard three-dimensional tensor format. Then, the U-Net++ network is used to automatically segment the lung region. Next, density feature constraints and morphological post-processing are combined to obtain the lung parenchyma region features. Then, three types of feature vectors are extracted from the lung parenchyma region: texture feature vector T, morphological feature vector M, and density distribution feature vector D; Finally, the three types of features are standardized and concatenated to form a fused feature vector F = [T, M, D], which serves as the input data for subsequent adaptive neural architecture search.

3. The lung infection image classification method based on adaptive neural architecture search according to claim 2, characterized in that, The extraction process of the three types of feature vectors is as follows: the texture feature vector T uses the gray-level co-occurrence matrix to calculate the contrast and correlation features, and combines the rotation invariant features of the local binary mode to calculate the microstructural features used to describe the lung tissue; the morphological feature vector M extracts the geometric features and morphological information of the lesions through connected component analysis; the density distribution feature vector D divides the range of Huntsfield unit HU values ​​into several intervals and counts the proportion of voxel distribution in each interval.

4. The lung infection image classification method based on adaptive neural architecture search according to claim 3, characterized in that, The specific implementation process of step two is as follows: Using the fused feature vector F as input, a neural architecture search space optimized for the characteristics of the feature vector is constructed. Temperature-adaptive Gumbel Softmax technology is used to optimize differentiable architecture parameters and generate a diverse pool of candidate neural architectures. The neural architecture search space uses the fused feature vector F as a unified input to all candidate network architectures, and the search goal is to construct a diverse set of neural architecture candidates that can process the fused feature vector F.

5. The lung infection image classification method based on adaptive neural architecture search according to claim 4, characterized in that, The neural architecture search space design comprises three layers: a micro-operation layer, a computational block structure layer, a cellular structure layer, and a macro-architecture layer. The micro-operation layer defines m candidate operations. The computational block structure layer consists of several candidate operations. Each cell in the cellular structure layer contains multiple computational blocks, and the connections between computational blocks are controlled by the architecture parameter α. Each block receives and processes the output features of the backbone network or information from the preceding blocks. The backbone network receives the fused feature vector F and outputs features into the macro-architecture layer. The overall network of the macro-architecture layer contains several cellular structure layers connected in series. After global average pooling and linear mapping, the output image classification probability is determined.

6. The lung infection image classification method based on adaptive neural architecture search according to claim 4, characterized in that, The temperature-adaptive Gumbel Softmax optimizes the search strategy by dynamically adjusting temperature parameters; for the i-th architecture parameter vector α i Generate the corresponding Gumbel noise Then calculate the temperature adaptive parameters. Then generate differentiable operation selection probabilities. ; During forward propagation, a single operation is performed using argmax, and the sampling result is a one-hot encoded vector. During backpropagation, gradient continuity is maintained.

7. The lung infection image classification method based on adaptive neural architecture search according to claim 6, characterized in that, The specific implementation process of the two-layer optimization strategy in step three is as follows: The two-layer optimization is implemented as an alternating iterative process of outer-layer optimization and inner-layer optimization, following a two-stage process of search and evaluation. The search stage includes six steps: defining the neural architecture search space, constructing the neural architecture search space, GumbelSoftmax sampling, updating neural architecture parameters, updating network weights, and generating an architecture pool. The outer-layer optimization fixes the network weights ω and updates the architecture parameters α using the fused feature vector F from the validation set. The inner-layer optimization fixes the architecture parameters α and updates the network weights ω using the feature vector F from the training set. Each training batch epoch performs multiple inner-layer optimizations and one outer-layer optimization. The evaluation stage includes four steps: candidate architecture selection, complete training, performance evaluation, and outputting the optimal model. The architecture evaluation adopts a surrogate model-based evaluation strategy, predicting architecture performance through training with the fused feature vector F.

Images