Lung infection image classification method based on adaptive neural architecture search

Abstract

The invention discloses a pulmonary infection image classification method based on adaptive neural architecture search, and the method comprises the steps: firstly obtaining multi-source chest CT image data, carrying out the preprocessing and feature extraction, and obtaining a fusion feature vector F; the method comprises the steps that firstly, a neural architecture search space based on temperature self-adaption Gumbel Softmax is constructed, a candidate architecture pool is generated, then a fusion feature vector F is used as input data, a multi-constraint-condition double-layer optimization strategy is adopted, and the neural architecture search space with the best performance is selected from the architecture pool through a Pareto optimal solution. And finally, based on the neural architecture with the optimal performance, training and testing are carried out by using the data set, and classification of the pulmonary infection images is realized. According to the method, the deep learning architecture for pulmonary infection multi-classification task optimization can be automatically found, and high-precision automatic classification of various pulmonary infection images is realized.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing and artificial intelligence technology, and in particular relates to a lung infection image classification method based on adaptive neural architecture search, which can identify images of various lung infections. Background Art

[0002] Pulmonary infections are a serious threat to human health worldwide. They include pneumonia caused by the novel coronavirus, bacterial pneumonia, viral pneumonia, tuberculosis, and other inflammatory lung diseases caused by various pathogens. Accurate and rapid classification of lung infections is crucial for developing targeted treatment plans and controlling disease spread.

[0003] Currently, diagnosis of lung infection relies primarily on chest CT imaging analysis and laboratory testing. However, different types of lung infection have overlapping imaging manifestations, making manual image interpretation inefficient and difficult to meet clinical needs during large-scale epidemics or when medical resources are limited.

[0004] Existing deep learning-based lung infection detection methods have significant technical defects. First, the ability to classify multiple diseases is insufficient. Existing methods mostly use a binary classification strategy, which cannot accurately distinguish between multiple types of lung infections. Secondly, the static nature of the architectural design is limited. Traditional methods rely on manually designed fixed network architectures and cannot adaptively adjust the network structure according to the pathological characteristics of different lung infection diseases. In addition, the problem of single feature representation is prominent. Existing methods are mainly based on single-modality CT image features and fail to fully utilize the multi-dimensional pathological information of lung infections. Finally, clinical adaptability is insufficient, and there is a lack of adaptive optimization for different hospitals, different equipment, and different scanning parameters. In actual clinical applications, performance often declines significantly.

[0005] Although neural architecture search technology provides new ideas for automatically discovering high-performance network architectures, the application of existing methods in the field of medical imaging still faces challenges such as improper search space design, difficulty in multi-objective optimization, and insufficient consideration of clinical constraints. Summary of the Invention

[0006] The purpose of the present invention is to provide a lung infection image classification method based on adaptive neural architecture search. This method, through key technical innovations such as multimodal feature fusion, temperature-adaptive Gumbel Softmax (a differentiable discrete sampling method) sampling, and multi-constraint neural architecture search, can automatically discover deep learning architectures optimized for multi-classification tasks of lung infection images, and achieve high-precision automatic classification of various lung infection images.

[0007] To achieve the above objectives, the present invention provides a lung infection image classification method based on adaptive neural architecture search, comprising the following steps:

[0008] Step 1: Preprocessing and feature extraction of multimodal medical image data: Obtain multi-source chest CT image data, perform preprocessing and feature extraction, and obtain the fusion feature vector F.

[0009] Receive multi-source chest CT image data, perform standardized preprocessing operations, extract multi-dimensional feature vectors related to lung infection, and form a unified feature representation space.

[0010] The multimodal data preprocessing process first standardizes the image formats, supporting automatic recognition and conversion of formats such as DICOM, NIfTI, PNG, and TIFF, converting all image data into a standard 3D tensor format. A U-Net++ network is then used to segment the lung parenchyma into a coarse region. Density feature constraints and morphological post-processing are then combined to produce a refined lung parenchyma region for subsequent analysis.

[0011] The feature extraction process extracts three types of feature vectors from the refined lung parenchyma region. The texture feature vector T uses the gray-level co-occurrence matrix to calculate features such as contrast and correlation, combined with the rotation-invariant features of local binary patterns, to describe the microstructural characteristics of lung tissue. The morphological feature vector M uses connected domain analysis to extract geometric features such as the area, perimeter, and eccentricity of the lesion, as well as morphological information such as edge roughness and spatial distribution. The density distribution feature vector D divides the Hounsfield unit (HU) value range into 64 intervals and calculates the voxel distribution percentage in each interval to reflect the density variation of lung tissue. The three features are normalized by Z-score and then concatenated to form a fused feature vector F = [T, M, D], which serves as input data for the subsequent adaptive neural architecture search.

[0012] Step 2: Construct a neural architecture search space based on temperature-adaptive Gumbel Softmax to generate a candidate architecture pool.

[0013] Taking the fused feature vector F extracted in step 1 as input, a neural architecture search space optimized for the characteristics of the feature vector is constructed. The temperature-adaptive Gumbel Softmax technology is used to achieve differentiable architecture parameter optimization, generating a diverse candidate neural architecture pool, which is a collection of candidate neural network architectures.

[0014] The neural architecture search space uses the fused feature vector F as the unified input for all candidate network architectures. The search goal is to construct a diverse set of candidate neural architectures that can effectively process the fused feature vector F. The neural architecture search space design comprises four layers: micro-operation layer, computational block structure layer, cellular structure layer, and macro-architecture layer. The micro-operation layer defines m candidate operations, including 1×1×1 3D convolution, 3×3×3 depthwise separable 3D convolution, residual connection, batch normalization, ReLU activation, and global average pooling. The computational block structure layer consists of several candidate operations. Each cell in the cellular structure layer contains multiple computational blocks, with connections between blocks controlled by an architectural parameter α. Each block receives and processes the output features of the backbone network or information from the previous block. The backbone network receives the fused feature vector F and outputs the features to the macro-architecture layer. The overall network in the macro-architecture layer consists of several cellular structure layers connected in series, which undergo global average pooling and linear mapping to output image classification probabilities.

[0015] The temperature-adaptive Gumbel Softmax sampling mechanism optimizes the search strategy by dynamically adjusting the temperature parameter. i , first generate Gumbel noise G i , and then calculate the temperature adaptive parameters:

[0016] ( / )^(t / T)

[0017] in , Represent the maximum and minimum values ​​of the temperature adaptation parameter, t is the current training round, T is the total number of training rounds, and generates a differentiable operation selection probability :

[0018] / )

[0019] During forward propagation, a single operation is performed via argmax, resulting in a one-hot vector sample. Gradient continuity is maintained during backward propagation. Compared to the hybrid operation strategy of the traditional DARTS (Differentiable Architecture Search) method, this mechanism reduces the computational complexity of each layer from O(K × C) to O(C), where K is the number of candidate operations and C is the number of feature channels, significantly improving search efficiency. This step outputs a pool of candidate neural architectures, providing sufficient architectural diversity for subsequent optimization selection.

[0020] Step 3: Using the fused feature vector F as input data, a multi-constraint two-level optimization strategy is adopted to select the neural architecture search space with the best performance from the architecture pool through the Pareto optimal solution.

[0021] Based on the candidate architecture pool generated in step 2, the fused feature vector F extracted in step 1 is used as training data. A multi-constraint two-layer optimization strategy is adopted, while considering multiple objectives such as classification accuracy, computational efficiency, memory usage, and clinical deployment constraints. The single network architecture with the best performance on the feature vector F is obtained through Pareto optimal solution selection.

[0022] During the multi-constraint optimization process, the accuracy loss L acc Based on the five-category accuracy calculation of the feature vector F on the validation set, F is input into the candidate architecture to obtain the predicted result and compare it with the true label. flops Evaluate the number of floating-point operations when processing the fused feature vector F, and require that it does not exceed the set number threshold. Memory constraint L memory Limit the GPU memory usage when processing the feature vector F to not exceed the set memory threshold. Latency constraint L latency It is required that the inference time from the input feature vector F to the output classification result does not exceed the set inference time threshold. Robustness loss L robustness Evaluate the performance stability of the same architecture on the fusion feature vectors F extracted from different hospitals.

[0023] The dual-layer optimization is implemented as an alternating iterative process of outer-layer optimization and inner-layer optimization, following a two-stage process of search and evaluation. The search stage includes six steps: neural architecture search space definition, neural architecture search space construction, Gumbel Softmax sampling, neural architecture parameter update, network weight update, and architecture pool generation. In the outer-layer optimization, the network weights ω are fixed and the architecture parameters α are updated using the fused feature vector F of the validation set. In the inner-layer optimization, the architecture parameters α are fixed and the network weights ω are updated using the fused feature vector F of the training set. Multiple inner-layer optimizations and one outer-layer optimization are performed in each epoch. The evaluation stage includes four steps: candidate architecture screening, full training, performance evaluation, and optimal model output. The architecture evaluation adopts an efficient evaluation strategy based on a surrogate model, which quickly predicts architecture performance by training a small number of feature vectors F, avoiding the need for full training of each candidate architecture.

[0024] This step outputs a well-verified optimal neural architecture from the architecture pool as the basis for subsequent training and deployment

[0025] Step 4: Based on the best performing neural architecture, use the dataset for training and testing to achieve classification of lung infection images.

[0026] The beneficial effects of the present invention are as follows:

[0027] The present invention can automatically discover a deep learning architecture optimized for multi-classification tasks of lung infection images, and achieve high-precision automatic classification of various lung infection images. The present invention integrates three types of features, texture, morphology, and density, through multimodal feature fusion technology, and improves the image classification accuracy by about 8-12% compared with a single feature method. The temperature-adaptive GumbelSoftmax technology shortens the search time from several days to several hours, greatly improving practicality. The multi-constraint optimization strategy achieves the optimal balance between accuracy and efficiency, and the accuracy of the five categories reached 94.2% in multi-center verification. The analysis time of a single CT image is shortened from 5-10 minutes for manual reading to within 3 seconds, significantly improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a data processing flow chart;

[0029] Figure 2 is the search space structure diagram;

[0030] Figure 3 A comparison chart of the temperature-adaptive Gumbel Softmax sampling mechanism;

[0031] Figure 4 Flowchart for neural architecture search. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0033] Example 1: Specific implementation of multimodal feature extraction

[0034] Combine Figure 1 , this embodiment describes in detail the technical implementation of the multimodal feature extraction step.

[0035] During the image data standardization process, input chest CT images are first checked for format compatibility. DICOM format data is parsed using a dedicated library to extract pixel matrices and scan parameters. NIfTI format preserves spatial orientation information, and PNG / TIFF format is used for depth reconstruction. All data is uniformly converted to a standard format, with a floating-point data type and a normalized size of 512 × 512 × the number of slices.

[0036] Automatic lung segmentation is performed using a trained U-Net++ model. The network input is a single-slice CT slice, and the output is a lung mask. The entire CT volume is processed layer by layer, and the segmentation results are smoothed using 3D morphological operations. The resulting lung mask is multiplied with the original CT image to extract the clean lung region for feature calculation.

[0037] Texture feature extraction is based on the segmented lung regions, calculating gray-level co-occurrence matrix features, local binary pattern features, wavelet texture features, and Gabor filter features, totaling 256 dimensions. Morphological feature extraction targets the geometric, edge, distribution, and shape moment features of lung lesions, totaling 128 dimensions. Density distribution features analyze the HU value distribution histogram, density gradient features, and regional connectivity features of different lung density regions, totaling 64 dimensions.

[0038] Feature fusion combines the three types of features into a 448-dimensional fused feature vector F. Z-score normalization is performed on each dimension of the feature, and the normalized feature vector is used as the input for subsequent neural architecture search.

[0039] Example 2: Implementation of Temperature Adaptive Gumbel Softmax Search Mechanism

[0040] Combine Figure 2 and Figure 3 ,This embodiment describes the detailed implementation of the search space construction and ,sampling mechanism.

[0041] The hierarchical search space design optimizes the characteristics of the 448-dimensional fused feature vector F. The candidate set of micro-operations includes 12 operations, including 1×1×1 3D convolution, 3×3×3 depthwise separable 3D convolution, residual connection, batch normalization, ReLU activation, and global average pooling. Each operation is designed to accept the 448-dimensional feature vector F or its transformation as input. Each cell in the internal structure contains multiple computational blocks (blocks 1 to 6). The first cell receives the feature vector F from step 1. Intermediate cells perform feature transformations, and finally, the output is sent to the classification layer through global average pooling and linear mapping. The macro-architecture layer uses a structure of six cells connected in series to construct a complete network architecture from the 448-dimensional input feature vector F to the 5-dimensional classification output.

[0042] Architectural parameterization uses a continuous architectural parameter α to represent discrete operation selections. For each operation node processing a feature vector F, an operation selection weight vector is defined, where each element represents the weight of selecting the corresponding operation. The temperature parameter is dynamically adjusted using an exponential decay strategy to ensure sufficient exploration in the early stages of the search and accurate convergence in the later stages when processing the feature vector F.

[0043] During the Gumbel Softmax sampling implementation, the same feature vector F is used for forward propagation for each candidate architecture. A single operation is selected through argmax, and the gradient is calculated using a continuous distribution during backward propagation. The core goal of architecture search is to find the network structure that can most efficiently process the 448-dimensional feature vector F and produce accurate classification results.

[0044] Example 3: Implementation of multi-constraint optimization

[0045] Combine Figure 4 ,This embodiment details the specific implementation of the multi-objective optimization strategy.

[0046] The multi-objective loss function design comprehensively considers five dimensions: classification accuracy, computational constraints, memory constraints, latency constraints, and robustness loss. The accuracy loss is calculated based on the accuracy of the five categories on the validation set. The computational constraint imposes a linear penalty on FLOPs (floating-point operations) exceeding a preset threshold. The memory constraint limits GPU memory usage. The latency constraint requires that inference time meet clinical requirements. The robustness loss evaluates the standard deviation of accuracy across different hospital datasets.

[0047] The two-layer optimization algorithm implements an alternating iterative process involving inner and outer optimization layers. The inner layer optimizes fixed architectural parameters and updates network weights, while the outer layer optimizes fixed network weights and updates architectural parameters. The optimization process uses gradient clipping to prevent gradient explosion and a learning rate scheduling strategy to improve convergence performance.

[0048] The NSGA-II algorithm is used to select Pareto optimal solutions for multi-objective optimization. Through population initialization, fast non-dominated sorting, crowding distance calculation, crossover mutation, and environment selection, the excellent solutions on the Pareto front are retained.

[0049] Example 4: Pulmonary Infection Five-Classification Network Training

[0050] This embodiment details the training process of the optimal architecture.

[0051] The dataset configuration includes training, validation, and test sets, covering five categories: pneumonia caused by COVID-19, bacterial pneumonia, viral pneumonia, tuberculosis, and normal lungs. The data comes from multiple tertiary hospitals and covers CT equipment from different manufacturers. The optimal architecture obtained by the search consists of four parts: a backbone network, six tandem cells, each cell containing several computational blocks, each computational block containing several candidate operations, a global average pooling layer, and a linear mapping layer, which maps from a 448-dimensional feature vector to a 5-dimensional classification output.

[0052] Training hyperparameters were set using the AdamW optimizer and a cosine annealing learning rate scheduling strategy. A multi-objective joint loss function was used as the loss function. Data augmentation included techniques such as random rotation, rescaling, elastic deformation, and intensity perturbation. Training performance was monitored every 10 epochs on the validation set, recording classification accuracy and precision-recall for each category. An early stopping strategy was used to prevent overfitting.

[0053] Based on the Pareto optimal architecture obtained in step three, formal training was performed using the complete dataset to achieve five-category output for five types of lung images: pneumonia caused by new coronavirus infection, bacterial pneumonia, viral pneumonia, tuberculosis, and normal lungs.

[0054] The training process uses the 448-dimensional fused feature vector F extracted in step 1 as input data and employs the optimal network architecture searched for in step 3. This architecture comprises a backbone network, six cell structures, a global average pooling layer, and a linear mapping layer, enabling end-to-end learning from feature vectors to lung infection image classification. A multi-objective joint loss function is used as the loss function. The optimization strategy uses cosine annealing learning rate scheduling, and training is performed for 300 epochs. Data augmentation is performed at the feature level, including feature perturbation and random dropout techniques. Regularization uses weight decay and early stopping strategies to prevent overfitting.

[0055] The output results include confidence scores for the five categories, the final image result, and an entropy-based uncertainty measure.

[0056] Example 5: System performance verification experiment

[0057] The present invention was systematically validated on a multi-center dataset. The training dataset consists of 10,000 chest CT images, including images of five disease types. The validation dataset is derived from independent datasets of five tertiary hospitals and is used to evaluate the generalization performance of the model. The test dataset is derived from 10 hospitals in different regions, covering different CT equipment. The comparative experimental design includes multiple baseline methods, including six existing image classification methods such as ResNet-50, DenseNet-121, EfficientNet-B3, 3D ResNet, DARTS, and PC-DARTS (differentiable architecture search method with partial channel gradient update). The performance evaluation indicators cover three dimensions: classification performance, computational efficiency, and robustness.

[0058] Table 1 Comparison of five-category performance

[0059]

[0060] Table 2 Detailed classification performance of each category of images

[0061]

[0062] Table 3 Comparison of computational efficiency

[0063]

[0064] The performance comparison results of the five-category classification are shown in Table 1, the detailed classification performance of each category of images is shown in Table 2, and the computational efficiency comparison is shown in Table 3. The experimental results show that the method of the present invention achieves an accuracy of 94.2%, a precision of 93.7%, a recall rate of 94.0%, an F1 score of 93.8%, and an AUC of 0.971 in the five-category task, which is significantly better than all baseline methods. In terms of computational efficiency, the number of floating-point operations (FLOPs) is 1.9G, the number of parameters is 8.7M, the inference time is 2.8 seconds, and the GPU memory occupancy is 4.2GB, which meets the requirements of clinical deployment. Cross-hospital robustness analysis shows that the average accuracy is 94.2%±0.6%, and the standard deviation is only 0.6%, showing excellent generalization performance.

[0065] Through systematic experimental verification, the technical advancement, clinical practicality and deployment feasibility of this invention in the task of lung infection image classification have been fully demonstrated.

Claims

1. A lung infection image classification method based on adaptive neural architecture search, characterized by: The following steps are involved: Step 1: Obtain multi-source chest CT image data, perform preprocessing and feature extraction, and obtain the fusion feature vector F; Step 2: Construct a neural architecture search space based on temperature-adaptive Gumbel Softmax to generate a candidate architecture pool; Step 3: Using the fused feature vector F as input data, a multi-constraint two-level optimization strategy is adopted to select the neural architecture search space with the best performance from the architecture pool through the Pareto optimal solution; Step 4: Based on the best performing neural architecture, use the dataset for training and testing to achieve classification of lung infection images.

2. The lung infection image classification method based on adaptive neural architecture search according to claim 1 is characterized in that: The specific process of feature extraction in step 1 is as follows: First, all image data are converted into a standard 3D tensor format. Then, the U-Net++ network is used to automatically segment the lung region. Then, density feature constraints and morphological post-processing are combined to obtain the lung parenchymal regional features. Then, three types of feature vectors are extracted from the lung parenchyma region: texture feature vector T, morphological feature vector M and density distribution feature vector D; Finally, the three types of features are normalized and concatenated to form a fused feature vector F = [T, M, D], which serves as the input data for subsequent adaptive neural architecture search.

3. The lung infection image classification method based on adaptive neural architecture search according to claim 2, characterized in that: The extraction process of the three types of feature vectors is specifically as follows: the texture feature vector T uses the gray-level co-occurrence matrix to calculate the contrast and correlation features, combined with the rotation-invariant features of the local binary pattern to calculate the microstructural features used to describe the lung tissue; the morphological feature vector M extracts the geometric features and morphological information of the lesion through connected domain analysis; the density distribution feature vector D divides the Hounsfield unit HU value range into several intervals, and calculates the voxel distribution proportion in each interval.

4. The lung infection image classification method based on adaptive neural architecture search according to claim 3 is characterized in that: The specific implementation process of step 2 is as follows: Taking the fused feature vector F as input, we construct a neural architecture search space optimized for the characteristics of this feature vector. We use temperature-adaptive Gumbel Softmax technology to achieve differentiable architecture parameter optimization and generate a diverse pool of candidate neural architectures. The neural architecture search space takes the fused feature vector F as the unified input of all candidate network architectures, and the search goal is to construct a diverse set of neural architecture candidates that can process the fused feature vector F.

5. The lung infection image classification method based on adaptive neural architecture search according to claim 4 is characterized in that: The neural architecture search space design includes three levels: micro-operation layer, computing block structure layer, cell structure layer and macro-architecture layer; the micro-operation layer defines m candidate operations; the computing block structure layer is composed of several candidate operations; each cell in the cell structure layer contains multiple computing blocks, and the connection between the computing blocks is controlled by the architecture parameter α. Each block receives and processes the output features of the backbone network or the information of the previous block; the backbone network receives the fused feature vector F, and the output features enter the macro-architecture layer. The overall network of the macro-architecture layer includes several cell structure layers connected in series, which are subjected to global average pooling and linear mapping to output the image classification probability.

6. The lung infection image classification method based on adaptive neural architecture search according to claim 4, characterized in that: The temperature adaptive Gumbel Softmax optimizes the search strategy by dynamically adjusting the temperature parameter; for the i-th architecture parameter vector α i , generate the corresponding Gumbel noise , and then calculate the temperature adaptive parameters , and then generate the differentiable operation selection probability ; During forward propagation, a single operation is performed through argmax, and the sampling result is a one-hot encoded one-hot vector, while maintaining gradient continuity during backward propagation.

7. The lung infection image classification method based on adaptive neural architecture search according to claim 4, characterized in that: The multiple constraints in step 3 include: Precision loss, calculated based on the class accuracy of the feature vector F on the validation set, that is, inputting F into the candidate architecture to obtain the predicted result and comparing it with the true label; Computational constraint: evaluates the number of floating-point operations when processing the fused feature vector F, requiring it not to exceed the set number threshold; Memory constraint, which limits the GPU memory usage when processing the fused feature vector F to not exceed the set memory threshold; Latency constraint, which requires that the inference time from inputting the fused feature vector F to outputting the classification result does not exceed the set inference time threshold; Robustness loss, which evaluates the performance stability of the same architecture on the fused feature vectors F extracted from different hospitals.

8. The lung infection image classification method based on adaptive neural architecture search according to claim 7, characterized in that: The specific implementation process of the two-layer optimization strategy in step three is as follows: The two-layer optimization is implemented as an alternating iterative process of outer-layer optimization and inner-layer optimization, and the overall process follows a two-stage process of search stage and evaluation stage; the search stage includes six steps: neural architecture search space definition, neural architecture search space construction, GumbelSoftmax sampling, neural architecture parameter update, network weight update and architecture pool generation; the outer-layer optimization fixes the network weight ω, and uses the fused feature vector F of the validation set to update the architecture parameter α; the inner-layer optimization fixes the architecture parameter α, and uses the feature vector F of the training set to update the network weight ω; multiple inner-layer optimizations and one outer-layer optimization are performed in each training batch epoch; the evaluation stage includes four steps: candidate architecture screening, complete training, performance evaluation and optimal model output. The architecture evaluation adopts an evaluation strategy based on a proxy model, and predicts the architecture performance through training of the fused feature vector F.

Images