A burn image deep intelligent classification method based on multi-feature extraction and selection

Abstract

The application provides a burn image deep intelligent classification method based on multi-feature extraction and selection, and relates to the technical field of computer vision. The method firstly deploys an improved ResNet-50 network Burn-ResNet51 and a pre-trained ViT-Base-16 in parallel, respectively extracts local texture details and global context information of a burn image; through systematic evaluation of ViT all 12 layers of encoder output, the intermediate layer features most discriminative to two classification and three classification tasks are selected, and the "black box" type only uses the final layer convention is broken. Then the double-branch deep features are spliced into a high-dimensional vector, and adaptive selection is carried out based on random forest OOB permutation importance. Finally, a cascade ensemble learning framework is adopted: preliminary probability evaluation is carried out by using a random forest, and then combined with a particle swarm optimization support vector machine or XGBoost for secondary threshold decision, and fine discrimination is realized for three classification and two classification tasks respectively; Grad-CAM is introduced to visualize the attention area, and the model interpretability is ensured.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a deep intelligent classification method for burn images based on multi-feature extraction and selection. Background Technology

[0002] In modern clinical diagnosis, accurate determination of burn depth remains a core element in determining patient treatment plans and prognosis. Traditional clinical assessment relies heavily on physicians' visual observation and palpation experience, which is highly subjective and inconsistent. Evidence-based studies show that the accuracy rate hovers only around 50%–80%, with differences of up to 30% between physicians of different qualifications. Primary healthcare institutions, lacking experienced specialists, suffer from high misdiagnosis rates, often causing patients to miss the optimal treatment window, leading to infections, sepsis, and even death. Although advanced equipment such as laser Doppler imaging can provide quantitative data, these devices are expensive, complex to operate, and have stringent environmental and personnel requirements, making them difficult to deploy in resource-constrained settings. Therefore, developing low-cost, high-precision, and easily deployable computer-aided diagnostic systems has become an urgent clinical need.

[0003] With the rise of deep learning, Convolutional Neural Networks (CNNs) are gradually replacing traditional manually designed features (such as HOG and LBP) due to their ability to automatically extract local texture details. Visual Transformers (ViTs), on the other hand, capture long-distance dependencies through self-attention mechanisms, providing a new approach to global context modeling. However, a single CNN is prone to "local blind spots," while a single ViT may ignore fine textures. Simply concatenating the two can lead to overfitting due to the curse of dimensionality and information redundancy, especially in the small sample scenarios commonly encountered in medical images, where performance degrades sharply. Furthermore, current applications of ViTs often remain at the level of "black box" access to the output of the last layer, ignoring the intermediate encoder layers containing more discriminative mid-level semantics for specific tasks. The feature fusion stage also lacks an efficient selection mechanism, making it difficult to achieve true synergistic enhancement. How to overcome the separation between local and global information, how to systematically mine the value of each layer within ViT, and how to train a strong generalization model on extremely small datasets have become key bottlenecks restricting the further clinical application of intelligent burn image classification.

[0004] In view of the above, this application is hereby submitted. Summary of the Invention

[0005] This invention provides a deep intelligent classification method for burn images based on multi-feature extraction and selection, which can at least partially improve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A deep intelligent classification method for burn images based on multi-feature extraction and selection, comprising:

[0008] Obtain a preset burn image dataset and perform data preprocessing on the burn image dataset to obtain input image data;

[0009] Parallel feature extraction processing is performed on the input image data to obtain a first deep feature vector extracted by the improved Burn-ResNet51 model and a second deep feature vector extracted by the ViT-Base-16 model;

[0010] The first and second depth feature vectors are concatenated, and the importance of the concatenated feature vectors is evaluated to select the feature subset with the highest contribution.

[0011] The feature subset is subjected to ensemble learning classification to obtain the classification results. The gradient weighted class activation mapping method is used to perform visualization analysis on the Burn-ResNet51 model and the ViT-Base-16 model to obtain the visualization results.

[0012] In summary, this clinical pain point, "deep intelligent classification of burn images," addressed by the proposed deep intelligent classification method for burn images based on multi-feature extraction and selection, constructs a complete few-shot learning framework encompassing data augmentation, parallel feature extraction, hierarchical feature optimization, fusion dimensionality reduction, and cascaded ensemble decision-making. By improving ResNet-50 to obtain Burn-ResNet51, local deep features are output at the global average pooling layer. Simultaneously, pre-trained ViT-Base-16 is used to evaluate and select the intermediate encoder features with the strongest discriminative power for binary and tri-class classification tasks, achieving complementary fusion of local and global information. Subsequently, random forest OOB permutation importance is used to compress the 1024-dimensional high-dimensional fusion features into an 8-dimensional core subset, eliminating redundancy and avoiding overfitting. Finally, a cascaded ensemble strategy of "RF probability evaluation + PSO-SVM / XGBoost secondary threshold decision-making" is adopted, combined with interpretable Grad-CAM heatmaps. The accuracy rates for binary and tri-class classification are significantly improved, far exceeding the current best levels. Moreover, the model is lightweight and does not require expensive hardware, so it can be directly embedded into existing imaging equipment in primary hospitals to achieve rapid, accurate, and low-cost auxiliary diagnosis of burn depth. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the intelligent deep classification method for burn images based on multi-feature extraction and selection provided in this embodiment of the invention.

[0014] Figure 2 This is a framework diagram of the intelligent deep classification method for burn images based on multi-feature extraction and selection provided in this embodiment of the invention.

[0015] Figure 3 This is the original ResNet-50 model structure diagram.

[0016] Figure 4 This is a diagram of the Burn-ResNet51 model structure provided in an embodiment of the present invention.

[0017] Figure 5 This is a structural diagram of the feature selection method based on random forest provided in the embodiments of the present invention.

[0018] Figure 6 This is a schematic diagram illustrating the impact of the number of features on the classification effect provided in an embodiment of the present invention.

[0019] Figure 7 These are feature heatmaps extracted using GradCAM from features extracted from ResNet-101, Burn-ResNet51, and ResNet-18 models, respectively, provided in this embodiment of the invention.

[0020] Figure 8 These are feature importance heatmaps created using GradCAM visualization technology for features extracted from Encoder-0 to Encoder-11 layers of ViT, as provided in this embodiment of the invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a deep intelligent classification method for burn images based on multi-feature extraction and selection, which can be executed by a deep intelligent classification device for burn images based on multi-feature extraction and selection (hereinafter referred to as the classification device), specifically, by one or more processors within the classification device, to implement the following method:

[0023] S1, Obtain the preset burn image dataset, and perform data preprocessing on the burn image dataset to obtain input image data;

[0024] Specifically, step S1 further includes: acquiring a dataset of burn images captured by a preset high-resolution camera, and standardizing the image size of the burn image dataset to a preset pixel size.

[0025] The adjusted burn image dataset is subjected to random transformation to achieve data augmentation, and the image pixel values ​​are standardized to obtain the input image data. Different standardization strategies are adopted for different deep learning models. For the ViT-Base-16 model, the standardized parameters pre-trained by ImageNet are used, and for the CNN model, the pixel values ​​are normalized to a preset range.

[0026] In this embodiment, the original burn image dataset is first preprocessed to obtain image data that meets the input requirements of deep networks. All experiments utilize a publicly available burn image dataset, named "Burns BIP_US dataset," jointly owned by a university's biomedical image processing group and a hospital. This dataset consists of 94 burn images of three different burn depths. The original images were captured using a high-resolution camera, at a distance of approximately 15 to 30 centimeters from the burn site, under soft lighting and stable focus conditions on the cleaned burned skin. The image resolution ranges from 84×71 to 830×536. The image resolutions range from 84×71 to 830×536.

[0027] Specifically, all images were uniformly scaled to 224×224 pixels. This size satisfies the GPU's requirements for parallel computing on power-two side lengths while maintaining the recognizability of lesion areas. Bicubic interpolation was used for scaling to reduce jagged edges and maintain smooth edges. The data augmentation stage then began: random rotations (0°–180°) and random scaling (0–10 times) were applied to the training set images sequentially, allowing each sample to achieve up to 20 times the potential change in geometry and photometric aspects, significantly expanding sample diversity and suppressing model overfitting. It is important to emphasize that all random transformations were performed using a recoverable seed method to ensure reproducibility during the validation and testing phases. Immediately after data augmentation, pixel values ​​are standardized: For the ViT-Base-16 branch, the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] corresponding to the ImageNet-1K pre-trained weights are directly used to ensure that the input distribution is consistent with the pre-training scenario, avoiding small-sample oscillations caused by fine-tuning from scratch; for the improved CNN branch Burn-ResNet51, pixel values ​​are linearly mapped to the [0,1] interval, and Z-score normalization (mean minus variance) is performed in the channel dimension to accelerate gradient convergence. After the above standardization process, stable, consistent-sized, and information-enhanced input image data is finally obtained, laying a high-quality data foundation for subsequent dual-path parallel feature extraction.

[0028] It should be noted that all experiments were conducted on a personal computer equipped with an Intel(R) i7-4770 (2.90 GHz) CPU, an NVIDIA GeForce RTX 2060 GPU, and 16GB of RAM.

[0029] S2, Parallel feature extraction processing is performed on the input image data to obtain the first deep feature vector extracted by the improved Burn-ResNet51 model and the second deep feature vector extracted by the ViT-Base-16 model;

[0030] Specifically, step S2 further includes: exchanging the input image data, inputting the exchanged input image data into the pre-improved Burn-ResNet51 model, and extracting deep features in the last global average pooling layer of the Burn-ResNet51 model to obtain the first deep feature vector.

[0031] The exchange process includes randomly rotating, randomly scaling, and resizing the input image data.

[0032] Obtain the original ResNet-50 model and insert three additional layers between the activation_49_relu layer and the avg_pool layer in the original ResNet-50 model architecture. The three additional layers are a Conv convolutional layer, a Batch_Normaliz batch normalization layer, and an Activation_Relu activation layer.

[0033] The mathematical expression for the Conv convolutional layer is: The mathematical expression for the Batch_Normaliz batch normalization layer is: , , The mathematical expression for the Activation_ReLU activation layer is: N is the output size, W is the input size, F is the kernel size, P is the padding size, and S is the stride. This represents the final batch normalization transformation of the Batch_Normaliz batch normalization layer. For a Each activation value in Perform a batch normalization transformation to be added to the network to manipulate any activations. and These are all parameters that need to be learned. for The i-th activation value in for The m-th activation value in for The normalized value, For a given set containing m activation values mini-batch For each middle arrive The mean, For each middle arrive variance It is a constant added to the mini-batch variance for numerical stability. Activation layer for Activation_ReLU;

[0034] After the avg_pool layer, the original three classification layers are replaced with three newly added related classification layers: a fully connected layer, a softmax activation function layer, and a ClassificationLayer classification layer, resulting in the improved Burn-ResNet51 model.

[0035] In this embodiment, compared to traditional manual methods, deep learning can automatically extract useful information from images, making it more adaptable to various scenarios. This automated feature extraction process can significantly reduce human error and capture complex features that may be invisible to the human eye. Theoretically, deep convolutional neural networks can automatically learn hierarchical features from low to high levels; the more layers the network has, the more complex the tasks it can handle, and the higher the level of features it can extract. Therefore, this method selects the ResNet-50 model as the base network for subsequent improvements based on the optimal results obtained after directly classifying burn images using several different deep convolutional neural networks (direct classification refers to classification using fully connected layers, softmax layers, and classification layers).

[0036] Parallel feature extraction is the foundation of this framework. Its core idea is to leverage the strengths of different neural network architectures to extract complementary feature information from the same medical image in parallel. A specially modified CNN branch (named Burn-ResNet51) is responsible for capturing local texture and details of the image, while the ViT branch is responsible for modeling long-range dependencies between image patches and obtaining global contextual information. First, the burn image undergoes simple transformations (including random rotation from 0° to 180°, random scaling from 0 to 10 times, and resizing to 224×224×3), then it is input into the Burn-ResNet51 network. Finally, deep features are extracted from the last global average pooling layer of this network.

[0037] The improved Burn-ResNet51 model is obtained by adjusting the structure and parameters of a pre-trained ResNet-50. Among other things, Figure 3 and Figure 4 It provides a basic network structure and a visual representation of the adjusted network. Figure 3 The original ResNet-50 architecture is shown; the red boxes and arrows in this model indicate the parts that need adjustment. Figure 4 The modified Burn-ResNet51 architecture after merging new layers is shown. The parts marked with blue boxes in this model are the parts that have been adjusted.

[0038] Specifically, the improvement steps for the Burn-ResNet51 model are as follows. First, the improved CNN pathway is described: To further enhance the network's ability to perceive local pathological features of burn wounds, this implementation constructs Burn-ResNet51 based on the original ResNet-50. The specific improvement is located after the 49th residual block and before the global average pooling layer: a Conv-BN-ReLU triple is inserted, where the Conv convolution kernel size is 3×3, stride is 1, padding is 1, and the number of output channels is determined to be 256 through grid search. This can enhance the semantic abstraction of the middle layer without significantly increasing the number of parameters. This is because the kernel size and the number of filters in the convolutional layer significantly affect the classification accuracy. The kernel size of the newly added convolutional layer in the Burn-ResNet51 network of this invention is set to 3×3. At the same time, since each convolutional filter contributes to a specific component of the feature vector, the dimension of the extracted feature vector is directly related to the number of convolutional filters used, and the number of feature maps determines the total number of extracted feature vectors. Especially in this invention, where the dataset is relatively small and the feature dimension of the deep features is not very high, the feature dimension and the corresponding number of filters become particularly important. Therefore, after relevant parameter tuning, the number of filters was ultimately set to 256 in this invention.

[0039] In the batch normalization algorithm of the newly added normalization layer, It is added as a constant to the mini-batch variance to maintain numerical stability. Therefore, the Batch_Normaliz layer uses... =0.0001 (binary classification) or 0.001 (triple classification) ensures numerical stability and accelerates convergence; the ReLU activation layer maintains sparse gradients to alleviate overfitting. Subsequently, the original fully connected layer used for ImageNet-1000 is replaced with a full_connected→softmax→ClassificationLayer triplet for 2 / 3 classification tasks, with weights initialized using Xavier and biases set to zero. During training, only newly inserted and replaced layers are unfrozen and fine-tuned; the frozen residual backbone retains the ImageNet pre-trained weights to overcome the oscillations and overfitting caused by de novo training in small sample scenarios. Fine-tuning uses driven SGD with an initial learning rate of 1e-3, cosine annealing to 1e-5, for a total of 80 epochs, with a batch size of 8. Experiments show that with only 20 training images, this improved structure increases the binary classification validation accuracy from 92.1% to 94.7%, indicating that the additional Conv-BN-ReLU module effectively enhances the ability to extract local high-order semantics.

[0040] During feature extraction, the input image first undergoes a slight "transformation process": random rotation of 0–30°, random scaling of 0.9–1.1 times, and bicubic interpolation to 224×224 to simulate subtle differences in clinical shooting angles and distances, further enhancing the model's robustness to changes in viewing angle. The image is then fed into Burn-ResNet51, where the 2048-dimensional tensor output from the last global average pooling layer is extracted as the first depth feature vector, denoted as fCNN. This vector incorporates rich local texture, edge, and mid-layer shape information, exhibiting high sensitivity to subtle differences in burn eschar, blisters, and dermal structures.

[0041] The parallel ViT pathway is responsible for capturing the global context. Using ViT-Base-16 pre-trained on ImageNet-1k, the image is divided into 196 16×16 patches, which are then linearly projected and positionally encoded before being input into a 12-layer Transformer encoder. To break away from the industry convention of "using only the last layer," this implementation extracts the 768-dimensional vector corresponding to the classification token (CLS token) layer by layer, resulting in 12 candidate features. Subsequently, through random forest importance evaluation, it was found that the output of the 4th layer is the most discriminative for binary classification tasks, while the 1st layer is optimal for tri-class classification tasks. Therefore, the corresponding CLS vector is used as the second deep feature vector, denoted as fViT. This strategy can bring an additional performance gain of approximately 3.8% on a training set of 20 images, demonstrating that the intermediate layers retain a compact semantics that is "neither too general nor too specialized," making it particularly suitable for small-sample medical scenarios. Ultimately, fCNN and fViT together constitute complementary dual-stream features. The former focuses on the texture of local lesions, while the latter emphasizes the relative relationship between the global wound layout and the surrounding healthy tissue, laying an information-rich and low-redundancy feature foundation for subsequent fusion and selection.

[0042] Specifically, step S2 further includes: dividing the input image data into multiple non-overlapping patches, considering the three channels of the image (RGB), flattening each patch into a one-dimensional vector and mapping it into a feature embedding vector through a linear projection layer.

[0043] A learnable embedding vector, called a token, is added to the beginning of the feature embedding vectors, and a learnable positional code is added to each embedding vector. The token does not correspond to any specific part of the image, but will interact with all patch embeddings in the subsequent self-attention calculation. Its corresponding output vector will be used as the aggregate representation of the entire image for the final classification task.

[0044] The embedded and encoded sequence is input into the Transformer encoder of the ViT-Base-16 model, and the feature vector corresponding to the token is extracted from the output of each encoder layer in turn to ensure that each input image data can obtain multiple global feature representations from different network depths, thus obtaining the second depth feature vector.

[0045] The ViT-Base-16 model comprises 12 cascaded, structurally identical encoder layers. Each encoder layer mainly consists of two sub-modules: a multi-head self-attention module and a multilayer perceptron. The mathematical expression for the multi-head self-attention module is as follows: Q is the query matrix, K is the key matrix, and V is the value matrix. Let T be the dimension of the key vector, and T be the transpose.

[0046] In this embodiment, the hierarchical feature optimization process is the key innovation of this method. It abandons the traditional approach of directly using the output of the last layer of ViT, and instead systematically explores all encoder layers within ViT, identifying the most discriminative intermediate layer features for a specific classification task through experimental evaluation. Unlike convolutional neural networks (CNNs), which excel at capturing local texture details, the Vision Transformer (ViT) model, through its self-attention mechanism, can effectively model long-distance dependencies in images, thereby capturing global contextual information crucial for diagnosis. Theoretically, ViT can treat an image as a sequence, analyzing the interrelationships between different regions, which is essential for understanding the overall layout of burn wounds and surrounding healthy tissue, or the distribution patterns of multiple wounds. Therefore, this method, while employing an improved CNN model to extract local features, simultaneously introduces the ViT model to extract global features, and through an innovative hierarchical extraction strategy, ensures that the acquired global features are the most discriminative.

[0047] In this embodiment, the global features are extracted from a ViT-Base-16 model pre-trained on the ImageNet-1K dataset. This model contains approximately 86 million parameters. Specifically, the standardized 224×224×3 input image is first uniformly divided into (224 / 16)×(224 / 16)=14×14=196 non-overlapping 16×16 pixel patches. Each patch contains information from the R, G, and B channels. After flattening, these patches form a 16×16×3=768-dimensional one-dimensional vector. Subsequently, each vector is mapped to a 768-dimensional feature embedding using a linear projection matrix, ensuring consistency with the hidden layer dimension of ViT-Base-16. To endow the model with the ability to aggregate the concept of the "whole image," a learnable classification token (CLS) with a dimension of 768 is inserted at the beginning of the 196 patch embedding sequences. This token does not correspond to any local region of the image, but it can interact globally with all patch tokens in the subsequent 12 layers of self-attention computation, thereby gradually absorbing the contextual information of the entire wound. Next, to compensate for the loss of spatial information caused by the scrambled patch order, the system superimposes a set of learnable one-dimensional positional encodings onto each of the 197 embedding vectors. The encoding values ​​are iteratively updated during training, enabling the network to distinguish the different spatial locations of the "central eschar" and the "edge healthy skin."

[0048] The embedded and encoded sequence is fed into the ViT-Base-16 Transformer encoder, which consists of 12 structurally identical sub-layers connected in series. Each sub-layer sequentially contains two core modules: Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP). The MHSA module is the core of ViT's global information capture, and its basic calculation formula includes... (Query) (key), The values ​​are three matrices derived from the linear transformation of the input sequence. The essence of this formula is to calculate the attention weight of each element (patch) in the sequence with respect to all other elements, thereby achieving the interaction and integration of global information. In ViT-Base-16, the MHSA module contains 12 parallel attention heads, enabling it to simultaneously focus on information from different representation subspaces.

[0049] Traditional ViT application paradigms typically use only the output of the last (11th layer) encoder as the final image features. However, a key technical insight of this invention is that, for specific medical image classification tasks, the features of the final layer may be too abstract, losing crucial mid-level texture and structural information for diagnosis. Conversely, the intermediate encoder layers may achieve an optimal balance between semantic and spatial information during the learning process. To verify this hypothesis and find the optimal feature source, this embodiment implements a systematic hierarchical feature extraction and evaluation method. Specifically, the preprocessed image is input into the ViT model, and the 768-dimensional feature vector corresponding to the token is extracted sequentially from the output of each encoder layer (from layer 0 to layer 11). This yields 12 global feature representations from different network depths for each input image. Subsequent experimental evaluations strongly support the above hypothesis. After fusing the features extracted from each layer with the CNN features, subsequent feature selection and classification experiments were conducted independently. The experimental results in Table 1 show that the model performance does not change monotonically with the increase of encoder layers, but peaks at the 4th encoder layer (Encoder-4), achieving an accuracy of 94.19% on the binary classification task; for the superclassification task, it peaks at the 1st encoder layer (Encoder-1), achieving an accuracy of 82.16%. In contrast, the performance of shallow layers (Encoder-0) and deep layers (such as Encoder-11) is significantly worse.

[0050] Table 1. Performance Comparison of ViT Encoder Layer Features (Based on Random Forest Classifier)

[0051]

[0052] The final experimental results form the core basis for selecting the intermediate layer as the feature source in this invention. Therefore, in a preferred embodiment of this invention, the ViT hierarchical feature extraction branch specifically extracts the 768-dimensional vector corresponding to the token from the outputs of Encoder-4 and Encoder-1, as the second deep feature vector representing the global information of the image.

[0053] S3: The first depth feature vector and the second depth feature vector are concatenated, and the importance of the concatenated feature vector is evaluated to select the feature subset with the highest contribution.

[0054] Specifically, step S3 further includes: concatenating the first depth feature vector and the second depth feature vector to obtain a high-dimensional combined feature vector;

[0055] Perform the high-dimensional combined feature vector Standardization processing, the standardization formula is: , The calculated Z-score is x, where x is the input sample. The pixel mean of the burn image dataset. is the pixel standard deviation of the burn image dataset;

[0056] The random forest algorithm is used to evaluate the feature importance of the standardized high-dimensional combined feature vectors and select the feature subset with the highest contribution.

[0057] Record the OOB error of features not used during random forest training. The feature values ​​in the OOB sample set are randomly transformed, and the OOB error is recalculated based on the changed feature values. ;

[0058] Feature importance is evaluated using a permutation importance metric, where the j-th feature... The mathematical expression for the feature importance is: , The number of trees in the forest.

[0059] In this embodiment, step S3 is responsible for integrating and refining the various features extracted by the preceding modules. First, local and global features are fused into a high-dimensional vector through feature concatenation. Then, a mechanism based on random forest importance evaluation is used to intelligently select the feature subset with the highest contribution to eliminate redundancy and reduce computational complexity.

[0060] Specifically, the first deep feature vector fCNN output by Burn-ResNet51 and the second deep feature vector fViT output by the ViT-Base-16 optimization layer are concatenated to form a 1024-dimensional high-dimensional combined feature vector that combines traditional visual attributes with deep learning abstract information. The concatenation uses a simple vector concatenation method, preserving both the local texture details of the CNN and the global contextual information of ViT, providing a sufficient pool of original information for subsequent "redundancy removal and selection." To avoid bias in importance evaluation due to differences in dimensionality, Z-score standardization is performed on the high-dimensional combined feature vector: first, the mean and standard deviation are calculated dimension-wise across the entire training set, and then the samples are standardized to ensure the consistency of data distribution and prevent information leakage.

[0061] The initial feature extraction process generates a large number of features. To improve model performance by selecting more effective features, this invention concatenates and combines different features, then uses a random forest-based method to evaluate and select the features. This method is commonly used to handle variable selection scenarios with dense features, and it requires fewer parameters and is easy to tune. In this implementation, a random forest is used to evaluate the feature importance of the normalized high-dimensional vector. The random forest consists of multiple decision trees, which are built from samples formed by random sampling with replacement from the training dataset.

[0062] Specifically, a forest containing multiple decision trees is constructed, with each tree trained based on Bootstrap samples. The remaining samples naturally form the out-of-bag (OOB) data. For the j-th feature, the original OOB error is first recorded; then, the value of that dimension in the OOB set is randomly shuffled to disrupt its statistical association with the label, and the prediction is repeated to calculate the new error. The average of the two differences is the ranking importance, which is an importance measure determined by the change in the average error when a feature randomly changes a value in the OOB set. The logical structure diagram of this feature selection method is as follows. Figure 5 As shown.

[0063] To select the optimal feature dimensions, this study used a Random Forest classifier with all features in experiments. Experiments showed that the model performance was optimal when the eight most important features were selected. This invention uses the RF classifier to select different numbers of features and then plots the results according to accuracy, as shown below. Figure 6 The diagram shown (the top image shows the case where burn image depth is divided into 3 categories, and the bottom image shows the case where burn image depth is divided into 2 categories) visually illustrates the impact of the number of features on classification accuracy; brighter colors indicate higher accuracy. In three-class classification (… Figure 6 (above) and binary classification ( Figure 6 In the following tasks, the performance peaked when the number of features was small (in the brightest color region), and subsequently increasing the number of features led to a performance decline. This process significantly reduced the feature dimension from 1024 dimensions to 8 dimensions, greatly improving the efficiency and robustness of the model.

[0064] S4. The feature subset is subjected to ensemble learning classification to obtain the classification results. The gradient weighted class activation mapping method is used to perform visualization analysis on the Burn-ResNet51 model and the ViT-Base-16 model to obtain the visualization results.

[0065] In this embodiment, this step receives a selected subset of features and, based on the specific classification task (such as binary or tri-class classification), employs an optimized ensemble learning classifier (such as a combined classifier consisting of random forest and XGBoost), and combines it with a specific, experimentally optimized decision threshold for final discrimination to ensure high accuracy and robustness of the classification results. Simultaneously, this invention also visualizes feature importance to create an importance heatmap, thereby providing an intuitive observation and analysis of the model's focus areas.

[0066] Specifically, step S4 further includes: inputting the feature subset into a preset random forest classifier to obtain a class probability score vector, the calculation formula of which is: , where x is the input sample For the j-th category, The total number of decision trees in the random forest. For indicator functions, For the first Each decision tree predicts the category of an input sample x.

[0067] Images with class probability scores less than a preset threshold are selected and input into a cascade classifier for secondary classification. The deep class of the cascade classifier is taken as the final classification result.

[0068] Specifically, for binary classification tasks, a cascaded classifier consisting of random forest and XGBoost is used; for tri-class classification tasks, a cascaded classifier consisting of random forest and SVM optimized by PSO is used.

[0069] In this embodiment, the selected feature subset is fed into a random forest classifier. Unlike directly outputting the final category, the core task at this stage is to generate a category probability score vector for each test sample, corresponding to the category probability score vector of the current burn image. This score vector is calculated by statistically analyzing the "voting" results of all decision trees in the forest. The indicator function indicates that its value is 1 when the internal condition is true, and 0 otherwise. This score vector quantifies the confidence that a sample belongs to each category, providing rich input information for the next stage of refined decision-making. For example, for a binary classification task (category 1 and category 2), if the output score of a sample is... This indicates that the model has a 90% confidence level in classifying the sample as belonging to class 1. This score vector, containing rich probabilistic information, will serve as the input to the second-stage classifier.

[0070] Next, the output category probability score vector itself is used as a new, highly condensed feature, which is then input into a second, more specialized classifier for the final decision. This cascaded structure can select images with scores below a set threshold based on the preliminary results of the first stage, input them into a new classifier for secondary classification, and finally take the deep category of the new classifier as the final result. For different classification tasks, this invention employs different combinations of cascaded classifiers and decision thresholds.

[0071] Specifically, to reduce the clinical risks that "barely correct" samples may pose, this implementation introduces a dynamic threshold mechanism: if the maximum class probability is lower than a preset threshold, the sample is classified as "questionable" and sent to the second-level classifier for secondary decision-making. For binary classification tasks (self-healing burns vs. non-self-healing burns), XGBoost is used in the cascade backend: 100 iterations, maximum depth 15, learning rate 0.5, and gradient boosting is used to finely adjust the decision boundary; an experimentally optimized decision threshold is used. Make the final judgment.

[0072] For the three-class classification task (superficial dermis, deep dermis, and full-thickness burns), the backend uses RBF-SVM optimized by particle swarm optimization (PSO): the PSO algorithm is used to optimize the RBF kernel function parameters of the SVM, with the C parameter ranging from [0.1, 100]. The parameter range is [0.001, 0.1]. Based on the SVM output, an experimentally optimized decision threshold is used. Make the final judgment.

[0073] Furthermore, to verify the effectiveness of the present invention, classification experiments were conducted, with system parameters set as mentioned in the preceding steps. After setting the necessary parameters, to more accurately evaluate the performance of the proposed method and further explore the relationship between these features and classification performance, the present invention designed several experiments to study these issues. Each group of experiments in the present invention was randomly repeated 10 times under the various settings described above. Then, their average classification accuracy was recorded as the final result for fair comparison. To evaluate the performance of the proposed method, accuracy, precision, recall, and F1 score were used as indicators to measure classification performance, and were defined as follows, where TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative errors, respectively. The formula is as follows: , , , .

[0074] The following describes the experiment in detail. This invention extracts and connects three different sets of features from burn images. Feature selection is performed on the feature data of 20 training images, and corresponding features are selected for the feature data of 74 test images. A random forest classifier and several cascaded classifiers derived from random forests are used, including a random forest combined with particle swarm optimization support vector machine cascade classifier (RF+PSO-SVM), a random forest combined with extreme gradient boosting cascade classifier (RF+XGBoost), and a random forest combined with k-nearest neighbor cascade classifier (RF+KNN) to classify burn depth in the burn images. Furthermore, the classification results are compared with other commonly used classifiers, including those using particle swarm optimization support vector machine (PSO-SVM), decision tree, k-nearest neighbor (KNN), long short-term memory (LSTM), one-dimensional convolutional neural network (1D-CNN), discriminative local sparse representation (DLSR), adaptive boosting algorithm for multi-label classification (AdaBoost.MH), random undersampling boosting algorithm (RUSBoost), and extreme gradient boosting (XGBoost). To determine the optimal classification strategy, this invention conducted a comprehensive performance comparison experiment on various single classifiers and cascade classifiers, and the results are shown in Table 2.

[0075] Table 2 shows the classification results of different classifiers and the classification accuracy of each category when selecting three features and classifying burn images into three and two categories.

[0076]

[0077] Table 2 clearly shows that cascaded classifiers (such as RF+PSO-SVM, RF+XGB) significantly outperform any single classifier (such as RF, XGBoost, etc.). Specifically, in binary classification tasks, the cascaded classifier composed of Random Forest and XGBoost achieved the highest accuracy of 98.51%; in tri-class classification tasks, the cascaded classifier composed of Random Forest and PSO-optimized SVM performed best with an accuracy of 88.24%. Therefore, this invention employs the optimal combination of cascaded classifiers according to different classification tasks. For cascaded classifiers, the final decision threshold... It has a significant impact on the classification results. This invention systematically evaluated the impact of different thresholds on model performance through a series of experiments, and the results are shown in Tables 3 to 5.

[0078] Table 3 shows the classification results of the RF+XGBoost classifier at different thresholds when classifying burn images into three and two categories.

[0079]

[0080] Table 4 shows the classification results of the RF+PSO-SVM classifier at different thresholds when classifying burn images into three and two categories.

[0081]

[0082] Table 5. Classification results of the RF+KNN classifier at different thresholds when classifying burn images into three and two categories.

[0083]

[0084] Specifically, step S4 further includes: for the Burn-ResNet51 model, using the gradient information of the target class, backpropagating to the last convolutional layer of the network, and calculating the weighted sum of these gradients on the feature map of the layer to generate a low-resolution class activation map, which is used to represent the spatial region that contributes the most to the final decision.

[0085] For the ViT-Base-16 model, an importance mapping at the patch level is generated by combining self-attention weights and gradient information backpropagated from the target layer.

[0086] Visualization results are obtained based on class activation graphs and importance mapping graphs.

[0087] In this embodiment, to intuitively understand and verify the decision-making basis of the feature extraction model proposed in this invention, the Gradient-weighted Class Activation Mapping (GradCAM) method is used for visual analysis of the model. This step aims to generate a heatmap, highlighting the key image regions that the model focuses on when making classification decisions, thereby providing intuitive evidence of the effectiveness of the technology selection in this invention.

[0088] For convolutional neural networks, the core principle of GradCAM is to utilize the gradient information of the target category (such as "deep dermal burn") and backpropagate it to the last convolutional layer of the network. By calculating the weighted sum of these gradients on the feature map of that layer, a low-resolution "class activation map" can be generated, which represents the spatial region that contributes the most to the final decision. The visualization results are as follows. Figure 7 As shown. Unlike CNNs, ViT models do not have traditional convolutional feature maps. Therefore, a visualization method adapted to its self-attention mechanism is required. The ViT-GradCAM method used in this invention generates an importance map at the patch level by combining self-attention weights and gradient information backpropagated from the target layer (such as the LayerNorm layer in the Transformer Block), as follows. Figure 8 As shown in the heatmap, the regions of interest (hotspots) for features extracted from different layers of ViT are focused on the broad pathological areas of the burn wound, while the regions of interest for ResNet series models are relatively concentrated. This visualization provides intuitive evidence supporting the present invention's choice of a combination of the two models as the feature source.

[0089] Specifically, in this embodiment, compared with the prior art, this method has the following advantages: 1. It achieves unprecedented high classification accuracy under extremely small sample conditions. Its most important achievement is that it still achieves extremely high classification accuracy even under the extreme condition of using only 20 training samples. In binary classification tasks, the accuracy reaches 98.51%; in tri-class classification tasks, the accuracy reaches 88.24%. Compared with the best model in the current field, the accuracy is improved by an average of about 12%, which fully demonstrates the excellent data efficiency and powerful generalization ability of the present invention framework. 2. It discovers a non-obvious optimal feature source, overturning the traditional understanding of only using the last layer of ViT features. For the first time, experiments have proven that the intermediate encoder layer (layer 4) of ViT is a better feature source than shallow and deep layers. This discovery has important technical guiding value and is the key to achieving high performance. 3. It significantly improves the efficiency and robustness of the model, reducing the 1025-dimensional feature vector to only 8 dimensions. This efficient dimensionality reduction not only avoids the loss of key information, but also significantly reduces the computational cost of subsequent classifiers by removing noise and redundant features, and enhances the model's generalization ability, effectively avoiding overfitting. This result proves that the fusion features extracted in this invention have high representativeness and information density. 4. An innovative deep feature extraction and classification decision mechanism was developed. Through structural improvements to ResNet-50 (constructing Burn-ResNet51), the extraction quality of local pathological features was significantly improved. Simultaneously, through cascaded classifiers and specific threshold selection, this invention designed optimal decision strategies for different classification tasks, further improving classification accuracy and demonstrating the refinement and innovation of the method design. 5. Unprecedentedly high classification accuracy was achieved under extremely small sample conditions. Even with only 20 training samples, extremely high classification accuracy was still achieved. As shown in Table 6, the accuracy of this invention reached 98.51% in binary classification tasks and 88.24% in tri-class classification tasks. Compared with the best results of related studies such as B. Rostami et al. (90.5% for binary classification and 77.8% for triclass classification) and B. Zhang et al. (85.86% for binary classification and 76.87% for triclass classification), the accuracy of this invention is significantly improved, with an average improvement of about 12%, which fully demonstrates the excellent data efficiency and strong generalization ability of the framework of this invention.

[0090] Table 6 compares the results of burn image classification with those of other studies when selecting three features.

[0091]

[0092] The significant synergistic effect of multi-source feature fusion was verified: This invention achieves significant performance gains by fusing traditional features, improved CNN local features, and ViT hierarchical global features. As shown in Table 7, the accuracy of binary classification using Burn-ResNet51 features alone is 92.84%, while the accuracy using ViT-base features alone is only 78.78%. This invention, by fusing, selecting, and classifying multiple features, ultimately achieves an accuracy of 98.51%, far exceeding the performance of any single feature source, strongly demonstrating the synergistic enhancement effect of the multi-source feature fusion strategy of this invention.

[0093] Table 7. Comparison of classification accuracy results using CNN features and ViT features separately.

[0094]

[0095] In summary, the deep intelligent classification method for burn images based on multi-feature extraction and selection aims to address the industry pain points of insufficient representational power of single models and the difficulty in training high-performance models under small sample conditions. By systematically mining and optimizing ViT intermediate layer features, and combining traditional image features, a specially modified CNN (Burn-ResNet51), efficient feature selection, and a task-customized cascaded ensemble learning classification process, this method can achieve high-precision and high-efficiency automatic classification, thus providing a high-performance and robust technical solution for computer-aided diagnosis.

[0096] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A deep intelligent classification method for burn images based on multi-feature extraction and selection, characterized in that, include: Obtain a preset burn image dataset and perform data preprocessing on the burn image dataset to obtain input image data; Parallel feature extraction processing is performed on the input image data to obtain a first deep feature vector extracted by the improved Burn-ResNet51 model and a second deep feature vector extracted by the ViT-Base-16 model; The first and second depth feature vectors are concatenated, and the importance of the concatenated feature vectors is evaluated to select the feature subset with the highest contribution. The feature subset is subjected to ensemble learning classification to obtain the classification result. Specifically, the feature subset is input into a preset random forest classifier to obtain a class probability score vector. Images with class probability scores less than a preset threshold are selected and input into a cascade classifier for secondary classification. The deep class of the cascade classifier is taken as the final classification result. For binary classification tasks, a cascaded classifier consisting of random forest and XGBoost is used; for tri-class classification tasks, a cascaded classifier consisting of random forest and SVM optimized by PSO is used. The gradient-weighted class activation mapping method was used to perform visualization analysis on the Burn-ResNet51 model and the ViT-Base-16 model, and the visualization results were obtained. Parallel feature extraction is performed on the input image data to obtain the second deep feature vector extracted by the ViT-Base-16 model, specifically: The input image data is divided into multiple non-overlapping small blocks. Considering the three channels of the image (RGB), each small block is flattened into a one-dimensional vector and mapped into a feature embedding vector through a linear projection layer. A learnable embedding vector, called a token, is added to the beginning of the feature embedding vectors, and a learnable positional code is added to each embedding vector. The token does not correspond to any specific part of the image, but will interact with all the small patch embeddings in the subsequent self-attention calculation. Its corresponding output vector will be used as the aggregate representation of the entire image for the final classification task. The sequence after embedding and encoding is input into the Transformer encoder of the ViT-Base-16 model, and the feature vector corresponding to the token is extracted from the output of each encoder layer in turn to ensure that each input image data can obtain multiple global feature representations from different network depths, thus obtaining the second depth feature vector. The first and second depth feature vectors are concatenated, and the importance of the concatenated feature vectors is evaluated to select the feature subset with the highest contribution. Specifically: The first and second depth feature vectors are concatenated to obtain a high-dimensional combined feature vector. The random forest algorithm is used to evaluate the feature importance of the standardized high-dimensional combined feature vectors and select the feature subset with the highest contribution.

2. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, Obtain a pre-defined burn image dataset and perform data preprocessing on the burn image dataset to obtain input image data, specifically: Acquire a dataset of burn images captured by a preset high-resolution camera, and standardize the image size of the burn image dataset to a preset pixel size. The adjusted burn image dataset is subjected to random transformation to achieve data augmentation, and the image pixel values ​​are standardized to obtain the input image data. Different standardization strategies are adopted for different deep learning models. For the ViT-Base-16 model, the standardized parameters pre-trained by ImageNet are used, and for the CNN model, the pixel values ​​are normalized to a preset range.

3. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, Parallel feature extraction is performed on the input image data to obtain the first deep feature vector extracted by the improved Burn-ResNet51 model, specifically: The input image data is transformed and then fed into a pre-improved Burn-ResNet51 model. Deep features are extracted from the last global average pooling layer of the Burn-ResNet51 model to obtain the first deep feature vector. The transformation process includes randomly rotating, randomly scaling, and adjusting the image size of the input image data.

4. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, The specific steps for improving the Burn-ResNet51 model are as follows: Obtain the original ResNet-50 model and insert three additional layers between the activation_49_relu layer and the avg_pool layer in the original ResNet-50 model architecture. The three additional layers are a Conv convolutional layer, a Batch_Normaliz batch normalization layer, and an Activation_Relu activation layer. The mathematical expression for the Conv convolutional layer is: The mathematical expression for the Batch_Normaliz batch normalization layer is: , , The mathematical expression for the Activation_ReLU activation layer is: N is the output size, W is the input size, F is the kernel size, P is the padding size, and S is the stride. This represents the final batch normalization transformation of the Batch_Normaliz batch normalization layer. For a Each activation value in Perform batch normalization transformation. and These are all parameters that need to be learned. for The i-th activation value in for The m-th activation value in for The normalized value, For a given set containing m activation values mini-batch For each middle arrive The mean, For each middle arrive variance It is a constant added to the mini-batch variance for numerical stability. Activation layer for Activation_ReLU; After the avg_pool layer, the original three classification layers are replaced with a fully connected layer, a softmax activation function layer, and a ClassificationLayer classification layer to obtain the improved Burn-ResNet51 model.

5. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, The ViT-Base-16 model comprises 12 cascaded and structurally identical encoder layers. Each encoder layer contains two sub-modules: a multi-head self-attention module and a multilayer perceptron. The mathematical expression for the multi-head self-attention module is as follows: Q is the query matrix, K is the key matrix, and V is the value matrix. Let T be the dimension of the key vector, and T be the transpose.

6. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, The random forest algorithm is used to evaluate the feature importance of the standardized high-dimensional combined feature vectors, specifically as follows: Record the OOB error of samples not used during random forest training. The feature values ​​in the OOB sample set are randomly transformed, and the OOB error is recalculated based on the changed feature values. ; Feature importance is evaluated using a permutation importance metric, where the j-th feature... The mathematical expression for the feature importance is: , The number of trees in the forest.

7. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, The formula for calculating the category probability score vector is: , where x is the input sample For the first Categories The total number of decision trees in the random forest. For indicator functions, For the first Each decision tree predicts the category of the input sample x.

8. The intelligent deep classification method for burn images based on multi-feature extraction and selection according to claim 1, characterized in that, The Burn-ResNet51 and ViT-Base-16 models were visualized using the gradient-weighted class activation mapping method, and the visualization results are as follows: For the Burn-ResNet51 model, gradient information of the target class is used to backpropagate to the last convolutional layer of the network, and the weighted sum of these gradient information on the feature map of the layer is calculated to generate a low-resolution class activation map. This activation map is used to represent the spatial region that contributes the most to the final decision. For the ViT-Base-16 model, an importance mapping at the patch level is generated by combining self-attention weights and gradient information backpropagated from the target layer. Visualization results are obtained based on class activation graphs and importance mapping graphs.

Images