Image classification method and device based on multi-scale feature fusion, equipment and medium

Abstract

The invention relates to an image classification method and device based on multi-scale feature fusion, equipment and a medium, and the method comprises the steps: obtaining a kidney CT image data set, carrying out the preprocessing of the kidney CT image data set, and obtaining a preprocessed kidney CT image data set; extracting features of the kidney CT image in the preprocessed kidney CT image data set to obtain initial features; performing multi-scale feature enhancement processing based on the initial features through a multi-scale attention residual module embedded in a pre-trained convolutional neural network to obtain enhanced features; performing pooling and classification processing based on the enhanced features to generate a target classification probability value; performing model training on the classification model by adopting a Focal-Dice composite loss function and the target classification probability value to generate a target classification model; and obtaining a to-be-processed kidney CT image, and performing image classification on the to-be-processed kidney CT image based on the target classification model to obtain a target classification result. According to the invention, the accuracy of image classification is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image classification method, apparatus, device and medium based on multi-scale feature fusion. Background Technology

[0002] Kidney diseases, a common urinary tract condition, encompass various pathological types such as renal cysts, kidney stones, and renal tumors. Early and accurate classification plays a crucial role in selecting clinical treatment plans and assessing patient prognosis. CT imaging, with its high resolution, can clearly present the anatomical structure of the kidneys and the details of lesions, making it a key diagnostic tool. However, the traditional method relying on manual interpretation by radiologists has significant drawbacks: the interpretation process is highly subjective and easily influenced by individual differences in experience. With the continuous expansion of medical image data, manual analysis is no longer sufficient to meet the demands of efficient and accurate diagnosis, prompting researchers to explore artificial intelligence technologies to improve the efficiency of medical image analysis.

[0003] In the application of deep learning to medical image analysis, the task of classifying kidney CT images still faces multiple technical bottlenecks. Traditional convolutional neural network models generally adopt a single-scale feature extraction strategy, which cannot effectively adapt to the scale diversity of lesions in kidney CT images. For example, kidney stones often appear as small or atypical stones, so the learning difficulty is unbalanced due to the complexity of lesion morphology in kidney diseases. Furthermore, the overall structure of the kidney spans a large area, and the single-scale feature extraction mechanism is prone to the loss of information on small lesions or blurred boundaries, significantly reducing the sensitivity to the recognition of complex morphological changes, resulting in low image classification accuracy. Summary of the Invention

[0004] The purpose of this application is to propose an image classification method, apparatus, device, and medium based on multi-scale feature fusion to improve the accuracy of image classification.

[0005] To address the aforementioned technical problems, embodiments of this application provide an image classification method based on multi-scale feature fusion, comprising: A kidney CT image dataset is acquired and preprocessed to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones. Initial features are obtained by extracting features of kidney CT images from the preprocessed kidney CT image dataset using a pre-trained convolutional neural network. By performing multi-scale feature enhancement processing based on the initial features using the multi-scale attention residual module embedded in the pre-trained convolutional neural network, enhanced features are obtained. Based on the enhanced features, pooling and classification processes are performed to generate target classification probability values; The classification model is trained using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model. A kidney CT image to be processed is acquired, and the kidney CT image to be processed is classified based on the target classification model to obtain the target classification result.

[0006] To address the aforementioned technical problems, embodiments of this application provide an image classification device based on multi-scale feature fusion, comprising: The data acquisition module is used to acquire a kidney CT image dataset and preprocess the kidney CT image dataset to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones. The feature extraction module is used to extract features of the kidney CT images in the preprocessed kidney CT image dataset through a pre-trained convolutional neural network to obtain initial features; The feature enhancement module is used to perform multi-scale feature enhancement processing based on the initial features through the multi-scale attention residual module embedded in the pre-trained convolutional neural network to obtain enhanced features; The image classification module is used to perform pooling and classification processing based on the enhanced features to generate target classification probability values; The model training module is used to train the classification model using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model. The classification result generation module is used to acquire the kidney CT image to be processed, and to perform image classification on the kidney CT image to be processed based on the target classification model to obtain the target classification result.

[0007] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a computer device, including one or more processors; and a memory for storing one or more programs, so that the one or more processors implement the image classification method based on multi-scale feature fusion as described above.

[0008] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the image classification method based on multi-scale feature fusion as described above.

[0009] This invention provides an image classification method, apparatus, device, and medium based on multi-scale feature fusion. The method includes: acquiring a kidney CT image dataset and preprocessing the dataset to obtain a preprocessed kidney CT image dataset, wherein the dataset includes two types of samples: normal kidneys and kidney stones; extracting features from the kidney CT images in the preprocessed dataset using a pre-trained convolutional neural network to obtain initial features; performing multi-scale feature enhancement processing based on the initial features using a multi-scale attention residual module embedded in the pre-trained convolutional neural network to obtain enhanced features; performing pooling and classification processing based on the enhanced features to generate a target classification probability value; training a classification model using a Focal-Dice composite loss function and the target classification probability value to generate a target classification model; acquiring a kidney CT image to be processed and classifying the image based on the target classification model to obtain a target classification result. This invention effectively captures lesion features at different scales in kidney CT images through multi-scale feature enhancement processing, avoiding the loss of information on small lesions. The Focal-Dice composite loss function is used to optimize the model training process. The Focal Loss function can solve the problem of learning difficulty imbalance caused by the complexity of lesion morphology, while the Dice Loss function can solve the problem of imbalance between key lesion areas and large background areas in the image. The combination of the two functions effectively improves the model's ability to discriminate complex lesions and its generalization performance, thereby improving the accuracy of image classification. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the implementation of the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 2 This is a flowchart illustrating the implementation of the first sub-process in the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 3 This is a flowchart illustrating the implementation of the second sub-process in the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 4 This is a flowchart illustrating the implementation of the third sub-process in the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 5 This is a flowchart illustrating the implementation of the fourth sub-process in the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 6 This is a flowchart illustrating the implementation of the fifth sub-process in the image classification method based on multi-scale feature fusion provided in this application embodiment; Figure 7 This is a schematic diagram of an image classification device based on multi-scale feature fusion provided in an embodiment of this application; Figure 8 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0015] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] It should be noted that the image classification method based on multi-scale feature fusion provided in this application embodiment is generally executed by a server, and correspondingly, the image classification device based on multi-scale feature fusion is generally configured in the server.

[0017] Please see Figure 1 , Figure 1 A specific implementation of an image classification method based on multi-scale feature fusion is shown.

[0018] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 1 Limited to the order of the processes shown, this method includes the following steps: S1: Obtain a kidney CT image dataset and preprocess the kidney CT image dataset to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones.

[0019] Specifically, obtaining a kidney CT image dataset can be understood as extracting a collection of CT images containing both normal kidneys and kidney stones from medical imaging databases or equipment. The dataset can be categorized and labeled manually or using automated annotation tools. Its primary purpose is to provide foundational data for model training and testing. The preprocessed kidney CT image dataset can be generated by normalizing, denoising, and resizing the original images. For example, histogram equalization can be used to improve image contrast, or image sizes can be standardized through cropping and scaling. The aim is to ensure the consistency and quality of the input data.

[0020] In one specific embodiment, the kidney CT image dataset comes from two datasets on the Kaggle public platform. Dataset 1, named "CT_SCAN," contains a root directory named "CT_SCAN," which is divided into two categories: "Kidney_stone" and "Normal," with 781 and 828 images respectively, totaling 1609 CT scan images. Dataset 2, named "CT image of kidney," contains a root directory named "Dataset," which is divided into two categories: "Normal" and "Stone," with 608 and 992 images respectively, totaling 1600 CT scan images. Both datasets contain samples of normal and abnormal kidneys, and all images have been labeled and confirmed by a professional medical annotation team, meeting the standards for medical image analysis and research.

[0021] S2: Extract the features of the kidney CT images in the preprocessed kidney CT image dataset using a pre-trained convolutional neural network to obtain initial features.

[0022] Specifically, the VGG16 network is used as a pre-trained convolutional neural network. By freezing the shallow convolutional weights of the VGG16 network to retain its general feature extraction capabilities, features of kidney CT images in the preprocessed kidney CT image dataset are extracted to obtain initial features.

[0023] S3: Enhanced features are obtained by performing multi-scale feature enhancement processing based on the initial features through the multi-scale attention residual module embedded in the pre-trained convolutional neural network.

[0024] Specifically, the multi-scale attention residual module can be implemented by designing different parallel path structures. For example, different sized convolutional kernels can be used to extract local, intermediate, and global features respectively, or a pyramid pooling module can be introduced to capture multi-scale contextual information. Its main function is to perform multi-level enhancement processing on the initial features. Please see Figure 2 , Figure 2 A specific implementation of step S3 is shown below: S31: Multi-scale features are obtained by extracting multi-scale features from the initial features using a multi-scale attention residual module embedded in the pre-trained convolutional neural network. S32: Attention features are generated based on the multi-scale features using a spatial attention mechanism. S33: The enhanced features are generated by performing feature weighting and residual connections based on the attention features and the multi-scale features.

[0025] Multi-scale feature extraction refers to the process of capturing image information from different receptive fields, which can be achieved by designing parallel convolutional paths with different kernel sizes or timings. In practical applications, these parallel convolutional paths can capture local detail features, intermediate-scale features, and global contextual features respectively, thus ensuring that the features of small lesions and complex morphological changes in kidney CT images are fully covered. Spatial attention mechanism can be understood as a method of dynamically adjusting feature weights, aiming to highlight information in key regions. It can generate attention features through a combination of global average pooling and global max pooling. Feature weighting is a technique to enhance salient information, achieved by element-wise multiplication of attention features with multi-scale features, aiming to improve the feature discriminativeness of lesion regions. Residual connection is used to preserve the integrity of the initial features and avoid information loss, usually achieved by adjusting the number of channels and adding them to the initial features.

[0026] Specifically, by clarifying the implementation path of multi-scale feature enhancement processing, the problem of insufficient feature extraction is effectively solved. First, the multi-scale feature extraction step can simultaneously capture local details, intermediate-scale features, and global contextual features, which avoids the limitations of single-scale feature extraction and ensures that the features of small lesions and complex morphological changes in kidney CT images are fully covered. Second, the spatial attention mechanism uses multi-scale contextual information to dynamically adjust feature weights, highlighting the spatial relationship between lesions and normal tissues, and enhancing the model's ability to focus on key regions. Finally, the combination of feature weighting and residual connections not only strengthens the salient information of the lesion region but also maintains the robustness of the feature representation, laying a high-quality feature foundation for subsequent classification processing. In addition, the above scheme, combined with a pre-trained convolutional neural network, can fully utilize the prior knowledge of the pre-trained model to further improve the accuracy of feature classification.

[0027] Please see Figure 3 , Figure 3 A specific implementation of step S31 is shown below: S311: Use the VGG16 network as the pre-trained convolutional neural network, and embed the multi-scale attention residual module after pre-setting convolutional blocks in the VGG16 network. S312: Extract features based on the initial features using the three parallel convolutional paths in the multi-scale attention residual module to obtain initial multi-scale features, wherein the initial multi-scale features include local detail features, intermediate-scale features, and global context features. S313: Concatenate the different initial multi-scale features along the channel dimension to obtain the multi-scale features.

[0028] The multi-scale attention residual module is a structure specifically designed to capture multi-scale feature information, which can be implemented using multiple parallel convolutional paths in conjunction with a spatial attention mechanism. This module aims to address the insufficient sensitivity of traditional single-scale feature extraction methods in identifying minute lesions and complex morphological changes. In practical applications, the VGG16 network is a widely used deep learning model in medical image analysis, which can effectively extract spatial features of images through multi-layer convolutional operations. Embedding the multi-scale attention residual module on this basis can optimize the feature extraction process for key levels of kidney anatomy, thereby avoiding the omission of minute lesions. Furthermore, local detail features focus on subtle textures and edge information in the image, medium-scale features focus on morphological boundary changes of lesions, and global context features aim to capture the structural correlations of the entire kidney region.

[0029] Specifically, by embedding multi-scale attention residual modules after specific convolutional blocks in the VGG16 network, effective integration of multi-scale features from kidney CT images was achieved. Specifically, three parallel convolutional paths are responsible for extracting local detail features, intermediate-scale features, and global contextual features, respectively. This design ensures comprehensive coverage of lesion features from subtle to macroscopic levels. For example, when processing kidney stones, the local path captures the minute texture details of the stone, the intermediate path handles the boundary changes between the stone and surrounding tissues, and the global path captures the contextual information of the entire kidney region. Subsequently, these features at different scales are concatenated along the channel dimension to generate fused multi-scale features. This approach not only preserves the independent information of each scale feature but also avoids the feature dilution problem caused by spatial dimension concatenation, thus providing richer and more discriminative input for subsequent classification decisions.

[0030] In one specific embodiment, the kernel sizes corresponding to a single parallel convolution path are 1×1, 3×3, and 5×5, respectively, capturing local detail features, medium-scale features, and global context features.

[0031] The embodiments of this application significantly improve the recognition capability of kidney images, while addressing the shortcomings of existing methods in multi-scale feature capture. Based on this, by combining the mature performance of pre-trained convolutional neural networks and the optimized design of multi-scale attention residual modules, a complete feature extraction process is formed, laying a solid foundation for subsequent classification tasks.

[0032] Please see Figure 4 , Figure 4 A specific implementation of step S32 is shown below: S321: Perform global average pooling and global max pooling on the multi-scale features respectively to obtain average pooling features and max pooling features. S322: Concatenate the average pooling features and the max pooling features to obtain concatenated features. S323: Generate attention features by performing convolution and activation processing on the concatenated features.

[0033] Global average pooling involves averaging all elements of the input feature map to generate a fixed-length feature vector. This can be achieved using standard average pooling or weighted average pooling, aiming to capture the overall distribution characteristics of lesion areas in kidney CT images. Global max pooling extracts the maximum response value from the input feature map. This can be achieved using standard max pooling or block max pooling, aiming to highlight local salient features such as small stones. Concatenation merges two feature vectors along a specific dimension into a higher-dimensional feature representation. This can be achieved through channel-dimensional or spatial-dimensional concatenation, aiming to integrate spatial information extracted by different pooling methods. Convolution is used to learn local dependencies in the spatial dimension. This can be implemented using single-layer or multi-layer convolutional structures, aiming to enhance feature representation capabilities to adapt to the complexity of kidney anatomy. Activation generates normalized attention weights, which can be implemented using the sigmoid or softmax activation functions, aiming to dynamically adjust the importance of different regions in the feature map.

[0034] Specifically, the generation process of spatial attention features was optimized through a dual-path pooling and feature fusion strategy. First, global average pooling generates an average response based on the overall distribution of the feature map, reflecting the global statistical characteristics of kidney tissue and aiding in the identification of large-scale lesion areas. Global max pooling, on the other hand, focuses on the most significant local response points in the feature map, highly sensitive to subtle abnormalities such as small stones, avoiding the shortcomings of single pooling methods that ignore crucial details. Subsequently, by concatenating the results of the two pooling methods, a more comprehensive spatial descriptor is constructed, enabling the model to simultaneously utilize both overall distribution and local peak information to accurately infer the spatial location and morphological relationships of lesions within the kidney. Finally, convolution and activation processing of the concatenated features not only enhances the feature representation ability to adapt to the complexity of kidney anatomy but also generates normalized attention weights, allowing the model to dynamically adjust the importance of each region in the feature map, highlighting stone lesion areas and suppressing background noise.

[0035] The embodiments of this application effectively solve the problem of insufficient capture of spatial context information in kidney CT images. By combining the complementary advantages of global average pooling and global max pooling, the spatial correlation characteristics between lesions and normal tissues in kidney CT images are fully captured, thereby improving the pertinence of feature enhancement and the reliability of classification decisions.

[0036] Please see Figure 5 , Figure 5 A specific implementation of step S33 is shown below: S331: Multiply the attention feature and the multi-scale feature element-wise to obtain a weighted feature. S332: Adjust the number of channels in the weighted feature to match the number of channels in the initial feature, obtaining the adjusted feature. S333: Perform a residual concatenation between the adjusted feature and the initial feature to generate the enhanced feature.

[0037] Weighted features refer to the feature representation obtained by selectively enhancing multi-scale features through an attention mechanism. This can be achieved through element-wise multiplication, aiming to highlight key information about lesion areas in kidney CT images while suppressing interference from irrelevant features. Adjusted features can be understood as weighted features after channel number adjustment, which can be achieved through 1x1 convolution operations or channel clipping, aiming to ensure consistency in feature dimensions and provide a structural basis for subsequent residual connections. Enhanced features refer to the final feature representation that fuses the initial features and attention-weighted multi-scale features, effectively preserving original contextual information and strengthening the discriminative ability of the features.

[0038] Specifically, firstly, attention features and multi-scale features are weighted using element-wise multiplication. This operation leverages the selective enhancement of important features by the attention mechanism, enabling the model to focus on key information in lesion areas of kidney CT images. Subsequently, adjusting the number of channels in the weighted features resolves the residual connectivity barrier caused by feature dimensionality mismatch. This not only ensures the feasibility of feature alignment but also prevents the loss of original feature information. Finally, the adjusted features are residually connected to the initial features. This design preserves the complete contextual information of the initial features and improves the robustness and hierarchy of the feature representation by fusing attention-weighted multi-scale features.

[0039] This application's embodiments effectively solve the problem of inconsistency between the number of feature channels after feature weighting and the initial number of feature channels by introducing a channel number adjustment step, thus avoiding feature information loss and model training instability. Simultaneously, the combined use of element-wise multiplication and residual connections not only enhances the effective fusion of multi-scale features but also significantly improves the accuracy of kidney CT image classification, particularly excelling in distinguishing subtle differences between kidney stones and normal tissue. Furthermore, this scheme, combined with the aforementioned multi-scale feature extraction and spatial attention mechanism, forms a complete feature enhancement path, thereby further improving the model's generalization ability and classification reliability.

[0040] S4: Based on the enhanced features, perform pooling and classification processing to generate target classification probability values.

[0041] Please see Figure 6 , Figure 6 A specific implementation of step S4 is shown below: S41: Perform global average pooling on the enhanced features to obtain a feature vector. S42: Output a target feature vector of dimension 1 based on the feature vector through a fully connected layer. S43: Classify the target feature vector using a sigmoid activation function to generate a target classification probability value.

[0042] Global average pooling refers to compressing multi-scale spatial information into a fixed-length feature representation by averaging the enhanced features across the spatial dimension. Its purpose is to eliminate spatial redundancy caused by the diversity of lesion morphology while retaining key discriminative features. In practice, this operation can be achieved by averaging all spatial locations of each feature channel, ensuring the generated feature vector is compact and robust. A fully connected layer can be understood as a linear mapping structure used to directly map the input feature vector to the target output space. Specifically, its function is to map the feature vector generated by global average pooling to the decision space of two types of samples: normal kidneys and kidney stones. In practice, fully connected layers can be implemented using matrix multiplication combined with bias terms, aiming to simplify the network structure and reduce interference from irrelevant parameters, thereby improving the model's sensitivity to small lesions.

[0043] The Sigmoid activation function is a non-linear transformation tool used to normalize the output of a target feature vector into probability values ​​between 0 and 1. Its purpose is to provide an intuitive probabilistic output, facilitating interpretation and diagnosis by clinicians. It should be noted that this application's embodiments pertain to image classification and do not directly perform medical or treatment diagnoses; rather, they only classify kidney CT images to provide doctors with categorized images for subsequent clinical diagnosis.

[0044] S5: The classification model is trained using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model.

[0045] Specifically, the Focal-Dice composite loss function can achieve a dynamic balance between the two loss functions by adjusting the weight coefficient λ. For example, the value of λ can be automatically adjusted based on the model's performance during training, or a fixed weight allocation scheme can be determined experimentally. Its main function is to alleviate the problem of class imbalance and improve the model's adaptability to complex lesion morphologies. The generation process of the target classification model can be achieved by setting different optimization algorithms and hyperparameter configurations, such as using the Adam optimizer or SGD optimizer for model training. The purpose is to improve the model's generalization ability and classification accuracy.

[0046] Furthermore, the Focal-Dice composite loss function is a weighted sum of the Focal Loss function and the Dice Loss function, and the expression for the Focal-Dice composite loss function is as follows: ; The formula for calculating the Focal Loss function is as follows: ; The formula for calculating the Dice Loss function is as follows: ; in, λ These are the weighting coefficients. y For real labels, To predict probabilities, γ As a regulating factor, α For class balancing parameters, ϵ For smoothing coefficients, The Focal Loss value. The value of Dice Loss. The first in the preprocessed kidney CT image dataset One sample, For the first The true label of each sample No. The predicted probability of a sample.

[0047] In the embodiments of this application, the Focal Loss function dynamically reduces the loss contribution of easily classified samples by adjusting the factor (γ), so that the model training is more focused on difficult-to-classify samples, such as small or atypical stones; the Dice Loss function guides the model to learn feature representations that are more consistent with the lesion morphology by measuring the overlap between the predicted area of ​​interest and the actual lesion area. The combination of the two effectively improves the model's ability to distinguish complex lesions and its generalization performance.

[0048] In one specific embodiment, a standard VGG16 deep convolutional neural network is used to classify kidney CT images. The specific implementation process includes: (1) using the kidney CT scan image dataset publicly available on the Kaggle platform, which includes two types of samples: normal kidneys and abnormal kidneys. All images have been labeled and confirmed by a professional medical annotation team and meet the standards for medical image analysis research. The dataset is divided into training set, validation set, and test set in a ratio of 7:1.5:1.5; (2) constructing a standard VGG16 classification model, loading ImageNet pre-trained weights, removing the original top-level classifier, and adding a global average pooling layer and a fully connected layer with an output dimension of 1; (3) training the model using the binary cross-entropy loss function and the Adam optimizer, with the learning rate set to 1×10. -5 The batch size is 32, and the training rounds are 15 epochs; (4) Evaluate the model performance on the independent test set and record the accuracy, precision, recall and F1 score.

[0049] In another embodiment, an improved VGG16 model with an embedded multi-scale attention residual module is constructed to classify kidney CT images. The specific implementation process includes: (1) using the same dataset partitioning method and preprocessing process as the previous embodiment to ensure consistent experimental conditions; (2) constructing a VGG16 model with an embedded multi-scale attention residual module, inserting a custom multi-scale attention residual module after the block4_pool layer of VGG16, which contains three parallel convolutional paths; (3) using the Focal-Dice composite loss function for model optimization, which combines the advantages of Focal Loss and Dice Loss, where λ is set to 1.0, α is set to 1.0, γ is set to 1.0, and ϵ is a smoothing coefficient of 10. -7 (4) Employ an early stopping mechanism and model checkpoint callback. When the validation set AUC index does not improve within 5 consecutive epochs, automatically stop training and save the best performing model weights. (5) Evaluate the performance of the improved model on the same test set and compare it with the previous embodiment.

[0050] The evaluation metrics used for the test set include: accuracy, precision, recall, and F1 score, and their calculation formulas are as follows: ; Where TP represents the number of true positive samples, TN represents the number of true negative samples, FP represents the number of false positive samples, FN represents the number of false negative samples, TPR is the true positive rate, and FPR is the false positive rate.

[0051] The performance comparison of each model on the test set is shown in Table 1: Table 1 shows the performance comparison results of each model on the test set. As shown in Table 1, the embodiments of this application significantly outperform other comparative models on all evaluation metrics. In particular, compared with the baseline model, the synergistic optimization effect of the multi-scale attention mechanism and the composite loss function is demonstrated. Cross-validation results on two independent renal CT datasets show that the method of this invention has stable generalization performance and demonstrates good clinical applicability.

[0052] To further optimize the performance of the Focal-Dice composite loss function, this application conducted a systematic sensitivity analysis on the key hyperparameter γ. As shown in Table 2, the impact of γ values ​​in the range of 0.5 to 3.0 on the model's classification performance was tested. Table 2 Sensitivity analysis of hyperparameter γ S6: Acquire the kidney CT image to be processed, and perform image classification on the kidney CT image to be processed based on the target classification model to obtain the target classification result.

[0053] In this embodiment, a kidney CT image dataset is first acquired and preprocessed to obtain a preprocessed dataset containing two types of samples: normal kidneys and kidney stones. Further, a pre-trained convolutional neural network is used to extract features from the preprocessed kidney CT images to generate initial features. While the pre-trained convolutional neural network quickly acquires basic visual representations through transfer learning, the initial features are still limited by a single-scale extraction strategy, making it difficult to fully capture multi-scale information of the lesion area.

[0054] To address the aforementioned issues, a multi-scale attention residual module is embedded in the pre-trained convolutional neural network to perform multi-scale feature enhancement based on the initial features. Specifically, this module simultaneously extracts local detail features, intermediate-scale features, and global contextual features, and uses a spatial attention mechanism to focus on the spatial association region between lesions and normal tissues, thereby specifically enhancing the feature responses of micro-stones and complex morphological changes. This significantly improves the recognition sensitivity of multi-scale features, generating enhanced features. Based on the enhanced features, global average pooling is used to integrate the global information of the multi-scale enhanced features, and a fully connected layer and a sigmoid activation function are used to output classification probability values, generating target probability values. This process ensures that the decision-making process fully relies on the optimized feature representation, enhancing the model's ability to discriminate kidney lesions. Furthermore, the classification model is trained using a Focal-Dice composite loss function combined with the target probability value, dynamically adjusting the loss weights. The Focal Loss component effectively alleviates the class imbalance between normal kidney and stone samples, while the Dice Loss component enhances the sensitivity to the morphological diversity of lesion areas. The weighted combination of these two components makes the model training more closely reflect the pathological characteristics of kidney CT images. Finally, the kidney CT images to be processed are acquired, and the trained target classification model is used to classify the images, yielding the target classification results. The overall technical solution, through the synergistic design of feature enhancement and loss functions, solves the problems of low sensitivity and insufficient classification accuracy in identifying small lesions in traditional methods, achieving automated and accurate identification of kidney stones.

[0055] In this embodiment, by integrating a multi-scale feature fusion mechanism, the problems of traditional methods in kidney CT image classification—namely, the difficulty in capturing multi-scale information through single-scale feature extraction, the neglect of spatial correlation between lesions and normal tissues, and the failure to optimize the loss function for lesion morphological diversity—are addressed. By embedding a multi-scale attention residual module into a pre-trained convolutional neural network, efficient identification of minute lesions and complex morphological changes is achieved. Simultaneously, the Focal-Dice composite loss function enhances the model's adaptability to class imbalance and lesion morphological diversity, thereby improving the accuracy of kidney CT image classification.

[0056] Please refer to Figure 7 As a response to the above Figure 1 To implement the method shown, this application provides an embodiment of an image classification device based on multi-scale feature fusion. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0057] like Figure 7 As shown, the image classification device based on multi-scale feature fusion in this embodiment includes: a data acquisition module 71, a feature extraction module 72, a feature enhancement module 73, an image classification module 74, a model training module 75, and a classification result generation module 76, wherein: The data acquisition module 71 is used to acquire a kidney CT image dataset and preprocess the kidney CT image dataset to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones. Feature extraction module 72 is used to extract features of kidney CT images in the preprocessed kidney CT image dataset through a pre-trained convolutional neural network to obtain initial features; Feature enhancement module 73 is used to perform multi-scale feature enhancement processing based on the initial features through the multi-scale attention residual module embedded in the pre-trained convolutional neural network to obtain enhanced features; Image classification module 74 is used to perform pooling and classification processing based on the enhanced features to generate target classification probability values; Model training module 75 is used to train the classification model using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model; The classification result generation module 76 is used to acquire the kidney CT image to be processed and perform image classification on the kidney CT image to be processed based on the target classification model to obtain the target classification result.

[0058] Furthermore, the feature enhancement module 73 includes: A multi-scale feature generation unit is used to extract multi-scale features from the initial features by using a multi-scale attention residual module embedded in the pre-trained convolutional neural network; An attention feature generation unit is used to generate attention features based on the multi-scale features through a spatial attention mechanism. The enhanced feature generation unit is used to generate the enhanced features by performing feature weighting and residual connection based on the attention features and the multi-scale features.

[0059] Furthermore, the multi-scale feature generation unit includes: The neural network determination subunit is used to use the VGG16 network as the pre-trained convolutional neural network and embed the multi-scale attention residual module after pre-setting convolutional blocks in the VGG16 network. An initial multi-scale feature generation subunit is used to extract features based on the initial features through three parallel convolutional paths in the multi-scale attention residual module to obtain initial multi-scale features, wherein the initial multi-scale features include local detail features, medium-scale features, and global context features; An initial multi-scale feature splicing subunit is used to splice different initial multi-scale features along the channel dimension to obtain the multi-scale features.

[0060] Furthermore, the attention feature generation unit includes: The pooling processing subunit is used to perform global average pooling and global max pooling on the multi-scale features respectively to obtain average pooling features and max pooling features. The pooling feature splicing subunit is used to splice the average pooling feature and the max pooling feature to obtain the spliced ​​feature; An activation processing subunit is used to generate attention features by performing convolution and activation processing on the concatenated features.

[0061] Furthermore, the enhanced feature generation unit includes: The feature weighting subunit is used to multiply the attention features and the multi-scale features element-wise to obtain weighted features; The channel number adjustment subunit is used to adjust the channel number of the weighted feature so that the channel number of the weighted feature is consistent with the channel number of the initial feature, thereby obtaining the adjusted feature; The residual connection subunit is used to perform a residual connection between the adjusted feature and the initial feature to generate the enhanced feature.

[0062] Furthermore, the image classification module 74 includes: A feature vector generation unit is used to perform global average pooling on the enhanced features to obtain feature vectors; The target feature vector generation unit is used to output a target feature vector of dimension 1 based on the feature vector through a fully connected layer; The classification processing unit is used to classify the target feature vector using the Sigmoid activation function to generate a target classification probability value.

[0063] Furthermore, the Focal-Dice composite loss function is a weighted sum of the Focal Loss function and the Dice Loss function, and the expression for the Focal-Dice composite loss function is as follows: ; The formula for calculating the Focal Loss function is: ; The formula for calculating the Dice Loss function is: ; in, λ These are the weighting coefficients. y For real labels, To predict probabilities, γ As a regulating factor, α For class balancing parameters, ϵ For smoothing coefficients, The Focal Loss value. The value of Dice Loss. The first in the preprocessed kidney CT image dataset One sample, For the first The true label of each sample No. The predicted probability of a sample.

[0064] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed] for details. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0065] Computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that... Figure 8Only a computer device 8 with three components—memory 81, processor 82, and network interface 83—is shown. It should be understood that implementing all shown components is not required; more or fewer components may be implemented alternatively. Those skilled in the art will understand that this computer device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices.

[0066] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0067] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the memory 81 may also include both internal storage units and external storage devices of the computer device 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the computer device 8, such as program code for image classification methods based on multi-scale feature fusion. In addition, the memory 81 may also be used to temporarily store various types of data that have been output or will be output.

[0068] In some embodiments, processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, processor 82 is used to run program code stored in memory 81 or process data, for example, to run the program code of the image classification method based on multi-scale feature fusion described above, to implement various embodiments of the image classification method based on multi-scale feature fusion.

[0069] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.

[0070] This application also provides another embodiment, namely, providing a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the image classification method based on multi-scale feature fusion as described above.

[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0072] Obviously, the embodiments described above are merely some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of protection of this application.

Claims

1. An image classification method based on multi-scale feature fusion, characterized in that, include: A kidney CT image dataset is acquired and preprocessed to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones. Initial features are obtained by extracting features of kidney CT images from the preprocessed kidney CT image dataset using a pre-trained convolutional neural network. By performing multi-scale feature enhancement processing based on the initial features using the multi-scale attention residual module embedded in the pre-trained convolutional neural network, enhanced features are obtained. Based on the enhanced features, pooling and classification processes are performed to generate target classification probability values; The classification model is trained using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model. A kidney CT image to be processed is acquired, and the kidney CT image to be processed is classified based on the target classification model to obtain the target classification result.

2. The image classification method based on multi-scale feature fusion according to claim 1, characterized in that, The step of performing multi-scale feature enhancement processing based on the initial features using the multi-scale attention residual module embedded in the pre-trained convolutional neural network to obtain enhanced features includes: Multi-scale features are obtained by extracting multi-scale features from the initial features using the multi-scale attention residual module embedded in the pre-trained convolutional neural network; Attention features are generated based on the multi-scale features using a spatial attention mechanism; The enhanced features are generated by performing feature weighting and residual connections based on the attention features and the multi-scale features.

3. The image classification method based on multi-scale feature fusion according to claim 2, characterized in that, The step of extracting multi-scale features from the initial features by using a multi-scale attention residual module embedded in the pre-trained convolutional neural network includes: The VGG16 network is used as the pre-trained convolutional neural network, and the multi-scale attention residual module is embedded after pre-setting convolutional blocks in the VGG16 network. The initial multi-scale features are obtained by extracting features based on the initial features through three parallel convolutional paths in the multi-scale attention residual module. The initial multi-scale features include local detail features, medium-scale features, and global context features. The different initial multi-scale features are spliced ​​along the channel dimension to obtain the multi-scale features.

4. The image classification method based on multi-scale feature fusion according to claim 2, characterized in that, The generation of attention features based on the multi-scale features through a spatial attention mechanism includes: Global average pooling and global max pooling are performed on the multi-scale features respectively to obtain average pooling features and max pooling features; The average pooling feature and the max pooling feature are concatenated to obtain the concatenated feature; Attention features are generated by performing convolution and activation processing on the concatenated features.

5. The image classification method based on multi-scale feature fusion according to claim 2, characterized in that, The step of generating the enhanced features by performing feature weighting and residual connection based on the attention features and the multi-scale features includes: The attention features and the multi-scale features are multiplied element-wise to obtain the weighted features; The number of channels in the weighted feature is adjusted so that the number of channels in the weighted feature is consistent with the number of channels in the initial feature, resulting in the adjusted feature; The adjusted features are residually concatenated with the initial features to generate the enhanced features.

6. The image classification method based on multi-scale feature fusion according to any one of claims 1 to 5, characterized in that, The process of pooling and classifying based on the enhanced features to generate target classification probability values ​​includes: The enhanced features are subjected to global average pooling to obtain a feature vector; The fully connected layer outputs a target feature vector of dimension 1 based on the feature vector. The target feature vector is classified using the Sigmoid activation function to generate target classification probability values.

7. The image classification method based on multi-scale feature fusion according to any one of claims 1 to 5, characterized in that, The Focal-Dice composite loss function is a weighted sum of the Focal Loss function and the Dice Loss function. The expression for the Focal-Dice composite loss function is as follows: ； The formula for calculating the Focal Loss function is as follows: ； The formula for calculating the Dice Loss function is as follows: ； in, λ These are the weighting coefficients. y For real labels, To predict probabilities, γ As a regulating factor, α For class balancing parameters, ϵ For smoothing coefficients, The Focal Loss value. The value of Dice Loss. The first in the preprocessed kidney CT image dataset One sample, For the first The true label of each sample No. The predicted probability of a sample.

8. An image classification device based on multi-scale feature fusion, characterized in that, include: The data acquisition module is used to acquire a kidney CT image dataset and preprocess the kidney CT image dataset to obtain a preprocessed kidney CT image dataset, wherein the kidney CT image dataset includes two types of samples: normal kidneys and kidney stones. The feature extraction module is used to extract features of the kidney CT images in the preprocessed kidney CT image dataset through a pre-trained convolutional neural network to obtain initial features; The feature enhancement module is used to perform multi-scale feature enhancement processing based on the initial features through the multi-scale attention residual module embedded in the pre-trained convolutional neural network to obtain enhanced features; The image classification module is used to perform pooling and classification processing based on the enhanced features to generate target classification probability values; The model training module is used to train the classification model using the Focal-Dice composite loss function and the target classification probability value to generate the target classification model. The classification result generation module is used to acquire the kidney CT image to be processed, and to perform image classification on the kidney CT image to be processed based on the target classification model to obtain the target classification result.

9. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image classification method based on multi-scale feature fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image classification method based on multi-scale feature fusion as described in any one of claims 1 to 7.

Images