Image classification method based on improved ResNet-Unet network
By using an improved ResNet-Unet network, combined with SE and KAN modules, the problems of low contrast and noise interference in thyroid ultrasound image classification were solved, improving the accuracy and consistency of nodule classification and enhancing the model's adaptability and generalization ability.
Patent Information
- Application Number
- CN202510930194.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-07
- Publication Date
- 2025-11-07
AI Technical Summary
Existing thyroid ultrasound image classification methods suffer from problems such as low contrast, noise interference, and insufficient expression of local features, making it difficult to effectively perform automatic detection and classification of nodules.
An improved ResNet-Unet network is adopted, which combines the SE module and the KAN module to enhance the channel and spatial attention mechanism. Conditional random fields and Bayesian optimization are combined to optimize the image classification model.
It significantly improved the accuracy and spatial consistency of nodule benign and malignant discrimination, enhanced the model's ability to discriminate low-contrast ultrasound images, reduced the cost of manual parameter tuning, and improved the model's generalization ability on different datasets.
Smart Images
Figure CN120912945A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, and in particular to an image classification method based on an improved ResNet-Unet network. BACKGROUND
[0002] Thyroid nodules are one of the common thyroid diseases worldwide. Clinically, ultrasound imaging is usually used as the preferred examination method to evaluate the size, shape, boundary characteristics and blood flow of the nodules, thereby assisting in the judgment of benignity and malignancy.
[0003] Due to the particularity of ultrasound imaging, thyroid ultrasound images usually have problems such as low contrast, speckle noise, tissue overlap and probe angle variation, making automatic detection and classification of nodules challenging. Traditional image analysis methods are prone to failure in complex scenarios and are difficult to adapt to the differences in ultrasound imaging of different individuals.
[0004] In recent years, ultrasound image classification methods based on deep learning have made some progress, in which convolutional neural networks (CNN) and Transformer structures are widely used for feature extraction and classification. However, traditional CNN mainly relies on local receptive fields and is difficult to effectively model global shape features. Transformer is prone to overfitting on small sample medical data. SUMMARY
[0005] In view of the shortcomings of existing methods, the present application solves the problems of low contrast, noise interference and insufficient expression of local features in existing thyroid ultrasound image classification methods.
[0006] The technical solution adopted by the present application is: an image classification method based on an improved ResNet-Unet network includes the following steps: Step 1: Collect thyroid ultrasound images; As a preferred embodiment of the present application, the image is normalized for pretreatment.
[0007] As a preferred embodiment of the present application, the normalized image is subjected to CLAHE operation.
[0008] Step 2: Construct an improved ResNet-Unet network; the encoder of the ResNet-Unet network adopts a ResNet50 network, and an SE module is introduced after each encoding layer of the first encoding layer Enc1 to the fourth encoding layer Enc4 of the ResNet50 network; As a preferred embodiment of the present application, a KAN module is introduced after the SE module of the fourth encoding layer Enc4, and the output features of the KAN model are connected to the first decoding layer Dec4 of the Unet network decoder; the encoding layers and the corresponding decoding layers are connected by a skip connection; As a preferred embodiment of the present application, the improved ResNet-Unet network classifier comprises: a global average pooling and a fully connected layer.
[0009] As a preferred embodiment of the present application, the improved ResNet-Unet network adopts an improved cross-entropy loss ; wherein, is the class weight, represents the number of samples of the class , is the classification probability.
[0010] As a preferred embodiment of the present application, the improved ResNet-Unet network adopts an AdamW optimizer.
[0011] As a preferred embodiment of the present application, the improved ResNet-Unet network adopts a conditional random field.
[0012] As a preferred embodiment of the present application, the image classification system based on the improved ResNet-Unet network comprises: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the image classification method based on the improved ResNet-Unet network.
[0013] As a preferred embodiment of the present application, the computer readable medium storing computer program code implements the image classification method based on the improved ResNet-Unet network when executed by a processor.
[0014] The present application has the following advantages: 1. The introduction of medical knowledge graph enhances the recognition accuracy of the segmentation model for low-contrast lesion regions; 2. Local region adaptive calculation is performed using KAN, so that the classification network pays more attention to key regions such as nodule boundaries and calcification regions, and improves the accuracy of nodule benignity and malignancy discrimination; 3. Post-processing is performed in combination with a conditional random field (CRF) to optimize the classification boundary, improve the spatial consistency of the classification result, and reduce the occurrence of misclassification; 4. The Bayesian optimization method is introduced to automatically adjust the hyperparameters, so that the model has better generalization ability on different data sets and reduces the cost of manual parameter adjustment; 5. The SE module is introduced to enhance the channel perception ability: by dynamically learning the channel importance weight, irrelevant channel noise is suppressed, and the sensitivity to key features such as nodule texture and calcification region is improved; 6. Form a double attention mechanism: SE module (channel attention) and KAN (spatial attention) cooperate to optimize feature expression from channel and spatial dimensions, significantly improving the model's discriminant ability for low-contrast ultrasound images. BRIEF DESCRIPTION OF DRAWINGS
[0015] Figure 1 is the improved ResNet-Unet network structure diagram based on the application; Figure 2 is an SE module schematic diagram; Figure 3 is a KAN module structure diagram; Figure 4 is an SE, KAN module fusion schematic diagram; Figure 5 is an ablation comparison experiment heat map of the improved ResNet-Unet. DETAILED DESCRIPTION
[0016] The application will be further described below in conjunction with the drawings and examples, which are simplified schematic diagrams and only illustrate the basic structure of the application in a schematic manner, and therefore only show the components related to the application.
[0017] As shown in Figure 1 , the image classification method based on the improved ResNet-Unet network includes the following steps: Step one, collect thyroid ultrasound images; Thyroid ultrasound images can be obtained from public data sets and clinical cases, or can be annotated after self-defined collection; Preprocess the image, including: The input image is normalized to [0, 1] in pixel value, and the normalized image is obtained.
[0018] CLAHE (Contrast Limited Adaptive Histogram Equalization) is used to improve the contrast of local regions of the image , avoiding loss of nodule region information; non-local mean filtering (NLM) is used to remove speckle noise in the ultrasound image, improving the distinguishability of structural information.
[0019] Step two, input the preprocessed image into the improved Unet network, the encoder of the improved Unet network uses ResNet50 for downsampling, and the decoder uses a four-layer structure Unet for upsampling; the layers of the encoder and the decoder are connected by skip connection. As shown in Figure 1, the down-sampling of the encoder comprises: an input layer, a first encoder layer Enc1, a second encoder layer Enc2, a third encoder layer Enc3, and a fourth encoder layer Enc4; the up-sampling of the decoder comprises: a fourth decoder layer Dec1, a third decoder layer Dec2, a second decoder layer Dec3, and a first decoder layer Dec4; each encoder layer is jump-connected with a corresponding decoder layer, that is, Enc1 is jump-connected with Dec1, Enc2 is jump-connected with Dec2, Enc3 is jump-connected with Dec3, and Enc4 is jump-connected with Dec4; the first decoder layer Dec4 is connected with a classifier, and the classifier comprises: a global average pooling and a fully connected layer; and a classification result of benign or malignant is outputted. The input layer is Conv1 and MaxPool.
[0020] The improved ResNet50 encoder parameters are as shown in Table 1. Table 1: Encoder structure parameters
[0021] The decoder parameters of the Unet network are as shown in Table 2. Table 2: Decoder structure parameters
[0022] The parameters of the classifier are as shown in Table 3. Table 3: Classifier structure parameters
[0023] The improvement of the present application is that the output feature maps of Enc1, Enc2, Enc3 and Enc4 are inputted into an SE module, and then the output feature map of Enc4 is inputted into the SE module and then inputted into a KAN module. As shown in Figure 4 , it is a fusion diagram of Enc4 and Dec4. The structure of the SE module (Squeeze-and-Excitation, channel attention mechanism) is as shown in Figure 2 The SE module is connected after stage1 to stage4 of the encoder resnet50. The SE module performs global average pooling on the feature map to generate a channel description vector: . Wherein, H is the height of the feature map, representing the number of pixels of the feature map in the vertical direction; W is the width of the feature map, representing the number of pixels of the feature map in the horizontal direction. is the spatial response map of the cth feature learned by the network. is the feature map The value at spatial position (i, j), i is the index in height direction, j is the index in width direction.
[0024] Learn inter-channel dependency through full connection layer: ; where, is the Sigmoid function, is the ReLU activation function, , is a learnable weight matrix (r is the compression ratio); Multiply the weight vector s with the original feature map channel by channel, and input the weighted feature into the skip connection and fuse with the decoder feature to get: ; where, is the channel weight, which represents the importance weight of the c th feature channel (generated by the Excitation step of the SE module); is the original feature channel, which is the c th spatial feature response map extracted by the convolutional layer; is the weighted feature channel, which is the c th output feature channel after weighting.
[0025] UNet network reconstructs the spatial resolution through upsampling, uses skip connection to combine shallow and deep features, and improves the segmentation accuracy: ; where, is the encoder feature, which is the deep output feature map of the encoder path; is the skip connection feature, which is the shallow feature map of the same level of the encoder; is the decoder output feature.
[0026] After the decoder of UNet, the global feature representation is obtained: ; where, is the number of layers of the network, represents the feature map of the th layer.
[0027] KAN (spatial attention) calculates local features, uses kernel function to calculate the similarity of different positions in feature space, and calculates attention weight according to the similarity to enhance local information; Let be the feature map extracted by stage4 of resnet50 network, and the The feature vector of a pixel point is represented as: ; wherein, is the feature dimension.
[0028] For any two pixel points in the feature map, KAN calculates the feature similarity: ; wherein, and are the feature vectors at pixels and respectively, is a dynamically adjusted scale parameter for controlling the influence range of the local neighborhood.
[0029] KAN adopts a normalization mechanism to calculate the attention distribution of each pixel point to its neighborhood pixels: ; wherein, represents the attention weight of pixel to pixel .
[0030] Based on the calculated attention matrix , KAN performs weighted summation on the local region features extracted by resnet50: ; wherein, is the feature vector after KAN processing, represents the local neighborhood range of pixel .
[0031] Finally, the expression form of the entire local feature map is: ; This process enables the model to dynamically focus on important local regions and enhance the understanding of nodule details; based on the features extracted by resnet50, it further enhances the expression ability of local region information, making the model pay more attention to key regions (nodule boundary, calcification area, internal texture features) in the ultrasound image; KAN calculates the similarity of local region features through the dynamic kernel method and performs adaptive weighting to optimize the classification effect.
[0032] The global features provided by the decoder of UNet and the locally enhanced features provided by KAN are fused through concatenation: ; wherein, , is the feature dimension extracted by UNet, is the feature dimension processed by KAN.
[0033] The classifier uses Global Average Pooling to further reduce the dimension and obtain a fixed-length feature vector for the final classification: ; Input into the fully connected layer (FC) and calculate the probability of thyroid nodules belonging to benign (Benign) or malignant (Malignant) by Softmax: ; where, is the learnable weight matrix, is the bias vector.
[0034] Softmax calculates the probability of two types of nodules: ; where, and are the scores of benign and malignant, respectively.
[0035] The classification error is calculated by cross-entropy loss: ; where, is the true label (benign = 0, malignant = 1), is the class probability predicted by the model.
[0036] To prevent the model from overfitting to a certain class, introduce the class balance factor : ; where, is the class weight, represents the number of samples of class .
[0037] During training, the AdamW optimizer is used to update the parameters: ; where, is the learning rate, is the second moment estimation, is the weight decay coefficient.
[0038] Calculate the final classification probability, if , it is determined as a malignant nodule, otherwise as a benign nodule.
[0039] The conditional random field (CRF) post-processing is further used to optimize the classification result, so as to ensure the spatial consistency of the classification result: ; Among them, is the classification confidence, represent the mutual influence of adjacent pixels.
[0040] The Bayesian optimization is used to select the optimal hyperparameter: ; Among them, represent the model classification accuracy, is the hyperparameter.
[0041] The ablation experiment is carried out on the method of the application, as shown in Table 1; Table 1 Ablation experiment
[0042] As Figure 5 and Table 1, the indicators of the network model of the application are obviously improved.
[0043] The application designs to improve the U-Net structure for multi-scale feature extraction, extracts deep semantic information through the encoder, restores spatial information through the decoder, and enhances local feature expression through the jump connection; the KAN calculation is introduced to calculate the local feature similarity between the encoder and the decoder, the feature expression of the key area (such as the nodule boundary and the calcification area) is strengthened through the kernel attention mechanism, and the discrimination ability of the classifier is enhanced; The global feature extracted by resnet50 and the local feature enhanced by KAN are fused, the adaptability of the model to different sizes and shapes of nodules is enhanced, and the robustness of the classification is improved; the attention weight is dynamically adjusted in the KAN calculation process, so that the model can adapt to the local feature change under different ultrasonic imaging conditions, and the classification accuracy is improved; in the optimization process of the classification network, the class balance strategy is combined, the unbalanced data is weighted, the misclassification of the minority class is reduced, and the performance of the model in the small sample environment is improved.
[0044] Based on the above ideal embodiments according to the application, through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the application. The technical scope of the application is not limited to the contents in the specification, and the technical scope must be determined according to the scope of claims.
Claims
1. An image classification method based on an improved ResNet-Unet network, characterized in that, The method comprises the following steps: Step one, collecting thyroid ultrasound images; Step two, constructing an improved ResNet-Unet network; the encoder of the ResNet-Unet network adopts a ResNet50 network, and an SE module is introduced after each encoding layer of the first encoding layer Enc1 to the fourth encoding layer Enc4 of the ResNet50 network.
2. The image classification method based on the improved ResNet-Unet network according to claim 1, wherein, A KAN module is introduced after the SE module of the fourth encoding layer Enc4, and the output features of the KAN model are connected with the first decoding layer Dec4 of the decoder of the Unet network; the encoding layers and the corresponding decoding layers are connected by a skip connection.
3. The image classification method based on the improved ResNet-Unet network according to claim 2, characterized in that, The classifier of the improved ResNet-Unet network comprises a global average pooling and a fully connected layer.
4. The image classification method based on the improved ResNet-Unet network according to claim 1, wherein, An improved cross-entropy loss is applied to the improved ResNet-Unet network. ;in, For category weights, Representative category The number of samples, denoted as the classification probability.
5. The image classification method based on the improved ResNet-Unet network according to claim 1, wherein, The improved ResNet-Unet network adopts an AdamW optimizer.
6. The image classification method based on the improved ResNet-Unet network according to claim 1, wherein, The improved ResNet-Unet network adopts a conditional random field.
7. The image classification method based on the improved ResNet-Unet network according to claim 1, wherein, The image is subjected to normalization preprocessing.
8. The image classification method based on the improved ResNet-Unet network according to claim 7, characterized in that, The normalized image is subjected to CLAHE operation.
9. An image classification system based on an improved ResNet-Unet network, characterized in that, The method comprises the following steps: a memory for storing instructions executable by a processor; a processor for executing instructions to implement the image classification method based on the improved ResNet-Unet network according to any one of claims 1-8.
10. A computer readable medium having stored thereon a computer program code, characterized in that, The computer program code implements the image classification method based on the improved ResNet-Unet network according to any one of claims 1-8 when executed by the processor.