A small sample image three-classification method, device and storage medium based on deep learning
Patent Information
- Application Number
- CN202411870037.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-04-04
AI Technical Summary
In the medical field, especially in the classification of diabetic foot ulcer (DFU), it is difficult for the prior art to effectively process small sample data, and traditional machine learning methods have problems such as manual feature extraction and time-consuming, which limits the multi-level classification and automated recognition of images.
A small sample image three-class classification method based on deep learning is adopted. By acquiring standardized acquired images and expanding data, it combines the ResNet50 model and CBAM module for training to realize multi-level classification of images.
The accuracy of three-classification images in small sample environments is improved, and the automatic identification of image classification is realized, the scope of application is significantly expanded, the dependence on experts is reduced, and the robustness of the model is enhanced.
Smart Images
Figure CN119762863A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and specifically to a method, device and storage medium for three-classification of small sample images based on deep learning. Background Art
[0002] In the medical field, the corresponding treatment methods can be better implemented after grading and classifying diseases according to medical images or pathological pictures. The recognition of medical images or pathological images is a complex and subjective judgment process, which generally requires experts in this field to make judgments. For some cases where image acquisition is difficult and the sample size is small, the number of people who can classify them in detail is small. For some cases with a large number of categories, the differences between multiple categories require professional judgment, and experts rarely have the leisure to deal with such a single matter. The above two factors make case classification difficult to implement in actual medical guidance and cover a small range of patients.
[0003] For example, the traditional classification method of diabetic foot ulcer (DFU) mainly relies on doctors to manually classify according to Wangner or Texas grading standards. Although traditional machine learning methods, such as SVM and KNN, can achieve certain results in binary classification tasks, they have the problems of manual feature extraction and long time consumption. At present, deep learning methods also have classification in the field of DFU, but they are still at the binary classification stage, and the effect of processing small sample data is not ideal. Summary of the invention
[0004] The present application provides a small sample image three-classification method, device and storage medium based on deep learning, which can perform detailed recognition on images with a small number of samples and realize multi-level classification.
[0005] The technical solution of this application is as follows:
[0006] In a first aspect, the present application provides a small sample image three-classification method based on deep learning, comprising the following steps:
[0007] S1) acquiring standardized collected images under each category, wherein the number of the categories is at least three;
[0008] S2) performing scaling, rotation, flipping, brightening, darkening, pixel shifting, and noise adding operations on each standardized collected image to expand the image data set;
[0009] S3) constructing a training model based on the ResNet50 model as the basic network, specifically: inserting a CBAM module into each residual block of the ResNet50 model; the CBAM module is inserted before the residual connection and acts on the output of the residual block;
[0010] S4) inputting the expanded image data set into the training model for model training, wherein the training model outputs the classification to which the standardized collected image belongs, and the training is completed;
[0011] S5) Inputting the image to be recognized into the trained model, and determining the category to which the image to be recognized belongs according to the output result.
[0012] Further, in step S1), the standardized captured image is an RGB image taken at a standard distance under natural light.
[0013] Further, in step S2), the scaling value of the standardized acquired image is a random number between 50% and 200%;
[0014] The rotation angle range of the standardized acquisition image is 0° to 90°, and the direction is clockwise or counterclockwise;
[0015] The inversion operations of the standardized acquisition images include horizontal flipping and vertical flipping;
[0016] The brightness increase range in the brightening operation of the standardized acquired image is 0 to 10% of the original image;
[0017] The brightness reduction range in the darkening operation of the standardized acquired image is 0 to 10% of the original image;
[0018] The pixel shift operation of the standardized acquisition image is to shift all pixels to the left or right by 10 pixels;
[0019] The types of noise added to the standardized acquired images are salt and pepper noise and Gaussian noise.
[0020] Further, in step S3), the CBAM module includes a channel attention module;
[0021] The channel attention module takes the feature map F∈R C×H×W As input, the global average pooling layer outputs F avg =GAP(F)∈R C×1×1 , output F through the global maximum pooling layer max =GMP(F)∈R C×1×1 After that, F avg With F max Add together to learn the weights and get the learning weight M c :
[0022] M c =σ(W1(W0(F avg ))+W1(W0(F max )));
[0023] In the formula, σ is the Sigmoid function, W0 and W1 are fully connected layers;
[0024] To M c Perform feature recalibration to obtain the channel-weighted feature map F′:
[0025] F′=M c ⊙F;
[0026] Where ⊙ represents channel-by-channel multiplication.
[0027] Further, in step S3), the CBAM module also includes a spatial attention module;
[0028] The spatial attention module takes the feature map F′ as input, performs global maximum pooling and average pooling operations on F′ in the channel dimension, and obtains feature maps of size H×W: max_pool =MaxPool(F′), F′ avg_pool =AvgPool(F′); max_pool and F′ avg_pool Connect and input to the 7×7 convolutional layer to obtain the spatial attention map M s ;
[0029] M s =σ(Conv7x7([F′ max_pool ; F′ avg_pool ]))
[0030] To M s Feature recalibration is performed to obtain the spatially weighted feature map F″.
[0031] In a second aspect, the present application provides a small sample image multi-classification device based on deep learning, comprising a processor and a memory storing program instructions, wherein the processor is configured to execute the above-mentioned small sample image multi-classification method based on deep learning when running the program instructions.
[0032] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the small sample image multi-classification method based on deep learning as described above.
[0033] Due to the adoption of the above technical solution, the beneficial effects of this application are as follows:
[0034] 1. This application combines the deep residual network with the transfer learning algorithm to improve the accuracy of image three-classification in a small sample environment.
[0035] 2. This application enables automated recognition of image classification, significantly improves the application scope of image classification while maintaining a high accuracy rate, reduces dependence on experts, and improves the efficiency of image application.
[0036] 3. This application expands small sample images from multiple aspects, fully considering the actual image acquisition environment, improving the generalization ability of the model, and enhancing the robustness of the model.
[0037] 4. This application is suitable for deployment on terminal devices with low computing resources and can be promoted in a wider area. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.
[0039] Figure 1 A small sample image multi-classification flowchart based on deep learning provided for this application;
[0040] Figure 2 This is an architectural diagram of the training model described in this application. DETAILED DESCRIPTION
[0041] Based on the background technology, this application provides a small sample image three-classification method based on deep learning, which is used for multi-level classification of small sample images. Taking the zero degree, mild and severe classification of DFU images as an example, this method is described in detail. This embodiment includes the following steps:
[0042] S1) Acquire standardized acquired images under each category, where the number of the categories is at least three.
[0043] In this embodiment, the standardized images were taken by doctors using a mobile phone in an indoor well-lit environment, with the shooting position 30 to 40 cm away from the ulcer surface and the shooting plane parallel to the ulcer surface. In this embodiment, a total of 146 pathological photos were taken, and after classification by experts, 27 were zero, 45 were mild, and 74 were severe.
[0044] S2) performing scaling, rotation, flipping, brightening, darkening, pixel shifting, and noise adding operations on each standardized collected image to expand the image data set.
[0045] The scaling value of the standardized acquisition image is a random number of 50% to 200%; the rotation angle range of the standardized acquisition image is 0° to 90°, and the direction is clockwise or counterclockwise; the inversion operation of the standardized acquisition image includes horizontal flipping and vertical flipping; the brightness increase range in the brightening operation of the standardized acquisition image is 0% to 10% of the original image; the brightness reduction range in the darkening operation of the standardized acquisition image is 0% to 10% of the original image; the pixel movement operation of the standardized acquisition image is to move all pixels 10 pixels to the left or right; the noise addition types of the standardized acquisition image are salt and pepper noise and Gaussian noise.
[0046] In this embodiment, the density of the salt and pepper noise is 0.05, the mean value of the Gaussian noise is 0, and the variance is 0.01.
[0047] In this embodiment, the image data set after the above expansion contains 3000 pictures.
[0048] S3) constructing a training model based on the ResNet50 model as the basic network, specifically: inserting a CBAM module into each residual block of the ResNet50 model; the CBAM module is inserted before the residual connection and acts on the output of the residual block;
[0049] The CBAM module includes a channel attention module and a spatial attention module;
[0050] The channel attention module takes the feature map F∈R C×H×W As input, the global average pooling layer outputs F avg =GAP(F)∈R C×1×1 , output F through the global maximum pooling layer max =GMP(F)∈R C×1×1 After that, F avg With F max Add together to learn the weights and get the learning weight M c :
[0051] M c =σ(W1(W0(F avg ))+W1(W0(F max )));
[0052] In the formula, σ is the Sigmoid function, W0 and W1 are fully connected layers;
[0053] Among them, R represents the real number field, C represents the channel, H and W are three-dimensional tensors, representing the height and width of the feature map;
[0054] To M c Perform feature recalibration to obtain the channel-weighted feature map F′:
[0055] F′=M c ⊙F;
[0056] Where ⊙ represents channel-by-channel multiplication.
[0057] The spatial attention module takes the feature map F′ as input, performs global maximum pooling and average pooling operations on F′ in the channel dimension, and obtains feature maps of size H×W: max_pool =MaxPool(F′), F′ avg_pool =AvgPool(F′);max_pool and F′ avg_pool Connect and input to the 7×7 convolutional layer to obtain the spatial attention map M s The spatial attention map output after convolution is 1×H×W, which represents the attention weight of each position. In the above step, the two pooling results are concatenated to obtain a feature map of size 2C×1×1. It is then input into a 7×7 convolution layer to generate the spatial attention map.
[0058] To M s Feature recalibration is performed to obtain the spatially weighted feature map F″.
[0059] In this embodiment, the CBAM module combines the channel and spatial attention mechanisms to improve the perception of key areas (such as ulcer surfaces). Deep learning belongs to black box design and can enhance the attention to ulcer areas.
[0060] S4) inputting the expanded image data set into the training model for model training, wherein the training model outputs the classification to which the standardized collected image belongs, and the training is completed.
[0061] S5) Inputting the image to be recognized into the trained model, and determining the category to which the image to be recognized belongs according to the output result.
[0062] In this application, data augmentation is used to allow the model to encounter more diverse samples during training, forcing it to learn more universal features and reduce its dependence on training data, thereby effectively avoiding overfitting.
[0063] The disclosed embodiment also provides a small sample image multi-classification device based on deep learning, including a processor and a memory. Optionally, the device may also include a communication interface and a bus. The processor, the communication interface, and the memory may communicate with each other through the bus. The communication interface may be used for information transmission. The processor may call the logic instructions in the memory to execute the small sample image multi-classification method based on deep learning of the above embodiment.
[0064] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
[0065] The memory, as a computer-readable storage medium, can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes the function application and data processing by running the program instructions / modules stored in the memory, that is, implementing the small sample image multi-classification method based on deep learning in the above embodiment.
[0066] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory.
[0067] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned small sample image multi-classification method based on deep learning.
[0068] Anything not described in this application can be achieved by adopting or drawing on existing technologies.
[0069] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A small sample image three-classification method based on deep learning, characterized in that: The following steps are included: S1) acquiring standardized collected images under each category, wherein the number of the categories is at least three; S2) performing scaling, rotation, flipping, brightening, darkening, pixel shifting, and noise adding operations on each standardized collected image to expand the image data set; S3) constructing a training model based on the ResNet50 model as the basic network, specifically: inserting a CBAM module into each residual block of the ResNet50 model; the CBAM module is inserted before the residual connection and acts on the output of the residual block; S4) inputting the expanded image data set into the training model for model training, wherein the training model outputs the classification to which the standardized collected image belongs, and the training is completed; S5) Inputting the image to be recognized into the trained model, and determining the category to which the image to be recognized belongs according to the output result.
2. According to the method of three-classification of small sample images based on deep learning in claim 1, it is characterized in that: In step S1), the standardized captured image is an RGB image taken at a standard distance under natural light.
3. According to the method of three-classification of small sample images based on deep learning in claim 1, it is characterized in that: In step S2), the scaling value of the standardized acquired image is a random number between 50% and 200%; The rotation angle range of the standardized acquisition image is 0° to 90°, and the direction is clockwise or counterclockwise; The inversion operations of the standardized acquisition images include horizontal flipping and vertical flipping; The brightness increase range in the brightening operation of the standardized acquired image is 0 to 10% of the original image; The brightness reduction range in the darkening operation of the standardized acquired image is 0 to 10% of the original image; The pixel shift operation of the standardized acquisition image is to shift all pixels to the left or right by 10 pixels; The types of noise added to the standardized acquired images are salt and pepper noise and Gaussian noise.
4. The method for three-classification of small sample images based on deep learning according to claim 1, characterized in that: In step S3), the CBAM module includes a channel attention module; The channel attention module takes the feature map F∈R C×H×W As input, the global average pooling layer outputs F avg =GAP(F)∈R C×1×1 , output F through the global maximum pooling layer max =GMP(F)∈R C×1×1 After that, F avg With F max Add together to learn the weights and get the learning weight M c : M c =σ(W1(W0(F avg ))+W1(W0(F max ))); In the formula, σ is the Sigmoid function, W0 and W1 are fully connected layers; To M c Perform feature recalibration to obtain the channel-weighted feature map F′: F′=M c ⊙F; Where ⊙ represents channel-by-channel multiplication.
5. The method for three-classification of small sample images based on deep learning according to claim 4, characterized in that: In step S3), the CBAM module also includes a spatial attention module; The spatial attention module takes the feature map F′ as input, performs global maximum pooling and average pooling operations on F′ in the channel dimension, and obtains feature maps of size H×W: max_pool =MaxPool(F′), F′ avg_pool =AvgPool(F′); max_pool and F′ avg_pool Connect and input to the 7×7 convolutional layer to obtain the spatial attention map M s ; M s =σ(Conv7x7([F′ max_pool ;F′ avg_pool ])) To M s Feature recalibration is performed to obtain the spatially weighted feature map F″.
6. A small sample image multi-classification device based on deep learning, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the small sample image multi-classification method based on deep learning as claimed in claim 5 when running the program instructions.
7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method as claimed in claim 5 is implemented.