An ultrasonic image classification method based on a small training dataset

Abstract

The present invention belongs to the technical field of image classification, and particularly relates to an ultrasonic image classification method based on a small training data set, which comprises the following steps: S1, data acquisition: acquiring ultrasonic image data and annotating its categories to complete the construction of the data set required for model training; S2, data preprocessing: the preprocessing includes normalization, data cutting, unifying the data scale, and augmenting the data set to ensure the model training effect; S3, recognition model: using deep learning related technologies to build a classification recognition model, inputting training data, and completing the construction of the parameter model; S4, model saving: when the loss function of the model no longer decreases, saving the model. By simultaneously using a generator for data augmentation and using a discriminator to train the classifier, combining the advantages of data augmentation and transfer learning, it can be used to process very limited training data and improve the classification effect of images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and particularly relates to an ultrasonic image classification method based on a small training data set. Background Art

[0002] Ultrasonic imaging is a widely used medical imaging modality and is often used in many fields of clinical applications and biomedical research. The most popular ultrasonic imaging mode is the B (brightness) mode, which generates an intensity image by sweeping the transmitted ultrasonic waves across a plane. However, it is extremely difficult to obtain a large data set in the context of medical images, especially ultrasonic images.

[0003] Problems or defects of the prior art: Currently, in applications with only a small amount of data sets, CNN-based models usually adopt the method of transfer learning. However, due to the large differences between natural images and ultrasonic images, the effect of transfer learning from the computer vision domain to the ultrasonic domain is limited, resulting in low image recognition accuracy and large errors. Summary of the Invention

[0004] In this application, an ultrasonic imaging device is used to diagnose relevant breast cancer patients and normal people to obtain corresponding lesion images and normal images, and all the obtained images are image-annotated. After completing the data annotation, the data is preprocessed, and the preprocessing includes segmentation and adding noise. The preprocessed data is input into the built ACGAN network for training the network model. When the model loss function no longer decreases, the model is saved to complete the model construction.

[0005] An ultrasonic image classification method based on a small training data set disclosed in this application includes the following steps:

[0006] S1. Data collection: Collect ultrasonic image data and annotate its category to complete the construction of the data set required for model training;

[0007] S2. Data preprocessing: The preprocessing includes normalization, data cutting, unifying the data scale, and expanding the data set to ensure the model training effect;

[0008] S3. Recognition model: Adopt deep learning-related technologies to build a classification recognition model, input the training data, and complete the construction of the parameter model;

[0009] S4. Model saving: When the loss function of the model no longer decreases, save the model.

[0010] Further, in step S1, an ultrasonic imaging device is used to diagnose relevant breast cancer patients and normal people to obtain corresponding lesion images and normal images, and all the obtained images are image-annotated.

[0011] Further, in the step S2, the specific method of data augmentation is as follows: the training images are processed by popular data enhancement techniques, including flipping, rotation (at different angles), and noise addition. The noise addition to the images is processed by the Gaussian formula, and the formula is as follows: P out = P in + x means + sigma * G(d), where Pout and Pin are each output and input pixel respectively, xmeans represents the average value, sigma represents the average variance, and G(d) is a Gaussian distribution random value of random numbers.

[0012] Further, in the step S2, the specific method of data segmentation is as follows: for the purpose of verification, by using 5-fold cross-validation, the dataset of 250 images is segmented into 5 segmented datasets, and each segmented dataset contains 50 images.

[0013] Further, in the step S3, an ultrasonic image classification and recognition model based on ACGAN is constructed. This model consists of a generator G and a discriminator D. The images in the training dataset are X = {xi, i = 1,..., I}, and their image labels are Y = {yi (yi ∈ {1,..., C}), where C represents the number of classes. The training data is input into the classification and recognition model. First, the generator G synthesizes the images in the dataset X. The generator generates synthetic images and performs class annotation on the obtained synthetic images Among them, the generator is implemented by using a multi-layer deep network composed of a series of permuted convolutional layers. The task of the discriminator D is to separate the synthetic images from the images in X, and at the same time it is designed to predict the correct class yi. Then, the training process of the recognizer is realized by using the multi-task learning process. After that, the discriminator D is designed to predict the correct class yi of the synthetic images. Then, the training process of the recognizer is realized by using the multi-task learning process.

[0014] Further, in the step S3, it also includes regularizing the convolutional layer, using batch normalization layers and activation function ReLU layers, and finally feeding back to two independent fully connected (FC) layers, corresponding to class prediction and source prediction. The generator G and the discriminator D are trained in an adversarial manner. G continuously optimizes its weights to effectively reproduce the distribution of the target data, and D continuously optimizes itself to identify the class (yi) of the images. In addition, the D optimizes its weights to effectively distinguish whether the images are real or synthetic. The generator G updates its weights through interaction with the discriminator D, learns useful semantic features related to the training images, improves the overall classification effect of the model, and finally realizes the parameter tuning of the model.

[0015] The present invention proposes an ultrasonic image classification method based on a small training dataset. By simultaneously using a generator for data augmentation and a discriminator to train a classifier, combining the advantages of data augmentation and transfer learning, it can be used to process very limited training data and improve the classification effect of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 System module block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] As Figure 1 shown, an ultrasonic image classification method based on a small training dataset disclosed in the present application includes the following steps:

[0019] S1. Data acquisition: Collect ultrasonic image data and label its categories to complete the construction of the dataset required for model training; Data acquisition: By using an ultrasonic imaging device to diagnose relevant breast cancer patients and normal people, obtain corresponding lesion images and normal images, and perform image annotation on all the obtained images. This dataset consists of 250 ultrasonic images, which come from two categories - benign and malignant. 150 are malignant, and the rest are benign. At the same time, there is no obvious visual difference between these two categories, making the classification challenging and making this classification task highly credible and fair.

[0020] S2. Data preprocessing: The preprocessing includes normalization, data cutting, unifying the data scale, and expanding the dataset to ensure the model training effect; Data augmentation: The training images are processed by popular data augmentation techniques, including flipping, rotation (at different angles), and noise addition. Among them, the noise addition to the images is processed by the Gaussian formula, and the formula is as follows: P out = P in + x means + sigma * G(d), where Pout and Pin are each output and input pixel, xmeans represents the average value, sigma represents the average variance, and G(d) is a Gaussian distribution random value of random numbers.

[0021] When loading the data for training, perform dynamic probabilistic transformation on the data. This transformation method does not transform all the training data, but rather transforms partial images at each time node. Different images are transformed at different time nodes. Although the purpose of this is to enhance the data, it also adds randomness to the training mechanism. However, since data augmentation is performed by dynamic transformation when loading the training data and cannot effectively increase the number of training images, data augmentation is not applied during testing.

[0022] Data splitting: For the purpose of verification, the dataset of 250 images is split into 5 split datasets through 5-fold cross-validation, with each split dataset containing 50 images. For each evaluation, 4 groups consisting of 200 images are used during the training process. For testing, the remaining 50 images will be used for model testing. In this way, 5 independent evaluations are performed and the average value of the accuracy exceeds the average level. The 200 training images are far less than the number of training images generally used in computer vision tasks. Therefore, this dataset is suitable for testing the ability of the method in this paper to process under-trained images in ultrasonic image analysis and reflects the classification performance and ability of the model in small data samples.

[0023] Data normalization: Perform Min-Max normalization on each piece of data.

[0024] S3. Recognition model: Use deep learning-related technologies to build a classification recognition model, input the training data, and complete the construction of the parameter model; Model construction: Build an ultrasonic image classification recognition model based on ACGAN. This model consists of a generator G and a discriminator D. The images in the training dataset are X = {xi, i = 1,..., I}, and their image labels are Y = {yi (yi ∈ {1,..., C}), where C represents the number of categories. Input the training data into the classification recognition model. First, the generator G synthesizes the images in the dataset X, and the generator generates synthetic images and perform class annotation on the obtained synthetic images where the generator is implemented using a multi-layer deep network consisting of a series of permuted convolutional layers for c. The task of the discriminator D is to classify the synthetic images ​It is separated from the images in X, and it is designed to predict the correct class yi. Then, the training process of the recognizer is realized by using the multi-task learning process. This learning mechanism helps the discriminator learn useful features to distinguish different classes and achieve implicit auxiliary data augmentation. The discriminator network consists of a set of convolutional layers. To further regularize the convolutional layers, batch normalization layers and ReLU activation function layers are used, and finally, it is fed back to two independent fully connected (FC) layers, corresponding to the prediction of the class and the prediction of the source. The generator G and the discriminator D are trained in an adversarial manner. G continuously optimizes its weights to effectively reproduce the distribution of the target data, and D continuously optimizes itself to identify the class (yi) of the image. In addition, the D optimizes its weights to effectively distinguish whether the image is real or synthetic. The generator G updates its weights through interaction with the discriminator D, learns useful semantic features related to the training images, improves the overall classification effect of the model, and finally realizes the parameter tuning of the model.

[0025] S4. Model saving: Save the model when the loss function of the model no longer decreases.

[0026] The above only elaborates on the preferred embodiments of the present invention in detail. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention, and all such changes should be included within the protection scope of the present invention.

Claims

1. An ultrasonic image classification method based on a small training data set, characterized in that: It includes the following steps: S1. Data acquisition: Acquire ultrasonic image data, label its categories, and complete the construction of the dataset required for model training; S2. Data preprocessing: The preprocessing includes normalization, data cutting, unifying the data scale, and augmenting the dataset to ensure the model training effect; S3. Recognition Model: Construct an ultrasonic image classification and recognition model based on ACGAN. This model consists of a generator G and a discriminator D. The images in the training dataset are X = {xi, i = 1,..., I}, and their image labels are Y = {yi (yi ∈ {1,..., C}), where C represents the number of categories. Input the training data into the classification and recognition model. First, the generator G synthesizes the images in the dataset X, and the generator generates synthetic images. And for the obtained synthetic images perform class annotation where the generator is implemented using a multi-layer deep network composed of a series of permuted convolutional layers. The task of the discriminator D is to separate the synthetic images from the images in X, and at the same time it is designed to predict the correct class yi. Then, the training process of the recognizer is realized by using the multi-task learning process; It also includes regularizing the convolutional layer, using the batch normalization layer and the activation function ReLU layer, and finally feeding back to two independent fully connected (FC) layers, corresponding to the prediction of the class and the prediction of the source. The generator G and the discriminator D are trained in an adversarial manner. G continuously optimizes its weights to effectively reproduce the distribution of the target data, and D continuously optimizes itself to identify the class (yi) of the image. In addition, the D optimizes its weights to effectively distinguish whether the image is real or synthetic. The generator G updates its weights interactively with the discriminator D, learns useful semantic features related to the training images, improves the overall classification effect of the model, and finally realizes the parameter tuning of the model; S4. Model saving: Save the model when the loss function of the model no longer decreases.

2. The ultrasonic image classification method based on a small training data set according to claim 1, characterized in that: In the step S1, by using an ultrasonic imaging device to diagnose relevant breast cancer patients and normal people, the corresponding lesion images and normal images are obtained, and all the obtained images are image-labeled.

3. The ultrasonic image classification method based on a small training data set according to claim 2, characterized in that: In the step S2, the specific method of data augmentation is as follows: in the training images, popular data enhancement techniques are used for processing, including flipping, rotation (at different angles), and noise addition. Among them, the noise addition to the images is processed through the Gaussian formula, and the formula is as follows: P out = P in + x means + sigma * G(d), where Pout and Pin are each output and input pixel respectively, xmeans represents the average value, sigma represents the average variance, and G(d) is the Gaussian distribution random value of random numbers.

4. A method for classifying ultrasonic images based on a small training dataset according to claim 3, characterized in that: In the step S2, the specific method of data segmentation is: for the purpose of verification, by using 5-fold cross-validation, the dataset of 250 images is divided into 5 segmented datasets, and each segmented dataset contains 50 images.

Images