Lung image classification method, classification network determination method and device

Abstract

This article discloses a method, device and storage medium for determining a lung image classification network, as well as a lung image classification method, device and storage medium. The method for determining the lung image classification network includes obtaining labeled lung image pictures to establish a lung image data set; pre-training the complete DenseNet network according to the pictures and labels in the data set to determine the first part of the network; extracting the image features of each picture in the data set according to the first part of the network, eliminating the collinearity of the image features, and determining the weight of each image feature; training a first linear classifier according to the image features and corresponding weights of the pictures in the data set to determine the second part of the network; combining the first part of the network and the second part of the network to determine the lung image classification network.

Description

Technical Field

[0001] The present disclosure relates to, but is not limited to, the field of image processing technology, and in particular to a method, device and storage medium for determining a lung image classification network, and a lung image classification method, device and storage medium. Background Art

[0002] Timely and accurate diagnosis of pneumonia is crucial for the treatment of patients. Severe pneumonia is not only more difficult to treat, but may also cause permanent damage to patients. CT imaging diagnosis of pneumonia has the advantages of high accuracy, fast speed, and the ability to determine the impact of pneumonia on the lungs, and has been widely studied by researchers. Although computer-aided diagnosis technology combining deep learning and computer vision has made great progress in recent years, and the accuracy of diagnosis has continued to improve, in some areas it has reached or even exceeded the accuracy of manual diagnosis, but computer-aided diagnosis technology still faces many problems in its application areas:

[0003] Lack of explainability. The medical field requires not only that doctors can make correct diagnoses, but also that the reasoning behind the diagnosis is correct. Deep learning methods cannot provide a clear and reasonable reasoning process except for the areas of concern.

[0004] Poor transferability. Even if the deep learning algorithm performs well, it does not have good transferability to images produced by different medical imaging devices of the same type. That is, a model trained on images produced by a group of devices performs well on its own dataset, but performs poorly on images produced by other similar devices. This means that the model may not use the most reasonable features.

[0005] Poor robustness. In the field of deep learning, neural networks are easily attacked by adversarial samples. Adding a disturbance that is imperceptible to humans to the input image will change the classification result. This feature is very dangerous for applications in the medical field.

[0006] New solutions need to be proposed for the above problems. Summary of the invention

[0007] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0008] The disclosed embodiments provide a method, device and storage medium for determining a lung image classification network, which can effectively improve the performance of the model, improve the robustness of the model, and give the model a certain degree of interpretability. The determined classification network / model can be used offline and can be applied to a computer-aided diagnosis system.

[0009] The embodiments of the present disclosure also provide a method, device and storage medium for lung image classification, which utilizes the determined classification network (model) to classify lung images to help perform auxiliary diagnosis.

[0010] The present disclosure provides a method for determining a lung image classification network, comprising:

[0011] Obtaining lung image pictures with labels to establish a lung image dataset; the labels include: positive sample labels and negative sample labels;

[0012] Pre-training a complete DenseNet network according to the images and labels in the data set, and removing the fully connected layer from the pre-trained DenseNet network to determine the first part of the network;

[0013] Extracting image features of each image in the data set according to the first partial network, eliminating collinearity of the image features, and determining a weight of each image feature;

[0014] Training a first linear classifier according to image features and corresponding weights of the pictures in the data set, and determining the trained first linear classifier as the second partial network;

[0015] The first partial network and the second partial network are combined to form the lung image classification network.

[0016] In some exemplary embodiments, before pre-training the complete DenseNet network according to the images and labels in the data set, the method further includes:

[0017] Preprocessing the images in the dataset includes at least one of the following:

[0018] Crop the images in the data set according to a preset size;

[0019] Flipping the images of a preset proportion in the data set in a preset direction;

[0020] The images in the dataset are normalized.

[0021] In some exemplary embodiments, pre-training a complete DenseNet network according to the images and labels in the data set includes:

[0022] According to the images and labels in the data set, the complete DenseNet network is trained using a stochastic gradient descent algorithm until the weights of the DenseNet network meet the preset convergence conditions.

[0023] In some exemplary embodiments, the use of a stochastic gradient descent algorithm to train a complete DenseNet network includes:

[0024] Use the stochastic gradient descent algorithm and the Focal Loss function to train the complete DenseNet network.

[0025] Among them, the Focal Loss loss function is:

[0026] Loss = -α*(1-l p ) γ *logl p

[0027] Among them, α,γ are the hyper parameters of Focal Loss, l p It is the probability that the complete DenseNet network determines that the picture input into the complete DenseNet network is a positive sample.

[0028] In some exemplary embodiments, extracting image features of each picture in the data set according to the first partial network, eliminating collinearity of the image features, and determining the weight of each image feature includes:

[0029] According to the first part of the network, extract image features of each picture in the data set respectively;

[0030] According to the image features of each picture, determine the pseudo features corresponding to each picture respectively;

[0031] Training a second linear classifier based on the image features and pseudo features of the pictures in the data set;

[0032] Determine the positive sample probability and negative sample probability of each image according to the trained second linear classifier;

[0033] Determine the weight of each image feature according to the positive sample probability and negative sample probability of each image;

[0034] The weight of each image feature is equal to the ratio of the probability of a positive sample belonging to the image to the probability of a negative sample.

[0035] In some exemplary embodiments, the image features of the picture include features of at least one dimension;

[0036] The determining of pseudo features corresponding to each picture according to the image features of each picture includes:

[0037] The following operations are performed for each image: a pseudo feature is generated for each dimensional feature of the image, and a set of pseudo features corresponding to the image is obtained; wherein the pseudo feature is a feature of the same dimension corresponding to an image randomly selected from other images in the data set other than the image itself;

[0038] The step of training a second linear classifier according to the image features and pseudo features of the pictures in the data set comprises:

[0039] The Focal Loss function is adopted, the image features of the pictures in the data set are used as negative samples, the pseudo features corresponding to the pictures are used as positive samples, and the stochastic gradient descent algorithm is used to train the second linear classifier.

[0040] In some exemplary embodiments, training a first linear classifier according to image features and corresponding weights of pictures in the data set includes:

[0041] The first linear classifier is trained using a Focal Loss loss function and a stochastic gradient descent algorithm;

[0042] When calculating the loss function value of each image, the value of the loss function Loss is multiplied by the corresponding weight w to obtain a new loss function:

[0043] Loss re =w*Loss

[0044] The obtained new loss function is used to perform back propagation and parameter update on the first linear classifier to determine the weight of the first linear classifier.

[0045] The present disclosure provides a lung image classification method, comprising:

[0046] Obtain lung images to be classified;

[0047] The lung image pictures to be classified are classified according to the lung image classification network determined by any of the above-mentioned methods for determining the lung image classification network.

[0048] An embodiment of the present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program for determining a lung image classification network or classifying lung images, and the processor is configured to read and run the computer program for determining a lung image classification network or classifying lung images to execute any of the above-mentioned methods for determining a lung image classification network or any of the above-mentioned methods for classifying lung images.

[0049] An embodiment of the present disclosure further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute any of the above-mentioned methods for determining a lung image classification network or any of the above-mentioned methods for classifying lung images when running.

[0050] Other aspects will be apparent upon reading and understanding the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a lung image classification method in an embodiment of the present disclosure;

[0052] Figure 2 An algorithm flow of an algorithm for eliminating collinearity in an embodiment of the present disclosure;

[0053] Figure 3 A pseudo code diagram of an algorithm for eliminating collinearity in an embodiment of the present disclosure;

[0054] Figure 4 is a flow chart of a method for determining a lung image classification network in another embodiment of the present disclosure;

[0055] Figure 5 The figure is a flow chart of a lung image classification method in another embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] To make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily without conflict.

[0057] The basis for the application of traditional deep learning models in the field of medical imaging is that the collected medical imaging training data and test data follow the same distribution. However, this assumption is difficult to meet in the field of medical imaging. The image data generated by different devices follow different distributions, so the deep learning model will not perform well. This difference in distribution also limits the transferability of deep learning models, which greatly limits the scope of application of the model. In order to solve this problem, many methods have been proposed, such as transfer learning. By modeling the distribution of test data before testing, the information of the distribution of test data is added to the training process during the training phase, so that the model can also obtain better performance on the test data. However, the information of the test set is often not easy to obtain, so the transfer learning method is limited.

[0058] The application of deep learning in pneumonia image classification tasks still has specific problems in network structure design. Different solutions need to be designed according to the specific situation of the data set in terms of skip connection selection, network depth and width, convolution kernel size and other parameters. Pneumonia image classification also has the problem of unbalanced data distribution, and it is often necessary to weight between categories or use special loss functions to ensure the performance of the model. Specifically, the issues that need to be considered are:

[0059] (1) Pneumonia images are larger in size, higher in resolution, and deeper in layers than general images. Networks that have more downsampling to fuse image features are more suitable for such features. However, the amount of data for pneumonia images is relatively small, so deepening the network will improve the network's fitting ability, which may lead to overfitting problems on the data set. In order to balance the problem of large image size and small data volume, it is necessary to select a neural network with appropriate depth to ensure a balance between fitting ability and feature fusion ability.

[0060] (2) Pneumonia images have a class imbalance problem. Doctors often recommend that patients whose clinical manifestations may be diagnosed with pneumonia take images, so the positive samples obtained are often more than the negative samples. This data distribution is completely different from the actual data distribution. Deep learning models will perform better on data sets with roughly the same number of samples in different categories, so directly applying traditional deep learning models may not achieve ideal results.

[0061] In view of the above problems, it is very important to adopt a neural network with appropriate width and depth when designing the network structure. In the experiment, a variety of different settings of network width and depth are tested respectively to determine the more appropriate width and depth. The network used in the embodiments of the present invention tests the number of layers and channels of the network, and selects the appropriate width and depth based on the performance on the training set and the validation set.

[0062] Manual diagnosis of pneumonia through images has the above limitations, and computer-assisted diagnosis to assist doctors in treatment is a good solution. Research on the classification task of pneumonia images helps to improve the performance of computer-assisted diagnosis technology. The solution of the disclosed embodiment studies the characteristics of pneumonia medical images, and specifically designs a deep learning model. Combined with the application of causal inference in the field of machine learning, the experiment of computer automatic identification and classification of pneumonia is studied.

[0063] The following step numbers do not limit the specific execution order, and the execution order of some steps can be adjusted according to the specific embodiment. The "first part of the network", "second part of the network", "first linear classifier", and "second linear classifier" involved in the following records are used to distinguish the networks or linear classifiers involved in different steps, and do not limit the priority, execution order or other attributes. It should be noted that the embodiments described here are only for illustration and are not intended to limit the solutions provided by the present disclosure. In the following description, in order to provide a thorough understanding of the present invention, a large number of specific details are explained. However, it is not necessary for a person of ordinary skill in the art to adopt these specific details to implement the solution of the present disclosure, and the relevant specific details can be implemented by using relevant technical solutions known to those skilled in the art.

[0064] The solution provided by the embodiment of the present disclosure can effectively improve the performance of the model, improve the robustness of the model, and give the model a certain degree of interpretability. The model can be used offline and can be applied to a computer-aided diagnosis system.

[0065] Studies have shown that collinearity between data is one of the reasons for poor model performance and weak generalization ability. From the perspective of reducing collinearity, the embodiment of the present disclosure utilizes a method of weighting data to improve the performance and generalization ability of the model and better meet the requirements of computer-aided diagnosis technology.

[0066] Embodiment 1

[0067] The present disclosure provides a lung image classification method, the process of which is as follows: Figure 1 As shown, including:

[0068] Step 1: Create a lung image dataset:

[0069] 5756 labeled lung images were obtained from the Internet to establish a lung image dataset, including normal lung images and lung images of pneumonia patients, that is, the corresponding labels include: positive sample labels and negative sample labels.

[0070] In some exemplary embodiments, a preset number of labeled lung images may be obtained through other means and used as sample images to establish a lung image dataset, without being limited to the above specific means and numbers.

[0071] Step 2, data preprocessing and data enhancement:

[0072] The lung images were randomly cropped into 224*224 pixel images to ensure that the length and width of the images were consistent; the images were horizontally flipped with a probability of 0.5 during the training process for data enhancement; the images in the lung image dataset were normalized to make the images in the dataset independent and identically distributed.

[0073] In some exemplary embodiments, the cropping is performed according to a preset size and position; or, the cropping is performed randomly according to a preset size. The preset size is not limited to the above-mentioned example size. The preset position is not limited to a specific position and can be set according to the sample picture situation.

[0074] In some exemplary embodiments, data enhancement includes: flipping preset proportion images in all sample images in a preset direction; the preset direction includes a horizontal direction or a vertical direction.

[0075] Step 3, network pre-training:

[0076] Use the images (pictures) and labels in the dataset to pre-train the complete DenseNet network. The training uses the stochastic gradient descent algorithm and the Focal Loss loss function. Train until the weights converge to obtain the weights of the DenseNet network.

[0077] Among them, the loss function

[0078] Loss = -α*(1-l p ) γ *logl p (1)

[0079] Among them, α, γ are the hyper parameters of Focal Loss, l p It is the probability that the complete DenseNet network determines that the picture input into the complete DenseNet network is a positive sample.

[0080] Step 4: Extract image features:

[0081] Remove the last fully connected layer from the pre-trained DenseNet network obtained in step 3 and use it as the first part of the network. Each image passes through the first part of the network to obtain the features of the low-dimensional lung image.

[0082] Step 5: Eliminate collinearity of image features:

[0083] The image features obtained in step 4 are weighted to eliminate the collinearity of data distribution through weighting.

[0084] The weight calculation process is as follows: Figure 2 As shown, including:

[0085] (a) Pseudo-feature generation: A pseudo-feature is generated for each dimensional image feature obtained in step 4. The pseudo-feature is randomly extracted from the features of non-self samples of the same type of feature in the data set. For each real image feature, a corresponding pseudo-feature is generated. For each set of real image features of an image, a corresponding set of pseudo-features is generated. In some exemplary embodiments, for example, if each image has N-dimensional image features, a pseudo-feature is generated for each dimension. These N pseudo-features constitute a set of pseudo-features for the image.

[0086] (b) Training a linear classifier: Use the stochastic gradient descent algorithm to train a linear classifier (denoted as the second linear classifier), adopt the Focal Loss loss function, take the image features of the real picture obtained in step 4 as negative samples, and take the pseudo features generated according to the image features of the real picture in (a) as positive samples, so that the linear classifier (the second linear classifier) ​​can correctly classify the two types of samples;

[0087] Among them, the loss function

[0088] Loss = -α*(1-l p ) γ *logl p (2)

[0089] Among them, α, γ are the hyper parameters of Focal Loss, l p is the probability that the linear classifier (the second linear classifier) ​​judges that the sample is a positive sample.

[0090] (c) Calculate sample weight: For each image feature of a real sample, the ratio of the probability that the linear classifier (the second linear classifier) ​​outputs that the sample is a positive sample to the probability that the sample is a negative sample is used as the weight of the sample:

[0091]

[0092] Where x refers to the sample input into the linear classifier (the second linear classifier), and w is the weight corresponding to the sample x.

[0093] In some exemplary embodiments, the process (algorithm) of step 5 is as follows: Figure 3 The pseudo code is shown in the figure.

[0094] Step 6: Determine the classification network and image classification:

[0095] The image features obtained in step 4 and the image feature weights obtained in step 5 are used to train a linear classifier (referred to as the first linear classifier), and this linear classifier (the first linear classifier) ​​is used as the second part of the network. The first part and the second part of the network are combined to determine a classification network capable of classifying lung images. New lung images are classified according to the classification network.

[0096] Among them, the linear classifier (the first linear classifier) ​​is trained using the stochastic gradient descent algorithm and the FocalLoss loss function. When calculating the value of the loss function of each sample, the value of the loss function is multiplied by the sample weight to obtain a new loss function:

[0097] Loss re =w*Loss (3)

[0098] Where w is the weight corresponding to the image feature of the sample (image) calculated in step 5.

[0099] Loss is the value of the loss function, which is calculated according to the following formula:

[0100] Loss = -α*(1-l p ) γ *logl p (4)

[0101] Among them, α,γ are the hyper parameters of Focal Loss, l p is the probability that the first linear classifier judges the sample to be a positive sample.

[0102] Then, back-propagation and parameter update are performed on the first linear classifier to obtain the weight of the linear classifier (the first linear classifier).

[0103] The classification network obtained by combining the first part of the network and the second part of the network can classify new images. The classification network is also called a classification model.

[0104] In some exemplary embodiments, the classification network obtained by combining the first partial network and the second partial network includes: directly connecting the first partial network and the second partial network to obtain the classification network shown; or, using the output of the first partial network as the input of the second partial network to obtain the classification network.

[0105] The present disclosure also provides relevant experimental data as follows:

[0106] Experimental setup:

[0107] This experiment uses the annotated lung image data mentioned in the first embodiment of this invention, and divides the data into different parts, with no overlap between the parts, different ratios of positive and negative samples in different parts, and different distributions of samples in different parts. The same part of the data is used to train the classification network (model) of the first embodiment of this invention, the common neural network model DenseNet, and the DenseNet that has used Focal Loss, and other parts of the data set are used for verification and testing, and the average accuracy and variance of the accuracy of the tests under different distributions and positive and negative sample ratios are calculated.

[0108] The experimental environment is Linux 18.04 system, Python 3.8 environment, Pytorch 1.4 version.

[0109] Experimental results and analysis:

[0110] The classification network (model) of the first embodiment of the present disclosure, the training neural network model DenseNet and the DenseNet with Focal Loss were trained using the same part of the data, and the weights of each network were obtained respectively. The other parts of the data set were used for verification and testing, and the average accuracy and variance of the accuracy of the tests under different distributions and positive and negative sample ratios were calculated. The performance of different models on the same test set was compared. The experimental results are recorded in Table 1.

[0111] Table 1 Test accuracy comparison table

[0112]

[0113] As can be seen from Table 1, the classification network (model) determined by the solution of the first embodiment of the present disclosure has an improved average accuracy rate and a reduced variance of the accuracy rate compared to the common model, and has a better classification effect on lung images. The performance on data with different distributions is relatively close, and has good transferability.

[0114] Embodiment 2

[0115] The disclosed embodiment also provides a method for determining a lung image classification network, such as Figure 4 As shown, including:

[0116] Step 41, obtaining lung image pictures with labels and establishing a lung image dataset; the labels include: positive sample labels and negative sample labels;

[0117] Step 42, pre-training the complete DenseNet network according to the images and labels in the data set, removing the fully connected layer from the pre-trained DenseNet network to determine the first part of the network;

[0118] Step 43, extracting image features of each picture in the data set according to the first partial network, eliminating collinearity of the image features, and determining the weight of each image feature;

[0119] Step 44, training a first linear classifier according to the image features and corresponding weights of the pictures in the data set, and determining the trained first linear classifier as the second partial network;

[0120] Step 45: Combine the first partial network and the second partial network to determine the lung image classification network.

[0121] In some exemplary embodiments, before pre-training the complete DenseNet network according to the images and labels in the data set, the method further includes:

[0122] Preprocessing the images in the dataset includes at least one of the following:

[0123] Crop the images in the data set according to a preset size;

[0124] Flipping the images of a preset proportion in the data set in a preset direction;

[0125] The images in the dataset are normalized.

[0126] In some exemplary embodiments, flipping the pictures of a preset proportion in the data set in a preset direction includes:

[0127] Flipping the images of a preset proportion in the data set horizontally;

[0128] Alternatively, images of a preset proportion in the data set are flipped vertically.

[0129] In some exemplary embodiments, pre-training a complete DenseNet network according to the images and labels in the data set includes:

[0130] According to the images and labels in the data set, the complete DenseNet network is trained using a stochastic gradient descent algorithm until the weights of the DenseNet network meet the preset convergence conditions.

[0131] In some exemplary embodiments, the use of a stochastic gradient descent algorithm to train a complete DenseNet network includes:

[0132] Use the stochastic gradient descent algorithm and the Focal Loss function to train the complete DenseNet network.

[0133] Among them, the Focal Loss loss function is:

[0134] Loss = -α*(1-l p ) γ *logl p

[0135] Among them, α,γ are the hyper parameters of Focal Loss, l p It is the probability that the complete DenseNet network determines that the picture input into the complete DenseNet network is a positive sample.

[0136] In some exemplary embodiments, extracting image features of each picture in the data set according to the first partial network, eliminating collinearity of the image features, and determining the weight of each image feature includes:

[0137] According to the first part of the network, extract image features of each picture in the data set respectively;

[0138] According to the image features of each picture, determine the pseudo features corresponding to each picture respectively;

[0139] Training a second linear classifier based on the image features and pseudo features of the pictures in the data set;

[0140] Determine the positive sample probability and negative sample probability of each image according to the trained second linear classifier;

[0141] Determine the weight of each image feature according to the positive sample probability and negative sample probability of each image;

[0142] The weight of each image feature is equal to the ratio of the probability of a positive sample belonging to the image to the probability of a negative sample.

[0143] In some exemplary embodiments, the image features of the picture include features of at least one dimension;

[0144] The determining of pseudo features corresponding to each picture according to the image features of each picture includes:

[0145] The following operations are performed for each image: a pseudo feature is generated for each dimensional feature of the image, and a set of pseudo features corresponding to the image is obtained; wherein the pseudo feature is a feature of the same dimension corresponding to an image randomly selected from other images in the data set other than the image itself;

[0146] The step of training a second linear classifier according to the image features and pseudo features of the pictures in the data set comprises:

[0147] The Focal Loss function is adopted, the image features of the pictures in the data set are used as negative samples, the pseudo features corresponding to the pictures are used as positive samples, and the stochastic gradient descent algorithm is used to train the second linear classifier.

[0148] In some exemplary embodiments, training a first linear classifier according to image features and corresponding weights of pictures in the data set includes:

[0149] The first linear classifier is trained using a Focal Loss loss function and a stochastic gradient descent algorithm;

[0150] When calculating the value of the loss function for each image, the value of the loss function is multiplied by the corresponding weight to obtain a new loss function:

[0151] Loss re =w*Loss

[0152] Among them, w is the corresponding weight, and Loss is the value of the loss function;

[0153] Loss is calculated according to the following formula:

[0154] Loss = -α*(1-l p ) γ *logl p

[0155] Among them, α,γ are the hyper parameters of Focal Loss, l p is the probability that the first linear classifier judges the sample to be a positive sample.

[0156] The obtained new loss function is used to perform back propagation and parameter update on the first linear classifier to determine the weight of the first linear classifier.

[0157] Embodiment 3

[0158] The disclosed embodiment also provides a lung image classification method, such as Figure 5 As shown, including:

[0159] Step 51, obtaining a lung image to be classified;

[0160] Step 52: classify the lung image to be classified according to the lung image classification network determined by the method described in any of the above embodiments.

[0161] Embodiment 4

[0162] The present disclosure also provides a device for determining a lung image classification network, including:

[0163] A sample acquisition module is configured to acquire lung image pictures with labels to establish a lung image dataset; the labels include: positive sample labels and negative sample labels;

[0164] The first part of the network determination module is configured to pre-train the complete DenseNet network according to the images and labels in the data set, and remove the fully connected layer from the pre-trained DenseNet network to determine the first part of the network;

[0165] A weight determination module, configured to extract image features of each picture in the data set according to the first partial network, eliminate collinearity of the image features, and determine the weight of each image feature;

[0166] A second partial network determination module is configured to train a first linear classifier according to image features and corresponding weights of the pictures in the data set, and determine the trained first linear classifier as the second partial network;

[0167] The classification network determination module is configured to combine the first partial network and the second partial network to determine the lung image classification network.

[0168] In some exemplary embodiments, the device further comprises a pre-processing module;

[0169] The preprocessing module is configured to preprocess the images in the data set before pre-training the complete DenseNet network according to the images and labels in the data set, including at least one of the following:

[0170] Crop the images in the data set according to a preset size;

[0171] Flipping the images of a preset proportion in the data set in a preset direction;

[0172] The images in the dataset are normalized.

[0173] In some exemplary embodiments, flipping the pictures of a preset proportion in the data set in a preset direction includes:

[0174] Flipping the images of a preset proportion in the data set horizontally;

[0175] Alternatively, images of a preset proportion in the data set are flipped vertically.

[0176] In some exemplary embodiments, the first part network determination module is further configured to:

[0177] According to the images and labels in the data set, the complete DenseNet network is trained using a stochastic gradient descent algorithm until the weights of the DenseNet network meet the preset convergence conditions.

[0178] In some exemplary embodiments, the first part network determination module is further configured to:

[0179] Use the stochastic gradient descent algorithm and the Focal Loss function to train the complete DenseNet network.

[0180] Among them, the Focal Loss loss function is:

[0181] Loss = -α*(1-l p ) γ *logl p

[0182] Among them, α,γ are the hyper parameters of Focal Loss, l p It is the probability that the complete DenseNet network determines that the picture input into the complete DenseNet network is a positive sample.

[0183] In some exemplary embodiments, the weight determination module is further configured to:

[0184] According to the first part of the network, extract image features of each picture in the data set respectively;

[0185] According to the image features of each picture, determine the pseudo features corresponding to each picture respectively;

[0186] Training a second linear classifier based on the image features and pseudo features of the pictures in the data set;

[0187] Determine the positive sample probability and negative sample probability of each image according to the trained second linear classifier;

[0188] Determine the weight of each image feature according to the positive sample probability and negative sample probability of each image;

[0189] The weight of each image feature is equal to the ratio of the probability of a positive sample belonging to the image to the probability of a negative sample.

[0190] In some exemplary embodiments, the image features of the picture include features of at least one dimension;

[0191] The weight determination module determines the pseudo features corresponding to each picture according to the image features of each picture, including:

[0192] The following operations are performed for each image: a pseudo feature is generated for each dimensional feature of the image, and a set of pseudo features corresponding to the image is obtained; wherein the pseudo feature is a feature of the same dimension corresponding to an image randomly selected from other images in the data set other than the image itself;

[0193] The step of training a second linear classifier according to the image features and pseudo features of the pictures in the data set comprises:

[0194] The Focal Loss function is adopted, the image features of the pictures in the data set are used as negative samples, the pseudo features corresponding to the pictures are used as positive samples, and the stochastic gradient descent algorithm is used to train the second linear classifier.

[0195] In some exemplary embodiments, the second network determination module is further configured to:

[0196] The first linear classifier is trained using a Focal Loss loss function and a stochastic gradient descent algorithm;

[0197] When calculating the value of the loss function for each image, the value of the loss function is multiplied by the corresponding weight to obtain a new loss function:

[0198] Loss re =w*Loss

[0199] Among them, w is the corresponding weight, and Loss is the value of the loss function;

[0200] Loss is calculated according to the following formula:

[0201] Loss = -α*(1-l p ) γ *logl p

[0202] Among them, α,γ are the hyper parameters of Focal Loss, l p is the probability that the first linear classifier judges the sample to be a positive sample.

[0203] The second partial network determination module is further configured to: use the obtained new loss function to perform back propagation and parameter update on the first linear classifier to determine the weight of the first linear classifier.

[0204] Embodiment 5

[0205] The present disclosure also provides a lung image classification device, including:

[0206] An image acquisition module, configured to acquire lung image pictures to be classified;

[0207] The classification module is configured to classify the lung image pictures to be classified according to the lung image classification network determined by the method described in any of the above embodiments.

[0208] An embodiment of the present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program for determining a lung image classification network, and the processor is configured to read and run the computer program for determining a lung image classification network to execute any of the above-mentioned methods for determining a lung image classification network.

[0209] An embodiment of the present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program for classifying lung images, and the processor is configured to read and run the computer program for classifying lung images to execute any of the above-mentioned methods for classifying lung images.

[0210] The present disclosure also provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute any of the above-mentioned methods for determining a lung image classification network when running.

[0211] The present disclosure also provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute any of the above-mentioned lung image classification methods when run.

[0212] The lung image classification network (model) determined by the solution provided by the present disclosure after removing collinearity based on sample weighting has the following advantages:

[0213] (1) The model has strong fitting ability, good classification performance and high accuracy;

[0214] (2) The model has strong generalization ability and can perform well on data with different distributions, which is more practical in medical clinical applications where the data distribution is different from the training data;

[0215] (3) The model can run end-to-end and can adapt to real-time classification and auxiliary diagnosis and treatment in clinical situations.

[0216] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A method for determining a lung image classification network, characterized in that: include, Obtaining lung image pictures with labels to establish a lung image dataset; the labels include: positive sample labels and negative sample labels; Pre-training a complete DenseNet network according to the images and labels in the data set, and removing the fully connected layer from the pre-trained DenseNet network to determine the first part of the network; Extracting image features of each image in the data set according to the first partial network, eliminating collinearity of the image features, and determining a weight of each image feature; Training a first linear classifier according to image features and corresponding weights of the pictures in the data set, and determining the trained first linear classifier as the second partial network; Combining the first partial network and the second partial network to determine the lung image classification network; The pre-training of the complete DenseNet network according to the images and labels in the data set includes: training the complete DenseNet network using a stochastic gradient descent algorithm according to the images and labels in the data set until the weights of the DenseNet network meet a preset convergence condition; The extracting, according to the first part of the network, image features of each picture in the data set, eliminating collinearity of the image features, and determining the weight of each image feature includes: According to the first part of the network, extract image features of each picture in the data set respectively; According to the image features of each picture, determine the pseudo features corresponding to each picture respectively; Training a second linear classifier based on the image features and pseudo features of the pictures in the data set; Determine the positive sample probability and negative sample probability of each image according to the trained second linear classifier; Determine the weight of each image feature according to the positive sample probability and negative sample probability of each image; The weight of each image feature is equal to the ratio of the probability of a positive sample belonging to the image to the probability of a negative sample.

2. The method according to claim 1, characterized in that Before pre-training the complete DenseNet network according to the images and labels in the data set, the method further includes: Preprocessing the images in the dataset includes at least one of the following: Crop the images in the data set according to a preset size; Flipping the images of a preset proportion in the data set in a preset direction; The images in the dataset are normalized.

3. The method according to claim 1, characterized in that The stochastic gradient descent algorithm is used to train the complete DenseNet network, including: Use the stochastic gradient descent algorithm and the Focal Loss function to train the complete DenseNet network. Among them, the Focal Loss loss function is: Loss=-α*(1-l p ) γ *logl p Among them, α,γ are the hyper parameters of Focal Loss, l p It is the probability that the complete DenseNet network determines that the picture input into the complete DenseNet network is a positive sample.

4. The method according to claim 1, characterized in that The image features of the picture include features of at least one dimension; The determining of pseudo features corresponding to each picture according to the image features of each picture includes: The following operations are performed for each image: a pseudo feature is generated for each dimensional feature of the image, and a set of pseudo features corresponding to the image is obtained; wherein the pseudo feature is a feature of the same dimension corresponding to an image randomly selected from other images in the data set other than the image itself; The step of training a second linear classifier according to the image features and pseudo features of the pictures in the data set comprises: The Focal Loss function is adopted, the image features of the pictures in the data set are used as negative samples, the pseudo features corresponding to the pictures are used as positive samples, and the stochastic gradient descent algorithm is used to train the second linear classifier.

5. The method according to claim 1 or 2, characterized in that: The step of training a first linear classifier according to the image features and corresponding weights of the pictures in the data set includes: The first linear classifier is trained using a FocalLoss loss function and a stochastic gradient descent algorithm; When calculating the loss function value of each image, the value of the loss function Loss is multiplied by the corresponding weight w to obtain a new loss function: Loss re =w*Loss The obtained new loss function is used to perform back propagation and parameter update on the first linear classifier to determine the weight of the first linear classifier.

6. A lung image classification method, characterized in that: include, Obtain lung images to be classified; The lung image pictures to be classified are classified according to the lung image classification network determined by the method according to any one of claims 1 to 5.

7. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program for determining a lung image classification network or performing lung image classification, and the processor is configured to read and run the computer program for determining a lung image classification network or performing lung image classification to execute the method described in any one of claims 1 to 5 or claim 6.

8. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method described in any one of claims 1 to 5 or claim 6 when executed.

Images