Method and system for identifying false positives of lung nodules based on multi-scale redundant convolution

Abstract

The application discloses a kind of lung nodules false positive discrimination method and system based on multiscale redundant convolution.First, training set is obtained from lung nodules CT image database;Data in training set is input into redundant convolution layer of multiscale redundant convolution neural network, and Maxout module obtains key feature map;Key feature map is input into multiscale convolution layer, and multiscale feature map is obtained;Multiscale feature map is sequentially input into activation function, pooling layer, full connection layer, and the probability that sample is false positive sample and positive sample is obtained after the processing of Softmax function;Probability can judge whether it is lung nodules false positive sample.The application can automatically filter key features for input data of different types of lung nodules, so as to more effectively train a prediction network from a mixed data set;The application can adapt to input data of different actual size lung nodules, improve accuracy while not needing to train multiple network models according to different sample sizes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and particularly relates to a lung nodule false positive identification method and system based on multi-scale redundant convolution. BACKGROUND

[0002] With the development of convolutional neural network technology, computer vision has achieved more and more results in the field of medical diagnosis. Among them, the representative is the CAD (Computer Aided Diagnosis) system, which identifies and segments the organ, lesion and other regions in the input image through the method of deep learning, for assisting the process of medical diagnosis.

[0003] The lung nodule diagnosis technology based on convolutional neural network mainly includes two steps: first, the coordinates and range of the lung nodule pre-selected region are determined through the neural network, or the image region where the lung nodule is located is directly generated through the end-to-end neural network. Generally, there are a large number of false positive samples in the lung nodule samples obtained in the first step, that is, they are identified as lung nodules, but do not have pathological nature. Therefore, it is necessary to screen the lung nodule samples obtained in the previous step through the prediction model to eliminate the false positive samples.

[0004] Different lung nodule sample extraction methods mainly differ in extraction efficiency, so the key to correct diagnosis lies in the identification and inhibition of false positive samples. The identification of false positive samples can be modeled as an image binary classification problem, that is, the input image is converted into a binary data indicating whether it is a false positive sample or not. The classical method based on convolutional neural network can map the image to a feature space through convolution, pooling and other operations, and finally obtain the prediction result through the fully connected layer and the Softmax function. This method needs to use a labeled lung nodule sample dataset to train the network, so as to obtain the prediction model.

[0005] However, lung nodules include solid nodules, pure ground glass nodules and mixed ground glass nodules. The three kinds of lung nodules differ in morphological characteristics, so if the same network is trained using a dataset containing three kinds of nodules mixed together, it will cause certain confusion and affect the accuracy of the prediction model.

[0006] One of the current prior arts is the patent "Lung nodule false positive sample suppression method based on 3DCNN". This technology first finds the coordinates of the preselected lung nodule sample, then expands the input CT image into 3D data through interpolation, and cuts out 40x40x40 sample data according to the preselected sample coordinates. After labeling the sample data, the positive samples are expanded through translation, scaling, rotation and other operations, so that the number of positive samples and false positive samples in the data set is balanced. The input of 3DCNN is 3D sample data, which is then processed through convolution layer, pooling layer, full connection layer, Dropout layer and Softmax operation to obtain the probability that the sample belongs to false positive sample. The prepared data set is used to train the network to obtain the prediction model, which can be used for lung nodule false positive sample suppression. The disadvantage of this technology is that the same size of sample data is used for training, which is not conducive to the recognition of lung nodules of different actual sizes; and different types of lung nodules are not distinguished.

[0007] The second current prior art is the patent "Lung nodule false positive screening method based on convolutional neural network". This technology first obtains the coordinates and radius of the lung nodule preselected area from the detection model, and marks it as positive or false positive sample. According to the lung nodule coordinates obtained by the detection model, 32x32x32, 64x64x64 and 96x96x96 3D slices are respectively cut out from the original CT image, and then the obtained slices are collected into three data sets according to the size. The convolutional neural network of this technology is composed of convolution layer, global average pooling layer, full connection layer and Softmax. Finally, the prepared three data sets are used to train the network to obtain the prediction model for lung nodule false positive sample suppression. The disadvantage of this technology is that multiple convolutional neural networks are used to process lung nodule samples of different sizes, which is complex in implementation, and the process of training the network requires preparing double training data. At the same time, this technology does not consider the influence of different types of lung nodules on network training. SUMMARY

[0008] The purpose of the present application is to overcome the shortcomings of the existing method, and a lung nodule false positive identification method and system based on multi-scale redundant convolution are proposed. The main problems solved by the present application are: first, the existing method uses a data set that mixes different types of lung nodule samples, which can easily confuse the network and reduce the accuracy of the prediction model; second, when dealing with lung nodule samples of different actual sizes, the existing method uses multiple independent convolutional neural networks for training, which is complex and difficult to collect enough training data in the case of few labeled samples.

[0009] To solve the above problems, the application provides a lung nodule false positive identification method based on a multi-scale redundant convolution, which comprises the following steps:

[0010] CT images are obtained from a lung nodule CT image database, and coordinates of a preselected region of a lung nodule in the CT images are obtained after a lung nodule detection model, then a preselected region sample with a size of 96x96x96 is obtained from the CT images according to the coordinates, and true and false positive sample labeling is performed, and meanwhile, data enhancement is performed on the labeled sample, so that a training set used for training a multi-scale redundant convolution neural network is obtained.

[0011] The data in the training set are input into the multi-scale redundant convolution neural network, first processed through a redundant convolution layer, and then the key features are extracted through a Maxout module to obtain a key feature map.

[0012] The key feature map is input into a multi-scale convolution layer in the multi-scale redundant convolution neural network, and after multi-scale convolution operation, a multi-scale feature map is obtained by splicing all the feature maps.

[0013] The multi-scale feature map is sequentially input into an activation function and a pooling layer in the multi-scale redundant convolution neural network, the output data are stretched to obtain a one-dimensional column vector, the obtained column vector is taken as the input of a full connection layer, and the probability of a sample being a false positive sample and a positive sample is obtained through a Softmax function.

[0014] The cross-entropy loss is calculated according to the probability of the false positive sample and the positive sample, and the multi-scale redundant convolution neural network is trained by using the cross-entropy loss function, and finally a trained lung nodule false positive sample prediction model is obtained.

[0015] A user inputs a CT image to be identified into the lung nodule false positive sample prediction model, and according to the probability of the output false positive sample and the positive sample, it is judged whether the input CT image is a lung nodule false positive sample.

[0016] Preferably, the lung nodule CT image database is from a LIDC-IDRI database, the LIDC-IDRI database is composed of chest medical images and the corresponding diagnostic result labeling, and the labeling of the lung nodule CT image database is divided into two categories of nodules and non-nodules.

[0017] Preferably, after the lung nodule detection model, the coordinates of the preselected region of the lung nodule in the CT images are obtained, then the preselected region sample with the size of 96x96x96 is obtained from the CT images according to the coordinates, and the true and false positive sample labeling is performed, and specifically:

[0018] According to the coordinates obtained by the lung nodule detection model, a 96x96xN sample is cut out from the CT image, where N is the number of slices, and then N slices are uniformly interpolated to obtain a 96x96x96 candidate sample; then according to the database annotation, it is determined whether there is a lung nodule in the candidate sample, if there is, it is marked as a positive sample, if there is not, it is marked as a false positive sample.

[0019] Preferably, the annotated sample is subjected to data enhancement to obtain a training set for training the multi-scale redundant convolutional neural network, specifically:

[0020] Since the lung nodule detection model only roughly finds the candidate region of the lung nodule in the original image, the obtained sample usually contains many false positive samples. In order to maintain the balance between the number of false positive samples and positive samples, the positive samples need to be rotated, translated and scaled. Rotation: after rotating the positive samples by a certain angle, they are added as additional positive samples to the training set. Translation: in the CT image, the center coordinates of the lung nodule candidate region are translated by a certain number of pixels up, down, left and right, and then the candidate sample is cut out and added to the training set. Scaling: a MxMx96 candidate sample is obtained from the CT image, and M>96, then the sample is uniformly sampled, blurred and re-obtained as a 96x96x96 sample and added to the training set.

[0021] Preferably, the data in the training set is input into the multi-scale redundant convolutional neural network, first processed by the redundant convolutional layer to obtain redundant features, and then the key features are extracted by the Maxout module to obtain the key feature map, specifically:

[0022] The redundant convolutional layer of the multi-scale redundant convolutional neural network is composed of KxC1x5x5x5 convolution kernels, where K represents the number of nodule categories, i.e. solid nodule, pure ground glass nodule and mixed ground glass nodule, so K is 3, C1 is the number of channels after processing by the redundant convolutional layer, and the convolution kernel size is 5x5x5.

[0023] The feature map extracted by the redundant convolutional layer contains K groups of redundant features, and the Maxout module is used to extract key features from the redundant features, and the formula is:

[0024]

[0025] Where G represents the feature map extracted by the redundant convolutional layer, c represents the channel of the feature map and c∈[1,C1], F c (x) represents the cth channel of the key feature map F(x), G j，c(x) represents the cth channel of the K group of redundant feature maps G(x), and the max function traverses the C1-dimensional channels from the K group of redundant feature maps and selects the strongest feature from each group of features;

[0026] The above formula runs C1 times, thereby extracting a group of C1-dimensional key feature maps F(x).

[0027] Preferably, after the multi-scale convolution operation extracts multi-granularity feature maps, the multi-scale feature maps are obtained by splicing all the obtained feature maps, specifically:

[0028] The multi-scale convolution operation includes D groups of convolution layers, and the key feature maps are respectively input into each group of convolution layers to obtain feature maps of the same size; D is 3, that is, a total of 3 groups of feature maps are obtained; the same size of convolution kernels is used in each group of convolution layers, and the sizes are 3x3x3, 5x5x5 and 9x9x9 respectively; the 3x3x3 convolution operation includes four convolution layers, and the number of convolution kernels is 16, 32, 64 and 128 respectively; the 5x5x5 convolution operation includes two convolution layers, and the number of convolution kernels is 32 and 128 respectively; the 9x9x9 convolution operation includes one convolution layer, and the number of convolution kernels is 128;

[0029] The 3 groups of feature maps are spliced by anchoring the first dimension as the splicing reference:

[0030]

[0031] Wherein, F1, F2 and F3 respectively represent the feature maps obtained by the three scale convolution layers, h represents a certain dimension of the feature map, and h∈[0, 128); by splicing three 128-dimensional feature maps, a 384-dimensional multi-scale feature map is obtained as the output.

[0032] Preferably, after the multi-scale feature maps are sequentially input into the activation function and the pooling layer in the multi-scale redundant convolution neural network, the output data is stretched to obtain a one-dimensional column vector, and the obtained column vector is taken as the input of the full connection layer, and the probability that the sample is a false positive sample and a positive sample is obtained by processing the Softmax function, specifically:

[0033] The activation function in the multi-scale redundant convolution neural network adopts the ReLU nonlinear activation function, which sets the data less than 0 to 0, and the data greater than or equal to 0 is not changed; the pooling layer in the multi-scale redundant convolution neural network adopts the 2x2x2 average pooling operation;

[0034] The data output after the activation function and the pooling layer can be obtained after stretching a 1xV column vector, and the full connection layer will perform convolution processing on the column vector and map it to the probability of different sample categories;Two full connection layers are used for processing, the first full connection layer uses a convolution kernel of Vx64, and the input data is converted into a 1x64 column vector;The second full connection layer uses a convolution kernel of 64x2, and the input data is converted into a 1x2 column vector, and the two values in the vector represent the probability weight values of the false positive sample and the positive sample respectively;After the two values obtained are processed by the Softmax function, the probabilities of the false positive sample and the positive sample can be obtained.

[0035] Preferably, the cross-entropy loss function is represented as:

[0036]

[0037] Wherein, T represents the total number of categories, and T is 2, that is, two categories of false positive samples and positive samples;p t represents the probability of t categories obtained by the multi-scale redundant convolutional neural network, t is 1 or 2, p1 is the probability of false positive samples, and p2 is the probability of positive samples;y t represents the different category penalty proportion obtained according to the actual label;L is the cross-entropy loss of the network output result.

[0038] Correspondingly, the application also provides a lung nodule false positive identification system based on multi-scale redundant convolution, comprising:

[0039] The training set generation unit is used for obtaining CT images from the lung nodule CT image database, obtaining the coordinates of the preselected region of the lung nodule in the CT image after the lung nodule detection model, then obtaining the preselected region sample with a size of 96x96x96 from the CT image according to the coordinates, and performing true and false positive sample labeling, and simultaneously performing data enhancement on the labeled samples, so as to obtain the training set used by the multi-scale redundant convolutional neural network.

[0040] The key feature map extraction unit is used for inputting the data in the training set into the multi-scale redundant convolutional neural network, first performing processing through the redundant convolutional layer, obtaining redundant features, then extracting key features through the Maxout module to obtain a key feature map;

[0041] The multi-scale feature map extraction unit is used for inputting the key feature map into the multi-scale convolutional layer in the multi-scale redundant convolutional neural network, extracting multi-granularity feature maps after multi-scale convolution operation, and splicing all the obtained feature maps to obtain a multi-scale feature map.

[0042] A probability calculation unit is configured to input the multi-scale feature maps into an activation function and a pooling layer in the multi-scale redundant convolutional neural network in sequence, stretch the output data to obtain a one-dimensional column vector, take the column vector as an input of a full connection layer, and obtain probabilities of the sample being a false positive sample and a positive sample through a Softmax function.

[0043] A network training unit is configured to calculate a cross-entropy loss according to the probabilities of the false positive sample and the positive sample, train the multi-scale redundant convolutional neural network by using a cross-entropy loss function, and finally obtain a trained lung nodule false positive sample prediction model.

[0044] A false positive discrimination unit is configured to input a CT image to be discriminated by a user into the lung nodule false positive sample prediction model, and judge whether the input CT image is a lung nodule false positive sample according to the probabilities of the false positive sample and the positive sample output.

[0045] The present application has the following beneficial effects:

[0046] The present application can automatically filter key features of input data of different types of lung nodules, thereby more effectively training a prediction network from a mixed data set. Meanwhile, the present application can adapt to input data of different actual sizes of lung nodules, improve the accuracy, and does not need to train multiple network models according to different sample sizes. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a general flowchart of a lung nodule false positive discrimination method based on multi-scale redundant convolution according to an embodiment of the present application;

[0048] Figure 2 is a two-dimensional schematic diagram of multi-scale convolution operation according to an embodiment of the present application;

[0049] Figure 3 is a structure diagram of a lung nodule false positive discrimination system based on multi-scale redundant convolution according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0051] Figure 1 is a general flowchart of a lung nodule false positive discrimination method based on multi-scale redundant convolution according to an embodiment of the present application,Figure 1 The method comprises:

[0052] S1, from the lung nodule CT image database, after the lung nodule detection model, the coordinates of the preselected region of the lung nodule in the CT image are obtained, then the preselected region sample with the size of 96*96*96 is obtained from the CT image according to the coordinates, and the true and false positive sample labeling is carried out, and the labeled sample is subjected to data enhancement, so as to obtain the training set used for training the multi-scale redundant convolutional neural network;

[0053] S2, input the data in the training set into the multi-scale redundant convolutional neural network, first pass through the redundant convolutional layer processing, obtain the redundant feature, then pass through the Maxout module to extract the key feature, and obtain the key feature map;

[0054] S3, input the key feature map into the multi-scale convolutional layer in the multi-scale redundant convolutional neural network, after the multi-scale convolutional operation extracts the multi-granularity feature map, the obtained all feature maps are spliced to obtain the multi-scale feature map;

[0055] S4, input the multi-scale feature map into the activation function and the pooling layer in the multi-scale redundant convolutional neural network in turn, stretch the output data to obtain a one-dimensional column vector, take the obtained column vector as the input of the full connection layer, and process it through the Softmax function to obtain the probability of the sample being a false positive sample and a positive sample;

[0056] S5, calculate the cross-entropy loss according to the probability of the false positive sample and the positive sample, and train the multi-scale redundant convolutional neural network by using the cross-entropy loss function, and finally obtain the trained lung nodule false positive sample prediction model;

[0057] S6, the user inputs the CT image to be identified into the lung nodule false positive sample prediction model, and judges whether the input CT image is a lung nodule false positive sample according to the probability of the output false positive sample and the positive sample.

[0058] Step S1 is as follows:

[0059] S1-1, the lung nodule CT image data comes from the LIDC-IDRI database, the LIDC-IDRI database is composed of chest medical images and the corresponding diagnostic result labeling, the labeling is divided into three categories: >=3mm nodules; <3mm nodules; >=3mm non-nodules. In the present application, it is simplified and labeled as two categories: nodules and non-nodules.

[0060] S1-2, the lung nodule detection model used in the present application can extract the center position coordinates of the lung nodule preselected region and the number N of slices contained in the region.

[0061] S1-3, according to the lung nodule detection model, a 96x96xN sample is cut out from the original image, then N slices are uniformly interpolated to obtain a 96x96x96 candidate sample, and then whether the lung nodule exists in the candidate sample is determined according to the database label, if the lung nodule exists, the candidate sample is marked as a positive sample, and if the lung nodule does not exist, the candidate sample is marked as a false positive sample.

[0062] S1-4, since the lung nodule detection model only roughly finds the candidate region of the lung nodule in the original image, the sample obtained usually contains more false positive samples, in order to keep the balance between the number of false positive samples and positive samples, data enhancement such as rotation, translation and scaling is needed for the positive samples. Rotation: after rotating the positive sample by a certain angle, the rotated sample is added to the training set as an additional positive sample. Translation: in the CT image, the center coordinates of the lung nodule candidate region are respectively translated by a certain number of pixel points upward, downward, left and right, and then the candidate sample is recut and added to the training set. Scaling: a MxMx96 candidate sample is obtained from the CT image, and M>96, then the sample is uniformly sampled, blurred and recut to obtain a 96x96x96 sample, which is added to the training set.

[0063] Step S2 is as follows:

[0064] S2-1, the redundant convolutional layer of the multi-scale redundant convolutional neural network is composed of KxC1x5x5x5 convolutional kernels, wherein K represents the number of nodule categories, i.e. solid nodule, pure ground glass nodule and mixed ground glass nodule, so K is 3, C1 is the number of channels after processing by the redundant convolutional layer, and the size of the convolutional kernel is 5x5x5.

[0065] S2-2, the feature map extracted by the redundant convolutional layer contains K groups of redundant features, and a Maxout module is used to extract key features from the redundant features, and the formula is:

[0066]

[0067] wherein G represents the feature map extracted by the redundant convolutional layer, c represents the channel of the feature map and c∈[1,C1], F c (x) represents the cth channel of the key feature map F(x), G j，c (x) represents the cth channel of the K groups of redundant feature maps G(x), and the max function traverses the C1-dimensional channel from the K groups of redundant feature maps and selects the highest intensity feature from each group of features;

[0068] The above formula runs C1 times, thereby extracting a group of C1-dimensional key feature maps F(x).

[0069] Step S3 is specifically as follows:

[0070] S3-1, the multi-scale convolution operation contains D groups of convolution layers, and the key feature maps are respectively input into each group of convolution layers to obtain feature maps of the same size; D is 3, that is, a total of 3 groups of feature maps are obtained; the same size of convolution kernels is used in each group of convolution layers, and the sizes are 3x3x3, 5x5x5 and 9x9x9 respectively; the 3x3x3 convolution operation contains four convolution layers, and the number of convolution kernels is 16, 32, 64 and 128 respectively; the 5x5x5 convolution operation contains two convolution layers, and the number of convolution kernels is 32 and 128 respectively; the 9x9x9 convolution operation contains one convolution layer, and the number of convolution kernels is 128. As shown in the figure, the final output size of the convolution operation using the three convolution kernels of 3x3, 5x5 and 9x9 is the same through different layers, and the same is true for three-dimensional convolution operation. Figure 2

[0071] S3-2, the three groups of feature maps are anchored to the first dimension as a splicing reference for splicing:

[0072]

[0073] Among them, F1, F2 and F3 respectively represent the feature maps obtained by the convolution layers of the three scales, and h represents a certain dimension of the feature map, and h∈[0, 128); by splicing the three 128-dimensional feature maps, a 384-dimensional multi-scale feature map is obtained as the output.

[0074] Step S4 is specifically as follows:

[0075] S4-1, the activation function step in the multi-scale redundant convolutional neural network adopts the ReLU nonlinear activation function, which sets the data less than 0 to 0, and the data greater than or equal to 0 is not changed; the pooling layer in the multi-scale redundant convolutional neural network adopts the average pooling operation of 2x2x2;

[0076] S4-2, after the data output after the activation function and the pooling layer is stretched, a 1xV column vector can be obtained, and the full connection layer will perform convolution processing on the column vector and map it to the probability of different sample categories; two full connection layers are used for processing, the convolution kernel used by the first full connection layer is Vx64, and the input data is converted into a 1x64 column vector; the convolution kernel used by the second full connection layer is 64x2, and the input data is converted into a 1x2 column vector, and the two numerical values in the vector respectively represent the probability weight values of the false positive sample and the positive sample of the input sample data; after the two numerical values obtained are processed by the Softmax function, the probabilities of the false positive sample and the positive sample can be obtained.

[0077] Step S5 is specifically as follows:​

[0078] The training process of the network calculates the cross-entropy loss according to the probability obtained in S4, updates the network parameters, and then repeats S2 to S4 again;

[0079] The cross-entropy loss function is represented as:

[0080]

[0081] Wherein, T represents the total number of categories, and T is 2, that is, two categories of false positive samples and positive samples; p t represents the probability of the t-th category obtained by the multi-scale redundant convolutional neural network, t is 1 or 2, p1 is the probability of false positive samples, and p2 is the probability of positive samples; y t represents the penalty proportion of different categories according to the actual label; L is the cross-entropy loss of the network output result.

[0082] Since the cost of missed diagnosis is higher in actual medical diagnosis, the penalty proportion of the network loss part for the false positive sample (actually a positive sample, but predicted as a false positive sample) is increased: if the sample is actually a false positive sample, then y in the cross-entropy loss function {0, 1}, the loss L=-0*log(p1)-1*log(p2); if the sample is actually a positive sample, then y in the cross-entropy loss function {16, 0}, the loss L=-16*log(p1)-0*log(p2).

[0083] In step S6, whether the input is a lung nodule false positive sample is judged according to the probability output from S4. In actual operation, since the cost of missed diagnosis is higher, the condition for judging false positive samples can be set as: the probability of false positive samples obtained by the model > 90%.

[0084] Correspondingly, the application also provides a lung nodule false positive identification system based on multi-scale redundant convolution, as shown in Figure 3 The lung nodule false positive identification system based on multi-scale redundant convolution comprises:

[0085] The training set generation unit 1 is used to obtain CT images from a lung nodule CT image database, obtain the coordinates of the preselected lung nodule region in the CT images after passing through a lung nodule detection model, then obtain the preselected region sample with a size of 96*96*96 from the CT images according to the coordinates, and perform true and false positive sample labeling, and at the same time, the labeled samples are subjected to data enhancement, so as to obtain the training set used for training the multi-scale redundant convolutional neural network.

[0086] The key feature map extraction unit 2 is configured to input data in the training set into the multi-scale redundant convolutional neural network, first pass through the redundant convolutional layer, obtain redundant features, then pass through the Maxout module to extract key features, and obtain key feature maps;

[0087] The multi-scale feature map extraction unit 3 is configured to input the key feature maps into the multi-scale convolutional layer in the multi-scale redundant convolutional neural network, extract multi-granularity feature maps through multi-scale convolutional operation, and splice all the obtained feature maps to obtain multi-scale feature maps.

[0088] The probability calculation unit 4 is configured to input the multi-scale feature maps into the activation function and the pooling layer in the multi-scale redundant convolutional neural network in sequence, stretch the output data to obtain a one-dimensional column vector, take the obtained column vector as the input of the full connection layer, and obtain the probabilities of the sample being a false positive sample and a positive sample through the Softmax function.

[0089] The network training unit 5 is configured to calculate the cross-entropy loss according to the probabilities of the false positive sample and the positive sample, train the multi-scale redundant convolutional neural network by using the cross-entropy loss function, and finally obtain a trained lung nodule false positive sample prediction model.

[0090] The false positive discrimination unit 6 is configured to input a CT image to be discriminated by a user into the lung nodule false positive sample prediction model, and judge whether the input CT image is a lung nodule false positive sample according to the probabilities of the false positive sample and the positive sample output.

[0091] Therefore, the application can automatically filter key features from input data of different types of lung nodules, thereby more effectively training a prediction network from a mixed data set. Meanwhile, the application can adapt to input data of different actual sizes of lung nodules, improve the accuracy, and does not need to train multiple network models according to different sample sizes.

[0092] The above describes in detail a lung nodule false positive discrimination method and system based on multi-scale redundancy convolution provided by the embodiment of the application. The principle and implementation mode of the application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range can be changed. In conclusion, the content of the specification should not be understood as a limitation of the application.

Claims

1. A method for lung nodule false positive differentiation based on multi-scale redundant convolution, characterized in that, The method comprises: Obtain CT images from a lung nodule CT image database, pass the CT images through a lung nodule detection model, obtain coordinates of a preselected region of lung nodules in the CT images, then obtain a preselected region sample with a size of 96x96x96 from the CT images according to the coordinates, and perform true and false positive sample labeling on the sample, and meanwhile perform data enhancement on the labeled sample, thereby obtaining a training set for training a multi-scale redundant convolutional neural network; Input the data in the training set into the multi-scale redundant convolutional neural network, first pass the data through a redundant convolutional layer to obtain redundant features, then pass the redundant features through a Maxout module to extract key features, and obtain a key feature map; Input the key feature map into a multi-scale convolutional layer in the multi-scale redundant convolutional neural network, extract multi-granularity feature maps through multi-scale convolutional operations, and splice all the obtained feature maps to obtain a multi-scale feature map; Input the multi-scale feature map into an activation function and a pooling layer in the multi-scale redundant convolutional neural network in sequence, stretch the output data, obtain a one-dimensional column vector, take the column vector as an input of a full connection layer, and perform processing through a Softmax function to obtain probabilities of false positive samples and positive samples; Calculate a cross-entropy loss according to the probabilities of the false positive samples and the positive samples, train the multi-scale redundant convolutional neural network by using a cross-entropy loss function, and finally obtain a trained lung nodule false positive sample prediction model; Input a CT image to be identified by a user, pass the CT image through a lung nodule detection model, obtain coordinates of a preselected region of lung nodules in the CT image to be identified, then obtain a preselected region sample with a size of 96x96x96 from the CT image to be identified according to the coordinates, and input the sample into the lung nodule false positive sample prediction model, and determine whether the input CT image to be identified is a lung nodule false positive sample according to probabilities of false positive samples and positive samples output by the lung nodule false positive sample prediction model; Specifically, the inputting of the data in the training set into the multi-scale redundant convolutional neural network comprises the following steps: The redundant convolutional layer of the multi-scale redundant convolutional neural network is composed of KxC1x5x5x5 convolutional kernels, wherein K represents the number of nodule categories, namely three categories of solid nodules, pure ground glass nodules and mixed ground glass nodules, so K is 3, and C1 is the number of channels after the redundant convolutional layer processing, and the convolutional kernels all have a size of 5x5x5; The feature map extracted by the redundant convolutional layer contains K groups of redundant features, and a Maxout module is used to extract key features from the redundant features, and the formula is as follows: wherein G represents the feature map extracted by the redundant convolution layer, c represents a channel of the feature map and c∈[1, C1], F c (x) represents the cth channel of the key feature map F(x), G j,c (x) represents the cth channel of the K groups of redundant feature maps G(x), and the max function traverses the C1-dimensional channels from the K groups of redundant feature maps and selects the strongest feature from each group of features. The above formula runs C1 times, thereby extracting a group of C1-dimensional key feature maps F(x); Specifically, the inputting of the key feature map into the multi-scale convolutional layer in the multi-scale redundant convolutional neural network, the extraction of multi-granularity feature maps through multi-scale convolutional operations, and the splicing of all the obtained feature maps to obtain a multi-scale feature map comprise the following steps: The multi-scale convolution operation comprises D groups of convolution layers, the key feature maps are respectively input into each group of convolution layers to obtain feature maps of the same size; D is 3, that is, 3 groups of feature maps are obtained in total; the same size of convolution kernels is used in each group of convolution layers, and the sizes are 3*3*3, 5*5*5 and 9*9*9 respectively; the 3*3*3 convolution operation comprises four convolution layers, and the numbers of convolution kernels are 16, 32, 64 and 128 respectively; the 5*5*5 convolution operation comprises two convolution layers, and the numbers of convolution kernels are 32 and 128 respectively; the 9*9*9 convolution operation comprises one convolution layer, and the number of convolution kernels is 128; The three groups of feature maps are anchored in the first dimension as a splicing reference to be spliced: wherein, F1, F2 and F3 respectively represent feature maps obtained by three scales of convolution layers, h represents a certain dimension of the feature map, and h [0, 128); by splicing three 128-dimensional feature maps, a 384-dimensional multi-scale feature map is obtained as an output; Specifically, after the multi-scale feature map is sequentially input into the activation function and the pooling layer in the multi-scale redundant convolution neural network, the output data is stretched to obtain a one-dimensional column vector, and the obtained column vector is taken as the input of the full connection layer, and the probability that the sample is a false positive sample and a positive sample is obtained through the processing of the Softmax function, specifically: The activation function in the multi-scale redundant convolution neural network adopts the ReLU nonlinear activation function, which sets the data less than 0 to 0, and does not change the data greater than or equal to 0; the pooling layer in the multi-scale redundant convolution neural network adopts the 2*2*2 average pooling operation; After the output data after the activation function and the pooling layer is stretched, a 1*V column vector can be obtained, and the full connection layer will perform convolution processing on the column vector and map it to the probability of different sample categories; two full connection layers are used for processing, the convolution kernel used by the first full connection layer is V*64, and the input data is converted into a 1*64 column vector; the convolution kernel used by the second full connection layer is 64*2, and the input data is converted into a 1*2 column vector, and the two numerical values in the vector respectively represent the probability weight values of the input sample data being a false positive sample and a positive sample; after the two numerical values are processed by the Softmax function, the probability of the false positive sample and the positive sample can be obtained.

2. The method of claim 1, wherein the method comprises: The lung nodule CT image database comes from the LIDC-IDRI database, which is composed of chest medical images and the corresponding diagnostic result annotations, and the annotations of the lung nodule CT image database are divided into two categories of nodules and non-nodules.

3. The method of claim 1, wherein the method further comprises: After the lung nodule detection model, the coordinates of the preselected region of the lung nodule in the CT image are obtained, then the preselected region sample with a size of 96*96*96 is obtained from the CT image according to the coordinates, and the true and false positive sample annotations are performed, specifically: According to the coordinates obtained by the lung nodule detection model, a 96*96*N sample is cut from the CT image, where N is the number of slices, then N slices are uniformly interpolated to obtain a 96*96*96 candidate sample; then according to the database annotation, it is determined whether the lung nodule exists in the candidate sample, if it exists, it is marked as a positive sample, if it does not exist, it is marked as a false positive sample.

4. The method of claim 1, wherein the method further comprises: The labeled sample is subjected to data enhancement to obtain a training set for training the multi-scale redundant convolutional neural network, specifically: In order to maintain the balance between the number of false positive samples and positive samples, the positive samples need to be subjected to rotation, translation and scaling data enhancement; Rotation: the positive samples are rotated by a certain angle respectively, and added to the training set as additional positive samples; translation: the center coordinates of the lung nodule candidate region are respectively translated by a certain number of pixel points in the CT image, and the candidate sample is recut and added to the training set; scaling: a M*M*96 candidate sample is obtained from the CT image, and M>96, then the sample is uniformly sampled, blurred and recut to obtain a 96*96*96 sample, and added to the training set.

5. The method of claim 1, wherein the method further comprises: The cross-entropy loss function is represented as: Wherein, T represents the total number of categories, and T is 2, i.e. two categories of false positive samples and positive samples; p t represents the probability of t categories obtained by the multi-scale redundant convolutional neural network, t is 1 or 2, p1 is the probability of false positive samples, and p2 is the probability of positive samples; y t represents the penalty proportion of different categories according to the actual label; L is the cross-entropy loss of the network output result.

6. A multi-scale redundant convolution based lung nodule false positive differentiation system, comprising: The system comprises: A training set generation unit is configured to obtain CT images from a lung nodule CT image database, and after the lung nodule detection model, the coordinates of the preselected region of the lung nodule in the CT image are obtained, then a 96*96*96 preselected region sample is obtained from the CT image according to the coordinates, and true and false positive sample labeling is performed, and the labeled sample is subjected to data enhancement to obtain a training set for training the multi-scale redundant convolutional neural network; A key feature map extraction unit is configured to input the data in the training set into the multi-scale redundant convolutional neural network, first pass through the redundant convolutional layer to obtain redundant features, then pass through the Maxout module to extract key features and obtain a key feature map; A multi-scale feature map extraction unit is configured to input the key feature map into the multi-scale convolutional layer in the multi-scale redundant convolutional neural network, extract multi-granularity feature maps after multi-scale convolution operation, and splice all the obtained feature maps to obtain a multi-scale feature map; A probability calculation unit is configured to input the multi-scale feature map into the activation function and the pooling layer in the multi-scale redundant convolutional neural network in sequence, stretch the output data to obtain a one-dimensional column vector, take the obtained column vector as the input of the full connection layer, and process it by the Softmax function to obtain the probability that the sample is a false positive sample and a positive sample; A network training unit is configured to calculate the cross-entropy loss according to the probability of the false positive sample and the positive sample, and train the multi-scale redundant convolutional neural network by using the cross-entropy loss function, and finally obtain a trained lung nodule false positive sample prediction model. The false positive identification unit is configured to input a CT image to be identified by a user into the false positive sample prediction model of the lung nodules, and determine whether the input CT image is a false positive sample of the lung nodules according to the output probabilities of the false positive sample and the positive sample. Specifically, the key feature map extraction unit is specifically configured to: The redundant convolutional layer of the multi-scale redundant convolutional neural network is composed of a convolutional kernel of KxC1x5x5x5, wherein K represents the number of nodule categories, i.e., three categories of solid nodules, pure ground glass nodules and mixed ground glass nodules, so K is 3, and C1 is the number of channels after processing by the redundant convolutional layer, and the convolutional kernel has a size of 5x5x5; The feature map extracted by the redundant convolutional layer contains K groups of redundant features, and the key features are extracted from the redundant features using a Maxout module, and the formula is: wherein G represents the feature map extracted by the redundant convolution layer, c represents a channel of the feature map and c e [1, C1], F c (x) represents the cth channel of the key feature map F(x), G j,c (x) represents the cth channel of the K groups of redundant feature maps G(x), and the max function traverses the C1-dimensional channels from the K groups of redundant feature maps and selects the strongest feature from each group of features. The above formula runs C1 times, thereby extracting a group of C1-dimensional key feature maps F(x); Specifically, the multi-scale feature map extraction unit is specifically configured to: The multi-scale convolutional operation contains D groups of convolutional layers, the key feature maps are input into each group of convolutional layers respectively, and the same size feature maps are obtained; D is 3, i.e., three groups of feature maps are obtained; the same size convolutional kernel is used in each group of convolutional layers, and the sizes are 3x3x3, 5x5x5 and 9x9x9 respectively; the 3x3x3 convolutional operation contains four convolutional layers, and the numbers of convolutional kernels are 16, 32, 64 and 128 respectively; the 5x5x5 convolutional operation contains two convolutional layers, and the numbers of convolutional kernels are 32 and 128 respectively; the 9x9x9 convolutional operation contains one convolutional layer, and the number of convolutional kernels is 128; The three groups of feature maps are anchored in the first dimension as a splicing reference for splicing: Wherein, F1, F2 and F3 represent the feature maps obtained by the convolutional layers of the three scales respectively, h represents a certain dimension of the feature map, and h∈[0, 128); by splicing the three 128-dimensional feature maps, a 384-dimensional multi-scale feature map is obtained as an output; Specifically, the probability calculation unit is specifically configured to: The activation function step in the multi-scale redundant convolutional neural network adopts a ReLU nonlinear activation function, which sets the data less than 0 to 0, and does not change the data greater than or equal to 0; the pooling layer in the multi-scale redundant convolutional neural network adopts a 2x2x2 average pooling operation; The data output after the activation function and the pooling layer can obtain a 1xV column vector after stretching, and the full connection layer will perform convolution processing on the column vector and map it to the probability of different sample categories; two full connection layers are used for processing, the convolution kernel used by the first full connection layer is Vx64, and the input data is converted into a 1x64 column vector; the convolution kernel used by the second full connection layer is 64x2, and the input data is converted into a 1x2 column vector, and the two numerical values in the vector respectively represent the probability weight values of the input sample data being a false positive sample and a positive sample; after the two numerical values obtained are processed by a Softmax function, the probabilities of the false positive sample and the positive sample can be obtained.

Images