A method for detecting lung nodules

Abstract

This invention proposes a method for detecting lung nodules, relating to the field of medical image target detection technology. It involves collecting chest CT image samples from multiple patients to construct a dataset; the CT images are CT slice sequence data; morphological operations are used to preprocess the CT images in the dataset; based on the CornerNet network structure, a lung nodule detection model is constructed, including a feature extraction network, a target detection head, and a false positive reduction network; the dataset is used to train the lung nodule detection model to obtain a trained lung nodule detection model; finally, the chest CT images to be detected are preprocessed, and the preprocessed CT images are input into the trained lung nodule detection model to obtain the location information of the lung nodules, effectively eliminating the interference of false positives and balancing the inference speed and accuracy of lung nodule detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image target detection, and more particularly to a lung nodule detection method. BACKGROUND

[0002] In recent years, the morbidity and mortality of lung cancer are in the forefront among all cancers, and lung cancer is an extremely dangerous cancer. One of the important reasons for the high mortality rate of lung cancer is that early lung nodules are small and not easy to be found by examination. Since the early symptoms of lung cancer are lung nodules, and most of the clinical symptoms of lung nodules are not obvious, it is easy to cause patients to miss the best treatment opportunity. In another aspect, the detection rate of lung nodules in early lung examination is closely related to the survival rate of lung cancer.

[0003] With the development of computer technology, hospitals introduce computer-aided diagnosis systems to assist in diagnosis to meet the growing needs of lung disease patients for early lung examination. In recent years, deep learning has been widely used in the field of lung nodule detection. The current mainstream lung nodule detection is based on anchor point target detection method, although the accuracy is high, but the training and reasoning speed is slow. In view of this, the prior art discloses a three-dimensional lung nodule detection model training method without anchor box, including: acquiring image data, label category and label coordinate point of three-dimensional lung nodule sample in data set and constructing three-dimensional lung nodule database; inputting the image data after data enhancement into a convolutional neural network for processing to obtain a plurality of feature maps; inputting the feature maps into a pre-set shared head network with two convolutional branches to output a predicted category, a predicted center degree and a predicted coordinate point, calculating a predicted confidence and a label center degree, and inputting the label coordinate point, the predicted coordinate point, the predicted confidence, the label category, the predicted center degree and the label center degree into a loss function, back propagation and model training. However, the above method effectively reduces the amount of calculation, but the accuracy is not as good as the detection system with preset anchor points, especially in the case of heavy lung nodule target detection task, there are usually many false positives in the results. SUMMARY

[0004] In order to solve the problem of current lung nodule detection that cannot balance the reasoning speed and accuracy, the present application proposes a lung nodule detection method, constructs a lung nodule detection model, improves the two corner points of the CornerNet network structure, and additionally adds a false positive reduction network to effectively reduce the false positives in the detection results.

[0005] In order to achieve the above technical effects, the technical scheme of the present application is as follows:

[0006] A lung nodule detection method, comprising:

[0007] S1. Collect chest CT image samples from multiple patients to construct a dataset, and divide the dataset into a training set, a validation set, and a test set; the CT images are CT slice sequence data;

[0008] S2. Preprocess the CT images in the dataset using morphological operations;

[0009] S3. Based on the CornerNet network structure, a lung nodule detection model is constructed. The lung nodule detection model includes a feature extraction network, a target detection head, and a false positive reduction network. The lung nodule detection model is trained using a training set, evaluated using a validation set, and tested using a test set to obtain a well-trained lung nodule detection model.

[0010] S4. Preprocess the chest CT image to be detected, and input the preprocessed CT image into the trained lung nodule detection model. The lung nodule detection model detects and outputs the location information of the lung nodules.

[0011] In this technical solution, the lung nodule detection model is improved based on the CornerNet network structure, which effectively reduces false positives in the detection results.

[0012] Preferably, in steps S2 and S4, the preprocessing includes the following steps:

[0013] S20. Set a pixel threshold, convert pixels outside the threshold range in the CT image into white and black respectively, and convert the CT image into a binary image;

[0014] S21. Remove patches connected to the image border to eliminate unnecessary background information;

[0015] S22. Mark connected regions in the binary image after plaque removal for subsequent restoration of the complete lung;

[0016] S23. Divide the two largest regions into the left lung and the right lung for subsequent segmentation;

[0017] S24. Identify the internal boundaries of the segmented left and right lungs and fill the small holes inside the lung mask;

[0018] S25. Calculate the minimum convex polygon using the non-zero pixels in the mask to obtain the convex hulls corresponding to the filled left and right lungs, thereby further determining the shape of the left and right lungs in order to better represent the complete shape of the lungs;

[0019] S26. Connect the left and right lungs together to obtain a binary mask of the complete lung structure;

[0020] S27. Use binary closing operation to fill the gaps in the image and connect it with the complete lung structure to restore the boundary information, so that the lung nodules at the boundary of the lung parenchyma can be completely preserved in the segmented image; after the binary closing operation, the result can better present the lung nodules and can be connected with the lung structure obtained in step S26, so that the lung nodules can be preserved and highlighted in the segmented lung;

[0021] S28. Obtain the segmented image from the original CT image using a binary mask;

[0022] S29. Perform zero-centering on the segmented image to obtain the preprocessed CT image.

[0023] The lung nodule detection model includes a feature extraction network, a target detection head, and a false positive reduction network. The data flow of the lung nodule detection model is as follows:

[0024] The preprocessed CT image is input into the feature extraction network to obtain the output feature map and features at each level.

[0025] The output feature map is labeled using the target detection head to obtain preliminary candidate target detection boxes for lung nodules;

[0026] The preliminary candidate target detection boxes output by the target detection head are concatenated with the output feature map output by the feature extraction network. The corresponding concatenated part is then removed from the original input CT image to serve as the input for reducing false positives.

[0027] The false positive reduction network is used to classify lung nodules and determine whether the lung nodules in the preliminary candidate box are false positives. If they are, the test results are filtered out; otherwise, the test results are output.

[0028] Preferably, the feature extraction network consists of a ResNet50 network and a U-shaped network with non-local operations;

[0029] The input image is fed into the feature extraction network. First, it passes through a convolutional layer to adjust the dimensions and extract preliminary features of lung nodules. Then, it enters a max pooling layer for downsampling, and then passes through four residual blocks of different input sizes for feature extraction to obtain features at each level. Finally, it passes through an average pooling layer to obtain the output feature map.

[0030] Preferably, the residual block structure of the ResNet50 network is as follows:

[0031] F = W3σ(W2σ(W1X))

[0032] y = F + X

[0033] Among them, W iLet σ represent the i-th convolutional layer, and let σ represent the ReLU activation function. The input image is represented by D, H, W, and C, which are its depth, height, width, and number of channels, respectively.

[0034] After the input image is processed by a convolutional layer to adjust its dimensions, the slices are grouped and calculated along the depth dimension. The dimension of the feature map before grouping is D, the number of groups is G, and the number of slices in each group is D′.

[0035]

[0036] The formula for the nonlocal operation of grouping is as follows:

[0037] Y=f(vec(θ(X)),vec(φ(X)))vec(g(X))

[0038] Among them, the output vec represents the slice grouping operation, f(·,·) represents the similarity calculation between feature maps, and θ(·), φ(·), and g(·) are implemented through 1×1×1 convolutions, as expressed in the following formulas:

[0039] θ(X)=XW θ

[0040] φ(X)=XW φ

[0041] g(X) = XW g

[0042] W θ W φ and W g These represent the weight matrices that need to be learned;

[0043] The formula for expressing the output feature map Z is as follows:

[0044] Z = concat(BN(YW)) + X

[0045] Where W represents a 1×1×1 convolution, BN represents a batch normalization process, concat represents concatenating the previously grouped slices in the depth dimension to restore them to the input size, and a residual connection is performed at the end of the network so that the feature extraction network can learn at least one identity mapping.

[0046] Preferably, the target detection head uses a target detection box that combines corner points and center points, including a corner point module and a center point module; the corner point module includes three branches: heatmap, embedding vector, and offset, and the center point module includes two branches: heatmap and offset; the feature map input to the target detection head is first subjected to cascaded corner pooling before entering the corner point module, and first subjected to center point pooling before entering the center point module.

[0047] Preferably, the cascaded corner pooling process includes: finding a maximum value along the feature map boundary direction (i.e., up and down, front and back, left and right directions), finding an internal maximum value along the direction of the maximum value, adding the two together to obtain the internal information of the corner point, making the corner point positioning more accurate, and also alleviating the situation that corner pooling is more sensitive to the boundary.

[0048] During the center point pooling process, the input feature map is divided into four branches. Pooling is performed on the top three branches in the left-right, up-down, and front-back directions respectively. Then, through residual connection and element-by-element summation, feature aggregation in six directions is achieved.

[0049] The final target detection box is generated from two text points and a pair of corner points to form a preliminary candidate target detection box. The preliminary candidate target detection box is determined to be the real target by checking whether there is a center point of the same category falling in the center region.

[0050] Here, by using cascaded corner pooling and center point pooling, the CornerNet network structure can adapt to lung nodule data while enhancing the feature extraction capability of corner points, thus obtaining more semantic information. At the same time, center points are introduced to assist corner points in locating lung nodules. The added center points can extract the center information of lung nodules. Based on the corner point bounding boxes, the target detection boxes can be corrected, which can initially reduce the false positives of the model output.

[0051] Preferably, the false positive reduction network consists of two downsampled convolutional layers, and during forward propagation, it is concatenated with two shallow features obtained after the first two convolutional layers of the feature extraction network, so that the feature map has semantic information and high-resolution information, which is beneficial for classifying false positives; the output of the false positive reduction network is connected to a fully connected layer to classify and judge the detected lung nodules.

[0052] Here, by setting up a false positive reduction network for downsampling, the target detection head obtains target detection boxes, which are then cropped from the input data and used as input. During this process, the target boxes are also concatenated with the target boxes at the corresponding level of the feature extraction network, so that the concatenated feature map has richer lung nodule features, which helps in the classification of lung nodule false positives.

[0053] Preferably, the loss function of the lung nodule detection model includes the loss of the three branches of the corner module, the loss of the two branches of the center module, and the loss of the false positive reduction network. The expression for the total loss function L is as follows:

[0054]

[0055] in, It is the heatmap loss at the corner points. It is the heatmap loss at the center point. and The embedding vector loss of the corner points, It is the offset loss at the corner point. It is the offset loss of the center point, L rCNN α, β, γ, and δ are hyperparameters that reduce false positives and decrease the network loss. They are the weights of the corresponding losses in the total loss.

[0056] Preferably, the heatmap loss is constructed as follows:

[0057] A 3D Gaussian sphere is generated, centered on the actual corner point and with the radius of the lung nodule as the radius. This sphere maps a corner point or center point of the object's actual bounding box onto the corresponding spherical region in the heatmap. The formula for generating the 3D Gaussian sphere is as follows:

[0058]

[0059] Where (x0, y0, z0) are the coordinates of the corner point or center point, and σ is selected as the Gaussian radius. c represents the category to which the corner point or center point belongs;

[0060] The heatmap loss function using a Gaussian sphere is shown below:

[0061]

[0062] Where, p cijk In the heatmap output feature map, the value of the pixel with coordinates (i, j, k) of the c-th category represents the probability that the point is a corner point or the center point; y cijk This represents the value of the pixel at coordinates (i, j, k) of the c-th category in the actual heatmap. For points near corners or the center, a non-zero value is assigned to nearby pixels using the method described above. In the calculation of negative sample loss, (1-y cijk ) b As weights for negative samples, points near the corners or center (within the Gaussian sphere) have smaller losses, which allows the network to move closer to the center as much as possible; a and b are hyperparameters;

[0063] The embedding vector loss includes Pull Loss and Push Loss, and the Pull Loss formula is as follows:

[0064]

[0065] in, and It is the value of the embedding vector of the top left and bottom right corner points, e k yes and The average value; the meaning of this loss function is: to bring the embedding vectors of corner points belonging to the same group closer together, so that the network can assign a closer embedding vector to corner points belonging to the same group.

[0066] The Push Loss formula is as follows:

[0067]

[0068] Where N is the number of groups of corner points, and Δ is a hyperparameter; the meaning of this loss function is to increase the distance between the embedding vectors of different groups of angles so that the network can more clearly distinguish the corner points of different groups in the future.

[0069] The offset loss formula is as follows:

[0070]

[0071]

[0072]

[0073] Among them, O k This indicates that the actual object detection bounding box has been downsampled to its original size. The offset that occurs during the rounding operation of pixel coordinates; This represents the predicted offset; the function is smoother when the Smooth L1 loss value is small, and the value can be limited when the value is large, which can effectively alleviate the gradient explosion problem.

[0074] The loss function of the false positive reduction network is as follows:

[0075] L rCNN =-(1-p t ) γ log(p t )

[0076] Here, p is defined as the probability that a false positive reduces the network output of a subsequent lung nodule as a true nodule, and y represents the decision variable of whether it is a true nodule. If it is, y takes the value 1, corresponding to p. t The value p represents the probability of a false positive reducing the network output; otherwise, y is 0. t The value is 1-p; γ is an adjustment factor used to balance the weights of easy and difficult samples.

[0077] The loss function of the false positive reduction network also adopts the Focal Loss loss function, which can alleviate the sample imbalance problem through weighting.

[0078] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0079] This invention proposes a method for detecting lung nodules. It involves collecting chest CT image samples from multiple patients to construct a dataset; the CT images are slice sequence data from CT scans; morphological operations are used to preprocess the CT images in the dataset; a lung nodule detection model is constructed based on the CornerNet network structure, including a feature extraction network, a target detection head, and a false positive reduction network; the dataset is used to train the lung nodule detection model to obtain a trained model; finally, the chest CT images to be detected are preprocessed, and the preprocessed CT images are input into the trained lung nodule detection model to obtain the location information of the lung nodules, effectively eliminating the interference of false positives and balancing the inference speed and accuracy of lung nodule detection. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the lung nodule detection method proposed in Embodiment 1 of the present invention;

[0081] Figure 2 This diagram illustrates the structure of the lung nodule detection model proposed in this invention.

[0082] Figure 3 This is a schematic diagram of the target detection box in the present invention;

[0083] Figure 4 This diagram illustrates the cascaded corner pooling method used in this invention.

[0084] Figure 5 This diagram illustrates the central pooling mechanism in this invention. Detailed Implementation

[0085] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0086] To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions;

[0087] It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0088] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0089] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0090] Example 1

[0091] like Figure 1 As shown in the figure, this embodiment proposes a method for detecting lung nodules, including:

[0092] S1. Collect chest CT image samples from multiple patients to construct a dataset, and divide the dataset into a training set, a validation set, and a test set; the CT images are CT slice sequence data;

[0093] In this embodiment, preliminary processing of the collected hospital patient CT slice sequence data is also required, including:

[0094] (1) Keep patient data confidential and remove important privacy information such as names;

[0095] (2) Adjust the CT image interval so that each slice voxel represents 1 cubic millimeter in reality;

[0096] (3) Adjust the window level and width of the CT image to observe lung tissue.

[0097] S2. Preprocess the CT images in the dataset using morphological operations; specifically, the preprocessing process includes the following steps:

[0098] S20. Binarization: Set a pixel threshold, convert pixels outside the threshold range in the CT image into white and black respectively, and convert the CT image into a binary image;

[0099] S21. Remove border patches: Remove patches connected to the image border to eliminate unnecessary background information;

[0100] S22. Mark connected regions: Mark connected regions in the binary image after plaque removal to facilitate subsequent restoration of the complete lung;

[0101] S23. Segmenting the left and right lungs: Divide the two largest regions into the left and right lungs for subsequent segmentation;

[0102] S24. Fill the mask: Identify the internal boundaries of the segmented left and right lungs and fill the small holes in the lung mask;

[0103] S25. Calculate the convex hulls of the left and right lungs: Calculate the minimum convex polygon using the non-zero pixels in the mask to obtain the convex hulls corresponding to the filled left and right lungs, so as to better represent the complete shape of the lungs;

[0104] S26. Connect the left and right lungs: Connect the left and right lungs together to obtain a binary mask of the complete lung structure;

[0105] S27. Restore boundary information: Use binary closing operation to fill the gaps in the image and connect it with the complete lung structure to restore the boundary information, so that the lung nodules at the boundary of the lung parenchyma can be completely preserved in the segmented image; after the binary closing operation, the result can better present the lung nodules and can be connected with the lung structure obtained in step S26, so that the lung nodules can be preserved and highlighted in the segmented lung;

[0106] S28. Obtain the segmented image: Use a binary mask on the original CT image to obtain the segmented image;

[0107] S29. Centering: Perform zero-centering on the segmented image to obtain the preprocessed CT image.

[0108] S3. Based on the CornerNet network structure, a lung nodule detection model is constructed. The lung nodule detection model includes a feature extraction network, a target detection head, and a false positive reduction network. The lung nodule detection model is trained using a training set, evaluated using a validation set, and tested using a test set to obtain a well-trained lung nodule detection model.

[0109] Specifically, the data flow of the lung nodule detection model is as follows:

[0110] The preprocessed CT image is input into the feature extraction network to obtain the output feature map and features at each level.

[0111] The output feature map is labeled using the target detection head to obtain preliminary candidate target detection boxes for lung nodules;

[0112] The preliminary candidate target detection boxes output by the target detection head are concatenated with the output feature map output by the feature extraction network. The corresponding concatenated part is then removed from the original input CT image to serve as the input for reducing false positives.

[0113] The false positive reduction network is used to classify lung nodules and determine whether the lung nodules in the preliminary candidate box are false positives. If they are, the test results are filtered out; otherwise, the test results are output.

[0114] S4. Preprocess the chest CT image to be detected. In this embodiment, the preprocessing process is the same as step S2. Input the preprocessed CT image into the trained lung nodule detection model, and the lung nodule detection model detects and outputs the location information of the lung nodules.

[0115] Example 2

[0116] In this embodiment, based on Embodiment 1, the structure and training process of the lung nodule detection model are further explained.

[0117] like Figure 2 As shown, a lung nodule detection model is constructed based on the CornerNet network structure. This lung nodule detection model includes a feature extraction network (corresponding to...). Figure 2 Backbone, target detection head, and false positive reduction network (corresponding to) Figure 2 Fals in e Positiv e Reduction);

[0118] The feature extraction network consists of a 3D ResNet50 network combined with a U-shaped network using nonlocal operations, and is composed of five parts: conv1, conv2 x, conv3 x, conv4 x, and conv5 x. The input to the feature extraction network first passes through a 7×7×7 convolutional layer, which extracts preliminary features of the lung nodules and expands the input channels from 1 (grayscale image) to 64 channels. Then, it enters a 3×3×3 max-pooling layer for downsampling, followed by feature extraction through four residual blocks of different input sizes, with 3, 4, 6, and 3 residual blocks respectively. Finally, it passes through an average pooling layer to obtain the extracted feature map. The residual block structure of the 3D ResNet50 network is as follows:

[0119] F = W3σ(W2σ(W1X))

[0120] y = F + X

[0121] Among them, W i δ represents the i-th convolutional layer, and δ represents the ReLU activation function. The input image is represented by D, H, W, and C, which are its depth, height, width, and number of channels, respectively.

[0122] The nonlocal operations for grouping are as follows:

[0123] After the input feature map undergoes dimensionality adjustment through convolutions, it is divided into slices along the depth dimension for computation. Before grouping, the feature map has dimension D, the number of groups is G, and the number of slices in each group is D′.

[0124]

[0125] The formula for the nonlocal operation of grouping is as follows:

[0126] Y=f(vec(θ(X)),vec(φ(X)))vec(g(X))

[0127] Among them, the output vec represents the slice grouping operation, and f(·,·) represents the similarity calculation between feature maps. In this embodiment, a simpler and easier-to-implement dot product method is adopted, and the specific implementation is as follows:

[0128] f(vec(θ(X)), vec(φ(X)))=vec(θ(X))vec(θ(X)) T

[0129] θ(·), φ(·), and g(·) are achieved through a 1×1×1 convolution, as expressed in the following formula:

[0130] θ(X)=XW θ

[0131] φ(X)=XW φ

[0132] g(X) = XW g

[0133] W θ W φ and W g These represent the weight matrices that need to be learned;

[0134] The formula for expressing the output feature map Z is as follows:

[0135] Z = concat(BN(YW)) + X

[0136] Where W represents a 1×1×1 convolution, BN represents a batch normalization process, concat represents concatenating the previously grouped slices in the depth dimension to restore them to the input size, and a residual connection is performed at the end of the network so that the feature extraction network can learn at least one identity mapping.

[0137] After passing through the feature extraction network, the output feature map is input into the target detection head to obtain preliminary candidate box information for lung nodules.

[0138] like Figure 3 As shown, the target detection head uses a target detection bounding box that combines corner points and center points, including a corner point module (corresponding to...). Figure 2 Comer Head) and center point module (corresponding Figure 2 The corner module includes three branches: heatmap, embedding vector, and offset; the center point module includes two branches: heatmap and offset. Before entering the two branches, a corresponding pooling process is performed: cascaded corner pooling for corner points and center point pooling for center points.

[0139] In cascaded corner pooling, a maximum value is first found along the feature map boundary directions (i.e., up / down, front / back, left / right). Then, an internal maximum value is found along the direction of the maximum value. Finally, the two maximum values ​​are added together, thus obtaining the internal information of the corner point. Taking front pooling as an example, the matrix searches for the maximum value from back to front. After finding the maximum value, cascaded corner pooling looks to the right and down to find two more maximum points, and their sum is the current maximum value. In this way, internal information can be obtained within the pooling layer, making the corner point location more accurate and alleviating the sensitivity of corner pooling to boundaries.

[0140] In this embodiment, as Figure 4 As shown, for Cascade Front Pooling, the object detection head performs top pooling and left pooling on the input feature map, then sums the results and connects them to the residuals, finally passing them through a 3D convolution. v After enhancing feature representation, front pooling is performed to obtain the final cascaded front pooling result. Other cascaded operations, such as cascaded left pooling and cascaded upper pooling, follow the same principle.

[0141] In terms of center pooling, the center point is pooled to obtain information in six directions: front, back, left, right, up, and down. Through center point pooling, the center point acquires richer semantic information, enabling better localization of lung nodules. For example... Figure 5 As shown, the network structure of center point pooling is similar to that of cascaded corner pooling. The input feature map is divided into four branches. The top three branches are pooled in the left-right, up-down, and front-back directions, respectively. In this way, the information of the features in the left-right, up-down, and front-back directions can be gathered together. Finally, through residual connections and element-wise summation, the features in the six directions are gathered together.

[0142] The final target detection box is generated from two text points and a pair of corner points to form a preliminary candidate target detection box. The preliminary candidate target detection box is determined to be the real target by checking whether there is a center point of the same category falling in the center region.

[0143] The false positive reduction network consists of two downsampled convolutional layers, which are concatenated with two shallow features obtained after the first two convolutional layers of the feature extraction network during forward propagation. The output of the false positive reduction network is connected to a fully connected layer, the main function of which is to classify the detected lung nodules and determine whether the input is a real lung nodule or a false positive obtained from the previous target detection framework.

[0144] By setting up a false positive reduction network for downsampling, the target detection head obtains target detection boxes, which are then cropped from the input data and used as input. During this process, the target boxes are also concatenated with the target at the corresponding level of the feature extraction network, resulting in a concatenated feature map with richer lung nodule features, which helps in the classification of lung nodule false positives.

[0145] Example 3

[0146] In this embodiment, the loss function of the lung nodule detection model is explained based on the above embodiments.

[0147] The loss function of the lung nodule detection model consists of heatmap loss, embeddings loss, offsets loss, and false positive reduction network loss. The expression for the total loss function L is as follows:

[0148]

[0149] in, It is the heatmap loss at the corner points. It is the heatmap loss at the center point. and The embedding vector loss of the corner points, It is the offset loss at the corner point. It is the offset loss of the center point, L rCN N α, β, γ, and δ are hyperparameters that reduce false positives and decrease the network loss. They are the weights of the corresponding losses in the total loss.

[0150] The heatmap loss is constructed as follows:

[0151] A 3D Gaussian sphere is generated, centered on the actual corner point and with the radius of the lung nodule as the radius. This sphere maps a corner point or center point of the object's actual bounding box onto the corresponding spherical region in the heatmap. The formula for generating the 3D Gaussian sphere is as follows:

[0152]

[0153] Where (x0, y0, z0) are the coordinates of the corner point or center point, and σ is selected as the Gaussian radius. c represents the category to which the corner point or center point belongs;

[0154] The heatmap loss function using a Gaussian sphere is shown below:

[0155]

[0156] Where, p cijkIn the heatmap output feature map, the value of the pixel with coordinates (i, j, k) of the c-th category represents the probability that the point is a corner point or the center point; y cijk This represents the value of the pixel at coordinates (i, j, k) of the c-th category in the actual heatmap. For points near corners or the center, a non-zero value is assigned to nearby pixels using the method described above. In the calculation of negative sample loss, (1-y cijk ) b As weights for negative samples, points near the corners or center (within the Gaussian sphere) have smaller losses, which allows the network to move closer to the center as much as possible; a and b are hyperparameters;

[0157] The embedding vector loss includes Pull Loss and Push Loss, and the Pull Loss formula is as follows:

[0158]

[0159] in, and It is the value of the embedding vector of the top left and bottom right corner points, e k yes and The average value; the meaning of this loss function is: to bring the embedding vectors of corner points belonging to the same group closer together, so that the network can assign a closer embedding vector to corner points belonging to the same group.

[0160] The Push Loss formula is as follows:

[0161]

[0162] Where N is the number of groups of corner points, and Δ is a hyperparameter; the meaning of this loss function is to increase the distance between the embedding vectors of different groups of angles so that the network can more clearly distinguish the corner points of different groups in the future.

[0163] The offset loss is used to calculate the small offset caused by the rounding operation when the target detection box is downsampled, and the formula is as follows:

[0164]

[0165]

[0166]

[0167] Among them, O k This indicates that the actual object detection bounding box has been downsampled to its original size. The offset that occurs during the rounding operation of pixel coordinates; This represents the predicted offset; the function is smoother when the Smooth L1 loss value is small, and the value can be limited when the value is large, which can effectively alleviate the gradient explosion problem.

[0168] The loss function of the false positive reduction network is as follows:

[0169] L rCNN =-(1-p t ) γ log(p t )

[0170] Here, p is defined as the probability that a false positive reduces the network output of a subsequent lung nodule as a true nodule, and y represents the decision variable of whether it is a true nodule. If it is, y takes the value 1, corresponding to p. t The value p represents the probability of a false positive reducing the network output; otherwise, y is 0. t The value is 1-p; γ is an adjustment factor used to balance the weights of easy and difficult samples.

[0171] The loss function of the false positive reduction network also adopts the Focal Loss loss function, which can alleviate the sample imbalance problem through weighting.

[0172] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A lung nodule detection method, characterized by, The application relates to a lung nodule detection method based on CornerNet network structure. S1. Collecting chest CT image samples of a plurality of patients, constructing a data set, dividing the data set into a training set, a verification set and a test set; the CT image is a CT slice sequence data; S2. Preprocessing the CT image in the data set through morphological operation; S3. Constructing a lung nodule detection model based on the CornerNet network structure, wherein the lung nodule detection model comprises a feature extraction network, a target detection head and a false positive reduction network; training the lung nodule detection model by using the training set, evaluating the lung nodule detection model by using the verification set, testing the effectiveness of the lung nodule detection model by using the test set, and obtaining the trained lung nodule detection model; the lung nodule detection model comprises a feature extraction network, a target detection head and a false positive reduction network, and the data flow direction of the lung nodule detection model is as follows: The preprocessed CT image is input into the feature extraction network to obtain an output feature map and each level feature; The output feature map is labeled by using the target detection head to obtain a preliminary candidate target detection frame of the lung nodule; The preliminary candidate target detection frame output by the target detection head is spliced with the output feature map output by the feature extraction network, and the corresponding part of the original input CT image is cut off as the input of the false positive reduction network; The lung nodule is classified by using the false positive reduction network to determine whether the lung nodule in the preliminary candidate frame is a false positive; if yes, the detection result is screened out; otherwise, the detection result is output; The target detection head adopts a target detection frame combining an angle point and a center point, and comprises an angle point module and a center point module; the angle point module comprises a heat map, an embedded vector and an offset three branches, and the center point module comprises a heat map and an offset two branches; the feature map input into the target detection head is subjected to cascade angle pooling before entering the angle point module and is subjected to center point pooling before entering the center point module; In the cascade angle pooling process, a maximum value is found along the boundary direction of the feature map, an internal maximum value is found along the maximum value direction, and the two values are added to obtain internal information of the angle point; In the center point pooling process, the input feature map is divided into four branches, and the three branches on the top are subjected to pooling in the left-right, up-down and front-back directions respectively, and then the six-direction feature collection is realized through residual connection and element-by-element addition; The final target detection frame generates a preliminary candidate target detection frame from a center point and a pair of angle points, and whether the preliminary candidate target detection frame is a real target is determined by checking whether a center point with the same category falls in the center area; The false positive reduction network is composed of two down-sampling convolution layers, and is spliced with two shallow features obtained after the first two convolution layers of the feature extraction network during forward propagation; the output end of the false positive reduction network is connected to a full connection layer to classify and judge the detected lung nodule; S4. Preprocessing the chest CT image to be detected, and inputting the preprocessed CT image into the trained lung nodule detection model; the lung nodule detection model detects and outputs the position information of the lung nodule.

2. The lung nodule detection method of claim 1, wherein, In steps S2 and S4, the preprocessing comprises the following steps: S20. Set a pixel threshold, convert pixels outside the threshold range in the CT image into white and black respectively, and convert the CT image into a binary image; S21. Remove the patches connected to the image frame; S22. Label the connected regions in the binary image after removing the patches; S23. Divide the two largest regions into left lung and right lung; S24. Identify the internal boundary of the segmented left lung and right lung, and fill the small holes in the lung mask; S25. Calculate the minimum convex polygon using the non-zero pixel points in the mask to obtain the convex hull corresponding to the filled left lung and right lung; S26. Connect the left lung and the right lung together to obtain a binary mask of the complete lung structure; S27. Fill the image gap using binary closing operation and connect with the complete lung structure to restore the boundary information; S28. Obtain the segmented image on the original CT image using the binary mask; S29. Perform zero centering processing on the segmented image to obtain the preprocessed CT image.

3. The lung nodule detection method of claim 2, wherein, The feature extraction network is composed of a ResNet50 network and a U-shaped network with non-local operation; The input image is input into the feature extraction network, first passes through the convolution layer for dimension adjustment and extracts the preliminary features of the lung nodule, then enters the max pooling layer for down sampling, then passes through four residual blocks with different input sizes for feature extraction, obtains the features at each level, and finally passes through an average pooling layer to obtain the output feature map.

4. The lung nodule detection method of claim 3, wherein, The input image is dimensionally adjusted by a convolution layer, and then slice grouping calculation is performed in the depth dimension, the dimension of the feature map before grouping is , the grouping group number is , and the number of slices in each group is .

5. The lung nodule detection method of claim 4, wherein, The loss function of the lung nodule detection model includes the loss of the three branches of the corner point module, the loss of the two branches of the center point module, and the loss of the false positive reduction network.

Images