A deep learning-based lung nodule detection method

Abstract

The application provides a lung nodule detection method based on deep learning, which comprises the following steps: A, performing parenchymal segmentation on a three-dimensional lung CT image; B, sequentially processing the lung parenchymal image after being divided into blocks to obtain a sparse matrix; C, extracting three-view information of the sparse matrix for false positive screening; D, building a network model; and E, using the trained network model to detect lung nodules and output the detection results. The three-view false positive auxiliary module is combined, directly embedded into an end-to-end framework, and the three-view self-attention information of the sparse three-dimensional image is used to help screen out false positive regions. On the one hand, the problem of learning attention in a complex 3D scene is solved. On the other hand, the detection and false positive region screening are designed into an end-to-end training model, the detection speed and detection accuracy are improved, the overall complexity of the model is reduced, the model training convergence speed is improved when the model is trained, and the loss function is trained uniformly.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and specifically relates to a method for detecting lung nodules. Background Technology

[0002] Lung cancer is one of the leading causes of death worldwide. The detection of lung nodules is of great significance in predicting lung cancer; however, due to the diverse morphology of lung nodules and the lack of clear distinction between them and normal tissue, even doctors need to carefully distinguish them to reach a definitive diagnosis. With the widespread application of artificial intelligence technology, deep learning methods have become a rapid detection approach for lung nodules. By training a network model on a dataset, automatic detection and identification of lung nodules can be achieved, improving efficiency and reducing the workload for doctors.

[0003] Lung nodule detection is a challenging 3D object detection problem, and limited GPU memory is one of the technical difficulties it faces. Furthermore, 3D data annotation is typically much more difficult than 2D data annotation, which can lead to overfitting in deep learning models. Additionally, lung CT images often contain a significant amount of external, irrelevant tissue, such as the patient's torso, and internal lung structures like blood vessels and trachea, which can easily be misidentified as lung nodules, resulting in a high number of false positives. Therefore, the predictions obtained using deep learning detection methods contain a large number of false positives, leading to a decrease in detection accuracy. Summary of the Invention

[0004] To address the problem of numerous false positives in existing lung nodule detection methods, which reduces detection accuracy, this invention proposes a deep learning-based lung nodule detection method, implemented using the following technical solution:

[0005] A deep learning-based method for detecting lung nodules includes:

[0006] Step A: Perform parenchymal segmentation on the 3D lung CT image and remove irrelevant tissue to obtain the lung parenchyma image;

[0007] Step B: After segmenting the lung parenchyma image into blocks, serialize it and input it into a fast 3D ViT network to extract features and output a multi-scale feature map; at the same time, remove pixel regions with gradients of 0 and small gray-level changes (gray-level changes range from 0 to 255; in this invention, the range of gray-level changes from 0 to 50 is defined as pixel regions with small gray-level changes) from the three-dimensional lung parenchyma block image to obtain a sparsified matrix.

[0008] Step C: Extract the three-view information of the sparse matrix for false positive screening;

[0009] Step D: Build the network model and train the network parameters;

[0010] Step E: Use the trained network model to detect lung nodules and output the detection results.

[0011] Furthermore, in step B, the construction process of the multi-scale feature map is as follows: using the last layer feature information of the fast 3D ViT as the backbone network output, feature maps of four scales are constructed respectively. There is no connection path between these four scale feature maps. Each layer is directly constructed from the last layer output of the backbone network. The size gradually increases to twice that of the previous layer. The size of the second layer is consistent with the output of the last layer of the backbone network.

[0012] Furthermore, the serialization process in step B is as follows: the segmented image vector is linearly transformed, the three-dimensional image is projected onto a two-dimensional space and then flattened into one dimension, and one-dimensional position information is embedded.

[0013] Furthermore, in step B, the sparse matrix generation process is as follows:

[0014] Step B1: Process the three-dimensional lung CT images;

[0015] Step B2: Initialize the level set equation;

[0016] Step B3: Calculate the mean gray values ​​of the foreground and background based on the initialized level set equation, and adjust each point of the level set using the previously calculated mean gray values ​​of the foreground and background.

[0017] Step B4: Update the level set equation and draw the initial outline based on the level set equation.

[0018] Furthermore, the segmentation method in step A is as follows:

[0019] Step A1: Crop the irrelevant areas in the 3D lung CT image, keeping only the area containing the lung parenchyma;

[0020] Step A2: Binarize the cropped image;

[0021] Step A3: Eliminate cavities in the lungs, extract the largest connected region of the lung parenchyma, and perform binarization and inversion processing on the image to obtain a mask image of the lung parenchyma.

[0022] Step A4: Multiply the mask image with the image obtained in step A1 to obtain a complete lung parenchyma image.

[0023] Furthermore, in step B, the encoding layer of the 3D ViT network consists of a normalization layer, a multi-head attention layer, a re-normalization layer, and a multi-layer perceptron layer.

[0024] Furthermore, in step C, the three perspectives are front view, top view, and side view.

[0025] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0026] 1. Considering the large amount of data in 3D images, this invention extracts feature image information and generates multi-scale feature maps through a fast 3D ViT network structure. Only the output of the last layer of the backbone network is used to construct four feature maps of different scales. In this way, the size of the feature output of each layer is not changed during the calculation process, and the semantic features are integrated into the output of the last layer. Only the output features of the last layer are used to construct multi-scale feature maps, which greatly improves the output speed.

[0027] 2. A three-view false positive removal auxiliary module is directly embedded into the end-to-end framework, utilizing the self-attention information from the sparsed 3D image's three views to aid in the removal of false positive regions. On one hand, this solves the problem of learning attention in complex 3D scenes. On the other hand, this invention embeds a module containing information for false positive removal into the detection framework, designing detection and false positive region removal as a single, end-to-end trainable model. This improves detection speed and accuracy, reduces the overall complexity of the model, and allows for unified loss function training, thus accelerating model convergence. Attached Figure Description

[0028] Figure 1 This is a schematic flowchart of the method for detecting lung nodules using deep learning in this invention;

[0029] Figure 2 This is a diagram illustrating the lung parenchyma segmentation steps of the present invention;

[0030] Figure 3 This is a network structure diagram of the deep learning method used in this invention to detect lung nodules;

[0031] Figure 4 This invention relates to the level set image sparsification process.

[0032] Figure 5 This is the process of the three-view self-attention algorithm of the present invention.

[0033] Figure 6 This is an example of the lung nodule detection results of the present invention.

[0034] Figure 7 The images show a comparison before and after the false positive region screening. Figure (a) shows the result without the false positive module, with the false positive nodules within the gray box. Figure (b) shows the result with the false positive module embedded. Detailed Implementation

[0035] This invention proposes a deep learning-based method for lung nodule detection. First, lung CT images are segmented into lung parenchyma, removing irrelevant tissue and retaining only the lung parenchyma. Then, the lung parenchyma image is segmented into blocks, serialized, and embedded with learnable one-dimensional location information before being input into a fast 3D ViT algorithm to extract features, outputting a multi-scale feature map. Simultaneously, areas with gradients of 0 (i.e., minimal grayscale variation) in the 3D lung parenchyma image blocks are removed, as these locations typically lack lesions. The resulting 3D image, after removing most of the background, becomes a sparse matrix. Three-view information from the sparse matrix is ​​extracted for false positive screening. Next, network parameters are trained, and the trained network model is used to detect lung nodules. Finally, the prediction result is output, including the coordinates and radius of the candidate region, and the probability of whether it is a lung nodule.

[0036] To facilitate understanding of the present invention, the following description is in conjunction with the accompanying drawings. Figure 1 The embodiments further describe the present invention, but the scope of protection of the present invention is not limited to the following specific embodiments.

[0037] I. Reference Figure 1 The main implementation steps of this invention are as follows:

[0038] Step 1: Statistical segmentation of lung CT images.

[0039] Lung CT images typically include the trunk, lung parenchyma, trachea, and pulmonary vessels. The lung parenchyma appears as a large, low-grayscale connected region on the image, while other unrelated structural regions have higher grayscale values. Unrelated lung tissues may also appear spherical, interfering with the detection of lung nodules. In this embodiment, lung parenchyma segmentation is performed on the CT images first.

[0040] The original dataset of lung CT images is defined as follows:

[0041] CTSet = {S i ∈R H×W×C ,i=1...N} (1)

[0042] Among them, S i Let S represent the i-th CT image sample. Each sample in the dataset is a 3D image. H×W is the spatial resolution, and C is the number of channels, i.e., the number of CT slices. R represents the 3D vector space. Each sample S... i Each is composed of multiple slices, namely:

[0043] S = {slice j ∈R H×W ,j=1...C} (2)

[0044] The consistency of CT scanning equipment (Computed Tomography) and the uniform instructions given by doctors to each patient regarding posture adjustments during the imaging process result in a regular layout for lung CT imaging: the lungs are centered in the image, surrounded by other unrelated tissues. For example... Figure 2 As shown, the lung parenchyma region is located at the center of the current sample slice, and the coordinate variable of the center of the lung parenchyma region (x... c ,y c It follows the following distribution:

[0045]

[0046] (x c ,y c It follows a two-dimensional Gaussian distribution or a two-dimensional normal distribution, with five parameters, from left to right: x... c Expectations, y c The expectation, x c The variance, y c The variance, x c and y c The correlation coefficients (the center coordinate variables of each slice) all follow the same distribution and are independent of each other, i.e., they are independently identically distributed. Meanwhile, the width and height variables W and H follow the following distributions respectively:

[0047] H~N(μ3,σ3)

[0048] W~N(μ4,σ4) (4)

[0049] They are also independently and identically distributed.

[0050] Based on the above, combined with Figure 2 The steps for lung parenchyma segmentation are as follows:

[0051] (a) to (b): Based on the distribution of width and height variables W and H, the estimated values ​​w and h are determined, and irrelevant areas in the original CT image are cropped, retaining only the area where the lung parenchyma is located.

[0052] (b) to (c): The cropped image is binarized. By selecting an appropriate threshold, the entire image exhibits a clear black and white effect, highlighting the contours of the lung parenchyma.

[0053] (c) to (d): Eliminate cavities in the lungs, extract the largest connected region of the lung parenchyma, and perform binarization and inversion processing on the image to obtain a mask image of the lung parenchyma.

[0054] (d) to (e): Multiply the masked image with image (b) to obtain the complete lung parenchyma image.

[0055] The preprocessing algorithm is as follows:

[0056] <1> Input the complete original lung CT image 'a', where a = image(original)

[0057] "Original" refers to the original lung CT image, "image()" is the input function, and "a" is the input to the detection model.

[0058] <2> Image a is cropped based on the difference in gray values ​​between the lung parenchyma region and other irrelevant regions, retaining only the lung parenchyma region to obtain image b, where b = crop(a,(h,w)).

[0059] `crop()` is the cropping function. `a` is the input image, `h` and `w` are the height and width dimensions of the cropped image, respectively, and `b` is the resulting image after cropping.

[0060] <3> Select an appropriate threshold T and binarize image b. Pixels in b with gray values ​​greater than or equal to T are classified as foreground and marked as black; pixels with gray values ​​less than T are classified as background and marked as white. This yields image c, where c = binarize(b, T).

[0061] binarize() is a binarization function, where T is the grayscale threshold for binarization, and c is the image obtained after binarization by b.

[0062] <4> Image (c) still contains many holes. By using connectivity calculations to mark the connected regions with the largest number of pixels, the largest connected region is extracted, and the holes are eliminated. Then, the binarized image is inverted to obtain the mask (d) of the lung parenchyma, where d = maxfill(c).

[0063] `maxfill()` represents the process of extracting the maximum connected component, abstracted as a function, where `d` is the image after removing holes.

[0064] <5> The lung parenchyma mask (d) is multiplied with the image (b) to output the segmented lung parenchyma image (e), where e = multiply(d,b).

[0065] <6> The algorithm has ended.

[0066] The second step is to process lung parenchyma images in blocks.

[0067] Considering that the Transformer architecture cannot directly process 3D images, it has high memory requirements during training. This embodiment divides the input image into blocks, as follows:

[0068] The lung parenchyma image dataset obtained after parenchyma segmentation is defined as follows:

[0069] SCTSet = {Seg i ∈R h×w×C ,i=1...N} (5)

[0070] The size of the image patch is p×p×c, that is, the size of the patches obtained after segmentation is p×p×c.

[0071] The size of the lung parenchyma image is Seg∈R h×w×C Divide it into p×p×c patches, so there will actually be N patches, where:

[0072]

[0073] The resulting patches have dimensions of N×p×p×c. Then, the 3D patches are flattened, i.e.:

[0074]

[0075] The next step is to perform a linear embedding on the obtained patches: a process of transforming high-dimensional data into low-dimensional data. A linear transformation is applied to each patch vector using a fully connected layer, resulting in a dimensionality of D after reduction.

[0076] X P =linear-embedding(P,(p 2 ×c,D)) (8)

[0077] Where X P ∈R N×D .

[0078] To ensure the spatial location information between input image patches, a position encoding vector also needs to be added to the image patch embedding.

[0079]

[0080] in,

[0081] The obtained Z∈R N×D The next step is to input the vector sequence into the network model.

[0082] The third step is to build a network model.

[0083] The network model mainly consists of feature extraction, sparsification algorithm, self-attention module, and convolutional network.

[0084] The backbone network for feature extraction is a fast 3D ViT network. The preprocessed features Z∈R are then processed. N×DThe input is fed into the encoding layer for processing. Each encoding layer mainly consists of four computational processes: normalization (Norm), multi-head attention, re-normalization (Norm), and multilayer perceptron (MLP). During feature extraction, L layers of this encoding structure are sequentially stacked. The feature extraction process can be represented as:

[0085] Z l =Encoder(Z) l-1 ),l=1,2,3...L (10)

[0086] Finally, Z L The output features are defined as F1 and F1∈R N×D The feature pyramid is created using feature F1.

[0087] like Figure 3 In order to detect all suspected nodule regions more comprehensively, the F1 feature is decoded by convolution or deconvolution to obtain multi-scale information and generate four different scale feature outputs for candidate detection, ensuring that nodules of various sizes can be detected.

[0088] F1 generates four feature maps: L1, L2, L3, and L4. Each layer is generated by F1, and there are no connecting paths between layers, reducing the computational cost of feature map generation. The size of L1 to L4 is reduced to half that of the previous layer, and the feature outputs at these four scales are parallel, with no top-down or bottom-up connecting paths between them.

[0089] The P (patches obtained after segmenting the lung parenchyma image) obtained in the previous step is simultaneously processed using the level set algorithm to obtain a sparse matrix containing lung nodules or other porous tissues. For lung nodule detection, the lung parenchyma image contains a large amount of background regions, which greatly interferes with the detection of lung nodules. This invention uses the level set algorithm to sparsify the image. All suspected lung nodule regions are divided within the entire 3D slice, and the original P becomes a sparse matrix.

[0090] To obtain more feature information and filter out false positive areas, three views of P (front view, side view, and top view) are input into the network to jointly learn the features of lung nodules from multiple perspectives.

[0091] This invention utilizes three-view calculations to construct a module for filtering false positive regions, enabling sufficient discrimination of false positive regions. For example... Figure 5As shown, A, S, and C represent the top view, side view, and front view of the 3D image, respectively. These are convolved with 3×3 to obtain Q, K, and V. To obtain the correlation between Q and K, they are flattened, multiplied, and then the softmax function is calculated to obtain the attention score. This attention score is then multiplied by the flattened output to obtain the output feature F2.

[0092] like Figure 5 As shown, the information from the three perspectives is input into the self-attention mechanism module. The attention score is calculated with the feature information from the front view as the main component and the feature information from the top view and the side view as secondary components. The output feature F2 is then fused into the multi-scale feature map.

[0093] The multi-scale features L1 to L4 output by features F2 and F1 are concat fused.

[0094] Based on the dimensions of the output features L1 to L4, the dimensions of feature F2 need to be adjusted so that the dimensions of F2 can be fused with L1 to L4 respectively. Here, convolution operations are used to adjust the dimensions of F2:

[0095]

[0096]

[0097] After fusion, new features are obtained. These features include those extracted from the coding layer and also utilize the features from the three perspectives to further select for the next detection step.

[0098] This invention employs a module embedded within the detection framework that contains information for false positive screening. The detection and false positive region screening are designed as a single, end-to-end trainable model, significantly improving detection speed and accuracy while reducing overall model complexity. Furthermore, a unified loss function can be used during model training, improving the convergence speed of the model.

[0099] After two layers of convolutional network computation, the output is a four-dimensional tensor, which represents the predicted proposals. The last two dimensions correspond to the anchor boxes and the regressors, respectively. The five regressor values ​​are (x, y, z, r, p). (x, y, z) represent the coordinate positions, r is the radius of the nodule, and p is the probability that the region is a nodule.

[0100] The L1 to L4 layers are each fed into a network consisting of two convolutional layers, then connected to a fully connected layer, and finally the softmax function is used to output the result.

[0101] f i =ConvNet(L i (13)

[0102] V i =FC(f i ), i = 1, 2, 3, 4 (14)

[0103] V i It is the output after the fully connected layer.

[0104] The network training algorithm is as follows:

[0105] <1> First, obtain a publicly available dataset of lung CT images, and select the training set from it as the sample data for training the model.

[0106] <2> All samples are segmented into lung parenchyma to obtain the samples to be input into the model.

[0107] <3> Image segmentation is performed, and the resulting image patches are serialized and input into the model. A learnable vector PE is embedded at the initialization position.

[0108] <4> The image sequence vector and the location vector are combined and input into the model. Through feature extraction and decoding, the model outputs candidate results for lung nodules, namely the coordinates of the suspected lung nodule region, the radius of the lung nodule, and the probability of whether it is a lung nodule.

[0109] # Loop, 1 to several times

[0110] according to <3> The sample sequence vector and position vector PE are input into the network, and the discriminant network parameters are calculated using stochastic gradient descent.

[0111]

[0112] X is the prediction result, corresponding to the predicted mask, and Y is the ground truth.

[0113] The generated network parameters are calculated using stochastic gradient descent.

[0114] #Loop ends

[0115] <5> The algorithm has ended.

[0116] Step 4: Construct multi-scale feature maps.

[0117] Because the computation of 3D images is quite extensive, a simple strategy was adopted as much as possible when constructing feature maps. Feature maps of various scales were created for lung nodules of different sizes.

[0118] The specific algorithm for the generation process is as follows:

[0119] F1 generates L1: a 3×3 convolution with a size that is half the size of the F1 convolution, i.e.: The number of channel dimensions is C1

[0120] L1 = conv1(F1, C1) (15)

[0121] F1 generates L2: a 3×3 convolution with the same size as F1, i.e.: The number of channel dimensions is C2

[0122] L2=conv2(F1,C2) (16)

[0123] F1 generates L3: a 3×3 deconvolution with a size twice that of F1, i.e.: The number of channel dimensions is C3

[0124] L3 = deconv3(F1, C3) (17)

[0125] F1 generates L4: a 3×3 deconvolution with a size four times that of F1, i.e.: Channel dimension C4

[0126] L4 = deconv4(F1, C4) (18)

[0127] The feature outputs at these four scales are parallel, and there are no connected paths between them from top to bottom or bottom to top. Since the feature maps in this invention are not connected or fused together, the calculation speed is relatively fast. Detecting lung nodules on multi-scale feature maps can detect more lung nodules of various sizes, improving detection accuracy.

[0128] Step 5: Sparsification of lung parenchyma images.

[0129] Because 3D images contain a lot of information, the model is susceptible to interference from background or irrelevant information during computation. Furthermore, in lung CT images, blood vessels and pores in the lungs can easily interfere with nodule detection. This invention processes the P (patches obtained after segmenting the lung parenchyma image) obtained in the previous step using a level set algorithm to obtain a sparse matrix containing lung nodules or other porous tissues. The contours of lung nodules or other internal tissues within each lung parenchyma slice image are determined using binarized images, as shown below. Figure 4 As shown in (b) and (c), this is a closed curve in a two-dimensional plane. The curve outline formed by combining multiple slices constitutes the surface curve of a lung nodule or other internal tissue. In this way, all suspected lung nodule regions can be divided within the entire three-dimensional slice, and the original P becomes a sparse matrix.

[0130] The specific algorithm is as follows:

[0131] <1> Read in any original slice of a lung CT image, convert the image to grayscale, normalize it, and transform it into a processable array type;

[0132] <2> Initialize the level set equation IniLSF;

[0133] <3> Based on the level set equation, draw the initial contour, that is, use the contour function to draw the contour line with LSF=0.

[0134] <4> Define the CV (Chan-Vese) function: CV(LSF,img,mu,nu,epison,step)

[0135] The parameters include the penalty term coefficient mu, the length constraint coefficient nu, the regularization parameter epision, the step size step, LSF (level set contour lines), and img (image to be processed).

[0136] <5> # Loop, iteration count: iterNum

[0137] ① Calculate (estimate) the mean gray values ​​of the foreground and background based on the initialized level set;

[0138] ② Adjust (evolve) each point of the level set using the previously estimated foreground and background mean values. According to the definition, foreground > 0, background < 0. If the gray value of the current point is close to the foreground estimate, then the value of the level set of that point should be increased accordingly, and vice versa.

[0139] ③ Update the level set equation

[0140] #Loop ends;

[0141] <7> The algorithm has ended.

[0142] This invention eliminates unnecessary interference from lung parenchyma images, facilitating the subsequent three-view self-attention mechanism operation, while simplifying convolution and saving computational resources during product calculation.

[0143] Step 6: Calculation of the self-attention mechanism in three-view drawings

[0144] The three views of P (front view, side view, and top view) are input into the network respectively, and joint learning is performed from multiple perspectives. For example... Figure 5 As shown, C (coronal), A (axial), and S (sagittal) represent three perspectives of P: the front view, the top view, and the side view, each with dimensions of p×p×c. A 3x3 convolution operation is used to project the features from these three perspectives onto the dimensions of the Query, Key, and Value:

[0145] Query = conv(A)

[0146] Key = conv(S)

[0147] Value = conv(C) (19)

[0148] in,

[0149] Next, we will flatten Q, K, and V into a two-dimensional space, that is:

[0150]

[0151] Calculate the attention weight W according to the self-attention formula:

[0152]

[0153] Where W∈[0,1] n×m .

[0154] The final output feature is the product of the self-attention weights W and V, that is:

[0155] F sa =W×V (22)

[0156] Then output F sa The input is fed into a feedforward neural network to obtain the final feature output F2∈R n×D .

[0157] Step 7: Use the trained network model to detect lung nodules and output the prediction results.

[0158] Figure 7 The comparison shows the detection results of a single slice of lung CT image before and after false positive removal. Figure a shows the detection results without false positive removal, with the accurate detection results in the white box and the false positive areas in the gray box. Figure b shows the results after adding the false positive removal module, which shows that the false positive areas have been screened out.

[0159] This invention employs a fast 3D ViT network structure to extract feature image information, using single-scale outputs to sequentially construct multi-scale feature maps, thus solving the problem of massive data volume in 3D images. 3D lung parenchyma images contain a large number of background points, only a small portion of which are important information contributing to the detection results. During training with multi-view attention, the large number of background points can introduce unexpected noise into the attention module. Sparsification of the segmented image, i.e., removing most of the background areas, can highlight the regions where lung nodules are located. A self-attention mechanism is used to extract three-view information from the lung parenchyma image for the screening of false positive regions.

[0160] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1.A deep learning-based lung nodule detection method, characterized by include: Step A: Perform parenchymal segmentation on the 3D lung CT image and remove irrelevant tissue to obtain the lung parenchyma image; Step B: After segmenting the lung parenchyma image into blocks, serialize it and input it into a fast 3D ViT network to extract features and output multi-scale feature maps; at the same time, remove pixel regions with gradients of 0 and little gray-level change in the three-dimensional lung parenchyma block image to obtain a sparse matrix. Step C: Extract the three-view information of the sparse matrix, input it into the self-attention mechanism module, output the features, and then fuse it with the multi-scale feature map to obtain the fused features; Step D: Build the network model and input the fused features into the network model to train the network parameters; Step E: Use the trained network model to detect lung nodules and output the detection results. 2.The deep learning-based lung nodule detection method of claim 1, wherein, In step B, the construction process of multi-scale feature maps is as follows: using the last layer feature information of the backbone network output by Fast 3D ViT, feature maps of four scales are constructed respectively. There is no connection path between these four scale feature maps. Each layer is directly constructed from the last layer output of the backbone network. The size gradually increases to twice that of the previous layer. The size of the second layer is consistent with the output of the last layer of the backbone network. 3.The deep learning-based lung nodule detection method of claim 1, wherein, The serialization process in step B is as follows: the segmented image vector is linearly transformed, the three-dimensional image is projected onto a two-dimensional space and then flattened into one dimension, and one-dimensional position information is embedded. 4.The deep learning-based lung nodule detection method of claim 1, wherein, In step B, the sparse matrix generation process includes contour extraction based on the level set algorithm, as follows: Step B1: Process the three-dimensional lung CT images; Step B2: Initialize the level set equation; Step B3: Calculate the mean gray values ​​of the foreground and background based on the initialized level set equation, and adjust each point of the level set using the previously calculated mean gray values ​​of the foreground and background. Step B4: Update the level set equation and draw the initial outline based on the level set equation. 5.The deep learning-based lung nodule detection method of claim 1, wherein, The segmentation method in step A is as follows: Step A1: Crop the irrelevant areas in the 3D lung CT image, keeping only the area containing the lung parenchyma; Step A2: Binarize the cropped image; Step A3: Eliminate cavities in the lungs, extract the largest connected region of the lung parenchyma, and perform binarization and inversion processing on the image to obtain a mask image of the lung parenchyma. Step A4: Multiply the mask image with the image obtained in step A1 to obtain a complete lung parenchyma image. 6.The deep learning-based lung nodule detection method of claim 1, wherein, In step B, the encoding layer of the 3D ViT network consists of a normalization layer, a multi-head attention layer, a re-normalization layer, and a multi-layer perceptron layer. 7.The deep learning-based lung nodule detection method of claim 1, wherein, In step C, the three perspectives are front view, top view, and side view.

Images