RT-DETR training method for pulmonary nodule detection

Abstract

The invention discloses an RT-DETR training method for pulmonary nodule detection, and relates to the technical field of medical and industrial combination. The RT-DETR training method for pulmonary nodule detection comprises the steps of firstly obtaining an original data set from an LUNA16 data set, then performing normalization and slicing on a CT image in the original data set, performing segmentation by using a K-means algorithm to generate a pulmonary parenchyma mask and VOC data to generate a required data set, and performing random distribution on the data set; a light-weight LSNet module is introduced to replace an original model Backbone, tiny textures are strengthened while a receptive field is expanded, an EDFFN is adopted to replace a two-layer feed-forward network in an original model feature interaction module, high-value information is reserved through frequency domain gating, and artifacts are suppressed; and training and testing the improved model by using a randomly distributed data set. The improved method can effectively, quickly and accurately detect the pulmonary nodules in the lung CT image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical engineering, in particular to an RT-DETR training method for lung nodule detection. BACKGROUND

[0002] Lung nodule is an important imaging manifestation of early lung cancer, which usually presents as small target features such as blurred boundary, varying size and similar density in CT images, and is difficult to accurately identify through traditional image processing methods. With the development of deep learning technology, target detection methods based on convolutional neural network (CNN) have been widely applied to lung nodule detection tasks, among which detectors represented by YOLO series, Faster Region-based Convolutional Neural Network (Faster-RCNN) and Single Shot MultiBox Detector (SSD) have achieved certain results. However, these methods generally have two problems: on the one hand, traditional CNN networks rely on limited receptive field and hierarchical structure, making it difficult to accurately model the context information of small targets in complex backgrounds, resulting in limited detection ability for micro lesions such as lung nodules; on the other hand, due to the limitation of structural design, these models have high requirements for computing resources, making it difficult to balance high precision and real-time performance.

[0003] In recent years, detection frameworks based on Transformer have gradually become a research hotspot, and their strong global modeling ability makes them perform well in target recognition tasks. RT-DETR (Real-Time Detection Transformer) as a representative, by introducing and simplifying the DETR (Detection Transformer) structure, significantly improves the inference speed and has strong feature expression ability. However, the traditional backbone ResNet (Residual Network) structure used by RT-DETR still faces challenges such as insufficient capture of local information and insufficient expression of feature layers when dealing with small targets. In addition, its feature interaction module (Attention-Integrated Feature Interaction, AIFI) mainly relies on attention mechanism in multi-scale feature fusion, lacking efficient channel modeling and nonlinear adjustment mechanism, which limits the further improvement of overall detection performance.

[0004] In order to improve the performance of RT-DETR in small target scenarios such as lung nodule detection, it is necessary to optimize the model from the two core parts of Backbone and feature fusion structure. Summary of the Invention

[0005] The purpose of the present invention is to provide an RT-DETR training method for lung nodule detection, which can effectively, quickly and accurately detect lung nodules in lung CT images.

[0006] The present invention achieves the purpose of the invention by adopting the following technical scheme: an RT-DETR training method for lung nodule detection, comprising the following steps: S1, obtaining lung CT images from the LUNA16 dataset to generate the required lung CT image original dataset; S2, preprocessing the lung CT image original dataset generated by S1; S3, improving the RT-DETR model; S4, using the preprocessed lung CT image dataset to train and test the improved RT-DETR model.

[0007] Furthermore, in S1, the LUNA16 dataset is subjected to size conversion and format conversion respectively.

[0008] Furthermore, in said S2, the specific steps of preprocessing the original data set of lung CT images generated by S1 are as follows: S21, normalizing the pixel grayscale values ​​of the original lung CT image; S22, dividing the three-dimensional lung CT image into three two-dimensional slice images along the z-axis at equal distances; S23, after normalizing the slice images, performing brightness correction on the image based on the average pixel grayscale value of the lung edge area; S24, performing three-category segmentation on the corrected image using the K-means clustering algorithm to obtain the air, lung parenchyma and high Grayscale center value of density tissue; S25. Calculate the binarization threshold based on the grayscale center value, and perform binary segmentation on the image to generate foreground (lung tissue) and background (non-lung tissue) areas; S26. Perform corrosion and expansion operations on the binary image to remove noise and retain the main connected areas, and extract the lung parenchyma area; S27. Generate a lung mask image based on the lung parenchyma area, and combine it with the corresponding original CT image and save it as a JPEG format image; S28. Divide the image file into a training set and a test set, of which the training set accounts for 80% and the test set accounts for 20%.

[0009] Furthermore, in S21, the formula for normalizing the pixel grayscale values ​​of the original lung CT image is as follows: ;in, is the original CT pixel value, min and max represent double cropping, 、 Indicates the minimum or maximum grayscale of the image. Represents the entire image domain.

[0010] Further, the specific steps of the RT-DETR model in S3 are as follows: S31, replacing the Backbone part in the RT-DETR model with a lightweight LSNet (Large-Small Feature Aware Network) structure to improve the accuracy and computational efficiency of feature extraction; S32, replacing the feature interaction structure in the RT-DETR model from the original AIFI (Attention-Integrated Feature Interaction) structure to the EDFFN (Efficient Dynamic Feed-Forward Network) structure to optimize the feature fusion efficiency and dynamic modeling capability.

[0011] Further, in the S31, the application of the LSNet structure specifically includes: sequentially passing the input image into multiple Large-Small Feature Extraction units in the LSNet, each unit including a feature extraction structure for image patching and channel mapping; the Large-Small Feature Extraction unit includes a large receptive field convolution structure and a local attention enhancement structure connected in turn, the large receptive field convolution structure is used to extract the global context features of the input image, and the attention enhancement structure is used to model the features in the local area; the multi-scale feature maps output by multiple units are respectively unified to a preset channel dimension through a linear mapping structure, and are flattened into a token sequence after position encoding; the token sequence is used as the initial input of the Transformer encoder, which replaces the Backbone module in the original RT-DETR model.

[0012] Further, in the S32, the improvement of the AIFI structure specifically includes: in the encoder structure of the RT-DETR model, the multi-head self-attention mechanism and the cross-scale fusion framework in the AIFI module are retained, and only the feedforward network structure therein is replaced with the EDFFN structure; the EDFFN structure includes one-dimensional Fourier transform on the up-sampled features to obtain frequency domain representation; in the frequency domain representation, a learnable mask is generated through a gating mechanism including spectral global pooling, linear transformation and activation operation, and element-wise weighting processing is performed on the spectral tensor; the weighted spectral tensor is subjected to one-dimensional inverse Fourier transform to restore the time domain features, and is regressed to the original channel dimension through linear transformation; the restored features and the output features of the self-attention are fused through residual connection to output the final feature representation.

[0013] Furthermore, in said S4, the improved RT-DETR model is trained and tested using the preprocessed lung CT image dataset, specifically: the divided training set is used to train the improved model, and the test set is used to test and analyze the trained model.

[0014] Beneficial Effects: The improved RT-DETR algorithm integrates the LSNet backbone and the EDFFN-AIFI encoder, significantly improving the detection rate of lung nodules while maintaining real-time inference. LSNet's large-core-small-core convolution combined with final-level self-attention expands the effective receptive field and enhances subtle textures. EDFFN introduces frequency-domain gating in the feedforward phase to preserve critical high-frequency information, suppress artifacts, and reduce false positives. The two work together to optimize multi-scale features, improving overall detection sensitivity and positioning accuracy.

[0015] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of an RT-DETR training method for lung nodule detection according to the present invention.

[0017] Figure 2 This is the LSNet structure diagram provided by the present invention.

[0018] Figure 3 This is a structural diagram of the EDFFN provided by the present invention. DETAILED DESCRIPTION

[0019] The present invention is further explained below through specific examples.

[0020] See also Figure 1 The embodiment of the present invention provides a technical solution: an RT-DETR training method for pulmonary nodule detection, comprising the following steps:

[0021] Data preparation: Since the LUNA16 dataset consists of MHD and RAW formats and cannot be directly trained by the model, its original files need to be generated as 640×640 images and converted to VOC format to facilitate model training.

[0022] Data preprocessing: First, the obtained lung CT images are normalized using the following formula:

[0023] ;

[0024] in, is the original CT pixel value, min and max represent double cropping, 、 Indicates the minimum or maximum grayscale of the image. The entire image domain is represented by the 3D CT image. Then, three binary images are cut out in the z direction at equal intervals: upper lung, middle lung, and lower lung. This not only reduces the computational effort but also ensures that all three anatomical levels are covered. On the basis of the normalized slice image, the overexposure problem in the image is corrected by extracting the average pixel value around the lung. The formula is as follows:

[0025] ;

[0026] in, is the rough edge of the lungs, represents the average edge grayscale, Represents the corrected pixels after replacement, and then the K-means algorithm is used to process the image. The formula is as follows:

[0027] Objective function:

[0028] ;

[0029] in, Indicates the Grayscale of pixels (from ), Represents pixels The cluster number to which it belongs is 1, 2, or 3. represents the center grayscale of the kth cluster, represents the sum of the squared distances from the cluster to the center;

[0030] Iteration steps:

[0031] ;

[0032] The cycle continues until the movement of each centroid is less than the threshold or the maximum number of iterations is reached, where the darkest center Air, medium gray Lung parenchyma, brightest For nodules or soft tissues, the central area of ​​the image is divided into foreground and background and their center values ​​are calculated respectively. The average value of the two is taken as the classification threshold. When judging the standardized image, if the pixel value is greater than the threshold, it is marked as foreground (1), otherwise it is background (0). The formula is as follows:

[0033] ;

[0034] Where T is the midpoint of the centroid of the lung parenchyma and soft tissue as the binary classification threshold, is a preliminary binary mask, 1 represents foreground and 0 represents background. Then, the image erosion operation is used to remove edge burrs and the dilation operation is used to fill the holes. The formula is as follows:

[0035] ;

[0036] ;

[0037] in, Indicates corrosion, can remove burrs and discrete points, represents expansion, which can fill small holes and restore the main contour, Indicates the connected regions, Indicates the number of pixels in the area, Represents the area threshold to achieve effective separation of foreground (radio-opaque tissue) and background (radio-transparent tissue, i.e., lung area), retaining only the lung parenchyma area. Finally, the image of this area is morphologically processed to generate a lung mask and saved together with the original image in JPEG format. 80% of the obtained images are randomly assigned as a training set, and the rest are used as a test set.

[0038] Model improvement: The Backbone part of the RT-DETR model is replaced with the more lightweight and accurate LSNet module; the Neck part of the RT-DETR model is improved by introducing the lightweight EDFFN structure to improve the original AIFI module of the RT-DETR model.

[0039] Backbone part improvement: replace the Backbone module of RT-DETR with LSNet module, and pass the input image into multiple Large-Small Feature Extraction units in the LSNet structure in sequence, such as Figure 2 As shown, the formula is as follows:

[0040] ;

[0041] Where I is the input image; Indicates that the image or the previous layer feature is cut into pieces and projected into the channel dimension Operation; is the sth level layer feature Layer characteristics; It is the Large–Small Feature Extraction unit; The size of the large kernel convolution used for this level; is the number of LSFE stacking layers at this level. The unit is designed through convolution and attention. First, a large receptive field convolution is used to extract global information. Then, the local attention module is used to achieve local enhancement of key areas. The formula is as follows:

[0042] ;

[0043] ;

[0044] in, is the LSFE input feature; Indicates depth separability and kernel size is The large receptive field convolution outputs the global feature G; Q, K, V are the query, key, and value vectors in the local window. is the activation function; A is the local attention enhancement feature; , is a learnable scaling factor; BN is normalized with residual connection for steady-state output , and then the multi-scale feature maps output by the LSNet module correspond to the semantic layers of the Backbone output in the original RT-DETR. The formula is as follows:

[0045] ;

[0046] in, It is a 1×1 linear mapping that unifies the features of each scale into the Transformer input channel C; Flatten the space dimension to its length Token sequence; is the position code of the corresponding scale; S is the number of scales. Finally, after unified mapping, it is input into the Transformer encoder as the initial representation of the target query stage, completing the function of replacing the original Backbone network structure, improving the diversity and resolution of feature expression, reducing the model calculation amount and improving the detection accuracy and inference speed. It is suitable for the detection task of lung nodules. The formula is as follows:

[0047] ;

[0048] in, Indicates concatenation by token dimension; Z is the total token sequence, length , the number of channels C, which constitutes the initial input to the Transformer encoder.

[0049] Neck part improvement: At the encoder level, the multi-head self-attention and cross-scale fusion framework of AIFI is retained, and only the two layers of feedforward network are replaced with EDFFN, such as Figure 3 As shown, EDFFN first performs a one-dimensional Fourier transform on the upsampled features. The formula is as follows:

[0050] ;

[0051] Among them, X is the feature after multi-head self-attention; Represents an upsampling operation; It is a one-dimensional discrete Fourier transform (FFT) that outputs a frequency domain tensor F. Then, in the frequency domain, learnable gating is used to suppress redundant frequency bands and enhance high-value frequency bands. The formula is as follows:

[0052] ;

[0053] in, is a multiplication element; G is a learnable gated mask, which is linearly transformed after spectral global pooling (POOL) , and activation function It is used to suppress redundant frequency bands and highlight high-value frequency bands, and then inversely transformed back to the time domain and linearly regressed to the original channel dimension. Finally, it is fused with the attention output in a residual manner. The formula is as follows:

[0054] ;

[0055] in, is the inverse FFT; Y is the time domain feature; W,b is the linear transformation; is the final output.

[0056] Model training and testing: The divided training set is used to train the improved model, and the test set is used to test and analyze the trained model.

[0057] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0058] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A RT-DETR training method for pulmonary nodule detection, characterized in that: The following steps are involved: S1. Obtain lung CT images from the LUNA16 dataset to generate the required lung CT image raw dataset; S2, preprocessing the original lung CT image dataset generated by S1; S3, improve the RT-DETR model; S4. Use the preprocessed lung CT image dataset to train and test the improved RT-DETR model.

2. The RT-DETR training method for pulmonary nodule detection according to claim 1, characterized in that In S1, the LUNA16 dataset is subjected to size conversion and format conversion respectively.

3. The RT-DETR training method for pulmonary nodule detection according to claim 1, characterized in that In S2, the specific steps of preprocessing the original lung CT image dataset generated in S1 are as follows: S21, normalizing the pixel grayscale values ​​of the original lung CT image; S22, dividing the three-dimensional lung CT image into three two-dimensional slice images at equal intervals along the z-axis direction; S23, after normalizing the slice image, performing brightness correction on the image based on the average pixel grayscale value of the lung edge area; S24. Perform three-category segmentation on the corrected image using the K-means clustering algorithm to obtain the grayscale center values ​​of air, lung parenchyma, and high-density tissue; S25, calculating a binarization threshold based on the grayscale center value, and performing binary segmentation on the image to generate foreground and background areas; S26, performing erosion and dilation operations on the binary image to extract the lung parenchyma area; S27, generating a lung mask image based on the lung parenchyma region, and combining the mask image with the corresponding original CT image to save the image as a JPEG format; S28. Divide the image files into a training set and a test set.

4. The RT-DETR training method for pulmonary nodule detection according to claim 3, characterized in that In the step S21, the formula for normalizing the pixel grayscale values ​​of the original lung CT image is as follows: ； in, is the original CT pixel value, min and max represent double cropping, 、 Indicates the minimum or maximum grayscale of the image. Represents the entire image domain.

5. The RT-DETR training method for pulmonary nodule detection according to claim 1, characterized in that In S3, the specific steps of the RT-DETR model are as follows: S31. Replace the Backbone part of the RT-DETR model with a lightweight LSNet structure; S32. Replace the feature interaction structure in the RT-DETR model from the original AIFI structure to the EDFFN structure.

6. The RT-DETR training method for pulmonary nodule detection according to claim 5, characterized in that In the S31, the application of the LSNet structure specifically includes: Pass the input image into multiple Large-Small Feature Extraction units in LSNet in sequence; The Large-Small Feature Extraction unit includes a large receptive field convolution structure and a local attention enhancement structure connected in sequence; The multi-scale feature maps output by multiple units are unified to the preset channel dimension through a linear mapping structure, and are flattened into a token sequence after position encoding; The token sequence serves as the initial input to the Transformer encoder.

7. The RT-DETR training method for pulmonary nodule detection according to claim 5, characterized in that In the S32, the improvement of the AIFI structure specifically includes: In the encoder structure of the RT-DETR model, the multi-head self-attention mechanism and cross-scale fusion framework in the AIFI module are retained, and only the feedforward network structure is replaced with the EDFFN structure; The EDFFN structure includes performing a one-dimensional Fourier transform on the upsampled features to obtain a frequency domain representation; In the frequency domain representation, a learnable mask is generated through a gating mechanism consisting of spectral global pooling, linear transformation, and activation operations, and element-wise weighted processing is performed on the spectral tensor; Perform one-dimensional inverse Fourier transform on the weighted spectral tensor to restore it to time domain features, and then regress it to the original channel dimension through linear transformation; The recovered features are fused with the aforementioned self-attention output features through residual connections to output the final feature representation.

8. The RT-DETR training method for pulmonary nodule detection according to claim 3, characterized in that In said S4, the improved RT-DETR model is trained and tested using the preprocessed lung CT image dataset. Specifically, the divided training set is used to train the improved model, and the test set is used to test and analyze the trained model.

Images