Lung CT image processing method and system based on multi-scale spatial semantic fusion

Abstract

The invention provides a lung CT image processing method and system based on multi-scale spatial semantic fusion, and the method comprises the steps: collecting a plurality of lung CT-DICOM images, and carrying out the quality inspection processing and format conversion, and obtaining an original sample image; performing preprocessing and lung parenchyma segmentation on the original sample image to obtain a standard sample image; determining positive and negative samples according to an image labeling result, performing random interference on the positive and negative samples to obtain an expanded sample image, and generating a training data set based on the expanded sample image and the standard sample image; the YOLOv12 network is improved based on an MSSF module and a C3K2MSSF module to obtain an improved YOLOv12-3D network, and the improved YOLOv12-3D network is trained through the training data set to obtain a target detection model; and inputting the to-be-detected CT image into the target detection model to obtain detection information. According to the invention, by improving a YOLOv12-3D network architecture, deep fusion of three-dimensional space features and multi-scale space semantic features is realized in a feature extraction stage, and the extraction rate and efficiency of small targets are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a lung CT image processing method and system with multi-scale spatial semantic fusion. Background Art

[0002] In the field of medical image analysis, although CT image detection technology based on deep learning has made significant progress, it still faces two fundamental challenges: insufficient three-dimensional feature modeling and insufficient sensitivity in detecting small targets. Traditional two-dimensional convolutional networks are constrained by the planar processing mechanism and cannot effectively capture the spatiotemporal dependencies between layers in CT sequences. Although three-dimensional convolutional models represented by 3DU-Net can integrate spatial context information, they have a high missed detection rate when faced with tiny lesions less than 5 mm in size due to insufficient texture feature resolution. In addition, the differences in layer thickness, equipment noise interference, and the uncontrollability of manual post-processing in multi-center CT data have further exacerbated the bottlenecks in the algorithm's generalization ability and computational efficiency, making it difficult to meet the urgent clinical needs for high-precision, real-time diagnosis.

[0003] Although the current mainstream feature pyramid network (FPN) has achieved multi-scale feature fusion, it has inherent defects in the balance between local details and global semantic information. Especially in lesion areas with complex texture features, it is easy to cause feature misjudgment and misclassification problems. When the two-dimensional detection framework (such as the YOLO series) is directly migrated to a three-dimensional scene, the model produces a large number of false positive detection results due to the limited receptive field range and low efficiency of cross-layer feature interaction. At the same time, the split design of data preprocessing and model training not only increases the consumption of computing resources, but also seriously hinders the transformation of the algorithm from laboratory to clinical application. A new multi-scale spatial semantic fusion lung CT image processing method and system is urgently needed to solve these problems. Summary of the Invention

[0004] The present invention provides a lung CT image processing method and system with multi-scale spatial semantic fusion to solve the problems of insufficient three-dimensional feature modeling, low efficiency of small target detection, and poor algorithm generalization in the prior art.

[0005] The present invention provides a lung CT image processing method with multi-scale spatial semantic fusion, comprising:

[0006] Step 1: Collect multiple lung CT-DICOM images, perform quality inspection and format conversion to obtain original sample images;

[0007] Step 2: Preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image;

[0008] Step 3: According to the image annotation results, determine the positive and negative samples, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and the standard sample images, generate a training data set;

[0009] Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and train the improved YOLOv12-3D network using the training dataset to obtain the target detection model;

[0010] Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow.

[0011] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 1 includes:

[0012] Anonymize the collected lung CT-DICOM images and perform quality inspection on the processed lung CT-DICOM data;

[0013] For lung CT-DICOM images that have passed the quality inspection, the dcm2niix tool is used to perform data format conversion to obtain the original sample images;

[0014] Based on the preset lung disease database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images cover all lung disease types;

[0015] If not, proceed with lung CT-DICOM image acquisition;

[0016] If yes, the lung CT-DICOM image acquisition is terminated.

[0017] Preferably, in a lung CT image processing method with multi-scale spatial semantic fusion, step 2 includes:

[0018] Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample image with an original pixel spacing greater than 1 mm to obtain a new sample image;

[0019] The new image and the sample image with the original pixel spacing less than or equal to 1 mm are denoised using the 3D non-local mean filtering algorithm to obtain denoised sample data;

[0020] Based on the preset CT value, the denoised sample data is binarized to obtain binary sample data;

[0021] Perform pixel-by-pixel AND operation on the binarized sample data and its corresponding original sample image to obtain a lung parenchyma image;

[0022] The lung region is cropped from the lung parenchymal image to obtain a lung ROI image, and the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

[0023] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 3 includes:

[0024] Based on the pre-trained nnU-Net target detection model, standard sample images are marked with candidate boxes to obtain marked sample images, and the marking results are sent to experts for review;

[0025] After the expert review is completed, the expert marked target box is compared with the candidate box to obtain the marked box intersection and union ratio;

[0026] The labeled sample images whose marked box intersection-union ratio is greater than or equal to the preset threshold are regarded as positive samples;

[0027] The labeled sample images whose marked box intersection-union ratio is less than the preset threshold are regarded as negative samples;

[0028] Randomly perform interference processing on each positive and negative sample to obtain the expanded sample image;

[0029] A training dataset is generated based on the expanded sample images and the standard sample images.

[0030] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 4 includes:

[0031] Based on the MSSF module and C3K2_MSSF module, the main network and feature map network of the YOLOv12 network are constructed respectively, and the improved YOLOv12-3D network is generated by combining the detection head module;

[0032] The improved YOLOv12-3D network is trained based on the training data set to obtain the target detection model. Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, the main network of the YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including:

[0033] Two 3D convolution downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input it into the first C3K2 module for high-level feature extraction to obtain the first feature map;

[0034] The first feature map is processed by the third 3D convolution downsampling module and then input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map;

[0035] After the second feature map is processed by the fourth 3D convolution downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map;

[0036] The third feature map is processed by the fifth 3D convolution downsampling module to obtain a feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network.

[0037] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, a feature map network of a YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including:

[0038] The construction of the first-layer feature fusion network is as follows:

[0039] The output feature map of the backbone network is input into the third C3K2_MSSF module for processing to obtain the fourth feature map;

[0040] After upsampling the fourth feature map and concatenating it with the third feature map output by the first-layer feature fusion network, it is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map;

[0041] Upsample the fifth feature map and concatenate it with the second feature map in the backbone network to obtain the highest resolution first-layer final concatenated fusion feature map.

[0042] The construction of the second-layer feature fusion network is as follows:

[0043] The final spliced ​​fusion feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fusion feature map;

[0044] The first fused feature map is processed by the sixth 3D convolution downsampling module and then spliced ​​with the fifth feature map output by the first layer network. The spliced ​​result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map;

[0045] The second fused feature map is processed by the seventh 3D convolution downsampling module and then spliced ​​with the fourth feature map output by the first layer network. The splicing result is input into the second C3K2 module for feature extraction to obtain the third fused feature map.

[0046] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, the CT image to be detected is input into the target detection model to obtain detection information of the patient's lung shadow, including:

[0047] The pre-processed CT image to be detected is input into the target detection module, and the three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module;

[0048] The detection head module analyzes the three-dimensional spatial features, determines potential targets, marks multiple potential targets based on candidate boxes, and outputs the three-dimensional bounding box parameters of the candidate boxes;

[0049] A three-dimensional non-maximum suppression algorithm is used to calculate the confidence level of each candidate box based on a preset threshold. The candidate box with the highest confidence level is selected as the target box, and overlapping detection results are eliminated.

[0050] According to the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size;

[0051] Based on the three-dimensional bounding box parameters and the real physical size, the detection information of the patient's lung shadow is generated.

[0052] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, the method further includes:

[0053] After obtaining the test information, the target case or related test data of the patient is obtained simultaneously, and based on the type of lung disease corresponding to the test information, various related possible abnormal information of the patient's test data is predicted;

[0054] Retrieve relevant possible abnormal information from the target case or related test data, and obtain the abnormality degree corresponding to the relevant abnormal information;

[0055] Based on the abnormality and the size of the lesion location corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the verification degree is sent to the patient's attending physician as diagnostic auxiliary information.

[0056] The present invention provides a lung CT image processing system with multi-scale spatial semantic fusion, comprising:

[0057] The data preparation and quality control module is used to collect various lung CT-DICOM images, perform quality inspection and format conversion, and obtain original sample images;

[0058] Image preprocessing and lung parenchyma segmentation module, used to preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image;

[0059] The training data set construction module is used to determine positive and negative samples based on the image annotation results, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and standard sample images, a training data set is generated;

[0060] The target detection model construction module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and then train the improved YOLOv12-3D network using the training dataset to obtain the target detection model;

[0061] The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain detection information of the patient's lung shadow.

[0062] Compared with the prior art, the present invention has at least the following beneficial effects:

[0063] The present invention collects a variety of lung CT-DICOM images and conducts quality inspection and format conversion to ensure that the original samples cover image data from different devices and scenarios, thereby reducing model deviations caused by single data. At the same time, the format unification reduces the difficulty of subsequent processing and reduces GPU pressure. Subsequently, the original samples are preprocessed and lung parenchyma segmented to remove irrelevant tissue interference and standardize image features so that the model focuses on the effective area, effectively improving the detection targeting and accuracy, avoiding false detection caused by the influence of extrapulmonary structures, and accurately determining positive and negative samples based on the annotation results. By expanding the data through random interference, not only the number of samples is increased, but also the simulation of complex clinical situations is achieved, so that the model learns more diverse feature patterns, which is conducive to alleviating the overfitting problem and enhancing the accuracy of the model. The generalization ability of targets of different shapes and positions is then improved based on the MSSF module and the C3K2_MSSF module. The YOLOv12 network is used to deeply extract three-dimensional spatial features by utilizing multi-scale spatial semantic fusion and attention mechanism, effectively solving the problems of insufficient three-dimensional modeling and low sensitivity of small targets in traditional methods. In combination with high-quality training data sets, the improved YOLOv12-3D network fully learns target features during training and achieves high-precision detection. Finally, the CT image to be detected is input into the trained target detection model, which can quickly output lung shadow detection information. Compared with traditional methods, the detection time is greatly shortened, and the small target detection ability is improved at the same time, which helps to provide accurate auxiliary diagnosis for early diseases and improve medical efficiency and diagnostic accuracy. The present invention improves the YOLOv12-3D network architecture, realizes the deep fusion of three-dimensional spatial features and multi-scale spatial semantic features in the feature extraction stage, constructs an end-to-end target detection model, and does not require manual design of features or complex post-processing steps. It can directly output target positioning and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0064] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0065] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0067] Figure 1 This is a flowchart of a lung CT image processing method with multi-scale spatial semantic fusion;

[0068] Figure 2 To improve the overall architecture of the YOLOv12-3D network;

[0069] Figure 3 This is the architecture diagram of the MSSF module;

[0070] Figure 4 This is the architecture diagram of the C3K2_MSSF module;

[0071] Figure 5 This is a structural diagram of a multi-scale spatial semantic fusion lung CT image processing system. DETAILED DESCRIPTION

[0072] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0073] The present invention provides a lung CT image processing method based on multi-scale spatial semantic fusion, such as Figure 1 Shown, including:

[0074] Step 1: Collect multiple lung CT-DICOM images, perform quality inspection and format conversion to obtain original sample images;

[0075] Step 2: Preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image;

[0076] Step 3: According to the image annotation results, determine the positive and negative samples, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and the standard sample images, generate a training data set;

[0077] Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and train the improved YOLOv12-3D network using the training dataset to obtain the target detection model;

[0078] Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow.

[0079] In this embodiment, a weighted combination of multiple losses is used to improve the YOLOv12-3D network training process to meet the regression and classification requirements of the detection task:

[0080] The bounding box (i.e. candidate box, target box, etc.) regression loss uses the CIoU loss function to comprehensively measure the overlap, center distance, and width-to-height ratio difference between the predicted box and the true box;

[0081] The target category classification loss uses focal loss to enhance the learning ability of difficult-to-detect targets and alleviate the imbalance of sample categories;

[0082] The total loss is the weighted sum of the above two losses to simultaneously optimize position regression, size prediction and category discrimination.

[0083] Improved YOLOv12-3D network training uses the Adam optimization algorithm to update network parameters. The Adam optimizer adaptively adjusts the learning rate, accelerating convergence and improving training stability.

[0084] The beneficial effects of the above technical solution: The present invention collects a variety of lung CT-DICOM images and performs quality inspection and format conversion to ensure that the original samples cover image data under different devices and scenarios, thereby reducing model deviations caused by single data; at the same time, the difficulty of subsequent processing is reduced by unifying the format, and the GPU pressure is reduced; thereafter, the original samples are preprocessed and the lung parenchyma is segmented to remove irrelevant tissue interference and standardize image features so that the model focuses on the effective area, effectively improving the detection targeting and accuracy, avoiding false detection caused by the influence of extrapulmonary structures, and accurately determining positive and negative samples based on the annotation results. By expanding the data through random interference, not only the number of samples is increased, but also the simulation of complex clinical situations can be achieved, so that the model can learn more diverse feature patterns, which is conducive to alleviating overfitting. The problem is solved by improving the generalization ability of targets of different shapes and positions. Then, the YOLOv12 network is improved based on the MSSF module and the C3K2_MSSF module. The multi-scale spatial semantic fusion and attention mechanism are used to deeply extract three-dimensional spatial features, effectively solving the problems of insufficient three-dimensional modeling and low sensitivity to small targets in traditional methods. In combination with high-quality training data sets, the improved YOLOv12-3D network can fully learn target features during training and achieve high-precision detection. Finally, the CT image to be detected is input into the trained target detection model, which can quickly output lung shadow detection information. Compared with traditional methods, the detection time is greatly shortened, and the small target detection ability is improved at the same time, which helps to provide accurate auxiliary diagnosis for early diseases and improve medical efficiency and diagnostic accuracy. The present invention improves the YOLOv12-3D network architecture, realizes the deep fusion of three-dimensional spatial features and multi-scale spatial semantic features in the feature extraction stage, constructs an end-to-end target detection model, and does not require manual design of features or complex post-processing steps. It can directly output target positioning and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0085] Example 2:

[0086] On the basis of Example 1, step 1 includes:

[0087] Anonymize the collected lung CT-DICOM images and perform quality inspection on the processed lung CT-DICOM data;

[0088] For lung CT-DICOM images that have passed the quality inspection, the dcm2niix tool is used to perform data format conversion to obtain the original sample images;

[0089] Based on the preset lung disease database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images cover all lung disease types;

[0090] If not, proceed with lung CT-DICOM image acquisition;

[0091] If yes, the lung CT-DICOM image acquisition is terminated.

[0092] In this embodiment, the number of lung CT-DICOM data is greater than 1,000, including lung CT-DICOM sequences from multiple centers and multiple vendors.

[0093] In this embodiment, the converted image format is NIfTI format.

[0094] The beneficial effects of the above technical solution: The present invention performs anonymization on CT-DICOM images, removes privacy fields such as patient name, ID, medical history, and only retains key diagnostic parameters (such as scanning time, layer thickness, etc.), which can effectively avoid the risk of medical data leakage and ensure the compliance of subsequent research and clinical applications. It also performs quality inspection on the anonymized CT data to screen for problems such as missing slices, inter-layer dislocation, respiratory motion artifacts, and marks low-quality data to ensure the continuity and integrity of the CT sequence input into the training model, avoid low-quality data contamination of the training set, effectively reduce the probability of false detection of the model due to noise or artifacts, and improve the reliability of subsequent three-dimensional feature modeling; use the dcm2niix tool to convert the DICOM format to the NIfTI format to overcome the problems caused by differences in DICOM file headers (different The system can eliminate compatibility issues caused by different metadata structures of manufacturers' devices, reduce the complexity of data preprocessing, effectively improve computing efficiency (such as reducing the time spent on GPU loading data), and detect whether the original samples cover all target disease types based on the preset database (including common disease types such as pneumonia, lung nodules, lung cancer, and emphysema). It can also ensure that the training set contains CT features of various lung lesions (such as ground-glass shadows, solid nodules, and cavitary lesions) to the greatest extent possible, and prevent the model from missing rare types of lesions due to "few diseases it has seen". At the same time, comprehensive disease type coverage enables the model to learn diverse pathological features, which is conducive to improving the improved YOLOv12-3D network to adapt to complex diagnostic needs in clinical practice and reduce the overfitting problem of the improved YOLOv12-3D network due to single data distribution.

[0095] Example 3:

[0096] On the basis of Example 1, step 2 includes:

[0097] Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample image with an original pixel spacing greater than 1 mm to obtain a new sample image;

[0098] The new image and the sample image with the original pixel spacing less than or equal to 1 mm are denoised using the 3D non-local mean filtering algorithm to obtain denoised sample data;

[0099] Based on the preset CT value, the denoised sample data is binarized to obtain binary sample data;

[0100] Perform pixel-by-pixel AND operation on the binarized sample data and its corresponding original sample image to obtain a lung parenchyma image;

[0101] The lung region is cropped from the lung parenchymal image to obtain a lung ROI image, and the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

[0102] In this embodiment, B-spline interpolation is used to resample the original sample image with an original pixel spacing greater than 1 mm to 1×1×1 mm. 3 Isotropic space effectively eliminates the 3D convolution bias caused by anisotropy.

[0103] In this embodiment, the preset CT value can be flexibly set, and the default value is -400HU.

[0104] During the binarization of lung CT slices, lung region candidates were extracted, and the two largest connected domains, the left lung and the right lung, were retained. A morphological closing operation (3×3×3 structure kernel) was used to repair small holes in the lungs, and an edge-preserving smoothing algorithm was used to eliminate the jagged boundaries of the mask.

[0105] In this embodiment, the size of the standard sample image is 256×256×128 voxels by default and can be flexibly set.

[0106] In this embodiment, before the lung region is cropped from the lung parenchymal image, the process further includes: calculating a three-dimensional bounding box based on the segmentation mask of the lung, and expanding the box by a 10-voxel error-tolerant boundary.

[0107] The beneficial effect of the above technical solution is that the present invention uses B-spline interpolation to resample the data with original pixel spacing greater than 1mm to 1×1×1mm. 3Isotropic space makes each voxel represent the same physical size in three-dimensional space, ensuring that the receptive field of the 3D convolution kernel on the x / y / z axis is consistent, effectively improving the modeling accuracy of inter-layer feature association, and for images with a layer thickness >1mm, the number of inter-layer slices can be virtually increased after resampling to avoid small targets (such as <5mm nodules) from being missed due to excessive layer spacing (for example, in an image with a layer thickness of 3mm, a 5mm nodule may only be distributed in 2 layers of slices, but after resampling to 1mm, it can be distributed in 5 layers, which is convenient for capturing continuous three-dimensional features); and the 3D non-local mean filtering algorithm is used to denoise the new image and the sample image with the original pixel spacing less than or equal to 1mm to obtain denoised sample data, reduce noise interference in the CT image, and help improve the recognition of details in the CT image, especially for small objects. The system can identify the lung lesions (such as ground-glass shadows, tiny nodules, etc.), and then binarize the denoised sample data based on the preset CT value to obtain the binarized sample data, and perform pixel-by-pixel AND operation on the binarized sample data and the corresponding original sample image to obtain the lung parenchyma image, retaining only the lung parenchyma area, so that the subsequent model training can focus on lung lesion detection, which can effectively reduce the false positive rate of the trained model (such as avoiding misjudging rib calcification as nodule), crop the lung area of ​​the lung parenchyma image, obtain the lung ROI image, and use the bilinear interpolation algorithm to adjust the lung ROI image to the preset size to obtain the standard sample image, so that the images in the training set have a uniform size, which can effectively avoid the network structure adjustment caused by the change of input dimension, thereby improving the stability of training and inference.

[0108] Example 4:

[0109] On the basis of Example 1, step 3 includes:

[0110] Based on the pre-trained nnU-Net target detection model, standard sample images are marked with candidate boxes to obtain marked sample images, and the marking results are sent to experts for review;

[0111] After the expert review is completed, the expert marked target box is compared with the candidate box to obtain the marked box intersection and union ratio;

[0112] The labeled sample images whose marked box intersection-union ratio is greater than or equal to the preset threshold are regarded as positive samples;

[0113] The labeled sample images whose marked box intersection-union ratio is less than the preset threshold are regarded as negative samples;

[0114] Randomly perform interference processing on each positive and negative sample to obtain the expanded sample image;

[0115] A training dataset is generated based on the expanded sample images and the standard sample images.

[0116] In this embodiment, the interference processing includes elastic deformation (σ=10, α=20), rotation (±15°), and mirror flipping; adding local pixel value perturbation (±15% window width / window position offset), and randomly adding CT simulation noise (Poisson-Gaussian mixed model). Positive and negative samples may be subjected to one or more interference processing.

[0117] In this embodiment, the preset threshold value ranges from (0, 1) and is 0.2 by default.

[0118] The beneficial effects of the above technical solution: The present invention uses a pre-trained nnU-Net model to automatically generate candidate frames for standard sample images, realizes the rapid marking of lung lesion targets (such as nodules, inflammatory areas), and then sends the marking results generated by the model to experts for review. The experts manually adjust the position and size of the frame or delete the false detection frame, which can avoid the systematic errors of pure automatic labeling and ensure that the positive sample is an accurate mark of the real target. After the expert review is completed, the expert-marked target frame is compared with the candidate frame, and then the positive and negative samples are determined based on the intersection and union of the marked frames, avoiding the model learning confusion caused by fuzzy negative samples in traditional methods, effectively improving the discrimination ability of the subsequent trained model, and finally randomly performing interference processing on each positive and negative sample to obtain an expanded sample image; based on the expanded sample image and the standard sample image, a training data set is generated, which effectively overcomes the problems of insufficient sample quantity and high labeling cost, and is also conducive to improving the generalization ability of the model. The model can learn the characteristics of the same lesion under different noise levels, window width settings, and scanning angles.

[0119] Example 5:

[0120] Based on Example 1, step 4 includes:

[0121] Based on the MSSF module and C3K2_MSSF module, the main network and feature map network of the YOLOv12 network are constructed respectively, and the improved YOLOv12-3D network is generated by combining the detection head module;

[0122] The improved YOLOv12-3D network is trained based on the training dataset to obtain the target detection model;

[0123] Among them, the improved YOLOv12-3D network is as follows Figure 2 As shown, it includes: main network, feature map network and detection head module;

[0124] Among them, the main network of the YOLOv12 network is built based on the MSSF module and the C3K2_MSSF module, including:

[0125] Two 3D convolution downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input it into the first C3K2 module for high-level feature extraction to obtain the first feature map;

[0126] The first feature map is processed by the third 3D convolution downsampling module and then input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map;

[0127] After the second feature map is processed by the fourth 3D convolution downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map;

[0128] The third feature map is processed by the fifth 3D convolution downsampling module to obtain a feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network.

[0129] Among them, the feature map network of the YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including:

[0130] The construction of the first-layer feature fusion network is as follows:

[0131] The output feature map of the backbone network is input into the third C3K2_MSSF module for processing to obtain the fourth feature map;

[0132] After upsampling the fourth feature map and concatenating it with the third feature map output by the first-layer feature fusion network, it is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map;

[0133] Upsample the fifth feature map and concatenate it with the second feature map in the backbone network to obtain the highest resolution first-layer final concatenated fusion feature map.

[0134] The construction of the second-layer feature fusion network is as follows:

[0135] The final spliced ​​fusion feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fusion feature map;

[0136] The first fused feature map is processed by the sixth 3D convolution downsampling module and then spliced ​​with the fifth feature map output by the first layer network. The spliced ​​result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map;

[0137] The second fused feature map is processed by the seventh 3D convolution downsampling module and then spliced ​​with the fourth feature map output by the first layer network. The splicing result is input into the second C3K2 module for feature extraction to obtain the third fused feature map.

[0138] In this embodiment, the MSSF (Multi-Scale Spatial Semantic Fusion) module, such as Figure 3 As shown, it consists of a three-way parallel feature extraction network, which achieves efficient feature modeling through multi-scale feature fusion and multi-head attention mechanism:

[0139] The first path divides the feature map into 2×2×2 blocks, passing through the local-global attention submodule and the spatial attention submodule;

[0140] The second path uses 3D convolution to downsample and upsample the input tensor, preserving the joint spatial-channel information;

[0141] The third path divides the feature map into 4×4×4 blocks to extract fine-grained details using the same submodule as the first path;

[0142] The three-way output features are fused at multiple scales by element-by-element addition. The fusion result is then divided into 8×8×8 blocks, and the multi-head attention mechanism is used to mine long-range correlations between blocks.

[0143] Among them, MSFF contains two key sub-modules:

[0144] The LocalGlobalAttention submodule performs channel mean pooling on the block features, generates attention weights after MLP nonlinear transformation, combines position encoding to preserve spatial relationships, and finally reconstructs features through linear transformation;

[0145] The SpatialAttention submodule generates a spatial attention map by splicing the channel mean and maximum pooling, and multiplies it element-by-element with the original input to achieve adaptive enhancement of key areas.

[0146] The MSSF module preserves local details while improving semantic consistency through global interaction to form a hierarchical feature expression.

[0147] In this embodiment, the C3K2_MSSF module has a network structure similar to the C3K2 module in the YOLOv12 model, but integrates the MSSF module described above to improve the feature extraction effect. Figure 4 As shown, the specific design is as follows:

[0148] First, the input features are transformed by a 3D convolution kernel to obtain a preliminary feature map;

[0149] Subsequently, the feature map is divided into multiple sub-feature maps in the channel dimension, and each sub-feature map is input into an MSSF module for processing.

[0150] Finally, the sub-feature maps processed by the MSSF module are re-joined in the channel dimension, and feature fusion is performed through a Conv3D convolution layer to output the final feature map of this module.

[0151] By splitting features and processing them in parallel, the C3K2_MSSF module can capture multi-scale information and strengthen important features through the attention mechanism, thereby improving the detection robustness of the model.

[0152] In this embodiment, the feature maps of three scales generated by the feature map network (i.e., the first fused feature map, the second fused feature map, and the third fused feature map) are processed by the 3D convolution module and then input into the detection head module.

[0153] In this embodiment, the detection head module is responsible for using the final feature map to locate and classify the target, and the output includes the target position coordinates and confidence information;

[0154] The detection head module is divided into two parallel branches: position regression and category prediction:

[0155] Positioning branch: This branch consists of two standard convolutional modules (Conv3D+BatchNorm+SiLU) connected in series to extract positioning information, and is finally connected to a 3D convolutional layer to output the 3D spatial coordinates of the target;

[0156] Position localization branch: This branch sequentially contains three 3D depth-wise separable convolutional layers, two standard convolutional modules, and one 3D convolutional layer to extract category-related features and predict the category probability of each candidate box.

[0157] The beneficial effects of the above technical solution: The present invention constructs the main network and feature network of the improved YOLOv12-3D network through the MSSF (multi-scale spatial semantic fusion) module and the C3K2_MSSF module, so that the improved YOLOv12-3D network can simultaneously capture the detailed features of small targets (<5mm) and the global structural features of large targets, solving the problem of imbalance in multi-scale target modeling of traditional 3D models (such as 3DU-Net). The addition of the C3K2_MSSF module divides the feature map into multiple sub-feature maps in the channel dimension. Each sub-branch is processed independently by the MSSF module and then spliced ​​and fused. Different channel sub-branches can learn differentiated feature representations (such as one channel focusing on edge detection and another focusing on texture analysis), enriching the expressive power of the feature space. The backbone network is composed of a cascade of multiple downsampling and feature extraction modules, which can gradually reduce the spatial resolution and expand the receptive field to adapt to the input of CT three-dimensional data, forming a hierarchical feature pyramid from shallow details to deep semantics, adapting to the multi-scale characteristics of lung lesions; then based on the different scale feature maps output by the backbone network, a two-layer top-down feature fusion network is constructed to generate a feature map for multi-scale detection. Compared with the traditional FPN that only fuses at a single scale, this design achieves a dynamic balance between local details and global semantics through bidirectional cross-layer interaction (up-sampling + down-sampling combination), avoids the disappearance of small target features in the deep network, and effectively improves the efficiency and accuracy of small target detection. The present invention systematically solves the core problems of low sensitivity to small targets, insufficient feature modeling, and insufficient computational efficiency of traditional three-dimensional detection models through the combination of multi-scale feature fusion architecture and attention enhancement mechanism. It not only optimizes the network structure of the detection model, but also achieves a coordinated improvement in detection accuracy, reasoning speed, and generalization ability through hierarchical feature design, cross-layer interaction mechanism, and lightweight strategy, providing a high-performance technical solution for automated auxiliary diagnosis of lung CT images.

[0158] Example 6:

[0159] Based on Example 1, the CT image to be detected is input into the target detection model to obtain detection information of the patient's lung shadow, including:

[0160] The pre-processed CT image to be detected is input into the target detection module, and the three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module;

[0161] The detection head module analyzes the three-dimensional spatial features, determines potential targets, marks multiple potential targets based on candidate boxes, and outputs the three-dimensional bounding box parameters of the candidate boxes;

[0162] A three-dimensional non-maximum suppression algorithm is used to calculate the confidence level of each candidate box based on a preset threshold. The candidate box with the highest confidence level is selected as the target box, and overlapping detection results are eliminated.

[0163] According to the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size;

[0164] Based on the three-dimensional bounding box parameters and the real physical size, the detection information of the patient's lung shadow is generated.

[0165] In this embodiment, the 3D bounding box parameters of each candidate box include center point coordinates (x, y, z), axial dimensions (length, width, height), and a confidence score.

[0166] The beneficial effects of the above technical solution: The present invention uses the MSSF module and the C3K2_MSSF module to perform cross-layer feature extraction on the three-dimensional volume data of the CT image, which can simultaneously capture lung shadows of different sizes, effectively improving the extraction rate and efficiency of small targets. The detection head module predefines potential targets of various sizes through the three-dimensional anchor frame mechanism, and combines the three-dimensional non-maximum suppression (3D-NMS) algorithm to eliminate overlapping detection frames with a preset intersection-over-union (IoU) threshold to ensure that each real lesion corresponds to only one optimal detection frame, which can effectively avoid missing labels and multiple labeling problems, and effectively improve the detection efficiency of lung CT images; finally, based on the three-dimensional voxel size of the target frame of the pixel spacing parameter of the CT image, the marking frame is automatically converted to the actual physical size to realize the rapid and automatic determination of the location and size of the lung lesions, providing reliable auxiliary diagnosis results for the diagnosis of lung diseases.

[0167] Example 7:

[0168] Based on Example 1, a multi-scale spatial semantic fusion lung CT image processing method further includes:

[0169] After obtaining the test information, the target case or related test data of the patient is obtained simultaneously, and based on the type of lung disease corresponding to the test information, various related possible abnormal information of the patient's test data is predicted;

[0170] Retrieve relevant possible abnormal information from the target case or related test data, and obtain the abnormality degree corresponding to the relevant abnormal information;

[0171] Based on the abnormality and the size of the lesion location corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the verification degree is sent to the patient's attending physician as diagnostic auxiliary information.

[0172] The beneficial effects of the above technical solution: After obtaining the lung shadow detection information, the present invention simultaneously collects the patient's target case and related test data (such as blood test indicators, lung function test results, past imaging data, etc.) to provide a basis for the integration of multi-dimensional data, and predicts a variety of possible abnormal information in the patient's test data based on the lung disease type determined by the test information. For example, when a malignant tumor shadow is detected in the lung, the system can predict that the patient may have abnormal indicators related to systemic symptoms such as anemia and hypoproteinemia, as well as abnormalities in other organ function indicators that may be caused by tumor metastasis, and can detect potential risks in advance. Afterwards, the predicted possible abnormal information is retrieved from the target case and related test data, and the abnormality degree is calculated, combined with the lesion location in the test information. and size, to quantitatively assess the severity of the patient's current lung disease, measure abnormal conditions with specific numerical values, and make the disease assessment more objective and accurate. For example, for patients with lung tumors, not only the size and location of the tumor are taken into consideration, but also the severity score is calculated comprehensively based on factors such as the degree of increase in tumor markers in the blood and the damage to lung function, providing doctors with an accurate reference for formulating personalized treatment plans and avoiding under- or over-treatment. At the same time, the predicted disease severity is sent to the attending physician in real time as diagnostic auxiliary information, assisting the doctor in quickly obtaining key information and helping the attending physician to capture key information in a timely and accurate manner during busy clinical work, which helps the doctor to make diagnostic decisions quickly, formulate treatment plans, and shorten the waiting time for patients to be diagnosed and treated.

[0173] Example 8:

[0174] The present invention provides a multi-scale spatial semantic fusion lung CT image processing system, such as Figure 5 Shown, including:

[0175] The data preparation and quality control module is used to collect various lung CT-DICOM images, perform quality inspection and format conversion, and obtain original sample images;

[0176] Image preprocessing and lung parenchyma segmentation module, used to preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image;

[0177] The training data set construction module is used to determine positive and negative samples based on the image annotation results, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and standard sample images, a training data set is generated;

[0178] The target detection model construction module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and then train the improved YOLOv12-3D network using the training dataset to obtain the target detection model;

[0179] The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain detection information of the patient's lung shadow.

[0180] The beneficial effects of the above technical solution: The present invention collects a variety of lung CT-DICOM images and performs quality inspection and format conversion to ensure that the original samples cover image data under different devices and scenarios, thereby reducing model deviations caused by single data; at the same time, the difficulty of subsequent processing is reduced by unifying the format, and the GPU pressure is reduced; thereafter, the original samples are preprocessed and the lung parenchyma is segmented to remove irrelevant tissue interference and standardize image features so that the model focuses on the effective area, effectively improving the detection targeting and accuracy, avoiding false detection caused by the influence of extrapulmonary structures, and accurately determining positive and negative samples based on the annotation results. By expanding the data through random interference, not only the number of samples is increased, but also the simulation of complex clinical situations can be achieved, so that the model can learn more diverse feature patterns, which is conducive to alleviating overfitting. The problem is solved by improving the generalization ability of targets of different shapes and positions. Then, the YOLOv12 network is improved based on the MSSF module and the C3K2_MSSF module. The multi-scale spatial semantic fusion and attention mechanism are used to deeply extract three-dimensional spatial features, effectively solving the problems of insufficient three-dimensional modeling and low sensitivity to small targets in traditional methods. In combination with high-quality training data sets, the improved YOLOv12-3D network can fully learn target features during training and achieve high-precision detection. Finally, the CT image to be detected is input into the trained target detection model, which can quickly output lung shadow detection information. Compared with traditional methods, the detection time is greatly shortened, and the small target detection ability is improved at the same time, which helps to provide accurate auxiliary diagnosis for early diseases and improve medical efficiency and diagnostic accuracy. The present invention improves the YOLOv12-3D network architecture, realizes the deep fusion of three-dimensional spatial features and multi-scale spatial semantic features in the feature extraction stage, constructs an end-to-end target detection model, and does not require manual design of features or complex post-processing steps. It can directly output target positioning and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0181] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A lung CT image processing method based on multi-scale spatial semantic fusion, characterized in that: include: Step 1: Collect multiple lung CT-DICOM images, perform quality inspection and format conversion to obtain original sample images; Step 2: Preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image; Step 3: According to the image annotation results, determine the positive and negative samples, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and the standard sample images, generate a training data set; Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and train the improved YOLOv12-3D network using the training dataset to obtain the target detection model; Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow.

2. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 1, characterized in that: Step 1 includes: Anonymize the collected lung CT-DICOM images and perform quality inspection on the processed lung CT-DICOM data; For lung CT-DICOM images that have passed the quality inspection, the dcm2niix tool is used to perform data format conversion to obtain the original sample images; Based on the preset lung disease database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images cover all lung disease types; If not, proceed with lung CT-DICOM image acquisition; If yes, the lung CT-DICOM image acquisition is terminated.

3. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 1, characterized in that: Step 2 includes: Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample image with an original pixel spacing greater than 1 mm to obtain a new sample image; The new image and the sample image with the original pixel spacing less than or equal to 1 mm are denoised using the 3D non-local mean filtering algorithm to obtain denoised sample data; Based on the preset CT value, the denoised sample data is binarized to obtain binary sample data; Perform pixel-by-pixel AND operation on the binarized sample data and its corresponding original sample image to obtain a lung parenchyma image; The lung region is cropped from the lung parenchymal image to obtain a lung ROI image, and the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

4. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 1, characterized in that: Step 3 includes: Based on the pre-trained nnU-Net target detection model, standard sample images are marked with candidate boxes to obtain marked sample images, and the marking results are sent to experts for review; After the expert review is completed, the expert marked target box is compared with the candidate box to obtain the marked box intersection and union ratio; The labeled sample images whose marked box intersection-union ratio is greater than or equal to the preset threshold are regarded as positive samples; The labeled sample images whose marked box intersection-union ratio is less than the preset threshold are regarded as negative samples; Randomly perform interference processing on each positive and negative sample to obtain the expanded sample image; A training dataset is generated based on the expanded sample images and the standard sample images.

5. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 1, characterized in that: Step 4 includes: Based on the MSSF module and C3K2_MSSF module, the main network and feature map network of the YOLOv12 network are constructed respectively, and the improved YOLOv12-3D network is generated by combining the detection head module; The improved YOLOv12-3D network is trained based on the training dataset to obtain the target detection model.

6. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 5, characterized in that: The main network of the YOLOv12 network is built based on the MSSF module and the C3K2_MSSF module, including: Two 3D convolution downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input it into the first C3K2 module for high-level feature extraction to obtain the first feature map; The first feature map is processed by the third 3D convolution downsampling module and then input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map; After the second feature map is processed by the fourth 3D convolution downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map; The third feature map is processed by the fifth 3D convolution downsampling module to obtain a feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network.

7. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 5, characterized in that: The feature map network of the YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including: The construction of the first-layer feature fusion network is as follows: The output feature map of the backbone network is input into the third C3K2_MSSF module for processing to obtain the fourth feature map; After upsampling the fourth feature map and concatenating it with the third feature map output by the first-layer feature fusion network, it is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map; Upsample the fifth feature map and concatenate it with the second feature map in the backbone network to obtain the highest resolution first-layer final concatenated fusion feature map. The construction of the second-layer feature fusion network is as follows: The final spliced ​​fusion feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fusion feature map; The first fused feature map is processed by the sixth 3D convolution downsampling module and then spliced ​​with the fifth feature map output by the first layer network. The spliced ​​result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map; The second fused feature map is processed by the seventh 3D convolution downsampling module and then spliced ​​with the fourth feature map output by the first layer network. The splicing result is input into the second C3K2 module for feature extraction to obtain the third fused feature map.

8. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 5, characterized in that: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow, including: The pre-processed CT image to be detected is input into the target detection module, and the three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module; The detection head module analyzes the three-dimensional spatial features, determines potential targets, marks multiple potential targets based on candidate boxes, and outputs the three-dimensional bounding box parameters of the candidate boxes; A three-dimensional non-maximum suppression algorithm is used to calculate the confidence level of each candidate box based on a preset threshold. The candidate box with the highest confidence level is selected as the target box, and overlapping detection results are eliminated. According to the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size; Based on the three-dimensional bounding box parameters and the real physical size, the detection information of the patient's lung shadow is generated.

9. The lung CT image processing method of multi-scale spatial semantic fusion according to claim 5, characterized in that: Also includes: After obtaining the test information, the target case or related test data of the patient is obtained simultaneously, and based on the type of lung disease corresponding to the test information, various related possible abnormal information of the patient's test data is predicted; Retrieve relevant possible abnormal information from the target case or related test data, and obtain the abnormality degree corresponding to the relevant abnormal information; Based on the abnormality and the size of the lesion location corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the verification degree is sent to the patient's attending physician as diagnostic auxiliary information.

10. A multi-scale spatial semantic fusion lung CT image processing system, characterized in that: include: The data preparation and quality control module is used to collect various lung CT-DICOM images, perform quality inspection and format conversion, and obtain original sample images; Image preprocessing and lung parenchyma segmentation module, used to preprocess the original sample image and segment the lung parenchyma to obtain a standard sample image; The training data set construction module is used to determine positive and negative samples based on the image annotation results, and randomly interfere with the positive and negative samples to obtain expanded sample images. Based on the expanded sample images and standard sample images, a training data set is generated; The target detection model construction module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and then train the improved YOLOv12-3D network using the training dataset to obtain the target detection model; The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain detection information of the patient's lung shadow.

Images