Multi-scale spatial semantic fusion lung CT image processing method and system

Abstract

The application provides a lung CT image processing method and system based on multi-scale space semantic fusion, which comprises the following steps: collecting multiple lung CT-DICOM images, and performing quality inspection processing and format conversion to obtain original sample images; pre-processing and lung parenchyma segmentation are performed on the original sample images to obtain standard sample images; positive and negative samples are determined according to the image labeling results, and the positive and negative samples are subjected to random interference to obtain expanded sample images; a training data set is generated based on the expanded sample images and the standard sample images; the YOLOv12 network is improved based on the MSSF module and the C3K2_MSSF module to obtain an improved YOLOv12-3D network, and the improved YOLOv12-3D network is trained by using the training data set to obtain a target detection model; and the CT image to be detected is input into the target detection model to obtain detection information. The improved YOLOv12-3D network architecture realizes deep fusion of three-dimensional space features and multi-scale space semantic features in the feature extraction stage, and significantly improves the extraction rate and efficiency of small targets.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for processing lung CT images using multi-scale spatial semantic fusion. Background Technology

[0002] In the field of medical image analysis, while deep learning-based CT image detection technology has made significant progress, it still faces two fundamental challenges: insufficient 3D feature modeling and inadequate sensitivity in detecting small targets. Traditional 2D convolutional networks, limited by planar processing mechanisms, struggle to effectively capture the spatiotemporal dependencies between layers in CT sequences. While 3D convolutional models, such as 3DU-Net, can integrate spatial contextual information, they suffer from high false negative rates when dealing with lesions smaller than 5mm due to insufficient texture feature resolution. Furthermore, the differences in slice thickness in multi-center CT data, equipment noise interference, and the uncontrollability of manual post-processing further exacerbate the bottlenecks in generalization ability and computational efficiency of algorithms, making it difficult to meet the urgent clinical demand for high-precision, real-time diagnosis.

[0003] While current mainstream Feature Pyramid Networks (FPNs) achieve multi-scale feature fusion, they inherently suffer from deficiencies in balancing local details and global semantic information, particularly in lesion regions with complex textures, easily leading to feature misjudgment and misclassification. Directly transferring two-dimensional detection frameworks (such as the YOLO series) to three-dimensional scenes results in numerous false positives due to limited receptive fields and inefficient cross-layer feature interactions. Furthermore, the fragmented design of data preprocessing and model training not only increases computational resource consumption but also severely hinders the translation of algorithms from laboratory to clinical applications. Therefore, a new multi-scale spatial semantic fusion method and system for lung CT image processing is urgently needed to address these issues. Summary of the Invention

[0004] This invention provides a lung CT image processing method and system based on multi-scale spatial semantic fusion, in order to solve problems such as insufficient three-dimensional feature modeling, low efficiency of small target detection, and poor algorithm generalization in the prior art.

[0005] This invention provides a method for processing lung CT images using multi-scale spatial semantic fusion, comprising:

[0006] Step 1: Acquire various lung CT-DICOM images, perform quality control processing and format conversion to obtain raw sample images;

[0007] Step 2: Preprocess the original sample images and segment the lung parenchyma to obtain standard sample images;

[0008] Step 3: Based on the image annotation results, determine the positive and negative samples, and randomly perturb the positive and negative samples to obtain augmented sample images. Based on the augmented sample images and the standard sample images, generate the training dataset.

[0009] Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and train the improved YOLOv12-3D network with the training dataset to obtain the object detection model;

[0010] Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow.

[0011] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 1 includes:

[0012] Anonymize the acquired lung CT-DICOM images and perform quality checks on the processed lung CT-DICOM data;

[0013] For lung CT-DICOM images that have passed quality inspection, the dcm2niix tool is used to perform data formatting and conversion to obtain the original sample images;

[0014] Based on a pre-set lung lesion database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images encompass all lung disease types.

[0015] If not, continue with lung CT-DICOM image acquisition;

[0016] If so, then terminate the lung CT-DICOM image acquisition.

[0017] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 2 includes:

[0018] Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample images with original pixel spacing greater than 1mm to obtain new sample images;

[0019] The new image and the original sample image with a pixel pitch of less than or equal to 1 mm are denoised using a 3D nonlocal mean filtering algorithm to obtain denoised sample data.

[0020] Based on the preset CT value, the denoised sample data is binarized to obtain binarized sample data.

[0021] The lung parenchyma image is obtained by performing a pixel-by-pixel AND operation between the binarized sample data and its corresponding original sample image.

[0022] The lung region is cropped from the lung parenchyma image to obtain the lung ROI image, and then the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

[0023] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 3 includes:

[0024] Based on the pre-trained nnU-Net object detection model, candidate boxes are used to label standard sample images to obtain labeled sample images, and the labeling results are sent to experts for review.

[0025] Once the experts have completed their review, the expert-marked target boxes are compared with the candidate boxes to obtain the intersection-union ratio of the marked boxes;

[0026] Image samples with a cross-union ratio (CUC) greater than or equal to a preset threshold are considered positive samples.

[0027] Image samples with a cross-union ratio (CUC) of less than a preset threshold are designated as negative samples.

[0028] Each positive and negative sample is randomly subjected to interference processing to obtain an augmented sample image;

[0029] A training dataset is generated based on the augmented sample images and the standard sample images.

[0030] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, step 4 includes:

[0031] The main network and feature map network of YOLOv12 are constructed based on the MSSF module and C3K2_MSSF module, respectively, and an improved YOLOv12-3D network is generated by combining the detection head module.

[0032] An object detection model is obtained by training the improved YOLOv12-3D network using a training dataset. Preferably, in a multi-scale spatial semantic fusion method for lung CT image processing, the main network of the YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including:

[0033] Two 3D convolutional downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input into the first C3K2 module for high-level feature extraction to obtain the first feature map;

[0034] After processing the first feature map through the third 3D convolutional downsampling module, it is input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map;

[0035] After processing the second feature map through the fourth 3D convolutional downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map;

[0036] The third feature map is processed by the fifth 3D convolutional downsampling module to obtain the feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network.

[0037] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, a feature map network of the YOLOv12 network is constructed based on the MSSF module and the C3K2_MSSF module, including:

[0038] The construction of the first-layer feature fusion network is as follows:

[0039] The output feature map of the backbone network is processed by the third C3K2_MSSF module to obtain the fourth feature map;

[0040] After upsampling the fourth feature map and concatenating it with the third feature map output by the first layer feature fusion network, the result is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map.

[0041] The fifth feature map is upsampled and then concatenated with the second feature map in the backbone network to obtain the highest resolution first layer final concatenated and fused feature map;

[0042] The construction of the second-layer feature fusion network is as follows:

[0043] The final stitched and fused feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fused feature map.

[0044] After processing the first fused feature map through the sixth 3D convolutional downsampling module, it is concatenated with the fifth feature map output by the first layer network, and the concatenation result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map.

[0045] The second fused feature map is processed by the seventh 3D convolutional downsampling module and then concatenated with the fourth feature map output by the first layer network. The concatenation result is then input into the second C3K2 module for feature extraction to obtain the third fused feature map.

[0046] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, the CT image to be detected is input into a target detection model to obtain detection information of the patient's lung shadows, including:

[0047] The preprocessed CT image to be detected is input into the target detection module, and three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module.

[0048] The detection head module analyzes the 3D spatial features, identifies potential targets, and marks multiple potential targets based on candidate boxes, outputting the 3D bounding box parameters of the candidate boxes.

[0049] A three-dimensional non-maximum suppression algorithm is adopted. Based on a preset threshold, the confidence level of each candidate box is calculated. The candidate box with the highest confidence level is taken as the target box, and overlapping detection results are eliminated.

[0050] Based on the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size.

[0051] Based on the 3D bounding box parameters and the actual physical dimensions, detection information of the patient's lung shadow is generated.

[0052] Preferably, in a multi-scale spatial semantic fusion lung CT image processing method, the method further includes:

[0053] After obtaining the test information, the patient's target case or related test data are obtained simultaneously. Based on the lung disease type corresponding to the test information, various possible abnormal information related to the patient's test data is predicted.

[0054] Retrieve relevant potential abnormal information from the target case or related test data, and obtain the degree of abnormality corresponding to the relevant abnormal information;

[0055] Based on the anomaly level and the size of the lesion corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the verification level is sent to the patient's attending physician as diagnostic auxiliary information.

[0056] This invention provides a multi-scale spatial semantic fusion lung CT image processing system, comprising:

[0057] The data preparation and quality control module is used to acquire various lung CT-DICOM images, perform quality control processing and format conversion, and obtain raw sample images.

[0058] The image preprocessing and lung parenchyma segmentation module is used to preprocess the original sample images and segment the lung parenchyma to obtain standard sample images.

[0059] The training dataset construction module is used to determine positive and negative samples based on the image annotation results, randomly perturb the positive and negative samples to obtain augmented sample images, and generate the training dataset based on the augmented sample images and the standard sample images.

[0060] The object detection model building module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and to train the improved YOLOv12-3D network with the training dataset to obtain the object detection model;

[0061] The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadows.

[0062] Compared with the prior art, the present invention has at least the following beneficial effects:

[0063] This invention acquires and performs quality control and format conversion on various lung CT-DICOM images to ensure that the original samples cover image data from different devices and scenarios, reducing model bias caused by limited data. Simultaneously, format standardization reduces the difficulty of subsequent processing and lowers GPU load. Subsequently, the original samples undergo preprocessing and lung parenchyma segmentation to remove irrelevant tissue interference and standardize image features, allowing the model to focus on effective regions, effectively improving detection specificity and accuracy, avoiding false detections caused by extra-lung structures, and accurately determining positive and negative samples based on annotation results. Data expansion through random interference not only increases the number of samples but also enables the simulation of complex clinical situations, allowing the model to learn more diverse feature patterns, which helps alleviate overfitting and enhances the model's ability to... This invention improves the YOLOv12 network by enhancing its generalization ability for targets of different shapes and locations. Based on the MSSF and C3K2_MSSF modules, it enhances the YOLOv12 network by utilizing multi-scale spatial semantic fusion and attention mechanisms to deeply extract 3D spatial features. This effectively addresses the shortcomings of traditional methods, such as insufficient 3D modeling and low sensitivity to small targets. Combined with a high-quality training dataset, the improved YOLOv12-3D network learns target features extensively during training, achieving high-precision detection. Finally, by inputting the CT image to be detected into the trained target detection model, it can quickly output lung shadow detection information, significantly shortening the detection time compared to traditional methods. This also improves the detection capability for small targets, contributing to accurate auxiliary diagnosis of early-stage diseases and enhancing medical efficiency and diagnostic accuracy. This invention improves the YOLOv12-3D network architecture by achieving deep fusion of 3D spatial features and multi-scale spatial semantic features during the feature extraction stage, constructing an end-to-end target detection model. It eliminates the need for manual feature design or complex post-processing steps, directly outputting target localization and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0064] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0065] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 A flowchart of a lung CT image processing method involving multi-scale spatial semantic fusion;

[0068] Figure 2 To improve the overall architecture of the YOLOv12-3D network;

[0069] Figure 3 This is a diagram of the MSSF module architecture;

[0070] Figure 4 The architecture diagram for the C3K2_MSSF module;

[0071] Figure 5 This is a structural diagram of a lung CT image processing system that incorporates multi-scale spatial semantic fusion. Detailed Implementation

[0072] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0073] This invention provides a method for processing lung CT images using multi-scale spatial semantic fusion, such as... Figure 1 As shown, it includes:

[0074] Step 1: Acquire various lung CT-DICOM images, perform quality control processing and format conversion to obtain raw sample images;

[0075] Step 2: Preprocess the original sample images and segment the lung parenchyma to obtain standard sample images;

[0076] Step 3: Based on the image annotation results, determine the positive and negative samples, and randomly perturb the positive and negative samples to obtain augmented sample images. Based on the augmented sample images and the standard sample images, generate the training dataset.

[0077] Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and train the improved YOLOv12-3D network with the training dataset to obtain the object detection model;

[0078] Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadow.

[0079] In this embodiment, a weighted combination of multiple losses was used to address the regression and classification requirements of the detection task during the improved YOLOv12-3D network training process:

[0080] The bounding box (i.e., candidate box, target box, etc.) regression loss uses the CIoU loss function to comprehensively measure the overlap between the predicted box and the ground truth box, the center distance, and the difference in aspect ratio.

[0081] The target category classification loss uses focus loss to enhance the learning ability for hard-to-detect targets and alleviate sample class imbalance.

[0082] The total loss is a weighted sum of the two losses mentioned above, in order to simultaneously optimize location regression, size prediction, and category discrimination.

[0083] The Adam optimization algorithm is used to update network parameters during the training of the improved YOLOv12-3D network. The Adam optimizer can adaptively adjust the learning rate, accelerate convergence, and improve training stability.

[0084] The beneficial effects of the above technical solution are as follows: This invention acquires multiple lung CT-DICOM images and performs quality inspection and format conversion to ensure that the original samples cover image data from different devices and scenarios, reducing model bias caused by single data sets. Simultaneously, format standardization reduces the difficulty of subsequent processing and lowers GPU load. Subsequently, the original samples undergo preprocessing and lung parenchyma segmentation to remove irrelevant tissue interference and standardize image features, enabling the model to focus on effective regions, effectively improving detection specificity and accuracy, avoiding false detections caused by extra-lung structures, and accurately determining positive and negative samples based on annotation results. Data expansion through random interference not only increases the number of samples but also enables the simulation of complex clinical situations, allowing the model to learn more diverse feature patterns, which helps alleviate overfitting. This invention addresses the problem of improving the generalization ability for targets of different shapes and locations. It then improves the YOLOv12 network based on the MSSF and C3K2_MSSF modules, utilizing multi-scale spatial semantic fusion and attention mechanisms to deeply extract 3D spatial features. This effectively solves the problems of insufficient 3D modeling and low sensitivity to small targets in traditional methods. Combined with a high-quality training dataset, the improved YOLOv12-3D network fully learns target features during training, achieving high-precision detection. Finally, the CT image to be detected is input into the trained target detection model, which can quickly output lung shadow detection information. Compared with traditional methods, this significantly shortens the detection time and improves the detection capability of small targets, contributing to accurate auxiliary diagnosis of early diseases and improving medical efficiency and diagnostic accuracy. This invention improves the YOLOv12-3D network architecture by achieving deep fusion of 3D spatial features and multi-scale spatial semantic features in the feature extraction stage, constructing an end-to-end target detection model. It eliminates the need for manual feature design or complex post-processing steps, directly outputting target localization and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0085] Example 2:

[0086] Based on Example 1, step 1 includes:

[0087] Anonymize the acquired lung CT-DICOM images and perform quality checks on the processed lung CT-DICOM data;

[0088] For lung CT-DICOM images that have passed quality inspection, the dcm2niix tool is used to perform data formatting and conversion to obtain the original sample images;

[0089] Based on a pre-set lung lesion database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images encompass all lung disease types.

[0090] If not, continue with lung CT-DICOM image acquisition;

[0091] If so, then terminate the lung CT-DICOM image acquisition.

[0092] In this embodiment, the number of lung CT-DICOM data is greater than 1,000, including lung CT-DICOM sequences from multiple centers and multiple vendors.

[0093] In this embodiment, the converted image format is NIfTI format.

[0094] The beneficial effects of the above technical solution are as follows: This invention anonymizes CT-DICOM images, removing privacy fields such as patient name, ID, and medical history, retaining only key diagnostic parameters (such as scan time and slice thickness). This effectively avoids the risk of medical data leakage, ensuring the compliance of subsequent research and clinical applications. Furthermore, the anonymized CT data undergoes quality testing, screening for issues such as missing slices, slice misalignment, and respiratory motion artifacts, and low-quality data is marked. This ensures the continuity and integrity of the CT sequences input into the training model, preventing low-quality data from contaminating the training set. This effectively reduces the probability of false detections due to noise or artifacts, improving the reliability of subsequent 3D feature modeling. The dcm2niix tool is used to convert the DICOM format to NIfTI format, overcoming the limitations caused by differences in DICOM file headers (different...). To address compatibility issues caused by differences in manufacturer device metadata structures, this method reduces the complexity of data preprocessing, effectively improves computational efficiency (e.g., reducing GPU data loading time), and, based on a pre-defined database (containing common disease types such as pneumonia, pulmonary nodules, lung cancer, and emphysema), detects whether the original samples cover all target disease types. This ensures that the training set contains CT features of various lung lesions (such as ground-glass opacities, solid nodules, and cavitary lesions) to the greatest extent possible, preventing the model from missing rare lesion types due to "limited disease exposure." Furthermore, comprehensive disease type coverage allows the model to learn diverse pathological features, which is beneficial for improving the YOLOv12-3D network's adaptability to complex diagnostic needs in clinical practice and reducing overfitting issues caused by a single data distribution in the improved YOLOv12-3D network.

[0095] Example 3:

[0096] Based on Example 1, step 2 includes:

[0097] Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample images with original pixel spacing greater than 1mm to obtain new sample images;

[0098] The new image and the original sample image with a pixel pitch of less than or equal to 1 mm are denoised using a 3D nonlocal mean filtering algorithm to obtain denoised sample data.

[0099] Based on the preset CT value, the denoised sample data is binarized to obtain binarized sample data.

[0100] The lung parenchyma image is obtained by performing a pixel-by-pixel AND operation between the binarized sample data and its corresponding original sample image.

[0101] The lung region is cropped from the lung parenchyma image to obtain the lung ROI image, and then the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

[0102] In this embodiment, B-spline interpolation is used to resample the original sample image with an original pixel spacing greater than 1mm to 1×1×1mm. 3 Isotropic space effectively eliminates 3D convolution bias caused by anisotropy.

[0103] In this embodiment, the preset CT value can be set flexibly, with the default value being -400HU.

[0104] During the binarization of lung CT slices, lung region candidates are extracted, and the two largest connected domains, the left and right lungs, are retained. A morphological closing operation is used to repair small holes in the lungs using a 3×3×3 structure kernel. The jagged boundaries of the mask are eliminated by using an edge-preserving smoothing algorithm.

[0105] In this embodiment, the standard sample image size is 256×256×128 voxels by default, but can be flexibly set.

[0106] In this embodiment, before cropping the lung region of the lung parenchyma image, the method further includes: calculating a three-dimensional bounding box based on the segmentation mask of the lung and expanding it outward by a 10voxel tolerance boundary.

[0107] The beneficial effects of the above technical solution are as follows: For data with an original pixel pitch > 1 mm, the present invention uses B-spline interpolation to resample to 1×1×1 mm. 3Isotropic space ensures that each voxel represents the same physical size in 3D space, guaranteeing consistent receptive fields of the 3D convolutional kernel along the x / y / z axes. This effectively improves the modeling accuracy of inter-layer feature associations. For images with a slice thickness >1mm, resampling can virtually increase the number of slices, preventing small targets (such as nodules <5mm) from being missed due to excessive inter-slice spacing (for example, in a 3mm slice image, a 5mm nodule might only be distributed across 2 slices, but after resampling to 1mm, it can be distributed across 5 slices, facilitating the capture of continuous 3D features). Furthermore, a 3D nonlocal mean filtering algorithm is used to denoise new images and sample images with original pixel spacing less than or equal to 1mm, obtaining denoised sample data to reduce noise interference in CT images. This is beneficial for improving the recognition of details in CT images, especially for small targets. The identification of markers (such as ground-glass opacities, micronodules, etc.) is performed. Then, based on preset CT values, the denoised sample data is binarized to obtain binarized sample data. The binarized sample data is then ANDed pixel by pixel with its corresponding original sample image to obtain a lung parenchyma image. Only the lung parenchyma region is retained, so that subsequent model training focuses on lung lesion detection, which can effectively reduce the false positive rate of the trained model (such as avoiding misclassification of rib calcification as nodules). The lung parenchyma image is cropped to obtain a lung ROI image. Then, the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image. This ensures that the images in the training set have a uniform size, which can effectively avoid network structure adjustments caused by changes in input dimensions, thereby improving the stability of training and inference.

[0108] Example 4:

[0109] Based on Example 1, step 3 includes:

[0110] Based on the pre-trained nnU-Net object detection model, candidate boxes are used to label standard sample images to obtain labeled sample images, and the labeling results are sent to experts for review.

[0111] Once the experts have completed their review, the expert-marked target boxes are compared with the candidate boxes to obtain the intersection-union ratio of the marked boxes;

[0112] Image samples with a cross-union ratio (CUC) greater than or equal to a preset threshold are considered positive samples.

[0113] Image samples with a cross-union ratio (CUC) of less than a preset threshold are designated as negative samples.

[0114] Each positive and negative sample is randomly subjected to interference processing to obtain an augmented sample image;

[0115] A training dataset is generated based on the augmented sample images and the standard sample images.

[0116] In this embodiment, the interference processing includes elastic deformation (σ=10, α=20), rotation (±15°), and mirror flipping; local pixel value perturbation (±15% window width / window level offset) is added, and CT simulation noise (Poisson-Gaussian hybrid model) is randomly added. Positive and negative samples may be subject to one type of interference processing or multiple types of interference processing.

[0117] In this embodiment, the preset threshold value ranges from (0, 1), and the default value is 0.2.

[0118] The beneficial effects of the above technical solution are as follows: This invention uses a pre-trained nnU-Net model to automatically generate candidate boxes for standard sample images, achieving rapid labeling of lung lesion targets (such as nodules and inflammatory areas). The labeling results generated by the model are then sent to experts for review. Experts manually adjust the position and size of the boxes or delete false detection boxes, which avoids the systematic errors of purely automated labeling and ensures that positive samples are accurately labeled as true targets. After the expert review is completed, the expert-labeled target boxes are compared with the candidate boxes, and then the positive and negative samples are determined according to the intersection-union ratio of the labeled boxes. This avoids the model learning confusion caused by blurred negative samples in traditional methods and effectively improves the discrimination ability of the subsequently trained model. Finally, each positive and negative sample is randomly subjected to interference processing to obtain expanded sample images. Based on the expanded sample images and standard sample images, a training dataset is generated, which effectively overcomes the problems of insufficient sample quantity and high labeling cost. At the same time, it is also conducive to improving the model's generalization ability. The model can learn the features of the same lesion under different noise levels, window width settings, and scanning angles.

[0119] Example 5:

[0120] Based on Example 1, step 4 includes:

[0121] The main network and feature map network of YOLOv12 are constructed based on the MSSF module and C3K2_MSSF module, respectively, and an improved YOLOv12-3D network is generated by combining the detection head module.

[0122] An object detection model was obtained by training the improved YOLOv12-3D network based on the training dataset;

[0123] Among them, improvements to the YOLOv12-3D network, such as Figure 2 As shown, it includes: a main network, a feature map network, and a detection head module;

[0124] The main network for building the YOLOv12 network, based on the MSSF module and the C3K2_MSSF module, includes:

[0125] Two 3D convolutional downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input into the first C3K2 module for high-level feature extraction to obtain the first feature map;

[0126] After processing the first feature map through the third 3D convolutional downsampling module, it is input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map;

[0127] After processing the second feature map through the fourth 3D convolutional downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map;

[0128] The third feature map is processed by the fifth 3D convolutional downsampling module to obtain the feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network.

[0129] The feature map network of the YOLOv12 network, constructed based on the MSSF module and the C3K2_MSSF module, includes:

[0130] The construction of the first-layer feature fusion network is as follows:

[0131] The output feature map of the backbone network is processed by the third C3K2_MSSF module to obtain the fourth feature map;

[0132] After upsampling the fourth feature map and concatenating it with the third feature map output by the first layer feature fusion network, the result is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map.

[0133] The fifth feature map is upsampled and then concatenated with the second feature map in the backbone network to obtain the highest resolution first layer final concatenated and fused feature map;

[0134] The construction of the second-layer feature fusion network is as follows:

[0135] The final stitched and fused feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fused feature map.

[0136] After processing the first fused feature map through the sixth 3D convolutional downsampling module, it is concatenated with the fifth feature map output by the first layer network, and the concatenation result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map.

[0137] The second fused feature map is processed by the seventh 3D convolutional downsampling module and then concatenated with the fourth feature map output by the first layer network. The concatenation result is then input into the second C3K2 module for feature extraction to obtain the third fused feature map.

[0138] In this embodiment, the MSSF (Multi-Scale Spatial Semantic Fusion) module, such as Figure 3 As shown, it consists of a three-way parallel feature extraction network, which achieves efficient feature modeling through multi-scale feature fusion and multi-head attention mechanism:

[0139] The first approach divides the feature map into 2×2×2 blocks, and uses the LocalGlobalAttention submodule and the SpatialAttention submodule for local attention;

[0140] The second path uses 3D convolution to downsample and upsample the input tensor, preserving the spatial-channel joint information;

[0141] The third approach divides the feature map into 4×4×4 blocks to extract fine-grained details using the same sub-modules as the first approach.

[0142] The three output features are added element by element to achieve multi-scale information fusion. Then the fusion result is divided into 8×8×8 blocks, and the long-range correlation between blocks is mined by multi-head attention mechanism.

[0143] MSFF contains two key sub-modules:

[0144] The LocalGlobalAttention submodule performs channel mean pooling on the block features, generates attention weights after MLP nonlinear transformation, preserves spatial relationships by combining position encoding, and finally reconstructs features through linear transformation.

[0145] The SpatialAttention submodule generates a spatial attention map by concatenating the channel mean and maximum pooling values, and then multiplies it element-wise with the original input to achieve adaptive enhancement of key regions.

[0146] While preserving local details, the MSSF module enhances semantic consistency through global interaction, forming a hierarchical feature representation.

[0147] In this embodiment, the C3K2_MSSF module has a network structure similar to the C3K2 module in the YOLOv12 model, but integrates the MSSF module described above to improve feature extraction performance, such as... Figure 4 As shown, the specific design is as follows:

[0148] First, the input features are transformed by a 3D convolution kernel to obtain a preliminary feature map;

[0149] Subsequently, the feature map is divided into multiple sub-feature maps along the channel dimension, and each sub-feature map is input into an MSSF module for processing.

[0150] Finally, the sub-feature maps processed by the MSSF module are reassembled along the channel dimension and fused through a Conv3D convolutional layer to output the final feature map of this module.

[0151] By segmenting and processing features in parallel, the C3K2_MSSF module can capture multi-scale information and enhance important features through an attention mechanism, thereby improving the model's detection robustness.

[0152] In this embodiment, the feature maps at three scales (i.e., the first fused feature map, the second fused feature map, and the third fused feature map) generated by the feature map network are processed by the 3D convolution module and then input into the detection head module.

[0153] In this embodiment, the detection head module is responsible for using the final feature map for target localization and classification, and outputting target location coordinates and confidence information.

[0154] The detection head module is divided into two parallel branches: location regression and category prediction.

[0155] Location branch: This branch consists of two standard convolutional modules (Conv3D+BatchNorm+SiLU) connected in series to extract location information, and finally a 3D convolutional layer to output the three-dimensional spatial coordinates of the target.

[0156] Location branch: This branch contains three 3D depthwise separable convolutional layers, two standard convolutional modules, and one 3D convolutional layer, which are used to extract class-related features and predict the class probability of each candidate box.

[0157] The beneficial effects of the above technical solution are as follows: This invention constructs the main network and feature network of the improved YOLOv12-3D network through the MSSF (Multi-Scale Spatial Semantic Fusion) module and the C3K2_MSSF module, enabling the improved YOLOv12-3D network to simultaneously capture the detailed features of small targets (<5mm) and the global structural features of large targets, solving the problem of imbalance in multi-scale target modeling of traditional 3D models (such as 3DU-Net). Furthermore, the addition of the C3K2_MSSF module divides the feature map into multiple sub-feature maps in the channel dimension. Each sub-branch is processed independently by the MSSF module and then spliced ​​and fused. Different channel sub-branches can learn differentiated feature representations (e.g., some channels focus on edge detection, while others focus on texture analysis), enriching the expressive power of the feature space. The backbone network consists of multiple cascaded downsampling and feature extraction modules, which can progressively reduce spatial resolution and expand the receptive field to adapt to the input of CT 3D data, forming a hierarchical feature pyramid from shallow details to deep semantics, adapting to the multi-scale characteristics of lung lesions. Then, based on the feature maps of different scales output by the backbone network, a two-layer top-down feature fusion network is constructed to generate feature maps for multi-scale detection. Compared with traditional FPN which only fuses at a single scale, this design achieves a dynamic balance between local details and global semantics through bidirectional cross-layer interaction (combination of upsampling and downsampling), avoiding the disappearance of small target features in deep networks and effectively improving the efficiency and accuracy of small target detection. This invention systematically solves the core problems of traditional 3D detection models, such as low sensitivity to small targets, insufficient feature modeling, and insufficient computational efficiency, by combining a multi-scale feature fusion architecture with an attention enhancement mechanism. It not only optimizes the network structure of the detection model, but also achieves a synergistic improvement in detection accuracy, inference speed, and generalization ability through hierarchical feature design, cross-layer interaction mechanism, and lightweight strategy, providing a high-performance technical solution for the automated assisted diagnosis of lung CT images.

[0158] Example 6:

[0159] Based on Example 1, the CT image to be detected is input into the target detection model to obtain detection information of the patient's lung shadows, including:

[0160] The preprocessed CT image to be detected is input into the target detection module, and three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module.

[0161] The detection head module analyzes the 3D spatial features, identifies potential targets, and marks multiple potential targets based on candidate boxes, outputting the 3D bounding box parameters of the candidate boxes.

[0162] A three-dimensional non-maximum suppression algorithm is adopted. Based on a preset threshold, the confidence level of each candidate box is calculated. The candidate box with the highest confidence level is taken as the target box, and overlapping detection results are eliminated.

[0163] Based on the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size.

[0164] Based on the 3D bounding box parameters and the actual physical dimensions, detection information of the patient's lung shadow is generated.

[0165] In this embodiment, the three-dimensional bounding box parameters of each candidate box include the center point coordinates (x, y, z), the dimensions of each axis (length, width, height), and the confidence score.

[0166] The beneficial effects of the above technical solution are as follows: This invention uses the MSSF module and the C3K2_MSSF module to perform cross-layer feature extraction on the three-dimensional volume data of CT images, which can simultaneously capture lung shadows of different sizes, effectively improving the extraction rate and efficiency of small targets. The detection head module predefines potential targets of various sizes through the three-dimensional anchor box mechanism and combines the three-dimensional non-maximum suppression (3D-NMS) algorithm with a preset intersection-over-union (IoU) threshold to eliminate overlapping detection boxes, ensuring that each real lesion corresponds to only one optimal detection box, which can effectively avoid the problems of missed and multiple labels, and effectively improve the detection efficiency of lung CT images. Finally, based on the pixel spacing parameter of the CT image and the three-dimensional voxel size of the target box, the marked box is automatically converted into the real physical size, realizing the rapid and automated determination of the location and size of lung lesions, providing reliable auxiliary diagnostic results for the diagnosis of lung diseases.

[0167] Example 7:

[0168] Based on Example 1, a lung CT image processing method using multi-scale spatial semantic fusion further includes:

[0169] After obtaining the test information, the patient's target case or related test data are obtained simultaneously. Based on the lung disease type corresponding to the test information, various possible abnormal information related to the patient's test data is predicted.

[0170] Retrieve relevant potential abnormal information from the target case or related test data, and obtain the degree of abnormality corresponding to the relevant abnormal information;

[0171] Based on the anomaly level and the size of the lesion corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the verification level is sent to the patient's attending physician as diagnostic auxiliary information.

[0172] The beneficial effects of the above technical solution are as follows: After acquiring lung shadow detection information, this invention simultaneously collects the patient's target case and related detection data (such as blood test indicators, pulmonary function test results, and past imaging data) to provide a basis for the integration of multi-dimensional data. Based on the lung disease type determined by the detection information, it predicts various possible abnormal information in the patient's detection data. For example, when a malignant tumor shadow is detected in the lung, the system can predict that the patient may experience abnormal indicators related to systemic symptoms such as anemia and hypoproteinemia, as well as abnormal indicators of other organ functions that may be caused by tumor metastasis. This allows for the early detection of potential risks. Subsequently, the system retrieves the predicted possible abnormal information from the target case and related detection data, calculates the degree of abnormality, and combines this with the lesion location in the detection information. The system quantifies the severity of a patient's current lung disease by measuring abnormalities with specific numerical values, making the assessment more objective and accurate. For example, for patients with lung tumors, the severity score is calculated by considering not only the size and location of the tumor but also the elevation of tumor markers in the blood and the extent of lung function impairment. This provides doctors with precise references for developing personalized treatment plans, avoiding undertreatment or overtreatment. Simultaneously, the predicted disease severity is sent to the attending physician in real time as diagnostic auxiliary information, helping doctors quickly obtain key information. This enables attending physicians to capture key information promptly and accurately during busy clinical work, facilitating rapid diagnostic decisions, treatment planning, and shortening the patient's waiting time for diagnosis and treatment.

[0173] Example 8:

[0174] This invention provides a multi-scale spatial semantic fusion lung CT image processing system, such as... Figure 5 As shown, it includes:

[0175] The data preparation and quality control module is used to acquire various lung CT-DICOM images, perform quality control processing and format conversion, and obtain raw sample images.

[0176] The image preprocessing and lung parenchyma segmentation module is used to preprocess the original sample images and segment the lung parenchyma to obtain standard sample images.

[0177] The training dataset construction module is used to determine positive and negative samples based on the image annotation results, randomly perturb the positive and negative samples to obtain augmented sample images, and generate the training dataset based on the augmented sample images and the standard sample images.

[0178] The object detection model building module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and to train the improved YOLOv12-3D network with the training dataset to obtain the object detection model;

[0179] The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadows.

[0180] The beneficial effects of the above technical solution are as follows: This invention acquires multiple lung CT-DICOM images and performs quality inspection and format conversion to ensure that the original samples cover image data from different devices and scenarios, reducing model bias caused by single data sets. Simultaneously, format standardization reduces the difficulty of subsequent processing and lowers GPU load. Subsequently, the original samples undergo preprocessing and lung parenchyma segmentation to remove irrelevant tissue interference and standardize image features, enabling the model to focus on effective regions, effectively improving detection specificity and accuracy, avoiding false detections caused by extra-lung structures, and accurately determining positive and negative samples based on annotation results. Data expansion through random interference not only increases the number of samples but also enables the simulation of complex clinical situations, allowing the model to learn more diverse feature patterns, which helps alleviate overfitting. This invention addresses the problem of improving the generalization ability for targets of different shapes and locations. It then improves the YOLOv12 network based on the MSSF and C3K2_MSSF modules, utilizing multi-scale spatial semantic fusion and attention mechanisms to deeply extract 3D spatial features. This effectively solves the problems of insufficient 3D modeling and low sensitivity to small targets in traditional methods. Combined with a high-quality training dataset, the improved YOLOv12-3D network fully learns target features during training, achieving high-precision detection. Finally, the CT image to be detected is input into the trained target detection model, which can quickly output lung shadow detection information. Compared with traditional methods, this significantly shortens the detection time and improves the detection capability of small targets, contributing to accurate auxiliary diagnosis of early diseases and improving medical efficiency and diagnostic accuracy. This invention improves the YOLOv12-3D network architecture by achieving deep fusion of 3D spatial features and multi-scale spatial semantic features in the feature extraction stage, constructing an end-to-end target detection model. It eliminates the need for manual feature design or complex post-processing steps, directly outputting target localization and classification results from CT sequence images, significantly improving the extraction rate and efficiency of small targets.

[0181] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for processing lung CT images using multi-scale spatial semantic fusion, characterized in that, include: Step 1: Acquire various lung CT-DICOM images, perform quality control processing and format conversion to obtain raw sample images; Step 2: Preprocess the original sample images and segment the lung parenchyma to obtain standard sample images; Step 3: Based on the image annotation results, determine the positive and negative samples, and randomly perturb the positive and negative samples to obtain augmented sample images. Based on the augmented sample images and the standard sample images, generate the training dataset. Step 4: Improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and train the improved YOLOv12-3D network with the training dataset to obtain the object detection model; Step 5: Input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadows; The C3K2_MSSF module is designed as follows: First, the input features are transformed by a 3D convolution kernel to obtain a preliminary feature map; Subsequently, the feature map is divided into multiple sub-feature maps along the channel dimension, and each sub-feature map is input into an MSSF module for processing. Finally, the sub-feature maps processed by the MSSF module are reassembled along the channel dimension and fused through a Conv3D convolutional layer to output the final feature map of this module. The main network of the YOLOv12 network is built based on the MSSF module and the C3K2_MSSF module, including: Two 3D convolutional downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input into the first C3K2 module for high-level feature extraction to obtain the first feature map; After processing the first feature map through the third 3D convolutional downsampling module, it is input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map; After processing the second feature map through the fourth 3D convolutional downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map; The third feature map is processed by the fifth 3D convolutional downsampling module to obtain the feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network. The feature map network of the YOLOv12 network, constructed based on the MSSF module and the C3K2_MSSF module, includes: The construction of the first-layer feature fusion network is as follows: The output feature map of the backbone network is processed by the third C3K2_MSSF module to obtain the fourth feature map; After upsampling the fourth feature map and concatenating it with the third feature map output by the first layer feature fusion network, the result is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map. The fifth feature map is upsampled and then concatenated with the second feature map in the backbone network to obtain the highest resolution first layer final concatenated and fused feature map; The construction of the second-layer feature fusion network is as follows: The final stitched and fused feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fused feature map. After processing the first fused feature map through the sixth 3D convolutional downsampling module, it is concatenated with the fifth feature map output by the first layer network, and the concatenation result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map. The second fused feature map is processed by the seventh 3D convolutional downsampling module and then concatenated with the fourth feature map output by the first layer network. The concatenation result is then input into the second C3K2 module for feature extraction to obtain the third fused feature map.

2. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 1, characterized in that, Step 1 includes: Anonymize the acquired lung CT-DICOM images and perform quality checks on the processed lung CT-DICOM data; For lung CT-DICOM images that have passed quality inspection, the dcm2niix tool is used to perform data formatting and conversion to obtain the original sample images; Based on a pre-set lung lesion database, the disease types corresponding to the original sample images are detected to determine whether all the currently collected original sample images encompass all lung disease types. If not, continue with lung CT-DICOM image acquisition; If so, then terminate the lung CT-DICOM image acquisition.

3. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 1, characterized in that, Step 2 includes: Extract the original pixel spacing of the original sample image, and use B-spline interpolation to resample the original sample images with original pixel spacing greater than 1mm to obtain new sample images; The new image and the original sample image with a pixel pitch of less than or equal to 1 mm are denoised using a 3D nonlocal mean filtering algorithm to obtain denoised sample data. Based on the preset CT value, the denoised sample data is binarized to obtain binarized sample data. The lung parenchyma image is obtained by performing a pixel-by-pixel AND operation between the binarized sample data and its corresponding original sample image. The lung region is cropped from the lung parenchyma image to obtain the lung ROI image, and the lung ROI image is adjusted to a preset size using a bilinear interpolation algorithm to obtain a standard sample image.

4. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 1, characterized in that, Step 3 includes: Based on the pre-trained nnU-Net object detection model, candidate boxes are used to label standard sample images to obtain labeled sample images, and the labeling results are sent to experts for review. Once the experts have completed their review, the expert-marked target boxes are compared with the candidate boxes to obtain the intersection-union ratio of the marked boxes; Image samples with a cross-union ratio (CUC) greater than or equal to a preset threshold are considered positive samples. Image samples with a cross-union ratio (CUC) of bounding boxes less than a preset threshold are designated as negative samples. Each positive and negative sample is randomly subjected to interference processing to obtain an augmented sample image; A training dataset is generated based on the augmented sample images and the standard sample images.

5. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 1, characterized in that, Step 4 includes: The main network and feature map network of YOLOv12 are constructed based on the MSSF module and C3K2_MSSF module, respectively, and an improved YOLOv12-3D network is generated by combining the detection head module. An object detection model is obtained by training the improved YOLOv12-3D network based on the training dataset.

6. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 5, characterized in that, The CT image to be detected is input into the target detection model to obtain detection information of the patient's lung shadows, including: The preprocessed CT image to be detected is input into the target detection module, and three-dimensional spatial features are extracted based on the MSSF module and the C3K2_MSSF module. The detection head module analyzes the 3D spatial features, identifies potential targets, and marks multiple potential targets based on candidate boxes, outputting the 3D bounding box parameters of the candidate boxes. A three-dimensional non-maximum suppression algorithm is adopted. Based on a preset threshold, the confidence level of each candidate box is calculated. The candidate box with the highest confidence level is taken as the target box, and overlapping detection results are eliminated. Based on the pixel spacing parameters in the CT image to be detected and the three-dimensional bounding box parameters of the target box, the voxel size of the target box is converted into the real physical size. Based on the 3D bounding box parameters and the actual physical dimensions, detection information of the patient's lung shadow is generated.

7. The lung CT image processing method based on multi-scale spatial semantic fusion according to claim 5, characterized in that, Also includes: After obtaining the test information, the patient's target case or related test data are obtained simultaneously. Based on the lung disease type corresponding to the test information, various possible abnormal information related to the patient's test data is predicted. Retrieve relevant potential abnormal information from the target case or related test data, and obtain the degree of abnormality corresponding to the relevant abnormal information; Based on the abnormality and the size of the lesion corresponding to the detection information, the severity of the patient's current lung disease is predicted, and the severity is sent to the patient's attending physician as diagnostic auxiliary information.

8. A lung CT image processing system with multi-scale spatial semantic fusion, characterized in that, include: The data preparation and quality control module is used to acquire various lung CT-DICOM images, perform quality control processing and format conversion, and obtain raw sample images. The image preprocessing and lung parenchyma segmentation module is used to preprocess the original sample images and segment the lung parenchyma to obtain standard sample images. The training dataset construction module is used to determine positive and negative samples based on the image annotation results, randomly perturb the positive and negative samples to obtain augmented sample images, and generate the training dataset based on the augmented sample images and the standard sample images. The object detection model building module is used to improve the YOLOv12 network based on the MSSF module and the C3K2_MSSF module to obtain the improved YOLOv12-3D network, and to train the improved YOLOv12-3D network with the training dataset to obtain the object detection model; The lung CT image detection module is used to input the CT image to be detected into the target detection model to obtain the detection information of the patient's lung shadows; The C3K2_MSSF module is designed as follows: First, the input features are transformed by a 3D convolution kernel to obtain a preliminary feature map; Subsequently, the feature map is divided into multiple sub-feature maps along the channel dimension, and each sub-feature map is input into an MSSF module for processing. Finally, the sub-feature maps processed by the MSSF module are reassembled along the channel dimension and fused through a Conv3D convolutional layer to output the final feature map of this module. The main network of the YOLOv12 network is built based on the MSSF module and the C3K2_MSSF module, including: Two 3D convolutional downsampling modules are set up consecutively after the CT image input module to downsample the input CT image, and then input into the first C3K2 module for high-level feature extraction to obtain the first feature map; After processing the first feature map through the third 3D convolutional downsampling module, it is input into the first C3K2_MSSF module for high-level semantic feature extraction to obtain the second feature map; After processing the second feature map through the fourth 3D convolutional downsampling module, it is input into the second C3K2_MSSF module for high-level semantic feature extraction to obtain the third feature map; The third feature map is processed by the fifth 3D convolutional downsampling module to obtain the feature map with the largest receptive field, and the feature map with the largest receptive field is used as the output feature map of the backbone network. The feature map network of the YOLOv12 network, constructed based on the MSSF module and the C3K2_MSSF module, includes: The construction of the first-layer feature fusion network is as follows: The output feature map of the backbone network is processed by the third C3K2_MSSF module to obtain the fourth feature map; After upsampling the fourth feature map and concatenating it with the third feature map output by the first layer feature fusion network, the result is input into the fourth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the fifth feature map. The fifth feature map is upsampled and then concatenated with the second feature map in the backbone network to obtain the highest resolution first layer final concatenated and fused feature map; The construction of the second-layer feature fusion network is as follows: The final stitched and fused feature map of the first layer is input into the fifth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the first fused feature map. After processing the first fused feature map through the sixth 3D convolutional downsampling module, it is concatenated with the fifth feature map output by the first layer network, and the concatenation result is input into the sixth C3K2_MSSF module for multi-scale spatial fusion processing to obtain the second fused feature map. The second fused feature map is processed by the seventh 3D convolutional downsampling module and then concatenated with the fourth feature map output by the first layer network. The concatenation result is then input into the second C3K2 module for feature extraction to obtain the third fused feature map.

Images