Deep learning-based chest ct image multi-abnormality recognition and positioning method and system

Abstract

The application discloses a chest CT image multi-abnormality recognition and positioning method and system based on deep learning, and the method comprises the following steps: customizing chest and abdominal CT abnormality labels, constructing a chest CT multi-abnormality positioning labeling data set according to the abnormality labels; based on the data set, training a YOLO model, and performing abnormality detection on a single tomographic image in an input CT sequence to output a preliminary detection result with a tomographic sequence number; extracting interlayer parameters of the CT sequence, and based on the interlayer parameters and the preliminary detection result, performing interlayer correlation verification and three-dimensional feature fusion to obtain a detection result after verification and fusion; and based on the detection result after verification and fusion, generating and outputting a final detection report containing abnormal three-dimensional information. The application can start from constructing a chest CT data set for multi-class abnormality positioning labeling, utilize a target recognition and positioning network in computer vision, and train an auxiliary diagnosis model capable of simultaneously recognizing and positioning common abnormal signs of chest CT.

Description

Technical Field

[0001] This invention relates to the fields of medical image recognition and artificial intelligence technology, and in particular to a method and system for identifying and locating multiple abnormalities in chest CT images based on deep learning. Background Technology

[0002] In recent years, AI (Artificial Intelligence) technology has been successfully applied in medical image analysis, and deep learning is one such method that can be effectively applied to intelligent medical image analysis. Deep learning is an important research method in the rapidly developing field of artificial intelligence, enabling machines to mimic human activities such as sight, hearing, and thinking. It has solved many complex pattern recognition problems, leading to significant progress in AI-related technologies.

[0003] Artificial intelligence-assisted diagnostic tools for chest CT scans are currently a hot research topic, with several commercially available AI-assisted software applications in clinical practice. However, these commercial AI-assisted software programs are generally only designed for single diseases or single tasks, such as the detection of pulmonary nodules, tuberculosis, rib fractures, and pneumonia. Especially in recent years, with the COVID-19 pandemic, many studies have focused on the early diagnosis of the virus to reduce its mortality rate. A literature search reveals few studies focusing on the identification and localization of multiple abnormalities on chest CT scans, primarily due to the lack of chest CT datasets containing multi-lesion localization labels. Currently available public CT datasets are mostly built for the classification or segmentation of single diseases, such as the COVID-19-CT-dataset, the LIDC-IDRI dataset for pulmonary nodules, the LUNA16 dataset, and the NSCLC dataset. Research by Meng Xianghong et al. has utilized convolutional neural network models to assist in the diagnosis of rib fractures in chest CT images. Chengdi Wang et al.'s research used NPL (Non-Plastic Reporting) technology to extract report keywords to build a CT dataset. They then trained the CT and public chest X-ray datasets using deep learning classification networks ResNet-3D-18 and ResNet-50 modules. Finally, they displayed the regions that determined the classification using heatmaps, indirectly providing location information. However, these studies have limitations. Deep learning classification networks only require a dataset with labeled categories, and NPL technology can quickly establish category labels by extracting report information. However, to obtain location information for multiple anomalies, a dataset containing multiple anomaly bounding boxes must be used, and deep learning localization networks such as YOLO, Faster R-CNN, and RestinaNet must be used to train the model. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to propose a deep learning-based method and system for identifying and locating multiple abnormalities in chest CT images. This method begins by constructing a chest CT dataset with annotated locations for various anomalies. It then utilizes a target recognition and localization network from computer vision to train an auxiliary diagnostic model capable of simultaneously identifying and locating common abnormal signs in chest CT images. This simulates the first step in a radiologist's image interpretation process: identifying and locating abnormalities, and generating a diagnostic report based on the doctor's comprehensive knowledge. Finally, an AI-assisted diagnostic platform is built to reduce the workload of radiologists and improve diagnostic efficiency.

[0005] One technical solution adopted in this invention is: a method for identifying and locating multiple abnormalities in chest CT images based on deep learning, the method comprising the following steps: S1: Customize chest and abdominal CT abnormality labels, and construct a dataset of various abnormality localization annotations in chest CT based on chest and abdominal CT abnormality labels; S2: Based on the dataset, train the YOLO model and perform anomaly detection on a single tomographic image in the input CT sequence, outputting preliminary detection results with tomographic numbers; S3: Extract the inter-slice parameters of the CT sequence, and based on the inter-slice parameters and the preliminary detection results, perform inter-slice correlation verification and three-dimensional feature fusion to obtain the verified and fused detection results; S4: Based on the detection results after verification and fusion, generate and output a final detection report containing abnormal 3D information.

[0006] Furthermore, step S1 includes the following sub-steps: S11: Collect chest CT images and perform de-identification processing; S12: Develop a classification and labeling system for abnormal signs covering multiple organs in the chest and upper abdomen; S13: Radiology physicians perform bounding box localization and annotation on all categories of abnormalities in CT images, which are then reviewed by higher-level physicians to form the gold standard annotation; S14: Use all chest CT images and their corresponding gold standard annotations as a dataset.

[0007] Furthermore, step S2 includes the following sub-steps: S21: Divide the dataset into training set, validation set and test set according to a preset ratio; S22: Construct an improved YOLO model, the improvement including downsampling optimization of the YOLO backbone network, adding a feature fusion network to the neck network, and configuring classification weights for multiple anomaly types in the head network; S23: Train the improved YOLO model using the training set, and adjust the hyperparameters using the validation set until the model performance meets the preset conditions; S24: Input a single tomographic image from the CT sequence to be detected into the trained model, and output preliminary detection results including anomaly type, anomaly bounding box coordinates, confidence level, and corresponding tomographic number.

[0008] Furthermore, step S3 includes the following sub-steps: S31: Extract the interslice spacing parameters and the three-dimensional spatial coordinates of each slice from the DICOM file of the CT sequence, and construct a spatial position matrix; S32: Based on the spatial location matrix, match the associated regions in adjacent faults for the abnormal bounding boxes in the preliminary detection results; S33: Calculate the feature similarity between any fault anomaly region and adjacent fault-related regions, and perform feature consistency verification based on a preset threshold to correct or eliminate preliminary detection results; S34: When the same type of anomaly is detected in consecutive adjacent faults but the intermediate fault is missed, the theoretical location of the anomaly in the missed fault is calculated by spatial interpolation and a local secondary detection is performed to supplement the detection results. S35: Perform feature weighted fusion on the continuous fault detection results that have passed the verification and belong to the same anomaly to generate a three-dimensional feature representation.

[0009] Furthermore, step S31 includes the following sub-steps: S311: Calculation of interlayer spacing: ; in, d This indicates the actual spatial distance between adjacent faults. Indicates the first i Z-axis coordinates of the fault Indicates the relationship with the first i The adjacent fault i +1 Z-axis coordinate of the fault; S312: Construction of Spatial Position Matrix ; in, N Indicates the number of faults. Indicates the first N The fault number, Indicates the first N The interlayer spacing of the fault, Indicates the first N Spatial coordinates of the fault.

[0010] Furthermore, step S32 includes the following sub-steps: S321: Define the first i The coordinates of the fault's abnormal boundary box are ; S322: Calculate the adjacent number i Coordinates of the associated region of fault +1: ; in, Indicates the first i +1 fault associated region coordinates, W Indicates the first i width of fault anomaly bounding box, H Indicates the first i Fault anomaly bounding box height, Δ x express x Spatial offset of direction, Δ y express y Spatial offset of direction.

[0011] Furthermore, step S33 includes the following sub-steps: S331: Feature Similarity Calculation: ; in, Indicates the first i Abnormal feature vectors of faults Indicates the first i+ 1. Anomaly feature vector of the fault, n Represents feature dimension and feature similarity. Sim The value range is [0,1]; S332: Feature consistency check, when Sim If the value is ≥0.7, it is determined that the features are consistent; otherwise, proceed to step S333. S333: Spatial Offset Verification: ; ; in, L Represents the spatial offset on a two-dimensional plane, { } indicates the first i The center coordinates of the fault's anomaly bounding box, { } indicates the first i The center coordinates of the associated region of fault +1 This represents the actual spatial offset. P Indicates the pixel spacing of a CT image; when If the condition is met, it is considered a genuine anomaly; otherwise, it is considered a false alarm.

[0012] Furthermore, the specific calculation process for the theoretical location of the anomaly in the missed detection tomography is as follows: ; in,{ } indicates the first i-1 The center coordinates of the abnormal boundary box of the fault.

[0013] Furthermore, the specific fusion method of step S35 is as follows: Let the feature vectors of the same anomaly in k consecutive faults be respectively... , , ..., The corresponding detection confidence levels are respectively , , ..., Then the feature vector after 3D fusion is: .

[0014] The second technical solution adopted in this invention is a deep learning-based system for identifying and locating multiple abnormalities in chest CT images, which includes the following modules: The dataset construction module is used to customize chest and abdominal CT abnormality labels and construct a dataset of various abnormality localization annotations in chest CT based on the chest and abdominal CT abnormality labels; The single-image detection module is used to train a YOLO model based on the dataset, perform anomaly detection on a single tomographic image in the input CT sequence, and output preliminary detection results with tomographic numbers. The inter-layer correlation verification module is used to extract inter-layer parameters of the CT sequence, and based on the inter-layer parameters and preliminary detection results, perform inter-layer correlation verification and three-dimensional feature fusion to obtain the verified and fused detection results. The report generation module is used to generate and output a final detection report containing abnormal 3D information based on the detection results after verification and fusion.

[0015] The present invention provides a method and system for identifying and locating multiple abnormalities in chest CT images based on deep learning, which has at least the following beneficial effects: 1. This solution constructs a labeled dataset covering 27 types of abnormalities and trains a unified improved YOLO model, enabling a single system to identify and locate the vast majority of common abnormalities in a single CT scan in one go. This simulates the physician's global image interpretation perspective, fundamentally avoiding the cumbersome process of alternating between multiple single-disease AI tools in clinical practice, significantly improving diagnostic efficiency, and greatly reducing the risk of missed diagnoses due to incomplete examinations.

[0016] 2. A "cross-layer detection" mechanism is introduced. This mechanism constructs a spatial matrix by extracting the inter-layer spacing and performs correlation matching and feature consistency verification on individual detection results across adjacent slices. This design fully utilizes the spatial and feature continuity of real lesions on continuous slices, effectively identifying and eliminating false positives in individual slices. Simultaneously, interpolation calculations and local re-detection supplement missed detections, thus systematically reducing false positives and false negatives, particularly improving the detection rate of small and atypical lesions.

[0017] 3. This solution not only outputs a two-dimensional location bounding box, but also generates a quantitative report containing the three-dimensional spatial location, volume, and distribution range of abnormalities through inter-layer verification and feature fusion. This provides precise metrical evidence for lesion size assessment and treatment follow-up comparison that traditional two-dimensional detection cannot provide, and the output results are more in line with the actual needs of clinical diagnosis, thus having greater auxiliary value.

[0018] 4. Several improvements were made to the detection model to address the characteristics of chest and abdominal CT images: optimized downsampling to preserve details, the use of an advanced feature pyramid network to fuse multi-scale information, and a custom loss function to balance multiple sample categories. These optimizations collectively enhanced the model's ability to extract and identify features of anomalies of different sizes, densities, and shapes, ensuring robustness and accuracy in detecting anomalies across all categories. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a deep learning-based method for identifying and locating multiple abnormalities in chest CT images according to the present invention.

[0020] Figure 2 for Figure 1 The sub-flowchart of step S1.

[0021] Figure 3 for Figure 1 The sub-flowchart of step S2.

[0022] Figure 4 for Figure 1 The sub-flowchart of step S3 in the middle.

[0023] Figure 5 The diagram shows the structural block of a deep learning-based chest CT image multi-abnormality identification and localization system according to the present invention. Detailed Implementation

[0024] The invention will now be further described with reference to the accompanying drawings.

[0025] Please see Figure 1 The diagram below illustrates a deep learning-based method for identifying and locating multiple abnormalities in chest CT images according to the present invention. The method may include the following steps: S1: Customize chest and abdominal CT abnormality labels, and construct a dataset of various abnormality localization annotations for chest CT based on these labels. The background model is constructed in the same way as the traditional construction method described above in step S1.

[0026] Please see Figure 2This S1 step may also include the following sub-steps: S11: Collect and de-identify chest CT images; Step S11 can retrospectively collect chest CT images from a hospital's radiology department within a specific time period, such as a total of 6000 cases. Collect the chest CT images and diagnostic results of the subjects, as well as their age, gender, and general information. Simultaneously, subjects with poor or incomplete chest CT images as determined by radiologists, subjects whose chest CT images are postoperative images, and subjects under 18 years of age are excluded. De-identification is performed on the subjects' images before collection to protect their privacy. The collected images from this center and multiple centers are encrypted and stored for two years before being destroyed.

[0027] S12: Establish a classification labeling system for abnormal signs covering multiple organs in the chest and upper abdomen; This S12 step, regarding the classification of various abnormalities on chest CT, is based on common abnormal signs. Since chest CT scans involve the lungs, mediastinum, thoracic cage, and upper abdominal organs, classification labels based on abnormal signs can be established as follows: Abnormalities in both lungs: pneumothorax, nodules, masses, atelectasis, consolidation, fibrosis, interstitial changes, cavitation, bullae, pleural thickening, bronchiectasis, and emphysema, totaling 12 categories; General abnormalities: effusion, calcification, pneumothorax, enlarged lymph nodes, and postoperative metallic structures, totaling 5 categories; Mediastinal abnormalities: space-occupying lesions and aortic dilatation, totaling 2 categories; Thoracic wall abnormalities: fractures and bone lesions, totaling 2 categories; Abnormalities in other areas: pneumothorax, solid organ lesions, cholecystitis, gallstones, peripancreatic effusion, and fatty liver, totaling 6 categories. A total of 27 types of abnormalities are marked on chest CT images.

[0028] S13: Radiologists create bounding boxes to locate all abnormalities in the CT images, and this is then reviewed by a higher-level radiologist to form the gold standard annotation. In this S13 step, all chest CT images are converted from DICOM format to JPEG format by lung window, mediastinal window, and bone window. For each image, an intermediate-level radiologist (with 10-20 years of clinical experience in chest imaging diagnosis) uses LebalImg software to create bounding boxes for all abnormalities. This is then verified by a higher-level radiologist (with over 20 years of clinical experience in chest imaging diagnosis). In cases of disputed abnormality categories or locations, the conclusion of the higher-level radiologist prevails. After two levels of annotation and review by physicians, the resulting chest CT abnormality localization labels can be considered the gold standard annotation.

[0029] S14: Compile all chest CT images and their corresponding gold standard annotations into a dataset. This S14 step constructs a chest CT annotation dataset containing various anomalies by combining all chest CT images included in the study from this center with their gold standard annotations.

[0030] S2: Based on the dataset, train the YOLO model and perform anomaly detection on a single tomographic image in the input CT sequence, outputting preliminary detection results with tomographic numbers. As a representative of single-stage object detection methods, the YOLO algorithm, with its unique fully connected layer design, can complete the entire process from feature extraction to classification and regression prediction in one step, demonstrating a significant advantage in detection speed compared to multi-stage object detection algorithms. YOLOv11, as the latest evolution of the YOLO series algorithms, inherits the high efficiency and speed characteristics of the YOLO series while undergoing in-depth technical innovation and optimization. This algorithm not only incorporates many cutting-edge technologies in the field of object recognition, such as advanced data augmentation strategies, decoupled head design, and anchorless structure, but also cleverly integrates with the YOLO network architecture, achieving a double leap in performance and speed.

[0031] Please see Figure 3 This is a sub-flowchart for step S2, which may include the following sub-steps: S21: Divide the dataset into training, validation, and test sets according to a preset ratio. In this S21 step, the constructed chest CT data with various anomaly localization annotations are divided into training, validation, and test sets in a 7:1:2 ratio. That is, the training set contains 4200 cases, the validation set contains 600 cases, and the test set contains 1200 cases. A deep learning object detection algorithm, such as the YOLO series, is used to train the model.

[0032] S22: Construct an improved YOLO model. The improvements include downsampling optimization of the YOLO backbone network, adding a feature fusion network to the neck network, and configuring classification weights for multiple abnormality types in the head network. This step S22 uses YOLOv11 as the base model and optimizes the neck and head of the model. A feature fusion network (using CSPNet+PANet structure) is added to the neck to improve the feature extraction capability for small lesions. A weight adjustment layer for classification and regression branches is added to the head. This weight adjustment layer consists of weight vectors with the same number of abnormality categories and is automatically optimized through backpropagation during network training.

[0033] The initial value of this weight vector is calculated and set based on objective quantitative indicators of various anomalies. The calculation formula is as follows: ; in, This represents the initial weight of the i-th type of anomaly; This represents the statistical incidence rate of the i-th type of abnormality in the target dataset (such as a publicly available chest CT dataset or an internal historical dataset), and is normalized. (The higher the value, the higher the incidence rate). This represents the reciprocal of the average detection accuracy of the i-th type of anomaly on the baseline model and the dataset, after normalization. (A larger value indicates lower detection accuracy and higher detection difficulty). and This represents the harmonic coefficient, used to balance the influence of incidence rate and detection accuracy on the weights. It can typically be set to... , .

[0034] For 27 abnormality types, classification weight vectors are pre-initialized based on their clinical incidence and imaging feature complexity. For example, the initial weight for pulmonary nodules is set to 1.2, the initial weight for common minor effusions is set to 0.8, and the initial weight for rib fractures is set to 1.0. This mechanism addresses the sample imbalance problem and improves the detection accuracy of key abnormalities by increasing the weight of difficult-to-detect or important abnormality categories in the loss function, making the model focus more on these categories during training. The model loss function is optimized by using the CIoU loss function to calculate the bounding box regression loss, combined with Focal Loss to address the sample imbalance problem of the 27 abnormalities, further improving detection accuracy.

[0035] S23: Train the improved YOLO model using the training set, and adjust the hyperparameters using the validation set until the model performance meets the preset conditions; In this step S23, the improved YOLO model is trained using the divided training set. The training parameters can be set as follows: learning rate 0.001, batch size 16, number of iterations 100 rounds, and an early stopping strategy (training stops if the validation set mAP does not improve for 10 consecutive rounds) until the model's detection accuracy (mAP) ≥ 90% and the false negative rate ≤ 5%.

[0036] S24: Input a single tomographic image from the CT sequence to be detected into the trained model, and output preliminary detection results including anomaly type, anomaly bounding box coordinates, confidence score, and corresponding tomographic number. Step S24 is a single-image detection process. Each preprocessed CT tomographic image is input into the trained improved YOLOv11 model, and the detection results for a single image are output, including: anomaly type (one of 27), bounding box coordinates, etc. , and record the confidence level (0-1, confidence level ≥0.5 is considered a valid detection result), and record the tomographic number and interslice spacing corresponding to each valid detection result. d .

[0037] S3: Extract the inter-slice parameters of the CT sequence, and based on the inter-slice parameters and preliminary detection results, perform inter-slice correlation verification and 3D feature fusion to obtain the verified and fused detection results. This S3 step, based on the improved YOLOv11 single-slice tomography detection model, extracts and utilizes the core inter-slice information of the CT sequence to construct a cross-slice detection model of "single-slice detection - inter-slice correlation verification - 3D feature fusion." This breaks through the limitations of independent single-slice tomography detection in existing technologies, and uses the inter-slice correlation information of continuous slices to verify, correct, and supplement missed detections of single-slice detection results, significantly reducing the missed detection rate and false detection rate.

[0038] Please see Figure 4 This is a sub-flowchart of step S3. Step S3 may also include the following sub-steps: S31: Extract the interslice spacing parameters and the three-dimensional spatial coordinates of each slice from the DICOM file of the CT sequence, and construct a spatial position matrix. Step S31 automatically extracts two core interslice information from the original DICOM format sequence file of the chest CT scan using the DICOM parsing library (pydicom 2.3.0). The specific implementation process is as follows: 1. Read DICOM sequence files: Use the pydicom.read_file() function to read all DICOM slices of a single CT in batches, store them as a slice list, and traverse the list to obtain the metadata information of each slice; 2. Extraction of interlayer spacing parameter d: The interlayer spacing value is read from the "SliceThickness" field of the DICOM metadata, in mm. If this field is missing, it is calculated from the "ImagePositionPatient" field (3D spatial coordinates) of the adjacent slice. 3. Slice numbering: For continuous slices of a single CT scan, number them from 1 to N according to the scanning sequence (from the apex of the lung to the base of the lung or from the upper segment of the abdomen to the lower segment) (N is the total number of slices in a single CT scan), and establish a one-to-one correspondence between slices and numbers; 4. Spatial location matrix construction: Based on the extracted inter-slice spacing parameter d and the fault number, and combined with the spatial coordinates of each slice, a correspondence between "fault number - inter-slice spacing - spatial coordinates" is established to form a spatial location matrix of continuous faults. This clarifies the spatial location of each fault image and the spatial correlation between adjacent faults, providing a quantitative spatial basis for subsequent inter-slice correlation verification.

[0039] Specifically, step S31 may include the following sub-steps: S311: Calculation of interlayer spacing: ; in, d This indicates the actual spatial distance between adjacent faults. Indicates the firsti Z-axis coordinates of the fault Indicates the relationship with the first i The adjacent fault i +1 Z-axis coordinate of the fault; S312: Construction of Spatial Position Matrix ; in, N Indicates the number of faults. Indicates the first N The fault number, Indicates the first N The interlayer spacing of the fault, Indicates the first N Spatial coordinates of the fault.

[0040] S32: Based on the spatial location matrix, match associated regions in adjacent faults for the abnormal bounding boxes in the preliminary detection results; the specific implementation process of this step S32 is as follows: for any fault image... i (Serial Number) i Valid detection results (anomaly type A, bounding box B, coordinates are...) First, the three-dimensional spatial coordinates of fault i are obtained through the spatial position matrix. Based on the CT scan direction (assuming it is the z-axis direction), calculate the adjacent fractures ( i -1、 i +1) spatial coordinates , Based on interlayer spacing parameters d Calculate the relationship between adjacent faults and the fault. i Spatial offset (Because the interlayer spacing is fixed), based on the changing patterns of interlayer positions in the human anatomical structures of different organs, determine... x , y directional offset Δ x Δ y (e.g., interlaminar displacement of lung lesions does not exceed) d / 2). For relatively fixed soft tissue organs such as the lungs and mediastinum, the maximum displacement of lesions between layers is relatively small, Δ x Δ y The empirical value is set to 0-3 pixels; for areas such as the thoracic cavity and upper abdominal organs that may shift slightly during respiration and scanning, the Δ value of the lesion is... x Δ y The empirical value is set to 0-5 pixels. The specific offset can be adaptively adjusted according to the original resolution of the CT image (e.g., take the upper limit when the resolution is below 1mm, and the lower limit when it is above 1mm); based on the tomographic... i Based on the abnormal bounding box B, in the adjacent fault ( i-1、 i In step +1), the coordinates of the associated regions B1 and B2 are calculated. To avoid incomplete matching of associated regions due to the above-mentioned inter-layer anatomical offset or slight deformation, the size of the associated region is set to 1.2 times the original bounding box B (i.e., the coordinates are expanded outward by 10% of the width and height). The purpose of this operation is to ensure that the corresponding real lesion areas in adjacent layers can be completely covered within the associated region, thereby providing an effective comparison area for calculating feature similarity in the subsequent step S33, and ensuring the effectiveness of the inter-layer feature consistency verification.

[0041] The specific calculation process for S32 is as follows: S321: Define the first i The coordinates of the fault's abnormal boundary box are ; S322: Calculate the adjacent number i Coordinates of the associated region of fault +1: ; in, Indicates the first i +1 fault associated region coordinates, W Indicates the first i width of fault anomaly bounding box, H Indicates the first i Fault anomaly bounding box height, Δ x express x Spatial offset of direction, Δ y express y Spatial offset of direction. It should be noted that the above "set the size of the associated region to 1.2 times that of the original bounding box B" is a qualitative description, which means "expand the coordinates of the four vertices of the original abnormal bounding box B outward by 10%", that is, "increase the length and width of the original abnormal bounding box B by 10%", that is, expand the area of ​​the original abnormal bounding box B to 1.44 times its original size.

[0042] S33: Calculate the feature similarity between any fault anomaly region and adjacent fault-related regions, and perform feature consistency verification based on a preset threshold to correct or eliminate preliminary detection results; the specific implementation process of this S33 step is as follows: firstly, the fault is extracted through the feature extraction network (Backbone part) of the improved YOLOv11 model. i The core features within the abnormal bounding box B (including grayscale distribution, texture features, and morphological features, which are the core distinguishing features for chest CT lesions) are used to obtain the feature vector. Simultaneously, the same method was used to extract adjacent faults. i -1、 i +1 The similar features within the associated regions B1 and B2 are used to obtain the feature vector. , The cosine similarity algorithm is used to calculate the feature similarity among the three images, with a similarity threshold of 0.7. Based on the similarity results, the detection results for each image are stratified and corrected. For cases where the feature similarity does not reach the threshold, the interlayer spacing is considered. d Calculate the spatial position offset to determine whether it conforms to the normal interlayer variation pattern of human anatomical structure, and then determine whether it is a false detection and correct the bounding box coordinates.

[0043] Specifically, step S33 may include the following sub-steps: S331: Feature Similarity Calculation: ; in, Indicates the first i Abnormal feature vectors of faults Indicates the first i+ 1. Anomaly feature vector of the fault, n Represents feature dimension and feature similarity. Sim The value range is [0,1]; S332: Feature consistency check, when Sim If the value is ≥0.7, it is determined that the features are consistent; otherwise, proceed to step S333. S333: Spatial Offset Verification: ; ; in, L Represents the spatial offset on a two-dimensional plane, { } indicates the first i The center coordinates of the fault's anomaly bounding box, { } indicates the first i The center coordinates of the associated region of fault +1 This represents the actual spatial offset. P Indicates the pixel spacing of a CT image; when If the interslice spacing is [value missing], it is considered a genuine anomaly; otherwise, it is a false alarm. It should be noted that in continuous computed tomography (CT) scans, if the interslice spacing is [value missing], it is considered a genuine anomaly. d Theoretically, the spatial displacement (spatial offset) of a continuous anatomical structure or lesion between two adjacent layers should be much smaller than that of a lesion. d To conservatively estimate and reduce false positives, the judgment threshold can be set to... d / 2, and set the threshold to d / 2 is a reasonable and conservative engineering experience value.

[0044] S34: When similar anomalies are detected in consecutive adjacent faults but missed by intermediate faults, the theoretical location of the anomaly in the missed fault is calculated by spatial interpolation, and a local secondary detection is performed to supplement the detection results; the specific implementation process of step S34 is to traverse all three consecutive faults ( i -1、 i , i +1), firstly, the fault is confirmed through the feature consistency verification submodule. i -1 and i +1 indicates whether the detection results are of the same type of real anomaly (feature similarity ≥ 0.7); if there is a tomography... i The anomaly was not detected, based on the interlayer spacing parameters. d For faults i -1 and i Spatial interpolation is performed using the coordinates of the anomaly's bounding box (+1) to calculate the location of the anomaly within the fault. i The theoretical location of the fault; i For the theoretical location region, a secondary detection is performed using the local focus detection mode of the YOLOv11 model (adjusting the detection weight of this region to improve feature extraction accuracy and detection sensitivity). Specifically, a higher feature sampling rate or a lightweight attention subnetwork is applied to the feature layer corresponding to the theoretical location region to increase the feature extraction weight of this local area. Simultaneously, the loss weight of the prediction result for this region is increased during loss calculation, thereby improving the model's detection sensitivity in this region. If the confidence level of the secondary detection is ≥0.4, the missed detection result is supplemented, correcting the continuity of the detection results and completely resolving the problem of single-image missed detection of cross-layer lesions. For example, the aforementioned local focus detection mode can employ a lightweight channel-spatial attention subnetwork, which receives the primary feature map extracted by the backbone network near the theoretical location region. As input, the following operations are performed in sequence: Channel attention: Global average pooling is performed on F to obtain channel descriptors, followed by a two-layer fully connected layer (the first layer compresses the number of channels to T / r, and the second layer restores it to T, where r is the reduction rate, which defaults to 16) and a sigmoid activation function to generate channel attention weights. And multiply it with the input feature F channel by channel.

[0045] Spatial attention: The features weighted by channel attention are then subjected to global average pooling and global max pooling along the channel dimension, and the two results are concatenated to form a 2-channel feature map; subsequently, a standard convolutional layer (7x7 kernel) and a sigmoid activation function are used to generate spatial attention weights. And multiply it position by position with the input features.

[0046] The output of this attention subnetwork is a feature map F' that enhances the relevance of the theoretical location region. During the inference phase, this feature map F' is input into the pre-trained YOLOv11 detection head for forward propagation to obtain a secondary detection result for that region. If the confidence of the secondary detection is ≥0.4, the missed detection result is supplemented, thus correcting the continuity of the detection results.

[0047] Specifically, the calculation process for the theoretical location of the anomaly in the missed detection tomography is as follows: ; in,{ } indicates the first i -1 represents the center coordinates of the anomaly bounding box of the fault. Furthermore, based on this fault... i Theoretical center coordinates { }, and refer to faults i -1 and i The average width of the abnormal bounding box in +1 Compared with average height Determine the fault i The theoretical region of the anomaly is: .

[0048] S35: Perform feature-weighted fusion on the verified continuous fault detection results belonging to the same anomaly to generate a three-dimensional feature representation. The specific implementation process of step S35 involves combining all valid detection results after inter-layer correlation verification with the spatial location matrix and inter-layer spacing parameters. d To perform 3D feature fusion, firstly, the feature extraction network of the trained improved YOLOv11 model is used to extract feature vectors of the same anomaly in continuous faults and perform weighted fusion (the weights are the detection confidence of each fault) to obtain the 3D feature vector of the anomaly. Then, based on the 3D feature vector and the corresponding 3D spatial coordinates of each detection result, the K-means clustering algorithm is used to cluster the continuous detection results of the same anomaly. After clustering, all detection boxes in the same category are selected. x , y , z The minimum and maximum coordinate values ​​in the direction are used as the starting and ending points of the three-dimensional bounding box of the anomaly, respectively, to determine the three-dimensional spatial location of the anomaly and the length, width and height of its bounding box; finally, the optimized detection results are output, including the anomaly type, three-dimensional spatial coordinates, volume, confidence level and the distribution range of continuous faults.

[0049] Specifically, the fusion method of step S35 is as follows: Let the feature vectors of the same anomaly in k consecutive faults be respectively... , , ..., The corresponding detection confidence levels are respectively , , ..., Then the feature vector after 3D fusion is: .

[0050] S4: Based on the detection results after verification and fusion, generate and output a final detection report containing abnormal 3D information.

[0051] Please see Figure 5 The diagram below shows the structural block of the deep learning-based chest CT image multiple anomaly identification and localization system of the present invention. The present invention also provides a deep learning-based chest CT image multiple anomaly identification and localization system, which is used to implement the aforementioned deep learning-based chest CT image multiple anomaly identification and localization method. Specifically, the system may include a dataset construction module 100, a single image detection module 200, an inter-slice correlation verification module 300, and a report generation module 400. Specifically: Dataset construction module 100 is used to customize chest and abdominal CT abnormality labels and construct a chest CT abnormality localization annotation dataset based on chest and abdominal CT abnormality labels; The single-image detection module 200 is used to train a YOLO model based on the dataset, perform anomaly detection on a single tomographic image in the input CT sequence, and output preliminary detection results with tomographic numbers. The inter-layer correlation verification module 300 is used to extract inter-layer parameters of the CT sequence, and based on the inter-layer parameters and preliminary detection results, perform inter-layer correlation verification and three-dimensional feature fusion to obtain the verified and fused detection results. The report generation module 400 is used to generate and output a final detection report containing abnormal three-dimensional information based on the detection results after verification and fusion.

[0052] The above description merely illustrates preferred embodiments of the present invention and is quite specific and detailed; however, it should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for identifying and locating multiple abnormalities in chest CT images based on deep learning, characterized in that, The method includes the following steps: S1: Customize chest and abdominal CT abnormality labels, and construct a dataset of various abnormality localization annotations in chest CT based on chest and abdominal CT abnormality labels; S2: Based on the dataset, train the YOLO model and perform anomaly detection on a single tomographic image in the input CT sequence, outputting preliminary detection results with tomographic numbers; S3: Extract the inter-slice parameters of the CT sequence, and based on the inter-slice parameters and the preliminary detection results, perform inter-slice correlation verification and three-dimensional feature fusion to obtain the verified and fused detection results; S4: Based on the detection results after verification and fusion, generate and output a final detection report containing abnormal 3D information.

2. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 1, characterized in that, Step S1 includes the following sub-steps: S11: Collect chest CT images and perform de-identification processing; S12: Develop a classification and labeling system for abnormal signs covering multiple organs in the chest and upper abdomen; S13: Radiology physicians perform bounding box localization and annotation on all categories of abnormalities in CT images, which are then reviewed by higher-level physicians to form the gold standard annotation; S14: Use all chest CT images and their corresponding gold standard annotations as a dataset.

3. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 1, characterized in that, Step S2 includes the following sub-steps: S21: Divide the dataset into training set, validation set and test set according to a preset ratio; S22: Construct an improved YOLO model, the improvement including downsampling optimization of the YOLO backbone network, adding a feature fusion network to the neck network, and configuring classification weights for multiple anomaly types in the head network; S23: Train the improved YOLO model using the training set, and adjust the hyperparameters using the validation set until the model performance meets the preset conditions; S24: Input a single tomographic image from the CT sequence to be detected into the trained model, and output preliminary detection results including anomaly type, anomaly bounding box coordinates, confidence level, and corresponding tomographic number.

4. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 1, characterized in that, Step S3 includes the following sub-steps: S31: Extract the interslice spacing parameters and the three-dimensional spatial coordinates of each slice from the DICOM file of the CT sequence, and construct a spatial position matrix; S32: Based on the spatial location matrix, match the associated regions in adjacent faults for the abnormal bounding boxes in the preliminary detection results; S33: Calculate the feature similarity between any fault anomaly region and adjacent fault-related regions, and perform feature consistency verification based on a preset threshold to correct or eliminate preliminary detection results; S34: When the same type of anomaly is detected in consecutive adjacent faults but the intermediate fault is missed, the theoretical location of the anomaly in the missed fault is calculated by spatial interpolation and a local secondary detection is performed to supplement the detection results. S35: Perform feature weighted fusion on the continuous fault detection results that have passed the verification and belong to the same anomaly to generate a three-dimensional feature representation.

5. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 4, characterized in that, Step S31 includes the following sub-steps: S311: Calculation of interlayer spacing: ； in, d This indicates the actual spatial distance between adjacent faults. Indicates the first i Z-axis coordinates of the fault Indicates the relationship with the first i The adjacent fault i +1 Z-axis coordinate of the fault; S312: Construction of Spatial Position Matrix ； in, N Indicates the number of faults. Indicates the first N The fault number, Indicates the first N The interlayer spacing of the fault, Indicates the first N Spatial coordinates of the fault.

6. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 5, characterized in that, Step S32 includes the following sub-steps: S321: Define the first i The coordinates of the fault's abnormal boundary box are ; S322: Calculate the adjacent number i Coordinates of the associated region of fault +1: ； in, Indicates the first i +1 fault associated region coordinates, W Indicates the first i width of fault anomaly bounding box, H Indicates the first i Fault anomaly bounding box height, Δ x express x Spatial offset of direction, Δ y express y Spatial offset of direction.

7. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 4, characterized in that, Step S33 includes the following sub-steps: S331: Feature Similarity Calculation: ； in, Indicates the first i Abnormal feature vectors of faults Indicates the first i+ 1. Anomaly feature vector of the fault, n Represents feature dimension and feature similarity. Sim The value range is [0,1]; S332: Feature consistency check, when Sim If the value is ≥0.7, it is determined that the features are consistent; otherwise, proceed to step S333. S333: Spatial Offset Verification: ； ； in, L Represents the spatial offset on a two-dimensional plane, { } indicates the first i The center coordinates of the fault's anomaly bounding box, { } indicates the first i The center coordinates of the associated region of fault +1 This represents the actual spatial offset. P Indicates the pixel spacing of a CT image; when If the condition is met, it is considered a genuine anomaly; otherwise, it is considered a false alarm.

8. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 7, characterized in that, The specific calculation process for the theoretical location of the anomaly in the missed detection fault is as follows: ； in,{ } indicates the first i -1 The center coordinates of the abnormal boundary box of the fault.

9. The method for identifying and locating multiple abnormalities in chest CT images based on deep learning as described in claim 8, characterized in that, The specific fusion method of step S35 is as follows: Let the feature vectors of the same anomaly in k consecutive faults be respectively... , ,..., The corresponding detection confidence levels are respectively , , ..., Then the feature vector after 3D fusion is: 。 10. A deep learning-based system for identifying and locating multiple abnormalities in chest CT images, characterized in that, The system includes the following modules: The dataset construction module is used to customize chest and abdominal CT abnormality labels and construct a dataset of various abnormality localization annotations in chest CT based on the chest and abdominal CT abnormality labels; The single-image detection module is used to train a YOLO model based on the dataset, perform anomaly detection on a single tomographic image in the input CT sequence, and output preliminary detection results with tomographic numbers. The inter-layer correlation verification module is used to extract inter-layer parameters of the CT sequence, and based on the inter-layer parameters and preliminary detection results, perform inter-layer correlation verification and three-dimensional feature fusion to obtain the verified and fused detection results. The report generation module is used to generate and output a final detection report containing abnormal 3D information based on the detection results after verification and fusion.

Images