Intermediate distance compensation-based pulmonary embolism detection and segmentation method and system

Abstract

The invention provides a pulmonary embolism detection and segmentation method based on intermediate distance compensation, and relates to the technical field of computer medical image processing, and the method comprises the steps: carrying out the pre-labeling and preprocessing of a lung CT image set, and obtaining a pre-labeling label and a standard image set; obtaining a thrombus label through the pre-labeling label; integrating the thrombus label and the standard image set, and inputting the integrated thrombus label and the integrated standard image set into a three-dimensional target detection network model for training to obtain a detection training model; performing detection and identification through the plurality of detection training models to obtain a plurality of preliminary detection results, and fusing the preliminary detection results into an identification result; the thrombus label, the standard image set and the lung CT image set are integrated and then input into a semi-supervised semantic segmentation model for model training, and a final segmentation model is obtained; and performing segmentation correction on the identification result through the final segmentation model and an intermediate distance compensation mechanism to obtain a combination result. According to the method, the pulmonary embolism is intelligently recognized through deep learning and image processing, and the detection accuracy of the pulmonary embolism is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer medical image processing, and particularly to a pulmonary embolism detection and segmentation method and system based on intermediate distance compensation. Background Art

[0002] Pulmonary embolism is a common and life-threatening disease, usually caused by a thrombus blocking the pulmonary artery or its branches, resulting in blocked blood supply to the lungs. The clinical manifestations of acute pulmonary embolism are diverse, ranging from asymptomatic to sudden dyspnea, chest pain, hypotension, and even sudden death. Early identification of pulmonary embolism is crucial, but traditional diagnostic methods, such as ventilation / perfusion scanning, CT pulmonary angiography (CTPA), and echocardiography, rely on the interpretation of experienced radiologists, and the diagnostic speed and accuracy are limited.

[0003] In recent years, intelligent analysis methods have been widely studied and applied in the field of medical imaging. As one of the key technologies, the automatic extraction technology of vascular centerlines has become an important means to improve the accuracy and efficiency of pulmonary embolism detection. By segmenting the pulmonary vessels, the structure of the pulmonary artery and its branches can be accurately depicted, providing rich spatial information for pulmonary embolism detection. The extraction of vascular centerlines can provide the geometric structure characteristics of the pulmonary vessels, helping to segment and locate the thrombus area.

[0004] Currently, the diagnosis of pulmonary embolism has the following deficiencies: 1. Complicated image interpretation process: Although existing imaging examinations such as CTPA can better show the location and size of the embolism, the interpretation of the images requires experienced doctors, and misjudgments may occur due to visual fatigue or subjective differences during the interpretation process; 2. Lack of intelligent analysis tools: Although imaging examinations can provide a large amount of information about the pulmonary artery and surrounding tissues, there are no widely used intelligent tools to automatically analyze and process these images, making it difficult to obtain accurate diagnostic results in a short time.

[0005] Similarly, the existing artificial intelligence-based auxiliary diagnosis methods also have the following deficiencies: 1. The cost of medical data annotation is high. The annotation of medical data, especially in diseases such as pulmonary embolism, often requires doctors or professionals to perform precise pixel-level annotation. Due to the huge amount of medical image data and high annotation requirements, the annotation cost and time consumption are huge, which limits the usability of the data. In addition, due to the diversity and complexity of diseases, the annotated data is often insufficient to cover all lesion situations, resulting in unstable performance of the model on different lesion samples. 2. The accuracy of existing segmentation models is insufficient. Many current medical image segmentation models, such as those based on convolutional neural networks (CNNs), although achieving good results in some tasks, still face many challenges in pulmonary embolism detection and segmentation. For example, the morphology and size of thrombi vary greatly, and the model may have difficulty adapting to these changes, resulting in inaccurate segmentation results. In addition, existing segmentation models often perform poorly when dealing with details, complex boundaries, and small thrombus regions, making it difficult to achieve precise segmentation. Therefore, it is highly necessary to design a pulmonary embolism detection and segmentation method and system based on intermediate distance compensation. Summary of the Invention

[0006] The purpose of the present invention is to provide a pulmonary embolism detection and segmentation method and system based on intermediate distance compensation, so as to intelligently identify the pulmonary embolism region through deep learning and image processing technologies, and improve the detection accuracy of pulmonary embolism by using the vascular centerline.

[0007] To achieve the above purpose, the present invention provides the following solutions:

[0008] A pulmonary embolism detection and segmentation method based on intermediate distance compensation, comprising the following steps:

[0009] Perform pre-annotation on the pulmonary CT image set to obtain pre-annotation labels; the pre-annotation includes: manual annotation and pixel-level annotation;

[0010] Obtain thrombus labels through the pre-annotation labels;

[0011] Perform preprocessing operations on the pulmonary CT image set to obtain a standard image set; the preprocessing operations include: adjusting window width and window level, adjusting image spacing, adjusting the image direction cosine matrix, image denoising, and hybrid cropping;

[0012] Integrate the thrombus labels and the standard image set into a detection data set, and input the detection data set into a three-dimensional object detection network model for model training to obtain multiple detection training models;

[0013] Use the multiple detection training models to respectively detect and identify the detection data set to obtain multiple preliminary detection results, and fuse the multiple preliminary detection results to obtain an identification result;

[0014] Integrate the thrombus label, standard image set, and lung CT image set into a segmentation data set, and train a semi-supervised semantic segmentation model with the segmentation data set to obtain a final segmentation model;

[0015] Segment the recognition result through the final segmentation model to obtain a segmentation result;

[0016] Correct the segmentation result through an intermediate distance compensation mechanism to obtain a merged result.

[0017] Optionally, obtain the thrombus label through a pre-annotated label. Specifically: Detect the coordinates of the lung CT image according to the pre-annotated label to obtain a thrombus bounding box, and obtain the thrombus label according to the six corner coordinate scalars of the thrombus bounding box; The thrombus bounding box is a cuboid structure.

[0018] Optionally, integrate the thrombus label and the standard image set into a detection data set, input the detection data set into a three-dimensional object detection network model for model training, and obtain multiple detection training models, including:

[0019] Divide the detection data set into a detection training set and a detection validation set;

[0020] Train the three-dimensional object detection network model with the detection training set to obtain multiple preliminary training models;

[0021] Extract features from the detection training set through a multi-scale feature fusion technique to obtain thrombus features;

[0022] Set multi-sized anchor boxes according to the thrombus features, and determine the thrombus area through the anchor boxes;

[0023] Optimize and train the preliminary training model through the thrombus area and the first loss function to obtain a detection training model.

[0024] Optionally, the first loss function includes: position loss and confidence loss; The calculation formula of the first loss function is: where GIoU is the first loss function, IoU is the IoU loss function, A is the optimal anchor box, B is the thrombus label, C is the smallest enclosing box formed by A and B, |C| is the area of the smallest enclosing box, and |C\(A∪B)| is the area after removing the union of A and B from C.

[0025] Optionally, integrate the thrombus label, standard image set, and lung CT image set into a segmentation data set, and train a semi-supervised semantic segmentation model with the segmentation data set to obtain a final segmentation model. Specifically: Input the segmentation data set into the semi-supervised semantic segmentation model, and train the model through a consistency regularization loss, an enhanced perturbation strategy, a pseudo-label generation strategy, cross-validation, and a second loss function to obtain a final segmentation model.

[0026] Optionally, the expression of the second loss function is: L = L l + λL u ; where L is the value of the second loss function, L l is the labeled loss function, L u is the unlabeled loss function, and λ is the weight coefficient;

[0027] The expression of the labeled loss function is: where B l is the number of labeled images, H is the hard cross-entropy loss function, is the predicted value of the i-th image, is the true label of the i-th image;

[0028] The expression of the unlabeled loss function is: where B u is the number of unlabeled images, is the i-th pseudo-label of the weakly augmented image, is the i-th pseudo-label of the strongly augmented image, τ is the preset confidence threshold, is the hard one-hot label.

[0029] Optionally, the recognition result is segmented by the final segmentation model to obtain a segmentation result. Specifically: the candidate region containing pulmonary embolism in the recognition result is selected as the input of the final segmentation model, a high-confidence pseudo-label is generated according to the unlabeled part in the input of the final segmentation model, and the recognition result is segmented by the high-confidence pseudo-label to obtain a segmentation result.

[0030] Optionally, the segmentation result is corrected by the intermediate distance compensation mechanism to obtain a merging result, including:

[0031] Calculating the intermediate distance between the segmentation results through a search algorithm;

[0032] Judging two thrombi in the segmentation result according to the comparison result between the intermediate distance and the preset threshold to obtain a judgment result; the judgment result is: the same thrombus or different thrombi;

[0033] Merging the two thrombi with the judgment result of the same thrombus to obtain a merging result.

[0034] Optionally, calculating the intermediate distance between the segmentation results through a search algorithm, specifically: extracting the thrombus boundary from the segmentation result, and calculating the pixel distance between the nearest points of two thrombi through the vascular centerline and the thrombus boundary to obtain the intermediate distance.

[0035] A pulmonary embolism detection and segmentation system based on intermediate distance compensation, comprising:

[0036] An image acquisition module, which is used to pre-label a set of lung CT images to obtain pre-labeled tags;

[0037] A preprocessing module, which is used to perform preprocessing operations on a set of lung CT images to obtain a set of standard images;

[0038] A detection module, which is used to integrate thrombus tags and a set of standard images into a detection data set, input the detection data set into a three-dimensional object detection network model for model training to obtain multiple detection training models; and respectively detect and identify the detection data set through multiple detection training models to obtain multiple preliminary detection results, and fuse the multiple preliminary detection results to obtain an identification result;

[0039] A segmentation module, which is used to integrate thrombus tags, a set of standard images and a set of lung CT images into a segmentation data set, train a semi-supervised semantic segmentation model through the segmentation data set to obtain a final segmentation model; and segment the identification result through the final segmentation model to obtain a segmentation result;

[0040] An accuracy compensation module, which is used to correct the segmentation result through an intermediate distance compensation mechanism to obtain a merged result;

[0041] A visualization and report generation module, which is used to perform three-dimensional visualization display on the merged result and generate a screening report.

[0042] The present invention discloses the following technical effects: The lung embolism detection and segmentation method and system based on intermediate distance compensation provided by the present invention includes: pre-labeling a set of lung CT images to obtain pre-labeled tags; obtaining thrombus tags through the pre-labeled tags; performing preprocessing operations on the set of lung CT images to obtain a set of standard images; integrating thrombus tags and the set of standard images into a detection data set, inputting the detection data set into a three-dimensional object detection network model for model training to obtain multiple detection training models; respectively detecting and identifying the detection data set through multiple detection training models to obtain multiple preliminary detection results, and fusing the multiple preliminary detection results to obtain an identification result; integrating thrombus tags, the set of standard images and the set of lung CT images into a segmentation data set, training a semi-supervised semantic segmentation model through the segmentation data set to obtain a final segmentation model; segmenting the identification result through the final segmentation model to obtain a segmentation result; correcting the segmentation result through an intermediate distance compensation mechanism to obtain a merged result. This method intelligently identifies lung embolism through deep learning and image processing, improving the detection accuracy of lung embolism. Description of the Drawings

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 Flow chart of the pulmonary embolism detection and segmentation method of the present invention;

[0045] Figure 2 Flow chart of the training of the 3D object detection network model of the present invention;

[0046] Figure 3 Detection training model diagram of the present invention;

[0047] Figure 4 Flow chart of the training of the semi-supervised semantic segmentation model of the present invention;

[0048] Figure 5 Flow chart of the correction of the segmentation result of the present invention;

[0049] Figure 6 First slice diagram of thrombus recognition in the embodiment of the present invention;

[0050] Figure 7 Second slice diagram of thrombus recognition in the embodiment of the present invention;

[0051] Figure 8 Third slice diagram of thrombus recognition in the embodiment of the present invention;

[0052] Figure 9 First slice diagram of the vacant part in the embodiment of the present invention;

[0053] Figure 10 Second slice diagram of the vacant part in the embodiment of the present invention;

[0054] Figure 11 3D diagram of the vacant part in the embodiment of the present invention;

[0055] Figure 12 First slice diagram of the merging result in the embodiment of the present invention;

[0056] Figure 13 Second slice diagram of the merging result in the embodiment of the present invention;

[0057] Figure 14 Third slice diagram of the merging result in the embodiment of the present invention;

[0058] Figure 15 3D diagram of the merging result in the embodiment of the present invention. Detailed implementation manners

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0061] As Figure 1 shown, the present invention provides a pulmonary embolism detection and segmentation method based on intermediate distance compensation, including the following steps:

[0062] Step 100: Perform pre-annotation on the lung CT image set to obtain pre-annotation labels; the pre-annotation includes: simple manual annotation and pixel-level annotation.

[0063] Specifically, doctors perform simple manual annotation on the slices of the first layer, the layer with the largest cross-section, and the last layer of the thrombus position in the lung CT image to obtain pre-annotation labels. The actual number of annotated slices is determined by the state of the thrombus in different CT images. For a small number of thrombi, a complete annotation method is adopted, that is, pixel-level annotation for each layer.

[0064] It should be noted that after doctors obtain the lung CT image data of patients, they only need to perform simple manual annotation on most of them, and only need to perform pixel-level annotation on the remaining small part of CT images for pulmonary embolism. Compared with the pixel-level annotation required for all data in fully supervised semantic segmentation, this method greatly reduces the burden of dataset construction in semantic segmentation tasks.

[0065] Step 200: Obtain thrombus labels through the pre-annotation labels.

[0066] Specifically, coordinate detection is performed on the lung CT image according to the pre-annotation labels to obtain an AABB thrombus bounding box with a cuboid structure, and then the two diagonal coordinates of the thrombus bounding box, that is, six scalars, are determined. A thrombus bounding box label (i.e., thrombus label) with the same data format as the lung CT image is constructed according to the obtained six scalars.

[0067] It should be noted that the thrombus bounding box label consists of the center point of the cuboid structure and its length, width, and height, and the process of obtaining the cuboid structure is coordinate detection.

[0068] Step 300: Preprocess the lung CT image set to obtain a standard image set. The preprocessing operations include: adjusting window width and window level, adjusting image spacing, adjusting the image direction cosine matrix, image denoising, and hybrid cropping.

[0069] Specifically, adjusting the window width and window level means: adjusting the window width and window level of the lung CT images to a unified range for observing the thrombus structure in the lung CT images. In this embodiment, the unified window width and window level are 350 / 40. Adjusting the image spacing means: standardizing the pixel spacing between different lung CT images to a unified standard spacing to ensure that the images input into the model have the same image pixel spacing. Adjusting the image direction cosine matrix means: unifying the image direction cosine matrix into the identity matrix. Image denoising means: using denoising techniques such as Gaussian filtering and median filtering to process the CT images to reduce noise interference and improve image quality. Hybrid cropping means: randomly cutting and pasting image regions between two images and mixing their labels simultaneously to enhance the robustness and generalization ability of the model.

[0070] Step 400: Integrate the thrombus labels and the standard image set into a detection data set, and input the detection data set into a 3D object detection network model for model training to obtain multiple detection training models. Specifically, as Figure 2 shown, it includes:

[0071] Step 401: Divide the detection data set into a detection training set and a detection validation set;

[0072] Specifically, use the cross-validation method to reasonably split the detection data set, thereby dividing the detection data set into a detection training set and a detection validation set.

[0073] Step 402: Train the 3D object detection network model through the detection training set to obtain multiple preliminary training models;

[0074] Specifically, input the detection data set into the 3D object detection model for training to establish the mapping relationship between CT image features and thrombus positions, providing a basis for thrombus detection. Multiple preliminary training models are generated through the cross-validation method. Each preliminary training model will be evaluated and its performance will be verified on different detection data sets.

[0075] It should be noted that during the model training process, the model performance is continuously monitored and evaluated to ensure that the model has good generalization ability, effectively reducing the risk of overfitting, so that the model still maintains a high detection accuracy on unseen test data.

[0076] More specifically, the 3D object detection network model of this embodiment adopts the nnDetection framework. The nnDetection framework is based on the improved RetinaNet structure and can automatically perform object detection to improve the detection accuracy and efficiency of pulmonary embolism. This framework has a highly modular design and supports automatic network configuration and optimization to adapt to different medical image data. During the training process, the nnDetection framework can automatically configure the hyperparameters and architecture of the neural network, including adjusting parameters such as the number of network layers, convolutional kernel size, and learning rate, so as to find the optimal model structure. This automatic configuration enables the model to better adapt to the CT image data of different patients and improves the robustness of pulmonary embolism detection. Among them, the RetinaUNet structure is used as the basic network, which combines the segmentation ability and object detection ability. Through feature extraction and feature fusion of lung CT images, this nnDetection framework structure can accurately locate the detection box position of pulmonary embolism and calculate the confidence level.

[0077] Step 403: Extract features from the detection training set through multi-scale feature fusion technology to obtain thrombus features;

[0078] Specifically, the multi-scale feature fusion technology uses multiple convolutional layers and pooling layers to extract feature maps from different scales, enabling the model to recognize the features of thrombus from details to the whole, ensuring accurate detection of thrombus in various sizes and shapes, and generating thrombus features (thrombus position and thrombus size). Among them, the low-level feature maps retain a higher spatial resolution, which helps to identify small thrombi; while the high-level feature maps provide stronger semantic information, which helps to locate large thrombi.

[0079] Step 404: Set multi-sized anchor boxes according to the thrombus features and determine the thrombus area through the anchor boxes;

[0080] Specifically, according to the extracted thrombus features, different-sized anchor boxes are set for each image region. Each anchor box can cover thrombus regions of different scales, and the three-dimensional spatial position of the thrombus is accurately located by calculating the overlap degree between the anchor box and the thrombus target. The position, size, and shape of the anchor box are adjusted according to the thrombus instances in the training data. And redundant anchor boxes are removed through the non-maximum suppression (NMS) algorithm, and the optimal anchor boxes are retained to ensure more accurate detection results.

[0081] It should be noted that the K-means clustering algorithm is used to generate more suitable anchor boxes for detection to improve the accuracy and positioning accuracy of the anchor boxes. Through the anchor box mechanism, aiming at the diversity of thrombi (including changes in shape, size, and position), the design of multi-sized anchor boxes is adopted, which enhances the detection ability of the model when facing different thrombus morphologies and improves the recognition and positioning accuracy of thrombi.

[0082] Step 405: Optimize and train the preliminary training model through the thrombus region and the first loss function to obtain a detection training model.

[0083] Specifically, appropriate loss functions are introduced during the training process, including position loss and confidence loss. The position loss is used to optimize the accuracy of the thrombus position, and the confidence loss is used to optimize the credibility of thrombus detection. During the training process, the model will continuously iterate and optimize to reduce these losses and improve the final detection effect.

[0084] Specifically, the confidence loss function uses the BCE loss function. The calculation formula of the first loss function is:

[0085]

[0086] Where GIoU is the loss function, IoU is the IoU loss function, A is the optimal anchor box, B is the thrombus label, C is the smallest enclosing box formed by A and B, |C| is the area of the smallest enclosing box, and |C\(A∪B)| is the area obtained by removing the union of A and B from C.

[0087] It should be noted that by introducing loss function optimization techniques in the optimization training process, combining confidence loss and position loss, the accuracy of the model in locating the thrombus region and the precision of judging the thrombus are improved.

[0088] Furthermore, a spatial attention mechanism is also introduced in the optimization training process to improve the model's attention to the thrombus region. The spatial attention mechanism enables the model to pay more attention to key regions during the detection process and improve the accuracy of thrombus localization. This mechanism can adaptively adjust the attention distribution of the model according to the spatial characteristics of the thrombus, so as to detect the thrombus more accurately in complex backgrounds.

[0089] As Figure 3 shown, the input image passes through the attention mechanism and the feature extraction network of the detection training model in sequence to obtain two outputs. One output is obtained through the regression task of the coordinates of the anchor box, and the other output is obtained by classifying the size of the thrombus or the anchor box. Both outputs use the coordinates and size of the anchor box for supervision and calculate the loss. The feature extraction network has a built-in feature pyramid structure, and layer-by-layer feature transfer is performed between different feature extraction networks through convolution + normalization + activation function, and feature transfer is also performed through skip connections.

[0090] Step 500: Detect and identify the detection dataset through multiple detection training models respectively to obtain multiple preliminary detection results, and fuse the multiple preliminary detection results to obtain the identification result.

[0091] Specifically, the outputs of multiple detection training models are fused through a weighted average or voting mechanism, further improving the accuracy and robustness of detection, reducing the impact of possible biases in the output of a single model on the results, and enhancing the ability to recognize complex thrombus morphologies.

[0092] Step 600: Integrate the thrombus labels, standard image set, and lung CT image set into a segmentation data set, and train a semi-supervised semantic segmentation model using the segmentation data set to obtain a final segmentation model. Specifically, Figure 4 as follows:

[0093] Step 601: Integrate the thrombus labels, standard image set, and lung CT image set into a segmentation data set;

[0094] Specifically, mix the thrombus labels, standard image set, and lung CT image set together to create a segmentation data set containing real labels and pseudo-labels. Through an adaptive pseudo-label generation strategy, initialize the prediction of unlabeled lung CT images to generate pseudo-labels for the unlabeled images. The quality of the pseudo-labels is dynamically adjusted according to the confidence of the model, and low-confidence pseudo-labels will be given lower weights to prevent negative impacts on training. Expand the anchor box by a certain proportion of pixels according to the size of the anchor box itself to better obtain the contour information of the thrombus; at the same time, control the thrombus segmentation within a smaller range, which is beneficial to obtaining better segmentation results with fewer network parameters.

[0095] Step 602: Perform semi-supervised learning optimization on the semi-supervised semantic segmentation model using the segmentation data set to obtain an optimized segmentation model;

[0096] Specifically, the semi-supervised semantic segmentation model in this embodiment adopts the Unimatch framework based on the U-Net network structure. The Unimatch framework improves the accuracy and robustness of semantic segmentation through a semi-supervised learning method that combines labeled data and unlabeled data. In the diagnosis of pulmonary embolism in lung CT images, the Unimatch framework uses the bounding box regions in the object detection results for segmentation, effectively reducing the segmentation calculation amount and improving the overall segmentation efficiency. The Unimatch framework performs feature learning through consistency regularization and feature alignment techniques, ensuring the effective utilization of unlabeled data, and enhancing the model's precise segmentation ability for thrombus regions through multi-scale feature extraction. The Unimatch framework also adopts a pseudo-label generation strategy, using the high-confidence segmentation results in unlabeled data as pseudo-labels, expanding the utilization range of labeled data, improving the generalization ability of the model, and enabling it to better adapt to the morphological characteristics of pulmonary embolism in different patients. The Unimatch framework inherits the powerful feature extraction and context information aggregation capabilities of U-Net, and finely segments the thrombus regions in the input CT images through an encoder-decoder structure, and further improves the model performance by combining semi-supervised learning techniques. The encoder part consists of multiple convolutional layers and pooling layers, which are used to extract multi-scale feature information of the image; the decoder part restores the spatial resolution of the image through upsampling and convolutional layers, and finally outputs a segmentation result with the same size as the input image.

[0097] Step 603: Perform cross-validation on the optimized segmentation model;

[0098] Specifically, cross-validation is used to evaluate the optimized segmentation model, and the validation set used comes from CT images of different data sources to ensure the generalization ability of the optimized segmentation model. The evaluation metrics used include the Dice coefficient, IoU (Intersection over Union), etc., to ensure that the model can maintain high accuracy in complex pulmonary embolism segmentation tasks.

[0099] Step 604: Further optimize the model after cross-validation through the second loss function to obtain the final segmentation model.

[0100] Specifically, the second loss function includes the labeled loss function L l and the unlabeled loss function L u in two parts. The labeled loss function is calculated through the hard cross-entropy loss function H, which is used to measure the difference between the prediction of the model for the labeled image and the true label. The expression is:

[0101]

[0102] Among them, B l is the number of labeled images, H is the hard cross-entropy loss function, is the predicted value for the i-th image, is the true label for the i-th image.

[0103] The unlabeled loss function realizes self-supervised learning through pseudo-labels generated by the model itself, and introduces diverse image enhancement methods (such as color enhancement, cropping, rotation, and scaling, etc.) and feature perturbation methods (such as Gaussian noise, feature pruning) to implement the enhanced perturbation strategy, so as to improve the robustness of the model and its adaptability to different pulmonary embolism morphologies. First, pseudo-labels are generated on weakly enhanced images, and then training is carried out on strongly enhanced images. This weak-to-strong consistency regularization strategy maintains the learning invariance on strongly enhanced images. The unlabeled loss function updates the optimized segmentation model only when the confidence of the pseudo-label exceeds a preset confidence threshold to reduce the influence of noisy pseudo-labels. The expression of the unlabeled loss function is:

[0104]

[0105] where B u is the number of unlabeled images, is the i-th pseudo-label of the weakly enhanced image, is the i-th pseudo-label of the strongly enhanced image, τ is the preset confidence threshold, is the hard one-hot label obtained through the argmax operation.

[0106] The second loss function balances the contributions of the labeled loss function and the unlabeled loss function to model training, expressed as: L = L l + λL u ; where L is the second loss function, λ is the weight coefficient, and λ is usually set to 1, indicating that the same weight is given to the labeled loss function and the unlabeled loss function.

[0107] It should be noted that training the model through the second loss function can effectively utilize limited labeled data and a large amount of unlabeled data, improve the performance of the model in semi-supervised semantic segmentation tasks, reduce the dependence on manual annotation, and lower the cost. It enables the model to effectively learn from unlabeled data while maintaining high accuracy, thereby enhancing the generalization ability of the model.

[0108] Step 700: Segment the recognition result through the final segmentation model to obtain the segmentation result;

[0109] Specifically, select the candidate regions containing pulmonary embolism in the recognition result as the input of the final segmentation model, generate high-confidence pseudo-labels based on the unlabeled part in the input of the final segmentation model, and segment the recognition result through the high-confidence pseudo-labels to obtain the segmentation result.

[0110] Step 800: Correct the segmentation result through an intermediate distance compensation mechanism to obtain a merging result.

[0111] Specifically, as Figure 5 shown, it includes:

[0112] Step 801: Calculate the intermediate distance between the segmentation results through a search algorithm.

[0113] Specifically, extract the boundary of each detected thrombus from the segmentation result. These boundaries are located outside the thrombus region. Using the blood vessel centerline as a reference, determine the position of the thrombus region in the blood vessel. Calculate the intermediate distance between all detected thrombus boundaries using a search algorithm, and then use a sorting algorithm to obtain the adjacent thrombi corresponding to each thrombus boundary that meet the intermediate distance. The intermediate distance refers to the pixel distance between the closest points of two thrombus boundaries along the direction of the blood vessel centerline in the segmentation result. The specific calculation method is as follows: Find the two closest boundary points of the thrombi along the blood vessel centerline, and obtain the intermediate distance by calculating the pixel distance between these two closest boundary points. If the two thrombi are not on the same blood vessel, also calculate the distance between them along the blood vessel centerline to obtain a global intermediate distance.

[0114] Step 802: Judge the two thrombi in the segmentation result according to the comparison result between the intermediate distance and a preset threshold to obtain a judgment result. The judgment result is: the same thrombus or different thrombi.

[0115] Specifically, the preset threshold is determined through experiments and empirical data. When the intermediate distance is less than the preset threshold, the two thrombi are judged to be the same thrombus, otherwise they are judged to be different thrombi. The preset threshold in this embodiment is 3 pixels.

[0116] Step 803: Merge the two thrombi whose judgment result is the same thrombus to obtain a merging result.

[0117] Specifically, merge the boundary regions of the two thrombi whose judgment result is the same thrombus according to the gray scale and gray scale change to form a large thrombus region. The merging operation is realized by filling pixel points between the two thrombus regions. The region for filling pixel points is determined by carefully analyzing the image gray scale characteristics and gray scale change of the region. First, calculate the gray scale mean, variance and texture information of the thrombus region to be merged to evaluate the consistency and naturalness of this region. Subsequently, judge the clarity of the boundary and the demarcation from the surrounding tissues by evaluating the gray scale change at the edge of the region to be merged, such as gradient or gray scale change rate. As Figures 6 - 8 shown, the same thrombus is misclassified into three thrombi, and the judgment result is that the three thrombi are the same thrombus, Figure 9 and Figure 10 both are the vacant parts between the three thrombi. The three-dimensional schematic diagram is asFigure 11 As shown Figures 12 - 14 All are the results of thrombus combination. The three-dimensional schematic diagram is as Figure 15 shown

[0118] Furthermore, after the combination is completed, the morphology of the combined thrombus is checked for rationality to ensure that the new thrombus area conforms to the actual anatomical structure in terms of spatial distribution. If the combined area is too large or the morphology is irregular, the morphology of the thrombus area is adjusted through further morphological operations (such as morphological dilation and erosion).

[0119] It should be noted that the combination strategy based on the intermediate distance of thrombus effectively eliminates the multiple segmentation situations caused by model errors or low segmentation accuracy, improves the accuracy of pulmonary embolism detection, and reduces the occurrence of false alarms.

[0120] Furthermore, this embodiment also performs visual display according to the combination result and generates a screening report. The volume rendering technology is used to convert the CT image into an intuitive three-dimensional view. Combining the detected thrombus position and confidence, the pulmonary embolism characteristics in the pulmonary CT image are accurately displayed, enabling doctors to directly observe the three-dimensional position distribution of pulmonary embolism. The thrombus and its confidence are also visually distinguished by means of transparency, color coding, and texture changes. Transparency is used to highlight the thrombus area of interest, and color coding is used to distinguish according to the confidence of thrombus detection, so as to facilitate doctors to quickly identify and judge the severity of pulmonary embolism. The screening report includes the three-dimensional view of the patient's pulmonary CT image, the confidence analysis of thrombus detection, the segmentation result, and the corresponding confidence, and is presented in the form of structured information, which can assist doctors in comprehensive clinical diagnosis and greatly improve the diagnosis efficiency.

[0121] The present invention also provides a pulmonary embolism detection and segmentation system based on intermediate distance compensation, including:

[0122] An image acquisition module, which is used to pre-label the pulmonary CT image set to obtain pre-labeled tags;

[0123] A preprocessing module, which is used to perform preprocessing operations on the pulmonary CT image set to obtain a standard image set;

[0124] A detection module, which is used to integrate the thrombus tags and the standard image set into a detection data set, input the detection data set into a three-dimensional object detection network model for model training to obtain multiple detection training models; and respectively detect and identify the detection data set through the multiple detection training models to obtain multiple preliminary detection results, and fuse the multiple preliminary detection results to obtain an identification result;

[0125] The segmentation module is used to integrate the thrombus labels, the standard image set, and the pulmonary CT image set into a segmentation data set, train a semi-supervised semantic segmentation model through the segmentation data set to obtain a final segmentation model, and segment the recognition result through the final segmentation model to obtain a segmentation result.

[0126] The accuracy compensation module is used to correct the segmentation result through an intermediate distance compensation mechanism to obtain a merged result.

[0127] The visualization and report generation module is used to perform three-dimensional visualization of the merged result and generate a screening report.

[0128] The beneficial effects of the present invention are as follows:

[0129] 1) By using a computer artificial intelligence deep learning algorithm to detect and identify pulmonary artery thrombus, the accuracy and reliability of the detection are significantly improved, and pulmonary embolism can be effectively identified in complex clinical images, helping doctors make a diagnosis more quickly and accurately.

[0130] 2) By designing multi-size anchor boxes according to the morphological characteristics of different thrombi, the detection ability of the model for various thrombi with different morphologies and sizes is improved, ensuring comprehensive and accurate thrombus localization, and effectively avoiding missed diagnosis and misdiagnosis.

[0131] 3) Through loss function optimization technology, the accuracy of the model in locating the thrombus area is improved, and the overall robustness of the model is enhanced, enabling consistent diagnostic effects in different types of cases.

[0132] 4) By adopting a semi-supervised learning method, by combining a small amount of labeled data and a large amount of unlabeled data, the value of the unlabeled data is fully utilized, the performance and generalization ability of the segmentation model are improved, and the manual labeling cost is reduced.

[0133] 5) By adopting the method of expanding the thrombus bounding box obtained by object detection by several pixels and inputting it into the segmentation model, this method enables the segmentation model to focus on the contour boundary information around small thrombi and obtain a more accurate segmentation result.

[0134] 6) By introducing a merging strategy based on the intermediate distance of thrombi, the problem of mis-segmentation in the segmentation result is effectively solved, the phenomenon of multiple segmentations caused by insufficient model accuracy is avoided, the accuracy of pulmonary embolism detection is improved, and the false alarm rate is reduced.

[0135] 7) By three-dimensionally visualizing the pulmonary embolism segmentation result, the spatial distribution of the thrombus can be intuitively presented, helping doctors better understand and analyze the characteristics of pulmonary embolism, and assisting clinical decision-making.

[0136] 8) Combining with the analysis of the blood vessel centerline provides a more accurate thrombus localization method for pulmonary embolism detection, improves the accuracy of the thrombus boundary, and ensures the refinement of detection and segmentation;

[0137] 9) The recognition ability of pulmonary artery thrombus is improved through the labeling technique, and the accuracy of segmentation is also improved.

[0138] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0139] Specific examples are applied in the present invention to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A pulmonary embolism detection and segmentation method based on intermediate distance compensation, characterized in that: The steps include: Pre-label the lung CT image set to obtain pre-labeled labels; The pre-labeling includes: manual labeling and pixel-level labeling; Obtaining a thrombus label through the pre-labeled label; Performing a preprocessing operation on the lung CT image set to obtain a standard image set; the preprocessing operation includes: adjusting the window width and window position, adjusting the image spacing, adjusting the image direction cosine matrix, image denoising and hybrid cropping; Integrating the thrombus label and the standard image set into a detection data set, inputting the detection data set into a three-dimensional target detection network model for model training, and obtaining multiple detection training models; Using a plurality of the detection training models to respectively detect and identify the detection data set, a plurality of preliminary detection results are obtained, and the plurality of preliminary detection results are fused to obtain a recognition result; Integrating the thrombus label, the standard image set, and the lung CT image set into a segmentation data set, and performing model training on a semi-supervised semantic segmentation model through the segmentation data set to obtain a final segmentation model; Segmenting the recognition result by using the final segmentation model to obtain a segmentation result; The segmentation result is corrected by an intermediate distance compensation mechanism to obtain a merged result.

2. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 1, characterized in that: The thrombus label is obtained through the pre-labeled label, specifically: coordinate detection is performed on the lung CT image according to the pre-labeled label to obtain a thrombus bounding box, and the thrombus label is obtained according to the 6 corner coordinate scalars of the thrombus bounding box; the thrombus bounding box is a rectangular parallelepiped structure.

3. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 1, characterized in that: The thrombus label and the standard image set are integrated into a detection data set, and the detection data set is input into a three-dimensional target detection network model for model training to obtain multiple detection training models, including: Dividing the detection data set into a detection training set and a detection verification set; Performing model training on the three-dimensional object detection network model using the detection training set to obtain multiple preliminary training models; Extracting features from the detection training set by using a multi-scale feature fusion technology to obtain thrombus features; Setting anchor frames of multiple sizes according to the thrombus characteristics, and determining the thrombus area by means of the anchor frames; The preliminary training model is optimized and trained using the thrombus area and the first loss function to obtain the detection training model.

4. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 3, characterized in that: The first loss function includes: position loss and confidence loss; the calculation formula of the first loss function is: Among them, GIoU is the first loss function, IoU is the IoU loss function, A is the optimal anchor box, B is the thrombus label, C is the minimum closed box formed by A and B, |C| is the area of ​​the minimum closed box, and |C\(A∪B)| is the area of ​​C after removing the union of A and B.

5. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 1, characterized in that: The thrombus label, the standard image set and the lung CT image set are integrated into a segmentation data set, and the semi-supervised semantic segmentation model is trained through the segmentation data set to obtain the final segmentation model. Specifically, the segmentation data set is input into the semi-supervised semantic segmentation model, and the model is trained through consistency regularization loss, enhanced perturbation strategy, pseudo-label generation strategy, cross-validation and second loss function to obtain the final segmentation model.

6. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 5, characterized in that: The expression of the second loss function is: L = L l +λL u ; Wherein, L is the second loss function value, L l is the labeling loss function, L u is the unlabeled loss function, λ is the weight coefficient; The expression of the labeling loss function is: Among them, B l is the number of labeled images, H is the hard cross entropy loss function, is the predicted value of the i-th image, is the true label of the i-th image; The expression of the unlabeled loss function is: Among them, B u is the number of unlabeled images, is the i-th pseudo label of the weakly enhanced image, is the i-th pseudo label of the strongly enhanced image, τ is the preset confidence threshold, It is a hard hot label.

7. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 1, characterized in that: The recognition result is segmented by the final segmentation model to obtain the segmentation result, specifically: a candidate area containing pulmonary embolism in the recognition result is selected as the input of the final segmentation model, a high-confidence pseudo-label is generated according to the unlabeled part in the input of the final segmentation model, and the recognition result is segmented by the high-confidence pseudo-label to obtain the segmentation result.

8. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 1, characterized in that: The segmentation result is corrected by an intermediate distance compensation mechanism to obtain a merged result, including: Calculate the intermediate distance between the segmentation results by using a search algorithm; According to the comparison result between the intermediate distance and the preset threshold, the two thrombi in the segmentation result are judged to obtain a judgment result; the judgment result is: the same thrombus or different thrombi; The two thrombi that are judged to be the same thrombi are merged to obtain a merged result.

9. The pulmonary embolism detection and segmentation method based on intermediate distance compensation according to claim 8, characterized in that: The intermediate distance between the segmentation results is calculated by a search algorithm, specifically: the thrombus boundary is extracted from the segmentation result, and the pixel distance between the two closest points of the thrombus is calculated through the blood vessel centerline and the thrombus boundary to obtain the intermediate distance.

10. A pulmonary embolism detection and segmentation system based on intermediate distance compensation, characterized in that: include: An image acquisition module, wherein the image acquisition module is used to pre-label a lung CT image set to obtain a pre-labeled label; A preprocessing module, the preprocessing module is used to perform a preprocessing operation on the lung CT image set to obtain a standard image set; A detection module, wherein the detection module is used to integrate the thrombus label and the standard image set into a detection data set, and input the detection data set into a three-dimensional target detection network model for model training to obtain multiple detection training models; and respectively performing detection and recognition on the detection data set through a plurality of the detection training models to obtain a plurality of preliminary detection results, and fusing the plurality of the preliminary detection results to obtain a recognition result; A segmentation module, wherein the segmentation module is used to integrate the thrombus label, the standard image set and the lung CT image set into a segmentation data set, perform model training on a semi-supervised semantic segmentation model through the segmentation data set to obtain a final segmentation model; and perform segmentation on the recognition result through the final segmentation model to obtain a segmentation result; A precision compensation module, the precision compensation module is used to correct the segmentation result through an intermediate distance compensation mechanism to obtain a merged result; A visualization and report generation module, wherein the visualization and report generation module is used to perform a three-dimensional visualization of the combined results and generate a screening report.

Images