Pulmonary blood vessel enhancement method based on bimodal imaging

Abstract

The invention relates to the technical field of medical image processing, in particular to a lung blood vessel enhancement method based on bimodal imaging, and the method comprises the steps: obtaining a fluorescence image of a lung blood vessel through fluorescence development; obtaining an infrared image of the lung blood vessel through infrared development; based on the intensity information of the fluorescence image and the infrared image and the morphological information of the blood vessel, the enhancement weights of the two modals are calculated, and a self-adaptive enhanced pseudo-color blood vessel image is obtained; performing multi-scale blood vessel enhancement filtering processing on the pseudo-color blood vessel image to obtain a blood vessel enhancement result; performing weight fusion on the blood vessel enhancement result, the fluorescence image and the infrared image to obtain a final blood vessel enhancement result; and outputting a final blood vessel enhancement result, and comprehensively capturing blood vessels with different depths and different scales by skillfully combining the advantages of fluorescence imaging and infrared imaging. The high sensitivity of the fluorescence imaging technology ensures accurate display of tiny blood vessels, and infrared imaging provides blood vessel structure information of deep tissues.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a pulmonary blood vessel enhancement method based on dual-modality imaging. Background Art

[0002] With the continuous development of medical imaging technology, the imaging and analysis of pulmonary blood vessels plays an increasingly important role in the diagnosis of lung diseases. Traditional pulmonary vascular imaging methods mainly rely on a single modality, such as X-ray angiography or CT angiography. Although these methods can provide certain vascular structure information, they have obvious limitations in displaying small blood vessels and complex vascular networks.

[0003] In recent years, researchers have begun to try to use multimodal imaging technology to improve the imaging quality of pulmonary blood vessels. Among them, the dual-modality method combining fluorescence imaging and infrared imaging has attracted widespread attention. Fluorescence imaging can provide highly sensitive vascular information, especially for the display of tiny blood vessels; while infrared imaging can provide vascular structure information of deep tissues. However, the current dual-modality imaging method still faces some challenges.

[0004] First, existing dual-modality imaging methods often simply superimpose or fuse the images of the two modalities, lacking in-depth analysis and optimization of the characteristics of different modalities. As a result, the final fusion result may not fully utilize the advantages of the two modalities, and may even introduce new image artifacts.

[0005] Secondly, in terms of vascular enhancement, existing methods mostly use fixed-parameter filtering algorithms, which are difficult to adapt to vascular structures of different scales and shapes. This one-size-fits-all approach often leads to over-enhancement or under-enhancement of certain vascular structures, affecting the accuracy of vascular identification.

[0006] Furthermore, most existing methods for blood vessel recognition and segmentation rely on traditional image processing algorithms, which are not very effective when dealing with complex backgrounds and low-contrast blood vessels. Although some studies have begun to try to introduce deep learning technology, how to effectively combine traditional image processing methods and deep learning technology is still a problem that needs to be solved.

[0007] Finally, existing methods often have difficulty in dealing with blood vessels of different thicknesses and depths. For example, some methods may perform well in enhancing major blood vessels, but poorly in dealing with small blood vessels; or they may be effective in dealing with superficial blood vessels, but have limited effect in enhancing deep blood vessels.

[0008] In view of the above problems, there is an urgent need for a method that can fully utilize the advantages of dual-modality imaging, achieve adaptive vascular enhancement, and effectively process blood vessels of different scales and depths, so as to improve the quality and accuracy of pulmonary vascular imaging. Summary of the invention

[0009] The present invention provides a method for enhancing pulmonary blood vessels based on dual-modal imaging, aiming to solve the above-mentioned problems existing in the prior art. The present invention innovatively combines fluorescence imaging and infrared imaging technology, and introduces adaptive weight calculation and multi-scale enhancement technology to achieve high-quality imaging and enhancement of pulmonary blood vessels.

[0010] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0011] A method for enhancing pulmonary blood vessels based on dual-modality imaging, comprising:

[0012] The acquisition steps include:

[0013] Fluorescent images of pulmonary vessels were obtained by fluorescence angiography;

[0014] Infrared imaging is used to obtain infrared images of the blood vessels in the lungs;

[0015] Processing steps include:

[0016] Based on the intensity information of the fluorescence image and the infrared image and the morphological information of the blood vessels, the enhancement weights of the two modalities are calculated to obtain an adaptively enhanced pseudo-color blood vessel image;

[0017] Performing multi-scale blood vessel enhancement filtering processing on the pseudo-color blood vessel image to obtain a blood vessel enhancement result;

[0018] The blood vessel enhancement result is weightedly fused with the fluorescence image and the infrared image to obtain a final blood vessel enhancement result;

[0019] Output steps include:

[0020] The final blood vessel enhancement result is outputted.

[0021] Preferably, the obtaining step specifically includes:

[0022] By intravenously injecting fluorescent nanoparticles, collecting fluorescent signals of pulmonary blood vessels under laser excitation, and obtaining the fluorescent image;

[0023] The infrared image is acquired by using laser imaging technology.

[0024] Preferably, the process of calculating the enhancement weight in the processing step specifically includes:

[0025] Performing grayscale stretching on the fluorescent image and the infrared image respectively to obtain enhanced images;

[0026] Performing pseudo color mapping on the enhanced image to obtain a pseudo color blood vessel image;

[0027] Obtaining an intensity enhancement result based on the intensity information of the pseudo-color blood vessel image;

[0028] The morphological enhancement results are obtained according to the vascular morphology and branching information;

[0029] The enhancement weight of each modality is calculated based on the intensity enhancement result and the morphology enhancement result.

[0030] Preferably, the multi-scale blood vessel enhancement filtering process specifically includes:

[0031] A multi-scale separation algorithm is used to perform convolution calculations using a series of Gaussian kernels with different radii to obtain a multi-scale image sequence;

[0032] Using the Laplace pyramid algorithm, multi-layer filtering is performed on the pseudo-color blood vessel image to decompose it into multiple groups of images to be enhanced;

[0033] Calculating the channel similarity and scale similarity between the images to be enhanced;

[0034] Based on the channel similarity and scale similarity, calculating the fusion weight of each image to be enhanced;

[0035] The enhanced image is reconstructed by weighted summation.

[0036] As an advantage, the method further comprises the following steps:

[0037] Using a segmentation threshold to perform binarization processing on the enhanced image, and extracting blood vessels and blood vessel centerlines;

[0038] Calculate characteristic information of blood vessel contour;

[0039] Calculating the direction of the blood vessel based on the characteristic information of the blood vessel contour;

[0040] The blood vessel centerline is corrected according to the blood vessel direction to obtain a final blood vessel enhancement result.

[0041] As an advantage, the method further comprises the following steps:

[0042] Designing a blood vessel recognition module to perform deep learning on the final blood vessel enhancement result and extract boundary features of each type of blood vessel in the image;

[0043] Based on the boundary features, blood vessel fusion is performed to obtain an original blood vessel fusion image;

[0044] The original vascular fusion image is subjected to vascular fusion correction to obtain a corrected vascular enhancement image.

[0045] Preferably, the design of the vascular recognition module specifically includes:

[0046] Filter the brightness gradient map of the blood vessel enhanced image to obtain the salient area corresponding to the foreground area;

[0047] Filter the brightness gradient map of the obvious blood vessel enhancement image to obtain the blood vessel boundary area;

[0048] synthesizing the notable region and the blood vessel boundary region to obtain an enhanced blood vessel contour region;

[0049] Using the blood vessel structure of the image before enhancement, correcting the blood vessel boundary points in the blood vessel contour area;

[0050] Dijkstra's algorithm is used to smooth the blood vessel contour curve in the corrected blood vessel contour area;

[0051] The smoothed vascular contour curve is projected onto the enhanced image to complete the vascular recognition process.

[0052] As an advantage, the method further comprises the following steps:

[0053] For the final blood vessel enhancement result, finding a corresponding area in the original image;

[0054] Mark the difference between the blood vessels in the enhanced result and the initial blood vessel fusion image in the corresponding original image to obtain the difference rate;

[0055] If the difference rate is less than a preset threshold, the algorithm is determined to be successful;

[0056] If the difference rate is greater than or equal to the preset threshold, the method returns to the acquisition step and executes the method again.

[0057] Preferably, the preset threshold is 10%.

[0058] As an advantage, the method further comprises the following steps:

[0059] Using the trained pulmonary blood vessel recognition model to recognize the final blood vessel enhancement result, to obtain a blood vessel classification result;

[0060] The training method of the pulmonary blood vessel recognition model includes:

[0061] Obtaining real blood vessel enhancement results in dual-modality images as training data;

[0062] The training data is used as input of a deep learning framework, and the blood vessel recognition model is trained using a cross entropy loss function and an optimization method.

[0063] The method of the present invention has the following significant technical effects:

[0064] First, the present invention achieves comprehensive capture of blood vessels at different depths and scales by cleverly combining the advantages of fluorescence imaging and infrared imaging. The high sensitivity of fluorescence imaging technology ensures accurate display of tiny blood vessels, while infrared imaging provides information on the vascular structure of deep tissues. This dual-modality combination not only expands the depth range of vascular imaging, but also improves the ability to identify complex vascular networks.

[0065] Secondly, the present invention introduces an innovative adaptive weight calculation algorithm. This algorithm can automatically adjust the enhancement strategy according to the characteristics of different images, effectively solving the limitations of the traditional fixed parameter method. Through the comprehensive analysis of image intensity information and vascular morphological characteristics, this method can intelligently determine the enhancement weight of each modality, thereby achieving accurate enhancement of different types of blood vessels. This adaptability greatly improves the robustness of the method, enabling it to adapt to images from different patients and different imaging conditions.

[0066] Furthermore, the present invention adopts a multi-scale blood vessel enhancement filtering technology. This technology can simultaneously process blood vessel structures of different scales, effectively resolving the contradiction of traditional methods in processing blood vessels of different thicknesses. By using a series of Gaussian kernels of different radii for convolution calculation, the present method can effectively enhance the main blood vessel structure while retaining the details of small blood vessels, thus achieving balanced enhancement of the full spectrum of blood vessels.

[0067] In addition, the present invention also cleverly combines traditional image processing technology and deep learning methods. By introducing a deep learning-based vascular recognition module, this method not only improves the accuracy of vascular recognition, but also enhances the ability to handle complex backgrounds and low-contrast blood vessels. This organic combination of traditional methods and artificial intelligence technology has opened up a new research direction for pulmonary vascular enhancement.

[0068] Finally, the present invention also designs a complete set of evaluation and optimization mechanisms. By calculating the difference rate between the enhanced result and the original image and setting a reasonable threshold, the method can automatically determine whether the enhancement effect meets the standard and reprocess it when necessary. This closed-loop optimization mechanism ensures the high quality and reliability of the final output result.

[0069] In summary, the pulmonary vascular enhancement method based on dual-modality imaging provided by the present invention effectively solves the problems existing in the prior art by innovatively combining a variety of advanced technologies. This method not only significantly improves the quality and accuracy of pulmonary vascular imaging, but also has strong adaptability and robustness. These advantages make this method have broad application prospects in the early diagnosis of lung diseases, surgical planning, and efficacy evaluation, and provide important technical support for improving the diagnosis and treatment of lung diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0071] Figure 2 A flow chart for constructing a digital model of the present invention.

[0072] Figure 3 The flowchart of the network characteristic calculation of the present invention.

[0073] Figure 4 A flow chart for scoring blood supply function of the present invention. DETAILED DESCRIPTION

[0074] like Figure 1-4 As shown, the present invention provides a method for enhancing pulmonary blood vessels based on dual-modality imaging. The method achieves high-quality imaging and enhancement of pulmonary blood vessels by combining fluorescence imaging and infrared imaging technology. The specific implementation of the present invention will be described in detail below.

[0075] First, the method of the present invention includes an acquisition step, a processing step, and an output step. In the acquisition step, a fluorescent image of the pulmonary blood vessels is acquired by fluorescent development, and an infrared image of the pulmonary blood vessels is acquired by infrared development. This dual-modality imaging technology can provide more comprehensive vascular information, laying a foundation for subsequent processing.

[0076] In the processing step, the method of the present invention first calculates the enhancement weights of the two modalities based on the intensity information of the fluorescence image and the infrared image and the morphological information of the blood vessels. This step is one of the core innovations of the present invention. By adaptively calculating the weights, it is possible to optimize and enhance according to the characteristics of different images. Specifically, the present invention adopts an adaptive weight calculation algorithm, and its formula is as follows:

[0077] W p =W c ·W s

[0078] I p =W c I S +W s I S

[0079] Among them, W p is the final calculated weight, I p is the pixel value of pixel r(i,j,k) after final enhancement, I S is the pixel value of pixel r(i,j,k) after grayscale transformation. c and W s Represent the weights calculated based on the number and area of ​​connected regions respectively.

[0080] Next, the method performs multi-scale blood vessel enhancement filtering on the obtained pseudo-color blood vessel image. This step uses multi-scale analysis technology, which can effectively enhance blood vessels of different thicknesses. Specifically, the present invention uses a series of Gaussian kernels of different radii for convolution calculation to obtain a multi-scale image sequence. The advantage of this method is that it can enhance blood vessels of different thicknesses at the same time, thereby improving the comprehensiveness of the enhancement effect.

[0081] Finally, this method weights the vascular enhancement results with the fluorescence image and infrared image to obtain the final vascular enhancement results. This fusion process fully utilizes the advantages of dual-modality imaging and can obtain clearer and more accurate vascular enhancement images.

[0082] In the output step, the method outputs the final blood vessel enhancement result, which can be directly used for subsequent medical diagnosis and analysis.

[0083] Preferably, in one embodiment of the present invention, the specific implementation of the acquisition step includes collecting the fluorescence signal of the pulmonary blood vessels under laser excitation by intravenous injection of fluorescent nanoparticles to obtain a fluorescent image. This method can clearly display the vascular structure, especially small blood vessels. At the same time, the method uses laser imaging technology to obtain infrared images, which can provide vascular information deep in the tissue.

[0084] In addition, the process of calculating the enhancement weight in the processing steps of the present invention specifically includes the following steps: First, grayscale stretching is performed on the fluorescent image and the infrared image respectively to obtain an enhanced image. This step can increase the contrast of the image and make the vascular structure more prominent. Secondly, pseudo-color mapping is performed on the enhanced image to obtain a pseudo-color vascular image. Pseudo-color mapping can help better distinguish different vascular structures. Then, the intensity enhancement result is obtained based on the intensity information of the pseudo-color vascular image, and the morphological enhancement result is obtained based on the vascular morphology and branching information. Finally, the enhancement weight of each modality is calculated based on the intensity enhancement result and the morphological enhancement result.

[0085] An important advantage of this adaptive weight calculation method is that the enhancement strategy can be automatically adjusted according to the characteristics of different images. For example, for images with low signal-to-noise ratio, a higher weight may be given to intensity information; while for images with more vascular branches, more morphological information may be considered. This flexibility enables the method to adapt to input images of different quality and characteristics, improving the robustness and applicability of the method.

[0086] In general, the pulmonary vascular enhancement method based on dual-modal imaging provided by the present invention achieves high-quality imaging and enhancement of pulmonary vessels by combining fluorescence imaging and infrared imaging technology, and adopting adaptive weight calculation and multi-scale enhancement technology. This method not only improves the clarity and accuracy of vascular imaging, but also can better display small vascular structures, providing important technical support for the diagnosis and treatment of lung diseases. In a preferred embodiment of the present invention, the multi-scale vascular enhancement filtering process includes a series of carefully designed steps to comprehensively improve the quality and recognizability of vascular images. First, the method adopts a multi-scale separation algorithm and uses a series of Gaussian kernels of different radii for convolution calculation to obtain a multi-scale image sequence. The advantage of this method is that vascular structures of different scales can be processed simultaneously, thereby achieving comprehensive enhancement of thick and thin blood vessels.

[0087] Specifically, the radius of the Gaussian kernel used in the present invention is usually between 1 and 10 pixels. This range is obtained through a large number of experimental verifications and can effectively cover the size range of most pulmonary blood vessels. For example, a Gaussian kernel with a radius of 1-2 pixels is suitable for enhancing the smallest blood vessels, while a Gaussian kernel with a radius of 8-10 pixels is used to enhance larger blood vessels.

[0088] Next, this method uses the Laplacian pyramid algorithm to perform multi-layer filtering on the pseudo-color vascular image and decompose it into multiple groups of images to be enhanced. The use of the Laplacian pyramid algorithm enables this method to analyze and enhance images at different spatial frequencies, which is especially important for structures with multi-scale characteristics such as blood vessels. Usually, this method uses 3 to 5 layers of Laplacian pyramids, and the number of layers is selected based on the trade-off between computational efficiency and enhancement effect.

[0089] After obtaining multiple groups of images to be enhanced, the method of the present invention calculates the channel similarity and scale similarity between these images to be enhanced. The purpose of this step is to evaluate the information relevance between different scales and different channels, and provide a basis for the subsequent fusion step. The calculation formula of channel similarity is as follows:

[0090]

[0091] Among them, C ri represents the channel similarity between the images to be enhanced in the i-th layer, M represents the total number of pixels, and R j (r) and R k (r) represents the probability that the rth pixel in the jth and kth Laplacian feature layers is a blood vessel, respectively.

[0092] Similarly, the calculation formula for scale similarity is:

[0093]

[0094] Among them, S i Represents the scale similarity between the i-th layer and the j-th layer.

[0095] Based on the calculated channel similarity and scale similarity, this method further calculates the fusion weight of each image to be enhanced. This step is the key to achieve adaptive enhancement, which can automatically adjust the enhancement strategy according to the importance of different scales and channels. The calculation of fusion weight takes into account multiple factors, including channel similarity, scale similarity, and local features of the image.

[0096] Finally, this method reconstructs the enhanced image by weighted summation. This weighted fusion method can make full use of information from different scales and channels to obtain more comprehensive and accurate vascular enhancement results.

[0097] In another embodiment of the present invention, the method further includes further processing and optimizing the enhanced image. First, the enhanced image is binarized using a segmentation threshold to extract blood vessels and blood vessel centerlines. The segmentation threshold here is usually set between 60% and 80% of the image grayscale value, and the specific value can be fine-tuned according to the characteristics of the image. Binarization can effectively separate blood vessels from the background and provide clear blood vessel contours for subsequent analysis.

[0098] Next, the method calculates the characteristic information of the blood vessel contour. This characteristic information includes but is not limited to the width, direction, and branch point location of the blood vessel. Based on this characteristic information, the method further calculates the direction of the blood vessel. The calculation of the blood vessel direction uses an improved principal component analysis algorithm, which can accurately capture the overall direction and local changes of the blood vessel.

[0099] Finally, according to the calculated vascular orientation, the method corrects the vascular centerline to obtain the final vascular enhancement result. The purpose of this step is to further improve the accuracy of the vascular centerline, especially the positioning accuracy at the intersection or branch of the vessel.

[0100] Preferably, the present invention further comprises a deep learning-based vascular recognition module. This module first performs deep learning on the final vascular enhancement result to extract the boundary features of each type of blood vessel in the image. The present invention adopts an improved U-Net network structure, which, while maintaining the original U-Net advantages, adds an attention mechanism to better capture the detailed features of the blood vessels.

[0101] Based on the extracted boundary features, this method performs vascular fusion to obtain the original vascular fusion image. This fusion process takes into account the characteristics of different types of blood vessels, such as the differences between arteries and veins, thereby obtaining a more comprehensive and accurate vascular network representation.

[0102] Finally, the method performs vascular fusion correction on the original vascular fusion image to obtain a corrected vascular enhancement image. The purpose of this step is to eliminate possible artifacts and discontinuities and further improve the quality and reliability of vascular enhancement results.

[0103] Through this series of carefully designed steps, the method of the present invention can achieve high-quality enhancement of pulmonary blood vessels, providing a reliable basis for subsequent medical diagnosis and analysis. This method can not only effectively enhance blood vessels of different scales, but also further improve the accuracy of blood vessel recognition through deep learning technology, reflecting the organic combination of traditional image processing technology and modern artificial intelligence methods. In a preferred embodiment of the present invention, the design of the blood vessel recognition module is further refined to achieve more accurate blood vessel recognition and contour extraction. First, the method filters the brightness gradient map of the blood vessel enhanced image to obtain the significant area corresponding to the foreground area. This step adopts an improved Gaussian filtering algorithm, and the filter kernel size is usually set to 5x5 or 7x7, depending on the resolution of the image and the thickness of the blood vessels. This filtering process can effectively eliminate noise while retaining the main features of the vascular structure.

[0104] Next, this method filters the brightness gradient map of the obvious blood vessel enhancement image to obtain the blood vessel boundary area. A directional filter is used here to better capture the directional information of the blood vessel edge. The direction of the filter is usually set to 8 or 16, covering a range of 0 to 360 degrees to adapt to the direction of blood vessels in different directions.

[0105] After obtaining the salient region and the vascular boundary region, the method of the present invention synthesizes the two regions to obtain an enhanced vascular contour region. The synthesis process adopts a weighted fusion method, and the weight coefficient is usually set to 0.6:0.4 or 0.7:0.3. The specific value can be fine-tuned according to the characteristics of the image. This synthesis method can effectively combine global saliency and local edge information to improve the accuracy of vascular contours.

[0106] Preferably, the method also uses the vascular structure of the image before enhancement to correct the vascular boundary points in the vascular contour area. The purpose of this step is to eliminate artifacts or distortions that may be introduced by the enhancement process. The correction process uses a method based on morphological operations, including opening and closing operations, and the size of the structural element is usually set to 3x3 or 5x5 to balance detail preservation and noise suppression.

[0107] After completing the boundary point correction, the method of the present invention uses Dijkstra's algorithm to smooth the vascular contour curve in the corrected vascular contour area. Dijkstra's algorithm is cleverly used here to find the optimal smoothing path, effectively eliminating the jagged edges and irregularities on the contour line. In the parameter setting of the algorithm, the ratio of the distance weight to the smoothness weight is usually 3:1 or 4:1, so as to achieve a good smoothing effect while maintaining the contour accuracy.

[0108] Finally, this method projects the smoothed vascular contour curve onto the enhanced image to complete the vascular recognition process. The projection process uses a bilinear interpolation algorithm to ensure that the contour line is accurately aligned with the original image. The output of this step is an enhanced image containing accurate vascular contour information, which provides a reliable basis for subsequent medical analysis and diagnosis.

[0109] In another embodiment of the present invention, the method further includes evaluating and optimizing the final blood vessel enhancement result. Specifically, the method finds the area corresponding to the enhancement result in the original image, and marks the difference between the enhanced result and the blood vessel in the initial blood vessel fusion image in the corresponding original image to obtain the difference rate. The calculation of this difference rate adopts the improved structural similarity index (SSIM) method, which not only considers the difference at the pixel level, but also considers the difference in structural information.

[0110] Preferably, the method sets a preset threshold value for judging the effect of the enhancement algorithm. If the difference rate is less than the preset threshold value, the algorithm is determined to be successful; if the difference rate is greater than or equal to the preset threshold value, the method is returned to the acquisition step and re-executed. This preset threshold value is usually set to 10%, which is determined based on a large amount of experimental data and expert experience, and can ensure the enhancement effect while avoiding over-processing.

[0111] It is worth noting that the method of the present invention also includes a deep learning-based vascular recognition model. This model uses the trained pulmonary vascular recognition model to identify the final vascular enhancement results and obtain vascular classification results. The training process of the model adopts an innovative method. First, the real vascular enhancement results are obtained in the dual-modal images as training data, and then these training data are used as inputs of the deep learning framework to train the vascular recognition model using the cross entropy loss function and optimization method.

[0112] Specifically, the deep learning model used in the present invention is based on an improved U-Net structure, with an added attention mechanism and residual connection to improve the model's ability to identify small blood vessels. During the model training process, the learning rate is initially set to 0.001 and is adjusted using a cosine annealing strategy; the batch size is set to 16 or 32, depending on the available computing resources; the number of training rounds is usually set to 100 to 200 rounds, and an early stopping strategy is used to avoid overfitting.

[0113] The use of the cross entropy loss function enables the model to better handle the unbalanced binary classification problem of blood vessel segmentation. Its mathematical expression is as follows:

[0114]

[0115] Where N is the number of samples, y i is the true label, is the model's predicted value.

[0116] Through this series of carefully designed steps and innovative technical methods, the dual-modality imaging-based pulmonary vascular enhancement method provided by the present invention can achieve high-quality and high-precision vascular enhancement and identification. This method not only improves the clarity and accuracy of vascular imaging, but also further improves the intelligent level of vascular identification by introducing deep learning technology, providing strong technical support for the accurate diagnosis and treatment of lung diseases.

[0117] The above description is only a preferred specific implementation manner of the present invention; however, the protection scope of the present invention is not limited thereto; any person familiar with the art who, within the scope disclosed by the present invention, makes equivalent replacements or changes based on the scheme and improved concepts of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A method for pulmonary vascular enhancement based on dual-modality imaging, characterized in that: include: The acquisition steps include: Fluorescent images of pulmonary vessels were obtained by fluorescence angiography; Infrared imaging is used to obtain infrared images of the blood vessels in the lungs; Processing steps include: Based on the intensity information of the fluorescence image and the infrared image and the morphological information of the blood vessels, the enhancement weights of the two modalities are calculated to obtain an adaptively enhanced pseudo-color blood vessel image; Performing multi-scale blood vessel enhancement filtering processing on the pseudo-color blood vessel image to obtain a blood vessel enhancement result; The blood vessel enhancement result is weightedly fused with the fluorescence image and the infrared image to obtain a final blood vessel enhancement result; Output steps include: The final blood vessel enhancement result is outputted.

2. The method according to claim 1, characterized in that The acquisition step specifically includes: By intravenously injecting fluorescent nanoparticles, collecting fluorescent signals of pulmonary blood vessels under laser excitation, and obtaining the fluorescent image; The infrared image is acquired by using laser imaging technology.

3. The method according to claim 1, characterized in that The process of calculating the enhancement weight in the processing step specifically includes: Performing grayscale stretching on the fluorescent image and the infrared image respectively to obtain enhanced images; Performing pseudo color mapping on the enhanced image to obtain a pseudo color blood vessel image; Obtaining an intensity enhancement result based on the intensity information of the pseudo-color blood vessel image; The morphological enhancement results are obtained according to the vascular morphology and branching information; An enhancement weight of each modality is calculated based on the intensity enhancement result and the morphology enhancement result.

4. The method according to claim 1, characterized in that The multi-scale blood vessel enhancement filtering process specifically includes: A multi-scale separation algorithm is used to perform convolution calculations using a series of Gaussian kernels with different radii to obtain a multi-scale image sequence; Using the Laplace pyramid algorithm, multi-layer filtering is performed on the pseudo-color blood vessel image to decompose it into multiple groups of images to be enhanced; Calculating the channel similarity and scale similarity between the images to be enhanced; Based on the channel similarity and scale similarity, calculating the fusion weight of each image to be enhanced; The enhanced image is reconstructed by weighted summation.

5. The method according to claim 4, characterized in that The following steps are also included: Using a segmentation threshold to perform binarization processing on the enhanced image, and extracting blood vessels and blood vessel centerlines; Calculate characteristic information of blood vessel contour; Calculating the direction of the blood vessel based on the characteristic information of the blood vessel contour; The blood vessel centerline is corrected according to the blood vessel direction to obtain a final blood vessel enhancement result.

6. The method according to claim 1, characterized in that The following steps are also included: Designing a blood vessel recognition module to perform deep learning on the final blood vessel enhancement result and extract boundary features of each type of blood vessel in the image; Based on the boundary features, blood vessel fusion is performed to obtain an original blood vessel fusion image; The original vascular fusion image is subjected to vascular fusion correction to obtain a corrected vascular enhancement image.

7. The method according to claim 6, characterized in that The design of the vascular recognition module specifically includes: Filter the brightness gradient map of the blood vessel enhanced image to obtain the salient area corresponding to the foreground area; Filter the brightness gradient map of the obvious blood vessel enhancement image to obtain the blood vessel boundary area; synthesizing the notable region and the blood vessel boundary region to obtain an enhanced blood vessel contour region; Using the blood vessel structure of the image before enhancement, correcting the blood vessel boundary points in the blood vessel contour area; Dijkstra's algorithm is used to smooth the blood vessel contour curve in the corrected blood vessel contour area; The smoothed vascular contour curve is projected onto the enhanced image to complete the vascular recognition process.

8. The method according to claim 1, characterized in that The following steps are also included: For the final blood vessel enhancement result, finding a corresponding area in the original image; Mark the difference between the blood vessels in the enhanced result and the initial blood vessel fusion image in the corresponding original image to obtain the difference rate; If the difference rate is less than a preset threshold, the algorithm is determined to be successful; If the difference rate is greater than or equal to the preset threshold, the method returns to the acquisition step and executes the method again.

9. The method according to claim 8, characterized in that The preset threshold is 10%.

10. The method according to any one of claims 1 to 9, characterized in that: The following steps are also included: Using the trained pulmonary blood vessel recognition model to recognize the final blood vessel enhancement result, to obtain a blood vessel classification result; The training method of the pulmonary blood vessel recognition model includes: Obtaining real blood vessel enhancement results in dual-modality images as training data; The training data is used as input of a deep learning framework, and the blood vessel recognition model is trained using a cross entropy loss function and an optimization method.

Images