NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation

Abstract

The invention discloses an NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation, and belongs to the field of computer vision, and the method comprises the following steps: converting a video into a frame image; preprocessing the frame image by using a contrast limited adaptive histogram equalization algorithm and neighborhood mean filtering; performing top-hat transformation on the preprocessed image to obtain a rough foreground image; taking a mean value of non-zero pixels as a threshold value, and performing adaptive segmentation frame by frame; accumulating the output binary image, and enhancing a tubular structure in the image by using Frangi filtering; and performing certain morphological processing on the image after Frangi filtering to obtain a final lymphatic vessel segmentation result. According to the NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation, the detection capability of low-contrast lymphatic vessels with tiny disturbance in the video is improved.

Description

Technical Field

[0001] The invention relates to the field of medical image processing, and in particular to a NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation. Background Art

[0002] The lymphatic system is a network with exchange and transport functions, consisting of blind-end blood vessels that serve as fluid exchange sites, blood vessels that collect and transport lymph, and lymph nodes that affect immunity and antigen transport. Since the lymphatic system has no central pump to drive flow circulation, its transport function is mainly achieved by the contraction of the collecting lymphatic vessels, so the measurement of lymphatic vessels is crucial to the function of the lymphatic system.

[0003] Existing imaging methods for in vivo lymphatic visualization are divided into two categories: invasive surgical exposure and non-invasive imaging. Invasive surgical exposure is not only time-consuming, but also causes certain damage to the patient's body, so it is not recommended. Among non-invasive imaging, lymphoscintigraphy is the longest-used method in clinical practice, but it has poor spatial resolution, high requirements for radioactive protection, and lacks the temporal resolution required for real-time imaging of lymphatic contractile function. It has gradually been replaced by other imaging methods. Computed tomography (CT) and magnetic resonance lymphography (MRL) are currently commonly used imaging methods. CT has relatively high temporal and spatial resolution, but low diagnostic sensitivity and is usually used in combination with other methods. MRL can be used for large-area high-resolution imaging. Relying on its relative non-invasiveness, it helps to identify abnormalities related to lymphatic dysfunction, but its disadvantages are low sensitivity and low specificity for abnormal lymph nodes, making it unsuitable for the detection of lymph node abnormalities. Near-infrared fluorescence imaging has the advantages of fast acquisition speed and high spatial resolution, can achieve real-time imaging, and it retains the in vivo environment of lymphatic functional activity, which is conducive to in vivo research on pathological processes and the effects of molecular regulation, and is widely used. However, the disadvantage of optical imaging is that photon scattering and absorption in the body lead to rapid attenuation of the excitation light, which affects the tissue penetration depth, which limits the application of near-infrared fluorescence imaging in lymphatic visualization. Shifting the wavelength to a longer excitation window will reduce absorption attenuation and tissue autofluorescence, effectively increasing the penetration depth of photons, thereby achieving higher fidelity and spatial resolution. Therefore, with the upgrade of hardware systems, near-infrared zone II fluorescence imaging has become a better choice for non-invasive visualization and quantification of lymphatic vessels.

[0004] Near-infrared II (NIR-II, 1000-1700nm) fluorescence imaging provides the best solution for functional lymphatic imaging studies, but requires a high level of operator expertise and a carefully controlled imaging environment to achieve clear, high-resolution imaging results. Imaging of lymphatic vessels can be challenging due to the presence of optical scattering, stray light interference, and fluorescence saturation caused by surface-aggregated contrast agents, which can result in low image contrast, uneven vascular brightness, incomplete morphology, and occlusion of deep targets. These challenges can hinder subsequent quantitative detection. Therefore, an automatic segmentation method is needed to ignore the effects of minor perturbations in video recordings and extract reliable lymphatic vessels from the background with fluorescent interference. Summary of the invention

[0005] The present invention aims to provide a NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation to solve the problem that near-infrared second-zone fluorescence imaging requires a high level of operator expertise and a carefully controlled imaging environment to achieve clear, high-resolution imaging results. This method uses a series of medical image processing methods for low-contrast near-infrared images to achieve automatic extraction of low-contrast lymphatic vessels, laying a foundation for quantitative measurement of lymphatic vessel contraction function and its use in the diagnosis and treatment of related diseases.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation includes the following steps:

[0008] S1, processing the video to obtain a frame image sequence;

[0009] S2, preprocess each frame of image using CLAHE algorithm and 3x3 neighborhood mean filtering;

[0010] S3, extracting brighter structural areas from the dark background through top hat processing to obtain a rough foreground image;

[0011] S4, taking the average value of non-zero pixels in each frame of the image as a threshold to perform adaptive segmentation of the image frame by frame;

[0012] S5, reading all images contained in the image sequence, repeating steps S2 to S4, and accumulating binary images of all images;

[0013] S6, using Frangi filtering to enhance the tubular structure in the accumulated binary image to obtain a fine foreground image;

[0014] S7. The image after Frangi filtering is subjected to certain morphological processing to smooth the outline of the lymphatic vessels, eliminate the small holes, fill the gaps, and obtain clear lymphatic vessels.

[0015] Furthermore, in step S2, each frame of the image is subjected to denoising, including the following steps:

[0016] S201, divide the image into 16x16 non-overlapping sub-blocks of equal size, and calculate the grayscale histogram and corresponding transformation function (cumulative histogram) of each non-overlapping sub-block:

[0017]

[0018] Where x is a discrete grayscale image, L is the grayscale level in the image (usually 256), and n i is the number of occurrences of gray level i, p x (i) is the probability of the i-th grayscale pixel appearing in the image;

[0019] Its cumulative distribution function corresponds to:

[0020]

[0021] S202, calculating the clipping threshold clipLimit, and reallocating the redundant pixels;

[0022] S203, performing histogram equalization on the contrast-limited histogram to obtain a mapping function, and using the mapping function to process the original image;

[0023] S204, define a 3x3 neighborhood window, slide it in the image after CLAHE processing, calculate the mean in the window, and perform filtering to remove noise.

[0024] Furthermore, in step S3, the calculation formula of the top hat processing is as follows:

[0025]

[0026] Where I is the input image, THF(x,y) is the image after top-hat filtering, B is the structural element, Indicates an open operation.

[0027] Furthermore, in step S4, the calculation formula of the threshold and the binary image is as follows:

[0028]

[0029] where N(i) is the number of pixels with intensity value i in the image THF(x,y), T is the calculated threshold, and I T (x,y) is the binary image after threshold segmentation.

[0030] Furthermore, in step S6, the image is smoothed using a Gaussian filter function to obtain a filtered image I, and the partial derivative function is calculated to construct an enhancement filter using the Hessian matrix. The formula of the Hessian matrix is ​​as follows:

[0031]

[0032] Find the eigenvalues ​​based on the Hessian matrix:

[0033]

[0034] The eigenvalues ​​and their corresponding eigenvectors represent the magnitude and direction of the curvature of the point along a certain direction;

[0035] The response function is constructed using the eigenvalue, and its formula is as follows:

[0036]

[0037] A multi-scale strategy is used to obtain response results at different scales. When the scale and local structure match, the maximum response result can be obtained, thereby determining whether the pixel belongs to a vascular structure.

[0038] Furthermore, in step S7, a 4x4 square structural element is first constructed, and a binary morphological dilation operation is used to connect the low-brightness areas of the broken lymphatic vessels; then the area of ​​each area in the binary image is calculated, the noise area with a smaller area is filtered out, and the holes and gaps in the binary image are filled to obtain the final lymphatic vessel segmentation image.

[0039] The principle and beneficial effects of this technical solution:

[0040] The preprocessing method based on CLAHE and neighborhood mean filtering adopted by the present invention can effectively enhance the contrast of lymphatic vessels, remove noise and smooth the image. Top hat processing can extract brighter foreground areas from the background. In this step, top hat processing can avoid smaller fluorescence interference in the image, alleviate the problem of uneven brightness, and enhance lymphatic vessels in brighter areas. The above processing removes most of the stray light and background noise, but the edge contrast of the lymphatic vessels is low, which makes segmentation difficult. Using the non-zero mean of each frame of the image as a threshold for frame-by-frame image segmentation can effectively remove the residual noise around the lymphatic vessels. The accumulated binary image can further remove the influence of small disturbances in the video, so that the segmentation result remains relatively stable. Frangi filtering and morphological processing can enhance the tubular structure in the image on the one hand, and smooth the contour on the other hand, eliminate small isolated areas and spikes, eliminate small holes, fill the gaps in the contour, and obtain lymphatic vessels with complete morphology. In general, the method proposed in the present invention is highly robust to noise, has good detection and segmentation capabilities for dim lymphatic vessels with uneven brightness, and the contours of the segmented lymphatic vessels are more consistent with the lymphatic vessel morphology of the original image. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of a NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation provided in an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the experimental effect provided in one embodiment of the present invention; wherein (a) is the input original video frame image, (b) is the preprocessed image, and (c) is the result of the output segmented image edge on the original image. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0046] like Figure 1 As shown, the NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation proposed in the present invention includes the following steps:

[0047] S1. Process the video to obtain a frame image sequence.

[0048] S2, denoising preprocessing is performed on each frame of image using the CLAHE algorithm and 3x3 neighborhood mean filtering, including the following steps:

[0049] S201, divide the image into 16x16 non-overlapping sub-blocks of equal size, and calculate the grayscale histogram and corresponding transformation function (cumulative histogram) of each non-overlapping sub-block:

[0050]

[0051] Where x is a discrete grayscale image, L is the grayscale level in the image (usually 256), and n i is the number of occurrences of gray level i, p x (i) is the probability of the i-th grayscale pixel appearing in the image.

[0052] Its cumulative distribution function corresponds to:

[0053]

[0054] S202, calculating the clipping threshold clipLimit, and reallocating the redundant pixels;

[0055] S203, performing histogram equalization on the contrast-limited histogram to obtain a mapping function, and using the mapping function to process the original image;

[0056] S204, define a 3x3 neighborhood window, slide it in the image after CLAHE processing, calculate the mean in the window, and perform filtering to remove noise.

[0057] S3, extract the brighter area from the dark background through top-hat processing to obtain a rough foreground image.

[0058]

[0059] Where I is the input image, THF(x,y) is the image after top-hat filtering, B is the structural element, Indicates an open operation.

[0060] S4. Taking the mean value of non-zero pixels in each frame of image as a threshold, adaptively segment the image frame by frame.

[0061]

[0062] Where N(i) is the number of pixels with intensity value i in the image THF(x,y), and T is the calculated threshold. T (x,y) is the binary image after threshold segmentation.

[0063] S5. Read all images contained in the image sequence, repeat steps S2 to S4, and accumulate binary images of all images.

[0064] S6. The image after Frangi filtering is processed by certain morphology to obtain clear lymphatic vessels.

[0065] First, use the Gaussian filter function to smooth the image, obtain the filtered image I, and calculate the partial derivative function. The enhancement filter is constructed through the Hessian matrix. The formula of the Hessian matrix is ​​as follows:

[0066]

[0067] Find the eigenvalues ​​based on the Hessian matrix

[0068]

[0069] The eigenvalues ​​and their corresponding eigenvectors represent the magnitude and direction of the curvature of the point along a certain direction.

[0070] The response function is constructed using the eigenvalue, and its formula is as follows:

[0071]

[0072] A multi-scale strategy is used to obtain response results at different scales. When the scale and local structure match, the maximum response result can be obtained, thereby judging whether the pixel belongs to the vascular structure, and thus taking the maximum value.

[0073] S7, the image after Frangi filtering is processed by certain morphology to smooth the outline of lymphatic vessels, eliminate the small holes, fill the gaps, and obtain clear lymphatic vessels;

[0074] First, a 4x4 square structure element is constructed, and the low-brightness areas of the broken lymphatic vessels are connected using binary morphological dilation operations. Then, the area of ​​each area in the binary image is calculated, the noise area with a smaller area is filtered out, and the holes and gaps in the binary image are filled to obtain the final lymphatic vessel segmentation image.

[0075] like Figure 2As shown, the obtained segmentation result effectively eliminates the influence of noise and uneven contrast, the contour edges are clear and smooth, and the image details are effectively retained, especially in the curves and branches of the structure, showing good segmentation performance and providing a reliable basis for subsequent processing.

[0076] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation, characterized in that: The following steps are involved: S1, processing the video to obtain a frame image sequence; S2, preprocess each frame of image using CLAHE algorithm and 3x3 neighborhood mean filtering; S3, extracting brighter structure areas from the dark background through top hat processing to obtain a rough foreground image; S4, taking the average value of non-zero pixels in each frame of the image as a threshold to perform adaptive segmentation of the image frame by frame; S5, reading all images contained in the image sequence, repeating steps S2 to S4, and accumulating binary images of all images; S6, using Frangi filtering to enhance the tubular structure in the accumulated binary image to obtain a fine foreground image; S7. The image after Frangi filtering is subjected to certain morphological processing to smooth the outline of the lymphatic vessels, eliminate the small holes, fill the gaps, and obtain clear lymphatic vessels.

2. The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation according to claim 1, characterized in that: In step S2, each frame of image is subjected to denoising, including the following steps: S201, divide the image into 16x16 non-overlapping sub-blocks of equal size, and calculate the grayscale histogram and corresponding transformation function (cumulative histogram) of each non-overlapping sub-block: Where x is a discrete grayscale image, L is the grayscale level in the image (usually 256), and n i is the number of occurrences of gray level i, p x (i) is the probability of the i-th grayscale pixel appearing in the image; Its cumulative distribution function corresponds to: S202, calculating the clipping threshold clipLimit, and reallocating the redundant pixels; S203, performing histogram equalization on the contrast-limited histogram to obtain a mapping function, and using the mapping function to process the original image; S204, define a 3x3 neighborhood window, slide it in the image after CLAHE processing, calculate the mean in the window, and perform filtering to remove noise.

3. The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation according to claim 1, characterized in that: In step S3, the calculation formula of the top hat processing is as follows: Where I is the input image, THF(x,y) is the image after top-hat filtering, B is the structural element, Indicates an open operation.

4. The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation according to claim 1, characterized in that: In step S4, the calculation formula of the threshold and the binary image is as follows: where N(i) is the number of pixels with intensity value i in the image THF(x,y), T is the calculated threshold, and I T (x,y) is the binary image after threshold segmentation.

5. The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation according to claim 1, characterized in that: In step S6, the image is smoothed using a Gaussian filter function to obtain a filtered image I, and the partial derivative function is calculated to construct an enhancement filter using the Hessian matrix. The formula of the Hessian matrix is ​​as follows: Find the eigenvalues ​​based on the Hessian matrix: The eigenvalues ​​and their corresponding eigenvectors represent the magnitude and direction of the curvature of the point along a certain direction; The response function is constructed using the eigenvalue, and its formula is as follows: A multi-scale strategy is used to obtain response results at different scales. When the scale and local structure match, the maximum response result can be obtained, thereby determining whether the pixel belongs to a vascular structure.

6. The NIR-II video lymphatic vessel segmentation method based on Frangi and multi-frame accumulation according to claim 1, characterized in that: In step S7, a 4x4 square structural element is first constructed, and the low-brightness areas of the broken lymphatic vessels are connected using a binary morphological dilation operation. Then, the area of ​​each area in the binary image is calculated, the noise area with a smaller area is filtered out, and the holes and gaps in the binary image are filled to obtain the final lymphatic vessel segmentation image.

Images