A method for enhancing a blood vessel image

Abstract

A blood vessel image enhancement method, comprising the following steps: step one: inputting an endoscopic blood vessel image; step two: separating the input blood vessel image into three RGB channel planes; step three: respectively performing adaptive stretching on each channel using a nonlinear mapping function; step four: applying multi-scale detail enhancement (MISCE) on the result of step three to further perform adaptive multi-scale enhancement, deblurring and dark area enhancement; step five: performing a contrast-limited histogram equalization on each channel using a contrast-limited histogram equalization (CLAHE) on the result of step four; and step six: fusing the three RGB channels processed above into a final enhanced result image. The present application can perform different adaptive enhancements on blood vessels of different levels, and obtain richer blood vessel images to assist doctors in diagnosing early cancer.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of auxiliary medical diagnosis and image enhancement, in particular to a blood vessel image enhancement method. BACKGROUND

[0002] After investigating the specific tissue information of the cancerous region, it is found that the cancerous region usually causes the composition of the tissue to change. Research shows that there is a large difference in the content of hemoglobin between normal tissue and cancerous tissue. Hemoglobin exists in large quantities in blood, and the blood vessels of malignant tumor tissue are more abundant, and the arrangement of blood vessels on tumor tissue is relatively chaotic, the degree of bending of blood vessels is large, the diameter is uneven, the branches are more, and the blood oxygen saturation of normal tissue and cancerous tissue also has a large difference. Therefore, as long as the suspicious cancerous region can be greatly enhanced in blood vessel information, it can help doctors to better diagnose. Early cancer often has no obvious clinical symptoms and signs, and the main manifestation under endoscopy is a small change in mucosal structure, so enhancing the blood vessel features to enlarge these characteristic information that can assist in diagnosis and the shape features that can be observed with the naked eye can help doctors to shorten the diagnosis time of early cancer, so as to avoid the best golden treatment time of cancer.

[0003] The currently mainstream technologies for endoscopic blood vessel image enhancement are NBI (narrow band imaging), FICE (flexible spectral imaging color enhancement), and I-Scan. They all enhance blood vessel features to achieve real-time processing. NBI is a hardware image enhancement technology that uses narrow-band light illumination, while FICE and I-Scan are electronic staining technologies. FICE reconstructs color images by calculating the reflection intensity at several specific wavelengths to enhance the target of interest. It needs to be strictly calibrated before use, and the color of the enhanced image is very different from the true color. The I-Scan technology not only can select different wavelength combinations to display images, but also introduces surface enhancement and contrast enhancement, but the algorithm is relatively complex.

[0004] From the spectral properties of hemoglobin (oxygenated and deoxygenated), it can be seen that there is a strong absorption peak at 410 nm, a secondary absorption peak at about 540 nm, and a weak small absorption peak at about 760 nm. As the wavelength increases, the penetration depth also increases. Mucosal blood vessels are generally divided into surface vessels, shallow vessels, and deep microvessels. The shorter the wavelength, the clearer the image of the shallow and surface capillaries, and the more detailed the image. Conversely, the fine vessels located in the deep layer of the mucosa layer can only have more detailed image information at longer wavelengths. Therefore, in order to obtain better blood vessel detail information, it is necessary to enhance different levels. The current narrow-band imaging technology (NBI) uses 415 nm and 540 nm narrow-band light illumination, which is a technology specially used to enhance the contrast of the blood vessel system and the shallow surface mucosa, and is based on the response spectrum of capillaries or veins at these two wavelengths. It can be seen that it loses the information of deeper vessels. SUMMARY

[0005] In order to overcome the shortcomings of the prior art, the present application provides a blood vessel image enhancement method, which can perform different adaptive enhancements on different levels of blood vessels to obtain more detailed blood vessel images to assist doctors in diagnosing early cancer.

[0006] The technical scheme adopted by the present application to solve the technical problems is:

[0007] A blood vessel image enhancement method, comprising the following steps:

[0008] Step 1: input an endoscopic blood vessel image;

[0009] Step 2: separate the input blood vessel image into three RGB channel planes;

[0010] Step 3: respectively perform adaptive stretching on each channel using a nonlinear mapping function;

[0011] Step 4: apply multi-scale detail enhancement (MISCE) to the results of step 3 for further adaptive multi-scale enhancement, deblurring, and dark area enhancement;

[0012] Step 5: use contrast-limited histogram equalization (CLAHE) to limit the contrast of the histogram of each channel based on the results of step 4;

[0013] Step 6: fuse the three RGB channels processed above to output the final enhanced result image.

[0014] Further, the processing procedure of step 3 is as follows:

[0015] Firstly, each channel plane pixel value is normalized to the range of (0.0, 1.0) for subsequent processing, through (0, 0) (1, 1) and only stretching the (0.2, 0.8) interval segment of the image;

[0016] Then, the pixel mean value g of each channel plane is calculated, and the adaptive image gain coefficient d is calculated by substituting formula (1) g ;

[0017]

[0018] Then, the pixel value is substituted into formula (2) for enhancement point by point;

[0019] f output1 (x, y) = [1 + dg*((f input (x, y)) -1 -1) 2 -1 (2)

[0020] Finally, the pixel values of the three channel planes are re-normalized to the regular pixel value interval of (0.0, 255.0).

[0021] Further, the processing procedure of step four is as follows:

[0022] First, the selection rule is determined by formula (3):

[0023]

[0024] Wherein, X, Y are the row height and column width of the input picture, k1, k2, k3 are the subsequent neighborhood size;

[0025] Calculate the neighborhood mean S k As a threshold to determine whether the current point is a large difference point, if yes, substitute it into the first formula of formula (4), otherwise substitute it into the second formula of (4);

[0026]

[0027] Wherein, B is the maximum value point of the global pixel, S k is the k neighborhood mean in (3) above, f output1 (x, y) is the calculation result in formula (2), and the following formula (5) is used to calculate A parameter,

[0028] Then, the following formula (5) is used to calculate A parameter,

[0029]

[0030] Wherein, x is pixel value, M is one parameter below which is suggested to be set as 6000-8000 according to experiment, and B is the maximum value point of global pixels;

[0031] Then, the selection of M value in the formula is followed, and the endoscope blood vessel image is set as 6000-8000;

[0032] Finally, the adaptive enhanced pixel values of three groups of different scales are obtained, and then the average value of the three groups of scales is taken.

[0033] Further, in step five, CLAHE algorithm is performed on three channels to perform a limited contrast adaptive histogram equalization, and the process is as follows:

[0034] Pretreatment, image block filling is performed;

[0035] For each block processing, the mapping relationship is calculated, and the contrast limit is used when calculating the mapping relationship.

[0036] The interpolation method is used to obtain the final enhanced image.

[0037] The technical concept of the present application is that according to the different waveband absorption spectrum (410nm, 540nm, 760nm) of hemoglobin, it can be inferred that the information of surface blood vessels is contained in the g channel information, the shallow blood vessels are in the b channel, and the deep blood vessel information is in the r channel. Therefore, adaptive enhancement is performed on them to obtain more detailed information about the blood vessel image, and it is no longer limited to one level of blood vessel information. In this way, adaptive enhancement is performed on the three channel planes, and a result with more blood vessel information is obtained.

[0038] Due to the spectral characteristics of hemoglobin and the fact that the longer the wavelength, the stronger the penetration, it can be known that surface, shallow and fine blood vessels are located in three different channel planes of GBR. Therefore, the image obtained needs to be processed by channel first, and then the adaptive enhancement in steps three and four needs to be performed on each channel. Finally, a unified processing is performed by using limited histogram equalization.

[0039] The beneficial effects of the present application mainly include: 1. Different adaptive enhancement can be performed on blood vessels of different levels to obtain more rich blood vessel images; 2. The real-time processing requirement of endoscopy is met. BRIEF DESCRIPTION OF DRAWINGS

[0040] Fig. 1 It is a flow chart of a blood vessel image enhancement method.

[0041] Fig. 2 It is an effect diagram of blood vessel image enhancement under normal illumination.

[0042] Fig. 3 is the effect picture of blood vessel image enhancement in dark environment. DETAILED DESCRIPTION

[0043] The application will be further described below with reference to the drawings.

[0044] Reference Figs. 1-3 A blood vessel image enhancement method, comprising the following steps:

[0045] Step one: input endoscopic blood vessel image;

[0046] Step two: separate the input blood vessel image into three RGB channel planes;

[0047] Step three: respectively perform adaptive stretching on each channel using a nonlinear mapping function;

[0048] The processing procedure of the step three is as follows:

[0049] First, normalize each separated channel plane pixel value to the range of (0.0, 1.0) for subsequent processing, pass (0, 0) (1, 1) and only stretch the (0.2, 0.8) interval segment of the image;

[0050] Then, find the pixel mean value g of each channel plane, substitute it into the following formula to calculate the adaptive image gain coefficient d g ;

[0051]

[0052] Then substitute the pixel value point by point into the following formula to perform one enhancement;

[0053] f output1 (x, y) = 1 + d g *((f input (x, y)) -1 - 1) 2 - 1 (2)

[0054] Finally, re-normalize the pixel values of the three channel planes to the conventional pixel value interval of (0.0, 255.0).

[0055] Step four: apply multi-scale detail enhancement MISCE on the result of step three to further perform adaptive multi-scale enhancement and deblurring and enhance dark areas;

[0056] The processing procedure of the step four is as follows:

[0057] First, determine the selection rule by the following formula;

[0058]

[0059] Wherein, X, Y are the row height and column width of the input picture, and k1, k2, k3 are the neighborhood size;

[0060] Calculate the neighborhood mean value S of three different scales k As a threshold to determine whether the current point is a large difference point, if yes, substitute into the first formula in the following formula, otherwise, substitute into the second formula in the following formula;

[0061]

[0062] Wherein, B is the maximum value of the pixel value, S k The k neighborhood mean value is calculated, and then the A parameter is calculated by the following formula,

[0063]

[0064] Wherein, x is the pixel value, M is an experimental suggestion of 6000-8000, and B is the maximum value of the pixel.

[0065] Then, the M value in the formula is selected, and the endoscopic blood vessel image is set to 6000-8000.

[0066] Finally, three groups of different scale adaptive enhanced pixel points are obtained, and the average value of the three groups of scales is taken.

[0067] Step five: using the contrast limited histogram equalization CLAHE of step four to perform a contrast limited histogram equalization on each channel;

[0068] In step five, the CLAHE algorithm is performed on three channels to perform a contrast limited adaptive histogram equalization, and the process is as follows:

[0069] Pretreatment, image block filling is performed;

[0070] For each block processing, the mapping relationship is calculated, and the contrast limit is used when calculating the mapping relationship.

[0071] The final enhanced image is obtained by using the interpolation method.

[0072] Step six: the RGB three channels processed above are fused and output as the final enhanced result image.

[0073] In the embodiment, Fig. 1 As shown in the flowchart of the blood vessel image enhancement method, the input blood vessel image is processed until an enhanced image containing more rich and easy-to-observe images is obtained, and the detailed operation is as follows:

[0074] Input image: input, output enhanced image: foutputi (x, y); (i represents the output of the step)

[0075] Separate channel planes: Ri, Gi, Bi; (i represents the channel component of the step)

[0076] Referring to Fig. 1 , first, collect the input image input from the image sensor containing the blood vessel endoscopic image from the image acquisition prototype;

[0077] First, separate the input image input into R0, G0, B0 three channel planes;

[0078] Then, normalize the pixel values of the R0, G0, B0 three channel planes to the range of (0.0, 1.0) to obtain the R1, G1, B1 three normalized channel planes;

[0079] Then, calculate the mean value of the pixel points of the R1, G1, B1 three channel planes, g1, g2, g3;

[0080] Then, calculate the adaptive gain coefficient d g : The adaptive gain coefficients of the three channels obtained and the pixel values of the three channels are brought into the following formula for enhancement to obtain R2, G2, B2: f output1 (x, y) = [1 + dg*((f input (x, y)) -1 -1) 2 ]-1;

[0081] Renormalize the R2, G2, B2 three components obtained above to the interval range of (0.0, 255.0) to obtain R3, G3, B3;

[0082] Bring the pixel size of the R3, G3, B3 three components obtained into the following calculation to obtain three different scale neighborhood sizes k1, k2, k3;

[0083]

[0084] Calculate the mean values S1, S2, S3 of the k1, k2, k3 three scale neighborhood sizes of each pixel point of the R3, G3, B3 three components, which are used as a threshold to determine whether the current pixel point is a point with large difference from the surroundings, if so, substitute into the first formula in the following formula, otherwise, substitute into the second formula in the following formula:

[0085]

[0086] Where B is the maximum value point of the global pixel, S kFor the above k neighborhood mean, then the A parameter is calculated using the following formula: Where x is the pixel value, M is an experimental suggestion of 6000-8000, and B is the maximum value point of the global pixel.

[0087] B, M are respectively the interval range of the pixel value size of the channel plane and an adjustable parameter, which is suggested to be set to 6000-8000 through a large number of endoscopic blood vessel image experiments.

[0088] Finally, three groups of adaptively enhanced pixel points of different scales are obtained, and then the average value of the three groups of scales is taken.

[0089] Thus, three channel components R4, G4, and B4 that have been adaptively enhanced by multi-scale enhancement are obtained.

[0090] Each channel is then processed in blocks, and the mapping relationship is calculated. The contrast limit is used when calculating the mapping relationship.

[0091] The interpolation method is then used to obtain the final three enhanced channel component images R5, G5, and B5.

[0092] Finally, R5, G5, and B5 are fused to obtain the final enhanced output image f output4 (x, y).

[0093] Fig. 2 、 Fig. 3 Respectively, as an embodiment of the present application, the effect diagram of the enhanced image of the image in the normal environment and the dark environment (the left is the original image, and the right is the enhanced effect diagram), the enhanced blood vessel information can be visually seen to be more abundant and more for the human eye, and some weak blood vessel details are better displayed. Table 1 compares the algorithm in this paper and the FICE endoscopic enhancement technology.

[0094]

[0095] Table 1

[0096] Tables 1-3 are the performances of the algorithm in this paper and the mainstream endoscopic enhancement technology (FICE, I-SCAN, NBI) in three evaluation indexes (entropy value, psnr (peak signal-to-noise ratio), and AG (average gradient)), which are better than the above three widely used proprietary technologies, and the blood vessel related information is better displayed not only by the naked eye but also by objective index evaluation.

[0097] Table 2 compares the algorithm in this paper and the I-SCAN endoscopic enhancement technology.

[0098]

[0099] Table 2

[0100] Table 3 compares the algorithm in this paper and NBI endoscopic enhancement technology.

[0101]

[0102] Table 3

[0103] The blood vessel image enhancement method of the embodiment mainly performs adaptive enhancement on three blood vessel information contained in different levels of different channel planes through steps three, four and five described above, and finally fuses to obtain an enhanced image. Experiments prove that the method can greatly enhance the blood vessel information visible to the naked eye and enrich some fine blood vessels and deep blood vessel information that are difficult to observe with the naked eye. Experiments prove that it also has good adaptability in dark environments of endoscopy. Comparison of four evaluation indexes (entropy value, signal-to-noise ratio, enhancement, and structural similarity) of some enhanced images of three mainstream proprietary technologies (NBI, FICE, and I-SCAN) widely used online and the enhanced images of the present application proves that the present application is superior to the above-mentioned technologies. Finally, experiments prove that the present application meets the real-time processing requirements of endoscopy.

[0104] The embodiments of the present application are merely a list of implementation forms of the inventive concept, and are only for the purpose of illustration. The protection scope of the present application should not be regarded as being limited to the specific forms described in the embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A blood vessel image enhancement method characterized by, The method comprises the following steps: Step one: input endoscopic blood vessel image; Step two: separate the input blood vessel image into three RGB channel planes; Step three: respectively adaptively stretch each channel with a nonlinear mapping function; Step four: apply multi-scale detail enhancement MISCE on the result of step three for adaptive multi-scale enhancement and deblurring and dark area lifting; Step five: limit the contrast of the histogram equalization of each channel using contrast limited histogram equalization CLAHE on the result of step four; Step six: fuse the above-processed three RGB channels into the final enhanced result image; The processing procedure of step three is as follows: First, normalize each separated channel plane pixel value to the range of (0.0, 1.0) for subsequent processing, pass (0, 0) (1, 1) and only stretch the (0.2, 0.8) interval segment of the image; Then, the pixel mean value g of each channel plane is calculated and substituted into formula (1) to calculate the adaptive image gain coefficient d g ; ； Then, replace the pixel value point by point into formula (2) for enhancement; ； Finally, re-normalize the pixel values of the three channel planes to the conventional pixel value interval of (0.0, 255.0); The processing procedure of step four is as follows: First, the neighborhood size of three different scales is calculated , , ; ； Wherein, X, Y are the row height and column width of the input picture, , , is the size of the neighborhood; Compute the mean of the neighborhood of three different scales As a threshold to determine whether the current point is a large difference point, if yes, put it into the first formula in formula (4), otherwise put it into the second formula in (4); ； wherein B is a maximum point of the global pixel, is the calculation result in the above (3), k neighborhood mean, is the calculation result in the above (3), the A parameter is calculated by using the following formula (5), ； Wherein, x is the pixel value, M is the interval range of the pixel value size of the channel plane, which is set to 6000-8000; Finally, three groups of adaptively enhanced pixel values of different scales are obtained, and the average value of the three groups of scales is taken.

2. The blood vessel image enhancement method of claim 1, wherein, In step five, CLAHE algorithm is used for three channels to perform adaptive histogram equalization with limited contrast, and the process is as follows: Preprocessing, image block filling; For each block processing, calculate the mapping relationship, and use contrast limitation when calculating the mapping relationship; Use interpolation method to obtain the final enhanced image.

Images