A method of blood vessel enhancement of an endoscopic image

Abstract

This invention discloses a method for enhancing blood vessels in endoscopic images, comprising: S1, reading the original endoscopic image and extracting the original blood vessel image based on the image features of the blood vessel region; S2, converting the color space of the original blood vessel image to the HSV color space and acquiring image information from the H, S, and V channels; S3, keeping the blood vessel image in the H channel unchanged, and performing enhancement processing on the original blood vessel images in the S and V channels respectively; S4, fusing the original blood vessel image in the H channel with the enhanced blood vessel images in the S and V channels to obtain the enhanced blood vessel image in the HSV color space, and converting it back to the original color space of the original blood vessel image. This invention achieves targeted enhancement of blood vessel images by extracting them in real time from acquired endoscopic images, reducing the amount of data processed and image noise while effectively highlighting the enhancement effect of the blood vessel region in the image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to a method for enhancing blood vessels in endoscopic images. Background Technology

[0002] Endoscopes are a key component in providing the visual system during minimally invasive surgery, and their image quality plays a crucial role. However, due to the narrowness of the human body's cavities, the complexity of the digestive tract structure, and especially the intricate distribution of blood vessels, which are often difficult to detect even beneath the mucosa, and the limited effective field of view of conventional endoscopes due to lens depth limitations, vascular features are often not prominent and appear light in existing endoscopic images, making them difficult to identify.

[0003] Currently, the main techniques for endoscopic vascular enhancement include: Retinex-based methods, adaptive sigmoid function-based methods, FICE-based methods, morphological operator-based methods, machine learning-based methods, curvature analysis-based methods, multi-scale segmentation-based methods, and CLAHE-based methods. However, these methods all suffer from over-enhancement or distorted vascular colors, and their real-time image enhancement processing is poor. For example, in the CLAHE-based method, due to contrast-limited adaptive histogram equalization (which divides regions with gray values ​​above a threshold into other gray value regions and performs non-linear logarithmic stretching on all pixels), this process can lead to color anomalies and color casts.

[0004] Furthermore, conventional image enhancement methods enhance the entire image, which is not only time-consuming but also introduces overall noise and doesn't significantly enhance useful information. Therefore, a processing method that is faster and provides better enhancement results is more beneficial for providing medical assistance. Summary of the Invention

[0005] Purpose of the invention: In order to solve the problems existing in the prior art, the present invention provides a method for enhancing blood vessels in endoscopic images. By extracting blood vessel images from the acquired endoscopic images, the method achieves targeted enhancement of blood vessel images, reduces the amount of data to be processed and image noise, and effectively highlights the enhancement effect of blood vessel regions in the image.

[0006] Technical solution: To achieve the above objectives, the present invention provides a method for enhancing blood vessels in endoscopic images, comprising the following steps:

[0007] S1. Read the original endoscope image and extract the original blood vessel image based on the image features of the blood vessel region;

[0008] S2. Convert the color space of the extracted original blood vessel image to the HSV color space and obtain the image information of the H, S, and V channels.

[0009] S3. Keep the original blood vessel image in the H channel unchanged, and perform enhancement processing on the original blood vessel images in the S and V channels respectively;

[0010] S4. The original blood vessel image in the H channel is fused with the enhanced blood vessel image in the S and V channels to obtain the enhanced blood vessel image in the HSV color space. The enhanced blood vessel image is then converted back from the HSV color space to the original color space of the original blood vessel image to obtain the enhanced endoscopic image after blood vessel enhancement.

[0011] Furthermore, the step of reading the original endoscopic image and extracting the original vascular image based on the image features of the vascular region specifically involves:

[0012] The image information of each channel of the original endoscope image in the original color space is obtained, and the original blood vessel image X1 is extracted based on the image features of the blood vessel region in the original color space.

[0013] Specifically, the original color space is the RGB color space, and the image features of the blood vessel region in the original color space include:

[0014] ① R(x,y) is the maximum value among R(x,y), G(x,y), and B(x,y);

[0015] ②R(x,y) is greater than the threshold K;

[0016] ③ The difference between G(x,y) and B(x,y) is less than the threshold Z, where R(x,y), G(x,y) and B(x,y) represent the pixel values ​​of the R, G, and B channels corresponding to the pixel coordinates (x,y) in the original endoscopic image, respectively.

[0017] Furthermore, the enhancement processing of the original blood vessel image in the V channel specifically includes:

[0018] A guided filtering algorithm is used to filter the original blood vessel image V(x,y) of the V channel to obtain the filtered blood vessel image Q(x,y) of the V channel; based on the correlation between the filtered blood vessel image Q(x,y) and the original blood vessel image V(x,y), the enhanced blood vessel image V′(x,y) of the V channel is generated.

[0019] Specifically, the enhanced vascular image of the V channel is as follows:

[0020] V′(x,y)=n×(V(x,y)-Q(x,y))+Q(x,y), where n is the image enhancement coefficient.

[0021] Preferably, the value of n is in the range of [2,5].

[0022] Furthermore, the input image P(x,y) and the guiding image I(x,y) of the guided filtering algorithm are both the original blood vessel image V(x,y) of the V channel.

[0023] Furthermore, the guided filtering algorithm uses a dynamic filtering window for guided filtering, that is: based on the original blood vessel image, region segmentation is performed to extract the central region of the blood vessel and the edge region of the blood vessel, and the filtering window size of the edge region of the blood vessel is set to be larger than the filtering window size of the central region of the blood vessel.

[0024] Specifically, the region segmentation based on the original blood vessel image includes:

[0025] Within the original color space, region segmentation is performed based on the image features of the central and edge regions of the blood vessels in the original blood vessel image.

[0026] Specifically, the original color space is the RGB color space, and the image features of the central region and the edge region of the blood vessel include:

[0027] When R(x,y)>G(x,y)+B(x,y), it means that the pixel coordinate (x,y) is in the central region of the blood vessel; otherwise, it means that the pixel coordinate (x,y) is in the edge region of the blood vessel. R(x,y), G(x,y) and B(x,y) represent the R, G and B channel pixel values ​​corresponding to the pixel coordinate (x,y) in the original image of the blood vessel, respectively.

[0028] Specifically, the filter window size for the blood vessel edge region is larger than the filter window size for the blood vessel center region, specifically:

[0029] If the size of the filtering window in the central region of the blood vessel is k×k, then the size of the filtering window in the edge region of the blood vessel is (k+Δ)×(k+Δ), where k and Δ are set parameters.

[0030] Preferably, k is an odd number, Δ is an even number, and the value of Δ ranges from (1, k+1).

[0031] Furthermore, the enhancement processing of the original blood vessel image in the S channel specifically includes:

[0032] The raw blood vessel image S(x,y) of channel S is segmented:

[0033] a. If S(x,y) > threshold m, then the enhanced blood vessel image of channel S is S′(x,y) = S(x,y);

[0034] b. If S(x,y)≤threshold m, then the enhanced blood vessel image of channel S′(x,y)=log(S(x,y)+h);

[0035] Where h is the logarithmic stretching factor.

[0036] Preferably, S min <m≤(S) max +2S min ) / 3, where S min and S max These are the minimum and maximum values ​​of S(x,y), respectively.

[0037] Beneficial effects:

[0038] 1. This invention only enhances the original image of the blood vessel area, rather than enhancing the entire image. Therefore, it reduces the amount of data processing, improves processing efficiency, has high real-time performance, eliminates the detrimental effects of useless noise on the image, and is more conducive to highlighting the enhancement effect of the blood vessel image.

[0039] 2. This invention converts the original blood vessel image to the HSV color space for enhancement processing, so as to separate color information and brightness information, retain the original color information, and enhance only the saturation and brightness information, so that there is no color distortion in the processing.

[0040] 3. This invention uses an improved guided filtering algorithm to enhance the brightness information of the original blood vessel image. Specifically, it uses a dynamic filtering window for guided filtering, which can achieve different enhancement effects according to different regions, ultimately achieving consistent brightness in the enhanced blood vessel image.

[0041] 4. This invention uses a segmented processing method to enhance the saturation information of the original blood vessel image. For cases with high saturation, the original saturation is maintained, and the color of the image will be more accurate. For cases with low saturation, a logarithmic stretching transformation is performed, which can significantly increase the difference between the blood vessel and the surrounding area. Attached Figure Description

[0042] Figure 1 This is an overall flowchart of the vascular enhancement method in an embodiment of the present invention;

[0043] Figure 2 This is an RGB three-channel data image of the blood vessel and surrounding area in an embodiment of the present invention;

[0044] Figure 3 This is an RGB three-dimensional model diagram constructed in the embodiments of the present invention;

[0045] Figure 4a , 4b This is a schematic diagram of segmenting the original blood vessel image in the S channel in an embodiment of the present invention;

[0046] Figure 5 This is a comparison of endoscopic images before and after vascular enhancement in an embodiment of the present invention. Detailed Implementation

[0047] The preferred embodiments of the present invention will now be described in conjunction with the accompanying drawings, which will more clearly and completely illustrate the technical solution of the present invention.

[0048] Reference Figure 1 This invention provides a method for enhancing blood vessels in endoscopic images, comprising the following steps:

[0049] S1. Read the original endoscope image and extract the original blood vessel image based on the image features of the blood vessel region;

[0050] The original endoscopic image described in this invention can be an original color image of the target body part, such as an RGB image; it can be an endoscopic image acquired in real time within a body cavity, such as an image of the digestive tract, respiratory tract, or urinary tract; or it can be an endoscopic image fused from a white light image (i.e., a visible light image). Preferably, the original endoscopic image in this invention is an RGB image.

[0051] Specifically, the original endoscope image to be processed is read and mapped to the RGB color space. The image information of the three channels R, G, and B, namely R(x,y), G(x,y) and B(x,y), are obtained respectively, which represent the pixel values ​​of the R, G, and B channels corresponding to the pixel coordinates (x,y) in the original endoscope image.

[0052] For example, pixel values ​​are represented by grayscale values, thereby obtaining RGB three-channel data of the raw endoscopic image. Figure 2 This embodiment contains RGB three-channel data of the blood vessel and surrounding area, where the values ​​of each pixel from top to bottom are B(x,y), G(x,y), and R(x,y).

[0053] Reference Figure 3 Based on the acquired RGB three-channel image data, an RGB three-dimensional model of the original endoscopic image is established, and RGB three-channel image features of different regions are extracted, for example:

[0054] (1) When R(x,y) is the maximum value among R(x,y), G(x,y) and B(x,y), and R(x,y) is greater than 100, and the difference between G(x,y) and B(x,y) is less than 10, it means that the pixel coordinate (x,y) is in the blood vessel region.

[0055] (2) When B(x,y) is the maximum value among R(x,y), G(x,y) and B(x,y), and the difference between R(x,y), G(x,y) and B(x,y) is less than 10, it means that the pixel coordinate (x,y) is within the instrument area.

[0056] (3) Pixel coordinates (x,y) that do not satisfy features (1) and (2) are in the organ or mucosal region.

[0057] During image enhancement, simultaneous enhancement of the mucosa or instrument area (especially the corner area) will not have a beneficial effect on the vascular image. Instead, it will introduce new noise, increase the amount of data to be processed, and prolong the image processing time. Therefore, this invention only enhances the vascular image, that is, it quickly extracts the original image of the vascular area through feature (1), that is, the original vascular image X1, and uses it as the target image for subsequent enhancement processing.

[0058] Existing blood vessel segmentation algorithms include edge detection, multi-scale segmentation, and machine learning, but these algorithms have poor real-time performance and cannot meet the real-time requirements of image enhancement. Therefore, this invention adopts a feature extraction method to segment blood vessels based on the image features of each channel in the original color space. The processing is simple and direct, and has high real-time performance.

[0059] S2. Convert the color space of the extracted original blood vessel image to the HSV color space and obtain the image information of the H, S, and V channels.

[0060] HSV and RGB are different representations of color spaces. The RGB color model is a device-oriented color space, and the RGB values ​​are not directly related to the three attributes of color. The HSV color model uses the three basic attributes of color—hue, saturation, and value—to represent color, and is a color model oriented towards visual perception. Because the channels of an RGB image contain not only color information but also brightness information, it is not conducive to preserving the original color information during image enhancement. Furthermore, HSV can better reflect human perception and discrimination of color. Therefore, converting an RGB image to an HSV image allows for the separation of color information and brightness information, preventing color distortion during processing.

[0061] Specifically, the original blood vessel image X1 is converted from the RGB color space to the HSV color space, and the image information of the hue channel H, saturation channel S, and luminance channel V is obtained, namely H(x,y), S(x,y), and V(x,y), which respectively represent the H, S, and V channel pixel values ​​corresponding to the pixel coordinates (x,y) in the original blood vessel image X1.

[0062] S3. Keep the original blood vessel image in the H channel unchanged, and perform enhancement processing on the original blood vessel images in the S and V channels respectively;

[0063] Specifically, a guided filtering algorithm is used to filter the original blood vessel image V(x,y) of the V channel to obtain the filtered blood vessel image Q(x,y) of the V channel; based on the correlation between the filtered blood vessel image Q(x,y) and the original blood vessel image V(x,y), the enhanced blood vessel image V′(x,y) of the V channel is generated.

[0064] The specific steps of the guided filtering algorithm are as follows:

[0065] ① Prepare the input image to be processed and the guide image for guided filtering, i.e., P(x,y) is the input image, I(x,y) is the guide image, and the output image is Q(x,y), where I(x,y)=P(x,y)=V(x,y).

[0066] ② Calculate the mean image and covariance image, that is, for the filter window, calculate the mean of I (I), the mean of P (P), the mean of I*P (IP), and the mean of I*I (II):

[0067]

[0068]

[0069] Where (x0, y0) represents the center point of the filter window, and the size of the filter window is k×k.

[0070] ③ Calculate the variance Var(I) of I and the covariance cov(I,P) between I and P:

[0071] Var(I)=Mean(II)-Mean(I)×Mean(I);

[0072] Cov(I,P)=Mean(II)-Mean(I)×Mean(P).

[0073] ④ Calculate the filter weights: Calculate the filter weight for each pixel based on the guide image. The filter weights determine the degree of influence each pixel has on its surrounding pixels during the filtering process. The calculation formula is as follows:

[0074]

[0075] Where A(x,y) is the filter weight corresponding to pixel coordinates (x,y), and ε is the design parameter (a small positive number) with a value range of 0.01 to 0.05. The smaller ε is, the more sensitive A is to the difference between the input image and the guide image; the larger ε is, the lower the sensitivity of A.

[0076] The constant B is calculated based on weight A:

[0077] B(x,y)=mean(P)-A(x,y)*mean(I);

[0078] Simultaneously perform mean smoothing on both A and B:

[0079]

[0080]

[0081] ⑤ Obtain the output image: Filter the input image using the calculated filter weights. The output image Q(x,y) of the guided filter is:

[0082] Q(x,y)=mean(A)×P(x,y)+mean(B).

[0083] Since the brightness of the edge region of a blood vessel is obviously not as high as that of the center region, if a fixed filter window is used for guided filtering, the brightness of different regions will still be inconsistent after enhancement. Therefore, this invention uses a dynamic filter window for guided filtering to achieve different enhancement effects.

[0084] Specifically, the filtering window in the central region of the blood vessel remains at k×k, while the filtering window in the edge region is increased to (k+Δ)×(k+Δ), where k is preferably an odd number, Δ is preferably an even number, and the value of Δ ranges from (1, k+1). The larger the filtering window, the greater the filtering weight, the better the filtering effect, and the more obvious the effect on image brightness enhancement, so as to achieve consistent brightness of blood vessel images in different regions after enhancement.

[0085] Furthermore, in order to distinguish between the central region and the edge region of the blood vessel, region segmentation can also be performed based on the image features of different blood vessel regions in the original blood vessel image X1. For example, when R(x,y)>G(x,y)+B(x,y), it means that the pixel coordinates (x,y) are in the central region of the blood vessel, and the size of the filtering window is k×k. Conversely, it means that the pixel coordinates (x,y) are in the edge region of the blood vessel, and the size of the filtering window is (k+2)×(k+2).

[0086] Furthermore, the enhanced vascular image of the V channel is as follows:

[0087] V′(x,y)=n×(V(x,y)-Q(x,y))+Q(x,y), where n is the image enhancement coefficient, preferably in the range [2,5]. When n is less than 2, the image enhancement effect is not obvious; when n is greater than 5, too much noise will be introduced during the image enhancement process.

[0088] Specifically, the original blood vessel image S(x,y) in the S channel is segmented: for cases where S is relatively high, the original S channel is preserved, resulting in more accurate colors; for cases where S is relatively low, a logarithmic stretching transformation is performed, which can significantly increase the difference between the blood vessel and its surroundings. The specific operation is as follows:

[0089] a. If S(x,y) ≥ threshold m, then the enhanced blood vessel image of channel S is S′(x,y) = S(x,y), as shown below. Figure 4a As shown;

[0090] b. If S(x,y) < threshold m, then the enhanced blood vessel image of channel S is S′(x,y) = log(S(x,y) + 1), as shown below. Figure 4b As shown. Preferably, m ≤ (S max +2S min ) / 3, where S min and S max These are the minimum and maximum values ​​of S(x,y), respectively. This is because in [S min ,S max The S value in the first 1 / 3 of the data will be more obvious when saturation enhancement is performed, and segmented processing will not affect the saturation intensity of the high saturation region.

[0091] S4. The original blood vessel image in the H channel is fused with the enhanced blood vessel image in the S and V channels to obtain the enhanced blood vessel image in the HSV color space. The enhanced blood vessel image is then converted back from the HSV color space to the original color space of the original blood vessel image to obtain the enhanced endoscopic image after blood vessel enhancement.

[0092] Specifically, the original blood vessel image H(x,y) of the H channel is fused with the enhanced blood vessel image V′(x,y) of the V channel and the enhanced blood vessel image S′(x,y) of the S channel to obtain the enhanced blood vessel image X2 in the HSV color space. Then, it is converted from the HSV color space to the RGB color space. Finally, the original blood vessel image X1 is replaced with the new enhanced blood vessel image X2 to obtain the enhanced endoscopic image after blood vessel enhancement.

[0093] like Figure 5 The image shows a comparison of endoscopic images before and after vascular enhancement. It can be seen that the vascular enhancement method provided by this invention has a good vascular enhancement effect, and the processing does not have problems such as color distortion. Moreover, the algorithm has high real-time performance, which is more conducive to providing medical assistance.

[0094] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications, substitutions, and improvements made by those skilled in the art to the technical solutions of the present invention based on the provided textual description and drawings, without departing from the design concept and spirit of the present invention, should all fall within the scope of protection of the present invention.

Claims

1. A method for enhancing blood vessels in endoscopic images, characterized in that, Includes the following steps: S1. Read the original endoscope image and extract the original blood vessel image based on the image features of the blood vessel region; S2. Convert the color space of the extracted original blood vessel image to the HSV color space and obtain the image information of the H, S, and V channels. S3. Keep the original blood vessel image in the H channel unchanged, and perform enhancement processing on the original blood vessel images in the S and V channels respectively; S4. The original blood vessel image in the H channel is fused with the enhanced blood vessel image in the S and V channels to obtain the enhanced blood vessel image in the HSV color space. The enhanced blood vessel image is then converted back from the HSV color space to the original color space of the original blood vessel image to obtain the enhanced endoscopic image after blood vessel enhancement. Specifically, a guided filtering algorithm is used to filter the original blood vessel image V(x,y) of the V channel to obtain the filtered blood vessel image Q(x,y) of the V channel; based on the formula V′(x,y)=n×(V(x,y)-Q(x,y))+Q(x,y), the enhanced blood vessel image V′(x,y) of the V channel is generated, where n is the image enhancement coefficient, and the value of n is in the range of [2,5]. The input image P(x,y) and the guided image I(x,y) of the guided filtering algorithm are both the original blood vessel image V(x,y) of the V channel. The guided filtering algorithm employs a dynamic filtering window for guided filtering. Specifically, it performs region segmentation based on the original blood vessel image, extracting the central and edge regions of the blood vessels. If the filtering window size for the central region is k×k, then the filtering window size for the edge region is (k+k). )×(k+ ), where k is an odd number, Even number, The range of values ​​for is (1, k+1). The enhancement processing of the original blood vessel image in the S channel specifically includes: The raw blood vessel image S(x,y) of channel S is segmented: a. If S(x,y)≥threshold m, then the enhanced blood vessel image of channel S′(x,y)=S(x,y); b. If S(x,y) < threshold m, then the enhanced blood vessel image of channel S is S′(x,y) = log(S(x,y) + h); Where h is the logarithmic stretching factor; <m≤( +2 ) / 3, of which and These are the minimum and maximum values ​​of S(x,y), respectively.

2. The vascular enhancement method according to claim 1, characterized in that, The process of reading the original endoscopic image and extracting the original vascular image based on the image features of the vascular region specifically involves: The image information of each channel of the original endoscope image in the original color space is obtained, and the original blood vessel image X1 is extracted based on the image features of the blood vessel region in the original color space.

3. The vascular enhancement method according to claim 2, characterized in that, The original color space is the RGB color space, and the image features of the blood vessel region in the original color space include: ① R(x,y) is the maximum value among R(x,y), G(x,y), and B(x,y); ②R(x,y) is greater than the threshold K; ③ The difference between G(x,y) and B(x,y) is less than the threshold Z, where R(x,y), G(x,y) and B(x,y) represent the pixel values ​​of the R, G, and B channels corresponding to the pixel coordinates (x,y) in the original endoscopic image, respectively.

4. The vascular enhancement method according to claim 1, characterized in that, The region segmentation based on the original blood vessel image specifically involves: Within the original color space, region segmentation is performed based on the image features of the central and edge regions of the blood vessels in the original blood vessel image.

5. The vascular enhancement method according to claim 4, characterized in that, The original color space is RGB color space, and the image features of the central and edge regions of the blood vessels include: When R(x,y)>G(x,y)+B(x,y), it means that the pixel coordinate (x,y) is in the central region of the blood vessel; otherwise, it means that the pixel coordinate (x,y) is in the edge region of the blood vessel. R(x,y), G(x,y) and B(x,y) represent the R, G and B channel pixel values ​​corresponding to the pixel coordinate (x,y) in the original image of the blood vessel, respectively.

Images