Endoscopic image blood vessel enhancement method and device, electronic equipment and storage medium

Abstract

The invention relates to a blood vessel enhancement method and device for an endoscope image, electronic equipment and a storage medium. The method comprises the steps that an initial endoscope image with RGB features is acquired; performing gain processing on a pixel value on a G channel of the initial endoscope image to obtain a first endoscope image; obtaining a first difference image based on the initial endoscope image and the first endoscope image; respectively fusing the pixel value of the first difference image with the pixel values of the R channel and the B channel on the first endoscope image to obtain a second endoscope image; acquiring a chromatic aberration image based on the second endoscope image and the first endoscope image; fusing the pixel value of the chromatic aberration image with the pixel value of the R channel on the second endoscope image to obtain a third endoscope image; and acquiring a saturation coefficient, and respectively adjusting pixel values of RGB channels on the third endoscope image by using the saturation coefficient to obtain a target endoscope image. The purposes of enhancing the blood vessel and optimizing the blood vessel display effect are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopic image processing, and in particular to a method, device, electronic equipment and storage medium for enhancing blood vessels of endoscopic images. Background Art

[0002] In minimally invasive surgery for gastrointestinal diseases, medical optical endoscopes can directly present dynamic images of blood vessels and tissues in organs such as the stomach and intestines, allowing doctors to directly observe lesion images and make diagnoses. Due to the dense folds and grooves in the organs and tissues of the human body and insufficient imaging light sources, the clarity of blood vessels and tissues in endoscopic imaging is low, so it is necessary to perform vascular enhancement processing on the images in the endoscopic image processing system.

[0003] In the field of medical image processing, there are many methods in the prior art that can achieve vascular enhancement processing, including Hessian operator, 3D Frangi filtering, multi-color space method, spectral transformation method, directional filter group and method based on deep learning features. In order to achieve better enhancement effects, these methods generally adopt relatively complex logical calculations, which requires a large amount of computing power resources in the image processing system, resulting in high requirements for the hardware module of image processing, thereby increasing the hardware cost of the image processing system and improving the overall cost of the image processing system.

[0004] At present, the endoscopic vascular image enhancement technologies mainly include Olympus's NBI (narrow band imaging), Fuji's FICE (flexible spectral imaging color enhancement) and Pentax's I-Scan technology. The above three vascular enhancement technologies can enhance vascular features in real time. NBI is a hardware image enhancement technology that uses narrow-band light illumination, while FICE and I-Scan are both electronic staining technologies. However, NBI and FICE can only select the reflection intensity of several fixed wavelengths to reconstruct color images. Not only does it require strict calibration of the endoscopic image processing system before use, but the color of the enhanced vascular image is quite different from the real color, which leads to the phenomenon of small blood vessel loss, that is, the small blood vessels at the end of the blood vessel cannot be clearly imaged, resulting in the inability of doctors to accurately identify lesions with subtle inflammation or lesions that are still in the early stages of the disease. Although I-Scan technology can select different wavelength combinations to display images, it can also enhance the vascular surface and enhance the overall image contrast, but the algorithm is relatively complex and will increase the hardware cost of the image processing system. Summary of the invention

[0005] (1) Technical issues to be solved

[0006] The present invention provides a method, device, electronic device and storage medium for enhancing blood vessels of endoscopic images, aiming to enhance blood vessels, especially small blood vessels, by improving the pixel value of the endoscopic image on the G channel, using a chromatic aberration image to improve the pixel value of the endoscopic image on the R channel, and then using the saturation coefficient of the endoscopic image on the B channel to synchronously adjust the pixel value of the endoscopic image on the RGB channel, so as to achieve the purpose of optimizing the blood vessel display effect.

[0007] (2) Technical solution

[0008] In a first aspect, an embodiment of the present invention provides a method for enhancing blood vessels in an endoscopic image, comprising:

[0009] Acquire an initial endoscopic image with RGB features;

[0010] Performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image;

[0011] Based on the initial endoscopic image and the first endoscopic image, acquiring a first difference image;

[0012] Based on the pixel values ​​of the first difference image, respectively adjust the pixel values ​​of all the pixels on the first endoscopic image on the R channel and the B channel to obtain a second endoscopic image;

[0013] acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image;

[0014] Based on the pixel values ​​of the chromatic aberration image, adjusting the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain a third endoscopic image;

[0015] Obtaining a saturation coefficient based on the pixel values ​​of all pixels on the third endoscopic image on the B channel;

[0016] Based on the saturation coefficient, the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel are adjusted respectively to obtain a target endoscopic image.

[0017] In a possible implementation, the pixel values ​​of all pixels on the initial endoscopic image on the G channel are subjected to gain processing to obtain a first endoscopic image, including

[0018] Normalizing the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first normalized image;

[0019] Performing mean processing on the pixel values ​​of all pixels on the first normalized image to obtain a first pixel mean;

[0020] Based on the first pixel mean, obtaining a gain coefficient;

[0021] Performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel based on the gain coefficient to obtain a second normalized image;

[0022] The second normalized image is denormalized to obtain the first endoscopic image.

[0023] In a possible implementation, the pixel values ​​of all pixels on the initial endoscopic image on the G channel are subjected to gain processing to obtain a first endoscopic image, including

[0024] Obtaining a first pixel mean value of all pixels on the initial endoscopic image on the G channel;

[0025] Normalizing the first pixel mean to obtain a normalized first pixel mean;

[0026] Based on the first pixel mean, obtaining a gain coefficient;

[0027] Performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel based on the gain coefficient to obtain a second normalized image;

[0028] The second normalized image is denormalized to obtain the first endoscopic image.

[0029] In a possible implementation, based on the pixel values ​​of the first difference image, the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel are adjusted to obtain a second endoscopic image, including

[0030] Fusing the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel, respectively, to obtain a first fused pixel value on the R channel of the first endoscopic image and a second fused pixel value on the B channel of the first endoscopic image;

[0031] Comparing the first fused pixel value with a pixel threshold;

[0032] Comparing the second fused pixel value with a pixel threshold;

[0033] Based on the comparison result of the first fused pixel value and the pixel threshold, the first fused pixel value of all pixels on the first endoscopic image in the R channel is adjusted, and based on the comparison result of the second fused pixel value and the pixel threshold, the second fused pixel value of all pixels on the first endoscopic image in the B channel is adjusted to obtain the second endoscopic image.

[0034] In a possible implementation, the first fused pixel value of all pixels on the first endoscopic image in the R channel is adjusted based on the comparison result of the first fused pixel value and the pixel threshold, and the second fused pixel value of all pixels on the first endoscopic image in the B channel is adjusted based on the comparison result of the second fused pixel value and the pixel threshold, including

[0035] Based on the fact that the first fused pixel value is greater than the pixel threshold, the pixel threshold is used to replace the first fused pixel value as the pixel value on the R channel; otherwise, the first fused pixel value is used as the pixel value on the R channel;

[0036] Based on the fact that the second fused pixel value is greater than the pixel threshold, the pixel threshold is used to replace the second fused pixel value as the pixel value on the B channel; otherwise, the second fused pixel value is used as the pixel value on the B channel.

[0037] In a possible implementation, acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image includes:

[0038] Obtaining a first maximum pixel value and a first minimum pixel value of all pixels on the second endoscopic image on the R channel, and performing mean processing on the first maximum pixel value and the first minimum pixel value to obtain a second pixel mean;

[0039] performing difference processing on pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image;

[0040] performing difference processing on pixel values ​​of the second endoscopic image and the first endoscopic image on channel B to obtain a third difference image;

[0041] The color difference image is obtained based on the first difference image, the second difference image, the third difference image and the second pixel mean.

[0042] In a possible implementation, the saturation coefficient is obtained based on the pixel values ​​of all pixels on the third endoscopic image on the B channel, including:

[0043] Obtaining the second maximum pixel value and the second minimum pixel value of all pixel points on the third endoscopic image on the B channel;

[0044] The saturation coefficient is obtained based on the second maximum pixel value and the second minimum pixel value.

[0045] In a possible implementation, the acquiring the saturation coefficient based on the second maximum pixel value and the second minimum pixel value includes:

[0046] Performing sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value;

[0047] performing difference processing on the second maximum pixel value and the second minimum pixel value to obtain a second pixel difference;

[0048] Comparing the first pixel sum value with a pixel threshold;

[0049] Based on the first pixel sum value being less than the pixel threshold, using a ratio of the second pixel difference value to the first pixel sum value as the saturation coefficient;

[0050] Based on the fact that the first pixel sum value is greater than or equal to the pixel threshold, the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value is obtained, and the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value are summed to obtain a second pixel sum value, and the ratio of the second pixel difference value to the second pixel sum value is used as the saturation coefficient.

[0051] In a possible implementation, based on the saturation coefficient, the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel are adjusted respectively to obtain a target endoscopic image, including

[0052] Obtaining the second maximum pixel value and the second minimum pixel value of all pixel points on the third endoscopic image on the B channel;

[0053] Performing mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean;

[0054] Performing difference processing on the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean value to obtain a third pixel difference value;

[0055] Performing product processing on the third pixel difference and the saturation coefficient to obtain a fourth pixel difference;

[0056] The third pixel mean, the third pixel difference, and the fourth pixel difference are processed by summing the values ​​to obtain the target endoscopic image.

[0057] In a possible implementation, the obtaining of an initial endoscopic image having RGB features includes:

[0058] acquiring an initial endoscopic image;

[0059] Performing denoising processing on the initial endoscopic image to obtain a denoised initial endoscopic image;

[0060] The de-noised initial endoscopic image is processed by a demosaicing algorithm to obtain an initial endoscopic image with RGB features.

[0061] In a second aspect, an embodiment of the present invention provides a blood vessel enhancement device, comprising:

[0062] An image acquisition module is configured to acquire an initial endoscopic image having RGB features;

[0063] An image gain module is configured to perform gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image;

[0064] a difference image processing module configured to perform difference processing on pixel values ​​on the G channel between the initial endoscopic image and the first endoscopic image to obtain a first difference image;

[0065] A first pixel value fusion module is configured to fuse the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively, to obtain a second endoscopic image;

[0066] a color difference image processing module configured to obtain a second pixel mean value of all pixel points on the second endoscopic image on the R channel; to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image; to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the B channel to obtain a third difference image; and to obtain the color difference image according to the first difference image, the second difference image, the third difference image and the second pixel mean value;

[0067] A second pixel value fusion module is configured to fuse the pixel values ​​on the color difference image with the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain the third endoscopic image;

[0068] a saturation coefficient acquisition module, configured to acquire a second maximum pixel value and a second minimum pixel value of all pixel points on the third endoscopic image on the B channel; and to perform sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value; and to perform difference processing on the second maximum pixel value and the second minimum pixel value to obtain a second pixel difference value; and to acquire a ratio of the second pixel difference value to the first pixel sum value as the saturation coefficient;

[0069] The third pixel value adjustment module is configured to obtain the second maximum pixel value and the second minimum pixel value of all pixel points on the third endoscopic image on the B channel; and to perform mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean; and to perform difference processing on the pixel values ​​of all pixel points on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean, respectively, to obtain a third pixel difference; and to perform product processing on the third pixel difference and the saturation coefficient to obtain a fourth pixel difference; and to perform sum processing on the third pixel mean, the third pixel difference and the fourth pixel difference to obtain the target endoscopic image.

[0070] In which, the saturation coefficient acquisition module is also configured to obtain the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value respectively based on the first pixel sum value being greater than or equal to the pixel threshold, and sum the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value to obtain a second pixel sum value, and use the ratio of the second pixel difference value to the second pixel sum value as the saturation coefficient.

[0071] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0072] In a fourth aspect, an embodiment of the present invention provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor.

[0073] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code, wherein when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0074] (3) Beneficial effects

[0075] In summary, by improving the pixel value of the endoscopic image on the G channel, using the chromatic aberration image to improve the pixel value of the endoscopic image on the R channel, and then using the saturation coefficient of the endoscopic image on the B channel to synchronously adjust the pixel value of the endoscopic image on the RGB channel, blood vessel enhancement, especially small blood vessel enhancement, is achieved, thereby achieving the purpose of optimizing the blood vessel display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0077] Figure 1 It is a flow chart of an image processing method of the present invention.

[0078] Figure 2 is another flow chart of an image processing method of the present invention.

[0079] Figure 3 is another flow chart of an image processing method of the present invention.

[0080] Figure 4 is another flow chart of an image processing method of the present invention.

[0081] Figure 5 is another flow chart of an image processing method of the present invention.

[0082] Figure 6 is another flow chart of an image processing method of the present invention.

[0083] Figure 7 is another flow chart of an image processing method of the present invention.

[0084] Figure 8 is another flow chart of an image processing method of the present invention.

[0085] Fig. 9 It is a structural schematic diagram of an invented image processing device.

[0086] Fig.10 It is a display effect diagram of the original image of the present invention.

[0087] Fig.11 It is a display effect diagram of the target image of the present invention.

[0088] In the figure:

[0089] 10-image acquisition module; 20-image gain module; 30-difference image processing module; 40-first pixel value fusion module; 50-color difference image processing module; 60-second pixel value fusion module; 70-saturation coefficient acquisition module; 80-third pixel value adjustment module. DETAILED DESCRIPTION

[0090] The following is a further detailed description of the embodiments of the present invention in conjunction with the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the described embodiments, and covers any modification, replacement and improvement of parts, components and connection modes without departing from the spirit of the present invention.

[0091] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0092] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0093] The image processing method of the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster of multiple server groups. Any electronic device can implement the image processing method of the embodiment of the present disclosure by a processor calling a computer-readable instruction stored in a memory.

[0094] Figure 1 A flow chart of a method for enhancing blood vessels in an endoscopic image according to the present invention is shown. Figure 1 As shown, the image processing method of the disclosed embodiment of the present invention may include the following steps S10-S80.

[0095] Step S10: Acquire an initial endoscopic image with RGB features.

[0096] In a possible implementation, the initial endoscopic image may be an original image directly obtained by controlling a white light camera or a fluorescent camera of a medical endoscope by an electronic device to capture an image of a target object, wherein the target object may be the acquired digestive tract (such as the stomach, rectum, esophagus, and colon, etc.), so the initial endoscopic image is a digestive tract image with RGB features (such as Fig.10 As shown), and the corresponding blood vessels and tissue information are presented on the digestive tract image. Of course, due to the different parameters of the medical endoscope camera, the data format of the original image directly obtained will also be different. For example, some existing medical endoscope camera devices are based on COMS image sensors to collect images of target objects, resulting in the image data of the original image acquired by the COMS image sensor being a BAYER array image, i.e., a RAW image, which cannot be directly visualized on the terminal display. Therefore, the format of the RAW image needs to be processed accordingly to obtain an initial endoscopic image with RGB features to meet the visualization requirements.

[0097] like Figure 2 As shown, the method for acquiring an initial endoscopic image with RGB features according to an embodiment of the present invention includes the following steps S110-S130.

[0098] S110, acquiring an initial endoscope image.

[0099] In a possible implementation, the CMOS image sensor in the camera of the medical endoscope is used to collect images of the target object to obtain an initial endoscope image. The image data of the initial endoscope image is a BAYER array image, i.e., a RAW image, as shown in the following table:

[0100] The letter in each box in the table represents a pixel on the RAW image, where B represents the pixel value of the blue channel, Gr represents the pixel value of the green channel in odd rows, Gb represents the pixel value of the green channel in even rows, and R represents the pixel value of the red channel. Therefore, the pixels on the RAW image only have the pixel value of one of the R, G, and B channels, that is, there is no pixel that has the pixel value of the RGB channels at the same time, that is, the RGB feature.

[0101] S120: Perform denoising processing on the initial endoscopic image to obtain a denoised initial endoscopic image.

[0102] In one possible implementation, during the process of acquiring the initial endoscopic image, more noise will be generated on the initial endoscopic image due to other interference factors such as the electromagnetic signal inside the endoscope camera, the light source host or the camera host. In order to reduce the impact of the noise on the quality of the initial endoscopic image, it is necessary to denoise the initial endoscopic image. Specifically, a Bayer denoising algorithm is used for processing, wherein the Bayer denoising algorithm includes mixed domain algorithms such as spatial domain method, transform domain method, time domain method and BM3D denoising. This embodiment uses the spatial domain method for denoising.

[0103] S130, performing a demosaicing algorithm process on the denoised initial endoscopic image to obtain an initial endoscopic image with RGB features.

[0104] In one possible implementation, the initial endoscopic image cannot be displayed directly due to its own format. In order to meet the requirements of subsequent image display and processing, a de-mosaicing algorithm is required for color reconstruction. The de-mosaicing algorithm includes a neighborhood interpolation method, an edge discrimination method, a residual interpolation method, a deep learning method, etc. In this embodiment, a more commonly used neighborhood interpolation method is used to process the pixel values ​​of all pixels on the R channel, the G channel and the B channel of the initial endoscopic image to obtain an initial endoscopic image with RGB features, that is, all pixels on the initial endoscopic image have pixel values ​​of the RGB channels at the same time. Specifically, the color interpolation algorithm includes a bilinear interpolation method or a neighboring pixel average method. Exemplarily, this embodiment For example, the pixel values ​​of all pixels on the original image are processed by the adjacent pixel mean method. According to the pixel value distribution arrangement in the above table, the corresponding calculation formula is used to process the pixel points with R features, the pixel points with Gr features, the pixel points with Gb features and the pixel points with B features so that the pixel points have RGB features. Specifically, except for the pixels in the 1st row and the 1st column and the K+1th row and the K+1th column, the pixels in other rows or columns have 8 adjacent pixels. The adjacent pixel mean method is similar for the pixel points with R features and the pixel points with B features. For example, the pixel point with R features in the 3rd row and the 3rd column is represented as R(3,3). The pixel at this position The pixel value of the R channel of the point is the pixel value of R(3,3), and the pixel value of the G channel of the pixel is the average of the values ​​of the four adjacent pixels with G channel characteristics, that is, G(3,3)=1 / 4*[Gr(3,2)+Gr(3,4)+Gb(2,3)+Gb(4,3)]. Similarly, the pixel value of the B channel of the pixel is the average of the values ​​of the four adjacent pixels with B channel characteristics, that is, B(3,3)=1 / 4*[B(2,2)+B(2,4)+B(4,2)+B(4,4)]. Similarly, taking the pixel with B characteristics in the 4th row and 4th column as an example, it is characterized as B(4,4). The pixel value of the B channel of the pixel at this position is the pixel value of B(4,4), and the The pixel value of the R channel of the pixel point is the average value of the values ​​of the four adjacent pixels with R channel characteristics, R(4,4)=1 / 4*[R(3,3)+R(3,5)+R(5,3)+R(5,5)], and the pixel value of the G channel of the pixel point is the average value of the values ​​of the four adjacent pixels with G channel characteristics, that is, G(4,4)=1 / 4*[Gb(4,3)+Gb(4,5)+Gr(3,4)+Gr(5,4)]. The adjacent pixel average method is similar for the pixel points with Gb characteristics and the pixel points with Gr characteristics. For example, the pixel point with Gr characteristics in the 3rd row and 4th column is represented as Gr(3,4), and the pixel value of the G channel of the pixel point at this position is Gr(3,4), and the pixel value of the R channel of the pixel is the average of the values ​​of the two adjacent pixels with R channel characteristics, that is, R(3,4)=1 / 2*[R(3,3)+R(3,5)], and the pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(3,4)=1 / 2*[B(2,4)+B(4,4)]. Taking the pixel with Gb characteristics at the 4th row and the 3rd column as an example, it is characterized as Gb(4,3). The G channel of the pixel at this position The pixel value is the pixel value of Gb(4,3), and the pixel value of the R channel of the pixel is the average of the values ​​of the two adjacent pixels with R channel characteristics, that is, R(4,3)=1 / 2*[R(3,3)+R(5,3)], and the pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(4,3)=1 / 2*[B(4,2)+B(4,4)], so that the pixel values ​​of all pixels on the initial endoscopic image have corresponding RGB features.

[0105] S20, performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image.

[0106] In one possible implementation, due to the absorption characteristics of blood glucose protein to green light, the pixel value on the G channel of the initial endoscopic image will be low, causing the blood vessels on the initial endoscopic image to be dark and resulting in low clarity. In order to highlight the blood vessels, especially small blood vessels, it is necessary to compensate the pixel value on the G channel of the initial endoscopic image. Specifically, the pixel value on the G channel of all pixels on the initial endoscopic image is gain processed. By increasing the pixel value on the G channel of the initial endoscopic image, the darker blood vessel part can be highlighted, thereby improving the clarity of the blood vessels, especially the clarity of small blood vessels.

[0107] like Figure 3 As shown, the method of performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain the first endoscopic image includes the following steps S210-S250.

[0108] S210 , normalizing the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first normalized image.

[0109] In a possible implementation, since it is necessary to compensate for the darkening of the blood vessels due to the green light absorbed by hemoglobin by performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel, in order to reduce the computational complexity in the gain processing, it is necessary to normalize the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first normalized image. Specifically, M=N / 255 is used for calculation, where M is the pixel value on the first normalized image, N is the pixel value of all pixels on the initial endoscopic image on the G channel, the value range of N is 0-255, and the value range of M is 0-1.

[0110] S220: Perform mean processing on the pixel values ​​of all pixels on the first normalized image to obtain a first pixel mean.

[0111] In one possible implementation, in order to further reduce the complexity of the calculation and improve the speed and efficiency of obtaining the pixel values ​​of all pixels on the first endoscopic image on the G channel for gain, the pixel values ​​of all pixels on the first normalized image are averaged to obtain a first pixel mean, and the calculation of multiple pixel values ​​on the first normalized image is converted into a single first pixel mean for calculation, which can greatly reduce the complexity of the calculation.

[0112] In a possible implementation, the complexity of the calculation can be greatly reduced by preferentially obtaining the first pixel mean of all pixels on the initial endoscopic image on the G channel, and then normalizing the first pixel mean to obtain a single first pixel mean for calculation by obtaining the normalized first pixel mean.

[0113] S230: Obtain a gain coefficient based on the first pixel mean.

[0114] In a possible implementation, in order to further reduce the complexity of calculation and improve the speed and efficiency of obtaining the pixel values ​​of all pixels on the first endoscopic image on the G channel for gain, the corresponding gain coefficient can also be obtained based on the first pixel mean. Specifically, the gain coefficient calculation formula is adopted: To obtain, where g ave is the first pixel mean, d g is the gain factor.

[0115] S240 , performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel based on the gain coefficient to obtain a second normalized image.

[0116] In a possible implementation, in order to increase the pixel values ​​of all pixels on the initial endoscopic image in the G channel, that is, to increase the brightness of the blood vessel part, the corresponding gain coefficient is used to perform gain processing on the pixel values ​​of all pixels on the initial endoscopic image in the G channel to obtain a second normalized image. Specifically, the mapping function is used. Calculate, where g(x,y) is the pixel value of the G channel on the normalized image, g * (x, y) is the pixel value of the G channel on the second normalized image.

[0117] S250: Perform denormalization processing on the second normalized image to obtain the first endoscopic image.

[0118] In one possible implementation, since the pixel values ​​of all pixels on the normalized image are between 0 and 1, which do not meet the display requirements of the color image, it is necessary to denormalize the pixel values ​​on the second normalized image to obtain the first endoscopic image with the brightness of the blood vessel part enhanced.

[0119] S30: Acquire a first difference image based on the initial endoscopic image and the first endoscopic image.

[0120] In a possible implementation, since blood glucose protein absorbs more green light, the pixel value on the G channel of the initial endoscopic image is relatively low. By performing gain processing on the pixel value on the G channel, the brightness of the blood vessel part on the initial endoscopic image can be quickly increased to enhance clarity. This is based on the fact that the lower pixel value of the G channel has a larger gain amplitude than that of the B channel and the R channel. Therefore, obtaining a first pixel difference between the pixel values ​​of the initial endoscopic image and the first endoscopic image on the G channel as the pixel value on the first difference image, compared with obtaining the difference between the pixel values ​​of the initial endoscopic image and the first endoscopic image on the B channel or the R channel, can ensure that the first pixel difference has a larger difference amplitude, that is, a larger numerical value, thereby improving the speed and efficiency of subsequent adjustment of the pixel values ​​on the B channel or the R channel of the first endoscopic image. Specifically, the corresponding pixel points on the G channel of the initial endoscopic image and the first endoscopic image are The pixel values ​​of the first endoscopic image are subjected to difference processing to obtain the first difference image. Exemplarily, the pixel values ​​of the corresponding pixels on the initial endoscopic image on the G channel are subtracted from the pixel values ​​of all pixels on the first endoscopic image on the G channel to obtain the first pixel difference of all pixels on the first endoscopic image on the G channel, while maintaining the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel unchanged. Since the pixel values ​​of all pixels on the first endoscopic image on the G channel are amplified after gain processing, it is possible to ensure that the pixel values ​​of all pixels on the first difference image, i.e., the first pixel difference, are positive numbers, and all pixels on the first difference image only retain the pixel difference between the initial endoscopic image and the first endoscopic image on the G channel, i.e., the first pixel difference, which can reduce the computational complexity of subsequently using the first difference image to adjust the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively.

[0121] S40. Based on the pixel values ​​of the first difference image, adjust the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively to obtain a second endoscopic image.

[0122] In a possible implementation, since blood glucose protein can also absorb a small amount of red light and less blue light, in order to better restore the color of the blood vessel part and ensure the authenticity of the highlighted blood vessel part, it is necessary to compensate the pixel values ​​on the R channel and the B channel of the initial endoscopic image, and by increasing the pixel values ​​on the R channel and the B channel of the initial endoscopic image or the first endoscopic image, the authenticity of the color of the blood vessel part is restored. Specifically, since the first difference image only has the pixel values ​​on the G channel of the initial endoscopic image and the first endoscopic image, The pixel difference of the corresponding pixel is superimposed and fused with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the pixel values ​​of all pixels on the initialization endoscopic image on the R channel and the pixel values ​​of all pixels on the first endoscopic image on the B channel, so as to compensate the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel, which can not only reduce the calculation complexity, but also improve the speed and efficiency of compensating the pixel values ​​on the R channel and the B channel on the initial endoscopic image or the first endoscopic image, based on the large difference amplitude of the first pixel difference.

[0123] like Figure 4 As shown, the method of adjusting the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively based on the pixel values ​​of the first difference image to obtain the second endoscopic image includes the following steps S410-S430.

[0124] S410. Fuse the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively to obtain a first fused pixel value on the R channel of the first endoscopic image and a second fused pixel value on the B channel of the first endoscopic image.

[0125] In a possible implementation, in order to better restore the color of the blood vessel part and ensure the authenticity of the highlighted blood vessel part, it is necessary to compensate the pixel values ​​on the R channel and the B channel of the initial endoscopic image accordingly. Therefore, the pixel values ​​of the first difference image and the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel are fused, that is, the pixel values ​​are superimposed to obtain the first fused pixel value and the second fused pixel value, thereby increasing the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel.

[0126] S420: Compare the first fused pixel value with a pixel threshold and compare the second fused pixel value with a pixel threshold.

[0127] In one possible implementation, since the first fused pixel value and the second fused pixel value may exceed the maximum pixel value of 255, and since the pixels exceeding the maximum pixel value cannot be displayed normally, it is necessary to add a pixel threshold, i.e., the maximum pixel value of 255, to identify whether the first fused pixel value and the second fused pixel value exceed the pixel threshold, so as to adjust the pixel values ​​of the corresponding pixels on the R channel and the B channel, and ensure that each pixel on the second endoscopic image can be displayed normally.

[0128] S430. Adjust the first fused pixel value of all pixels on the first endoscopic image in the R channel based on the comparison result between the first fused pixel value and the pixel threshold, and adjust the second fused pixel value of all pixels on the first endoscopic image in the B channel based on the comparison result between the second fused pixel value and the pixel threshold, to obtain the second endoscopic image.

[0129] In a possible implementation, since the comparison result between the first fused pixel value and the pixel threshold and the comparison result between the second fused pixel value and the pixel threshold have been obtained, in order to ensure that each pixel point on the second endoscopic image can be displayed normally, it is necessary to make corresponding adjustments to the first fused pixel value and the second fused pixel value that exceed the pixel threshold so that the first fused pixel value and the second fused pixel value do not exceed the pixel threshold, that is, the maximum pixel value. Specifically, based on the first fused pixel value being greater than the pixel threshold, the pixel threshold is used to replace the first fused pixel value as the pixel value on the R channel; otherwise, the first fused pixel value is used as the pixel value on the R channel; similarly, based on the second fused pixel value being greater than the pixel threshold, the pixel threshold is used to replace the second fused pixel value as the pixel value on the B channel; otherwise, the second fused pixel value is used as the pixel value on the B channel.

[0130] S50: acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image.

[0131] In one possible implementation, since the pixel values ​​of all pixels on the second endoscopic image on the R channel and the B channel are superimposed and fused with the corresponding first pixel difference, the red component of the R channel, the blue component of the B channel, and the green component of the G channel have a small difference in brightness, resulting in a low color contrast of the second endoscopic image, making it difficult to highlight the difference between blood vessels and tissues. Moreover, since the color difference image has a significant effect in improving the color contrast of blood vessels, it can highlight the red characteristics of blood vessels, thereby improving the contrast of blood vessels. Therefore, it is necessary to obtain the color difference image.

[0132] like Figure 5As shown, the method for acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image includes steps S510-S540.

[0133] S510. Obtain a first maximum pixel value and a first minimum pixel value on the R channel of all pixels on the second endoscopic image, and perform mean processing on the first maximum pixel value and the first minimum pixel value to obtain a second pixel mean.

[0134] In one possible implementation, since all pixel points on the second endoscopic image have a large number of pixel values ​​on the R channel, by performing mean processing on the large number of pixel values, the computational complexity can be reduced and the speed of pixel value processing can be improved. Specifically, the first maximum pixel value and the first minimum pixel value of all pixel points on the second endoscopic image on the R channel are averaged to obtain a second pixel mean. Compared with performing mean processing on the pixel values ​​of all pixel points on the second endoscopic image on the R channel, the computational complexity can be further reduced and at the same time, the necessary calculation factors can be provided for obtaining pixel values ​​on the chromatic aberration image.

[0135] S520: Perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image.

[0136] In one possible implementation, since the pixel values ​​of the color difference image are calculated based on the color difference formula, it is necessary to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain the second pixel difference, thereby providing the necessary calculation factors for the color difference formula.

[0137] S530: Perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the B channel to obtain a third difference image.

[0138] In one possible implementation, since the pixel values ​​of the color difference image are calculated based on the color difference formula, it is necessary to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the B channel to obtain a third pixel difference, thereby providing the necessary calculation factors for the color difference formula.

[0139] S540: Obtain the color difference image based on the first difference image, the second difference image, the third difference image and the second pixel mean.

[0140] In a possible implementation, since the color difference algorithm is used to obtain the pixel values ​​of the corresponding pixels on the color difference image, it is necessary to first determine the pixel difference values ​​corresponding to the G channel, R channel and B channel of the color difference image between the two images, and obtain the pixel average of the maximum pixel value and the minimum pixel value on any one of the two images as a calculation factor. Specifically, the average formula is used. The second pixel mean is calculated, where C1 is the first maximum pixel value of all pixels on the second endoscopic image on the R channel, and C2 is the first minimum pixel value of all pixels on the second endoscopic image on the R channel. Then the color difference calculation formula is used: The color difference pixel value is calculated to obtain the color difference image, where is the second pixel mean, ΔC is the pixel value of all pixels on the color difference image, ΔR is the second pixel difference, ΔG is the first pixel difference, and ΔB is the third pixel difference.

[0141] S60. Based on the pixel values ​​of the chromatic aberration image, adjust the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain a third endoscopic image.

[0142] In one possible implementation, since the color difference image can enhance the contrast of blood vessel color, the pixel value of the color difference image is used to enhance the pixel value of all pixel points on the second endoscopic image on the R channel, thereby enhancing the red features of the blood vessels to enhance the contrast of blood vessel color. Specifically, the pixel value on the color difference image is superimposed and fused with the pixel value of all pixel points on the second endoscopic image on the R channel to obtain the third endoscopic image.

[0143] In a possible implementation, since the pixel values ​​of all pixels on the fused third endoscopic image on the R channel may exceed the maximum pixel value of 255, in order to ensure that the pixel values ​​of all pixels on the third endoscopic image on the R channel can be displayed normally, it is necessary to add a pixel threshold, i.e., the maximum pixel value of 255, to identify whether the pixel values ​​of all pixels on the fused third endoscopic image on the R channel exceed the pixel threshold, so as to adjust the pixel value size of the corresponding pixel point. Specifically, based on the fact that the pixel values ​​of all pixels on the fused third endoscopic image on the R channel are greater than the pixel threshold, the pixel threshold is used to replace the pixel values ​​of all pixels on the fused third endoscopic image on the R channel; otherwise, the pixel values ​​of all pixels on the fused third endoscopic image on the R channel are used as the pixel value of the pixel on the R channel.

[0144] S70: Obtaining a saturation coefficient based on the pixel values ​​of all pixels on the third endoscopic image on the B channel.

[0145] In a possible implementation, since the pixel values ​​of the R channel on the first endoscopic image and the pixel values ​​of the R channel on the second endoscopic image are enhanced respectively, the pixel values ​​of the R channel on the third endoscopic image may have a red oversaturation phenomenon, which will cause the loss of vascular details and make it difficult to clearly display the original vascular details. Therefore, it is necessary to adjust the pixel values ​​of the R channel on the third endoscopic image to avoid the oversaturation phenomenon, so as to prevent the loss of vascular detail features. Since blue is darker than red, and based on the absorption characteristics of vascular proteins to blue light, it is insensitive to vascular reflection, so it will not destroy the imaging of small blood vessels. By reducing the pixel value of the red channel through the saturation coefficient of the blue channel, the color of the blood vessels on the third endoscopic image can be made less bright, thereby achieving a color balance effect. Since the pixel values ​​of the G channel have been gain processed on the first endoscopic image, if the saturation coefficient of the green channel, i.e., the G channel, is used, it will affect the imaging clarity of the small blood vessels on the third endoscopic image. Therefore, it is necessary to obtain the saturation coefficient of the blue channel, i.e., the B channel.

[0146] like Figure 6 As shown, the method for obtaining the saturation coefficient based on the pixel values ​​of all pixels on the third endoscopic image on the B channel includes steps S710-S720.

[0147] S710. Obtain the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel.

[0148] In one possible implementation, since the saturation coefficient of the B channel will not destroy the imaging of small blood vessels and can make the blood vessel color display effect on the third endoscopic image balanced, it is necessary to obtain the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel to provide a parameter factor for subsequent saturation coefficient calculation.

[0149] S720. Obtain the saturation coefficient based on the second maximum pixel value and the second minimum pixel value.

[0150] In a possible implementation, since there are multiple methods for calculating image saturation, among which the calculation method based on RGB value and the HSV color space method are both relatively common methods, this embodiment adopts a calculation method similar to the RGB value, specifically, by obtaining the difference between the second maximum pixel value and the second minimum pixel value and obtaining the sum of the second maximum pixel value and the second minimum pixel value and obtaining the pixel threshold, i.e., the difference between the maximum pixel value and the second maximum pixel value and obtaining the pixel threshold, i.e., the difference between the maximum pixel value and the second minimum pixel value, and based on the ratio of the difference between the second maximum pixel value and the second minimum pixel value and the sum of the second maximum pixel value and the second minimum pixel value as the saturation coefficient; or based on the ratio of the difference between the second maximum pixel value and the second minimum pixel value and the sum of the pixel threshold, i.e., the difference between the maximum pixel value and the second maximum pixel value and the difference between the pixel threshold, i.e., the difference between the maximum pixel value and the second minimum pixel value, as the saturation coefficient.

[0151] like Figure 7 As shown, the method for acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image includes steps S7210-S7250.

[0152] S7210. Perform sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value.

[0153] In a possible implementation, due to the need to adopt a calculation method similar to the existing RGB value, it is necessary to perform sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value, and use the first pixel sum value as a calculation factor.

[0154] S7220. Perform difference processing on the second maximum pixel value and the second minimum pixel value to obtain a fourth pixel difference.

[0155] In a possible implementation, due to the need to adopt a calculation method similar to the existing RGB value, it is necessary to perform difference processing on the second maximum pixel value and the second minimum pixel value to obtain a fourth pixel difference, and use the fourth pixel difference as a calculation factor.

[0156] S7230. Compare the first pixel sum value with the pixel threshold.

[0157] In a possible implementation, since the first pixel and value may be of two types: exceeding the pixel threshold, i.e., the maximum pixel value 255, and being less than or equal to the pixel threshold, i.e., the maximum pixel value 255, different calculation methods are required to obtain the saturation coefficient.

[0158] S7240. Based on the fact that the first pixel sum value is less than the pixel threshold, use the ratio of the fourth pixel difference value to the first pixel sum value as the saturation coefficient.

[0159] In a possible implementation, for the first type in which the first pixel sum is less than the pixel threshold, the ratio of the fourth pixel difference to the first pixel sum is used as the saturation coefficient. Specifically, the first saturation coefficient formula is used: The saturation coefficient is calculated, where max is the second maximum pixel value, min is the second minimum pixel value, v=max+min, is the first pixel sum value, max-min is the fourth pixel difference value, and s takes a value of 0-1.

[0160] S7250. Based on the fact that the first pixel sum value is greater than or equal to the pixel threshold, obtain the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value, respectively, and sum the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value to obtain a second pixel sum value, and use the ratio of the fourth pixel difference value to the second pixel sum value as the saturation coefficient.

[0161] In a possible implementation, for the second type in which the first pixel sum is greater than or equal to the pixel threshold, the ratio of the fourth pixel difference to the second pixel sum is used as the saturation coefficient. Specifically, the second saturation coefficient formula is used: The saturation coefficient is calculated, where max is the second maximum pixel value, min is the second minimum pixel value, v=max+min, is the first pixel sum value, max-min is the fourth pixel difference value, 255 is the pixel threshold, and s takes a value of 0-1.

[0162] S80, based on the saturation coefficient, adjusting the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel respectively to obtain a target endoscopic image.

[0163] In a possible implementation, in order to avoid the red oversaturation phenomenon in the pixel values ​​of the R channel on the third endoscopic image, the saturation coefficient is used to adjust the pixel values ​​on the R channel, the B channel and the G channel. By reducing the pixel value of the R channel and increasing the pixel value of the B channel, a color balance effect can be achieved, thereby highlighting the red characteristics of blood vessels, especially the red characteristics of small blood vessels, and optimizing the blood vessel display effect (such as Fig.11 shown).

[0164] like Figure 8As shown, the method for acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image includes steps S810-S850.

[0165] S810. Obtain the second maximum pixel value and the second minimum pixel value of all pixel points on the third endoscopic image on the B channel.

[0166] In one possible implementation, since the saturation coefficient of the B channel will not destroy the imaging of small blood vessels and can make the blood vessel color display effect on the third endoscopic image balanced, it is necessary to obtain the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel to provide calculation factors for the subsequent adjustment of the pixel values ​​of all pixels on the third endoscopic image on the R channel, G channel and B channel.

[0167] S820. Perform mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean.

[0168] In one possible implementation, in order to further reduce the complexity of calculation and improve the speed and efficiency of fusing the pixel values ​​of all pixels on the third endoscopic image on the R channel, G channel and B channel respectively, it is necessary to perform mean processing on the second maximum pixel value and the second minimum pixel value to obtain the third pixel mean.

[0169] S830, performing difference processing on the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean value to obtain a fifth pixel difference.

[0170] In a possible implementation, since the pixel values ​​of all pixels on the third endoscopic image in the R channel have undergone two pixel value fusions, in order to avoid the red oversaturation phenomenon in the pixel values ​​of the R channel on the third endoscopic image, the pixel values ​​of the R channel need to be lowered. Similarly, since the pixel values ​​of all pixels on the third endoscopic image in the G channel and the B channel have also undergone one pixel value fusion, in order to achieve the effect of color balance, the pixel values ​​on the G channel and the B channel also need to be adaptively lowered. Therefore, the pixel values ​​of all pixels on the third endoscopic image in the R channel, the G channel and the B channel are respectively subtracted from the third pixel mean to obtain the fifth pixel difference. Specifically, the difference formula is used. The fifth pixel difference corresponding to the R channel, G channel and B channel is calculated respectively, where max is the second maximum pixel value, min is the second minimum pixel value, B1, G1 and R2 are the pixel values ​​of all pixels on the third endoscopic image on the R channel, G channel and B channel respectively.

[0171] S840, performing product processing on the fifth pixel difference and the saturation coefficient to obtain a sixth pixel difference.

[0172] In a possible implementation, in order to further lower the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel, and the B channel, the fifth pixel difference and the saturation coefficient are multiplied to obtain a sixth pixel difference. Since the saturation coefficient is a constant less than 1 and greater than 0, the fifth pixel difference can be further lowered. Specifically, the corresponding calculation formula is used. Calculate the sixth pixel difference corresponding to the R channel, G channel and B channel respectively, where max is the second maximum pixel value, min is the second minimum pixel value, R2, G1 and B1 are the pixel values ​​of all pixels on the third endoscopic image on the R channel, G channel and B channel respectively, and s is the saturation coefficient, and the value range is 0-1.

[0173] S850, performing sum processing on the third pixel mean, the fifth pixel difference, and the sixth pixel difference to obtain the target endoscopic image.

[0174] In a possible implementation, since the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel are averaged, the pixel values ​​of all pixels on the target endoscopic image are initially lowered, and since the fifth pixel difference and the sixth pixel difference are respectively the second lowering and the third lowering of the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel, the third pixel mean, the fifth pixel difference and the sixth pixel difference are summed to obtain the target pixel value, so that the target pixel value obtained after the fusion of the third pixel mean, the fifth pixel difference and the sixth pixel difference can be overall lowered, thereby obtaining the target endoscopic image with balanced color and prominent blood vessel red features (such as Fig.11 Specifically, the corresponding pixel value fusion algorithm formula is used Calculate the target pixel values ​​R3, G2 and B2 corresponding to the R channel, G channel and B channel respectively, where max is the second maximum pixel value, min is the second minimum pixel value, R2, G1 and B1 are the pixel values ​​of all pixels on the third endoscopic image on the R channel, G channel and B channel respectively, and s is the saturation coefficient, ranging from 0 to 1.

[0175] Fig. 9 A schematic diagram of the structure of a blood vessel enhancement device of the present invention is shown. Fig. 9As shown, a blood vessel enhancement device according to a disclosed embodiment of the present invention comprises

[0176] An image acquisition module is configured to acquire an initial endoscopic image having RGB features;

[0177] An image gain module is configured to perform gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image;

[0178] a difference image processing module configured to perform difference processing on pixel values ​​on the G channel between the initial endoscopic image and the first endoscopic image to obtain a first difference image;

[0179] A first pixel value fusion module is configured to fuse the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively, to obtain a second endoscopic image;

[0180] a color difference image processing module configured to obtain a second pixel mean value of all pixel points on the second endoscopic image on the R channel; to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image; to perform difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the B channel to obtain a third difference image; and to obtain the color difference image according to the first difference image, the second difference image, the third difference image and the second pixel mean value;

[0181] A second pixel value fusion module is configured to fuse the pixel values ​​on the color difference image with the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain the third endoscopic image;

[0182] a saturation coefficient acquisition module, configured to acquire a second maximum pixel value and a second minimum pixel value of all pixels on the third endoscopic image on the B channel; and to perform sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value; and to perform difference processing on the second maximum pixel value and the second minimum pixel value to obtain a fourth pixel difference value; and to acquire a ratio of the fourth pixel difference value to the first pixel sum value as the saturation coefficient;

[0183] The third pixel value adjustment module is configured to obtain the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel; and to perform mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean value; and to perform difference processing on the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean value to obtain a fifth pixel difference value; and to perform product processing on the third pixel difference value and the saturation coefficient to obtain a sixth pixel difference value; and to perform sum processing on the third pixel mean value, the fifth pixel difference value and the sixth pixel difference value to obtain the target endoscopic image;

[0184] In which, the saturation coefficient acquisition module is also configured to obtain the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value respectively based on the first pixel sum value being greater than or equal to the pixel threshold, and sum the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value to obtain the second pixel sum value, and use the ratio of the fourth pixel difference value to the second pixel sum value as the saturation coefficient.

[0185] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0186] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0187] The embodiment of the present disclosure also proposes an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory. The electronic device can be provided as a server or a terminal device. For example, the electronic device includes a processing component, which further includes one or more processors, and a memory resource represented by a memory, which is used to store instructions executable by the processing component, such as an application. The application stored in the memory may include one or more modules, each corresponding to a set of instructions. In addition, the processing component is configured to execute instructions to perform the above method, and the electronic device may also include a power supply component configured to perform power management of the electronic device, a wired or wireless network interface configured to connect the electronic device to a network, and an input / output interface (I / O interface). The electronic device can operate based on an operating system stored in the memory, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0188] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0189] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0190] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0191] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0192] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0193] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0194] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0195] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0196] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0197] It should be clear that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present invention is not limited to the specific steps and structures described above and shown in the figures. In addition, for the sake of brevity, a detailed description of known methods and technologies is omitted here.

[0198] The above is only an embodiment of the present application and is not limited to the present application. For those skilled in the art, the present application may have various changes and variations without departing from the scope of the present invention. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for enhancing blood vessels in an endoscopic image, characterized in that: include Acquire an initial endoscopic image with RGB features; Performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image; Based on the initial endoscopic image and the first endoscopic image, acquiring a first difference image; Based on the pixel values ​​of the first difference image, respectively adjust the pixel values ​​of all the pixels on the first endoscopic image on the R channel and the B channel to obtain a second endoscopic image; acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image; Based on the pixel values ​​of the chromatic aberration image, adjusting the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain a third endoscopic image; Obtaining a saturation coefficient based on the pixel values ​​of all pixels on the third endoscopic image on the B channel; Based on the saturation coefficient, the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel are adjusted respectively to obtain a target endoscopic image.

2. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: The pixel values ​​of all pixels on the initial endoscopic image on the G channel are subjected to gain processing to obtain a first endoscopic image, including Normalizing the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first normalized image; Performing mean processing on the pixel values ​​of all pixels on the first normalized image to obtain a first pixel mean; Based on the first pixel mean, obtaining a gain coefficient; Performing gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel based on the gain coefficient to obtain a second normalized image; The second normalized image is denormalized to obtain the first endoscopic image.

3. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: Based on the pixel values ​​of the first difference image, the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel are adjusted to obtain a second endoscopic image, including Fusing the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel, respectively, to obtain a first fused pixel value on the R channel of the first endoscopic image and a second fused pixel value on the B channel of the first endoscopic image; Comparing the first fused pixel value with a pixel threshold; Comparing the second fused pixel value with a pixel threshold; Based on the comparison result of the first fused pixel value and the pixel threshold, the first fused pixel value of all pixels on the first endoscopic image in the R channel is adjusted, and based on the comparison result of the second fused pixel value and the pixel threshold, the second fused pixel value of all pixels on the first endoscopic image in the B channel is adjusted to obtain the second endoscopic image.

4. The blood vessel enhancement method of an endoscopic image according to claim 3, characterized in that: The first fused pixel value of all pixels on the first endoscopic image in the R channel is adjusted based on the comparison result of the first fused pixel value and the pixel threshold, and the second fused pixel value of all pixels on the first endoscopic image in the B channel is adjusted based on the comparison result of the second fused pixel value and the pixel threshold, including Based on the fact that the first fused pixel value is greater than the pixel threshold, using the pixel threshold to replace the first fused pixel value as the pixel value on the R channel; Otherwise, the first fused pixel value is used as the pixel value on the R channel; Based on the fact that the second fused pixel value is greater than the pixel threshold, using the pixel threshold to replace the second fused pixel value as the pixel value on the B channel; Otherwise, the second fused pixel value is used as the pixel value on the B channel.

5. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: The step of acquiring a chromatic aberration image based on the second endoscopic image and the first endoscopic image includes: Obtaining a first maximum pixel value and a first minimum pixel value of all pixels on the second endoscopic image on the R channel, and performing mean processing on the first maximum pixel value and the first minimum pixel value to obtain a second pixel mean; performing difference processing on pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image; performing difference processing on pixel values ​​of the second endoscopic image and the first endoscopic image on channel B to obtain a third difference image; The color difference image is obtained based on the first difference image, the second difference image, the third difference image and the second pixel mean.

6. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: The saturation coefficient is obtained based on the pixel values ​​of all pixels on the third endoscopic image on the B channel, including Obtaining the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel; The saturation coefficient is obtained based on the second maximum pixel value and the second minimum pixel value.

7. The blood vessel enhancement method of an endoscopic image according to claim 6, characterized in that: The obtaining of the saturation coefficient based on the second maximum pixel value and the second minimum pixel value comprises: Performing sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value; performing difference processing on the second maximum pixel value and the second minimum pixel value to obtain a fourth pixel difference; Comparing the first pixel sum value with a pixel threshold; Based on the first pixel sum value being less than the pixel threshold, using a ratio of the fourth pixel difference value to the first pixel sum value as the saturation coefficient; Based on the fact that the first pixel sum value is greater than or equal to the pixel threshold, the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value is obtained, and the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value are summed to obtain the second pixel sum value, and the ratio of the fourth pixel difference value to the second pixel sum value is used as the saturation coefficient.

8. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: Based on the saturation coefficient, the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel are adjusted respectively to obtain a target endoscopic image, including Obtaining the second maximum pixel value and the second minimum pixel value of all pixels on the third endoscopic image on the B channel; Performing mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean; Performing difference processing on the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean value to obtain a fifth pixel difference value; Performing a product processing on the fifth pixel difference and the saturation coefficient to obtain a sixth pixel difference; The third pixel mean, the fifth pixel difference, and the sixth pixel difference are processed by summing the values ​​to obtain the target endoscopic image.

9. The blood vessel enhancement method of an endoscopic image according to claim 1, characterized in that: The obtaining of an initial endoscopic image having RGB features comprises: acquiring an initial endoscopic image; Performing denoising on the initial endoscope image to obtain a denoised initial endoscope image; The de-noised initial endoscopic image is processed by a demosaicing algorithm to obtain the initial endoscopic image with RGB features.

10. A blood vessel enhancement device, characterized in that: include An image acquisition module is configured to acquire an initial endoscopic image having RGB features; An image gain module is configured to perform gain processing on the pixel values ​​of all pixels on the initial endoscopic image on the G channel to obtain a first endoscopic image; a difference image processing module configured to perform difference processing on pixel values ​​on the G channel between the initial endoscopic image and the first endoscopic image to obtain a first difference image; A first pixel value fusion module is configured to fuse the pixel values ​​of the first difference image with the pixel values ​​of all pixels on the first endoscopic image on the R channel and the B channel respectively, to obtain a second endoscopic image; a chromatic aberration image processing module, configured to obtain a second pixel mean value of all pixels on the second endoscopic image on the R channel; and for performing difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the R channel to obtain a second difference image; and for performing difference processing on the pixel values ​​of the second endoscopic image and the first endoscopic image on the B channel to obtain a third difference image; and for obtaining the color difference image according to the first difference image, the second difference image, the third difference image and the second pixel mean; A second pixel value fusion module is configured to fuse the pixel values ​​on the color difference image with the pixel values ​​of all pixels on the second endoscopic image on the R channel to obtain the third endoscopic image; a saturation coefficient acquisition module, configured to acquire a second maximum pixel value and a second minimum pixel value of all pixel points on the third endoscopic image on the B channel; The sum is used to perform sum processing on the second maximum pixel value and the second minimum pixel value to obtain a first pixel sum value; and is used to perform difference processing on the second maximum pixel value and the second minimum pixel value to obtain a fourth pixel difference; and for obtaining a ratio of the fourth pixel difference value to the first pixel sum value as the saturation coefficient; A third pixel value adjustment module is configured to obtain a second maximum pixel value and a second minimum pixel value of all pixel points on the third endoscopic image on the B channel; and is used to perform mean processing on the second maximum pixel value and the second minimum pixel value to obtain a third pixel mean; and for performing difference processing on the pixel values ​​of all pixels on the third endoscopic image on the R channel, the G channel and the B channel and the third pixel mean value to obtain a fifth pixel difference; and for performing product processing on the fifth pixel difference and the saturation coefficient to obtain a sixth pixel difference; And used to perform sum processing on the third pixel mean, the fifth pixel difference and the sixth pixel difference to obtain the target endoscopic image. In which, the saturation coefficient acquisition module is also configured to obtain the difference between the pixel threshold and the second maximum pixel value and the second minimum pixel value respectively based on the first pixel sum value being greater than or equal to the pixel threshold, and sum the difference between the pixel threshold and the second maximum pixel value and the difference between the pixel threshold and the second minimum pixel value to obtain the second pixel sum value, and use the ratio of the fourth pixel difference value to the second pixel sum value as the saturation coefficient.

11. An electronic device, characterized in that: comprising a processor; and a memory for storing processor executable instructions, wherein the processor is configured to implement the method of any one of claims 1 to 9 when executing the instructions stored in the memory.

12. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images