Blood vessel enhancement method, device and equipment for white light image of endoscope and storage medium

Abstract

The invention relates to a blood vessel enhancement method, device and equipment for a white light image of an endoscope and a storage medium. The method comprises the following steps: acquiring a white light image to be processed; based on the to-be-processed white light image, adjusting pixel values of all pixel points on the to-be-processed white light image on the R channel, the G channel and the B channel to obtain a first white light image; based on the first white light image, adjusting pixel values of all pixel points on the first white light image on an R channel, a G channel and a B channel to obtain a second white light image; performing image fusion on the first white light image and the second white light image to obtain a third white light image; and based on the third white light image, adjusting pixel values of all pixel points on the third white light image on the R channel, the G channel and the B channel to obtain a target white light image. According to the method, the pixel values of the G channel and the B channel are increased, and the influence of the pixel value on the B channel is eliminated, so that the blood vessel features on the white light image can be enhanced, and the definition of blood vessel details is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopic image processing, and in particular to a method, device, equipment and storage medium for blood vessel enhancement of an endoscopic white light image. Background Art

[0002] In recent years, the rapid development of endoscopic image processing technology has brought significant changes to the medical field. The core of this is the rapid update of image processing algorithms. The core of these algorithms is to deeply analyze and process the images obtained by the endoscope, thereby improving the accuracy and efficiency of diagnosis. The advancement of image processing algorithms has greatly improved the clarity and contrast, that is, the brightness, of endoscopic images. In low-light or dim light environments or with limited field of view, these image processing algorithms can automatically optimize image quality, thereby helping doctors better observe and analyze lesions. In short, with the continuous advancement of endoscopic image processing algorithm technology, it can not only improve the quality of medical services, but also promote the process of intelligent medical care, laying the foundation for future precision medicine.

[0003] At present, endoscopic image processing technology has been widely used in diagnosing tumors or cancerous lesions in the natural cavity of the human body. With the increasing incidence and mortality of digestive tract diseases (gastric cancer, rectal cancer, esophageal cancer and colon cancer) in China, the requirements for endoscopic image processing technology have also increased. Although the current endoscopic image processing technology has greatly improved the accuracy of diagnosing digestive tract diseases, the accuracy of discovering and confirming the early symptoms of such digestive tract diseases is not very high. In the process of forming such tumors, blood vessels will proliferate, and the early cancerous sites will have changes in color and texture structure. However, under conventional white light or in narrow cavities, it is difficult to directly observe and discover the color and texture characteristics of early cancerous sites through an endoscope, which increases the difficulty of discovering vascular lesions in early cancerous sites.

[0004] In addition, blood vessels are mainly distributed in the mucosal layer and submucosal layer. Since white light is composed of three primary colors of red, green and blue, with wavelengths of 605nm, 540nm and 415nm respectively, and the hemoglobin in the blood vessels has different absorption abilities for primary colors of different wavelengths, in the white light mode, hemoglobin has a characteristic absorption peak in the green light band, but not in the red and blue light bands, and hemoglobin absorbs less red light and more blue light, making the display clarity of the vascular features in the image in the white light mode on the G channel significantly higher than that on the R and B channels, and because the brightness on the R channel is high and the brightness on the B channel is low, the vascular features on the image acquired in the white light mode are not well presented. Summary of the invention

[0005] (1) Technical issues to be solved

[0006] The present invention provides a blood vessel enhancement method, device, equipment and storage medium for an endoscopic white light image, aiming to perform gain processing on the pixel values ​​of all pixel points on the G channel and the B channel of the white light image by using a gamma index algorithm, remove the pixel values ​​of all pixel points on the R channel of the white light image after the gain processing, fuse the pixel values ​​of all pixel points on the white light image before the gain processing on the B channel to the R channel of the white light image after the gain processing, and then perform gain processing on the pixel values ​​on the fused image by using the gamma index algorithm, so as to enhance the blood vessel features on the white light image, improve the clarity of the blood vessel details, and thus improve the accuracy of discovering early cancerous sites.

[0007] (2) Technical solution

[0008] In a first aspect, an embodiment of the present invention provides a method for enhancing blood vessels in an endoscopic white light image, comprising:

[0009] Acquire a white light image to be processed;

[0010] Based on the white-light image to be processed, adjusting the pixel values ​​of all pixels on the white-light image to be processed on the R channel, the G channel, and the B channel, respectively, to obtain a first white-light image;

[0011] Based on the first white light image, adjusting the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel, respectively, to obtain a second white light image;

[0012] performing image fusion on the first white-light image and the second white-light image to obtain a third white-light image;

[0013] Based on the third white light image, adjusting the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel, respectively, to obtain a target white light image;

[0014] The first white light image and the second white light image are fused to obtain a third white light image, which includes:

[0015] Removing pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the R channel and the B channel to obtain a first channel image;

[0016] Removing pixel values ​​of all pixels on the second white light image on the R channel to obtain a second channel image;

[0017] Merging the pixel values ​​on the B channel or the G channel of the first channel image into the R channel of the second channel image to obtain the third white light image;

[0018] or

[0019] Acquire a blank image, wherein the blank image has the same size as the first white light image and the second white light image;

[0020] Merging the pixel values ​​of the B channel or the G channel on the first white light image to the corresponding R channel on the blank image;

[0021] Merging the pixel values ​​of the G channel on the second white light image to the corresponding G channel on the blank image;

[0022] And the pixel values ​​of the B channel on the second white light image are merged into the corresponding B channel on the blank image to obtain the third white light image.

[0023] In a possible implementation, based on the white light image to be processed, adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel to obtain a first white light image, including

[0024] Acquire a white balance gain coefficient, wherein the white balance gain coefficient includes a first gain coefficient, a second gain coefficient, and a third gain coefficient;

[0025] Based on the first gain coefficient, the second gain coefficient and the third gain coefficient, the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel and the B channel are adjusted respectively to obtain the first white light image.

[0026] In a possible implementation, based on the white light image to be processed, adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel to obtain a first white light image, including

[0027] Decomposing the white light image to be processed by RGB channels to obtain corresponding R channel image, G channel image and B channel image;

[0028] Performing white balance gain processing on the R channel image, the G channel image, and the B channel image to obtain an R channel gain image, a G channel gain image, and a B channel gain image;

[0029] Perform RGB channel inverse decomposition on the R channel gain image, the G channel gain image, and the B channel gain image to obtain the first white light image.

[0030] In a possible implementation manner, obtaining the white balance gain coefficient includes:

[0031] Acquire a white light test image;

[0032] Based on the white light test image, obtaining a first average pixel value in the R channel, a second average pixel value in the G channel, and a third average pixel value in the B channel of the white light test image;

[0033] Obtaining the first gain coefficient based on a ratio of the second average pixel value to the first average pixel value;

[0034] obtaining the second gain coefficient based on a ratio of the second average pixel value to the second average pixel value;

[0035] The third gain coefficient is obtained based on a ratio of the second average pixel value to the third average pixel value.

[0036] In a possible implementation, the pixel values ​​of all pixels on the to-be-processed white light image on the R channel, the G channel, and the B channel are adjusted based on the first gain coefficient, the second gain coefficient, and the third gain coefficient, respectively, including:

[0037] Multiplying the first gain coefficient by the pixel values ​​of all pixels on the R channel of the white light image to be processed to obtain a first gain pixel value;

[0038] Multiplying the second gain coefficient by the pixel values ​​of all pixels on the G channel of the white light image to be processed to obtain a second gain pixel value;

[0039] Multiplying the third gain coefficient by the pixel values ​​of all pixels on the B channel of the white light image to be processed to obtain a third gain pixel value;

[0040] Comparing the first gain pixel value with the preset pixel threshold, and assigning the first gain pixel value as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed based on the first gain pixel value being less than the preset pixel threshold; otherwise, assigning the preset pixel threshold as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed;

[0041] Comparing the second gain pixel value with the preset pixel threshold, and assigning the second gain pixel value as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed based on the second gain pixel value being less than the preset pixel threshold; otherwise, assigning the preset pixel threshold as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed;

[0042] The third gain pixel value is compared with the preset pixel threshold, and based on the third gain pixel value being less than the preset pixel threshold, the third gain pixel value is assigned as the pixel value of the corresponding pixel point on the B channel of the white light image to be processed; otherwise, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the B channel of the white light image to be processed.

[0043] In a possible implementation, based on the first white light image, adjusting the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain the second white light image includes:

[0044] Acquire a first gamma index coefficient, where the first gamma index coefficient includes a first R channel gamma index coefficient, a first G channel gamma index coefficient, and a first B channel gamma index coefficient;

[0045] Normalizing the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain a first normalized image, wherein the first normalized image includes a first R channel normalized image, a second G channel normalized image, and a third B channel normalized image;

[0046] Based on the first gamma index coefficient, processing the first normalized image using a gamma index algorithm to obtain a first normalized gain image;

[0047] The first normalized gain image is denormalized to obtain the second white light image.

[0048] In a possible implementation, acquiring the white light image to be processed includes:

[0049] Get the original image;

[0050] Based on the original image, a white light image to be processed having RGB features is obtained.

[0051] In a possible implementation, based on the third white light image, adjusting the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a target white light image includes:

[0052] Acquire a second gamma index coefficient, wherein the second gamma index coefficient includes a second R channel gamma index coefficient, a second G channel gamma index coefficient, and a second B channel gamma index coefficient;

[0053] Normalizing the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a second normalized image, wherein the second normalized image includes a second R channel normalized image, a second G channel normalized image, and a second B channel normalized image;

[0054] Based on the second gamma index coefficient, processing the second normalized image by using a gamma index algorithm to obtain a second normalized gain image;

[0055] The second normalized gain image is subjected to denormalization processing to obtain the target white light image.

[0056] In a second aspect, an embodiment of the present invention provides a blood vessel enhancement device, comprising:

[0057] An image acquisition module, used for acquiring a white light image to be processed;

[0058] A first image processing module, used for adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel, respectively, to obtain a first white light image; and used for adjusting the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel, respectively, to obtain a second white light image;

[0059] An image fusion module, used to remove the pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the pixel values ​​of the R channel and the B channel to obtain a first channel image, and to remove the pixel values ​​of all pixels on the second white light image on the R channel to obtain a second channel image; and to fuse the pixel values ​​on the B channel or the G channel of the first channel image to the R channel of the second channel image to obtain the third white light image; or

[0060] Used to fuse the pixel values ​​of the B channel or the G channel on the first white light image to the corresponding R channel on the blank image; and used to fuse the pixel values ​​of the G channel on the second white light image to the corresponding G channel on the blank image; and used to fuse the pixel values ​​of the B channel on the second white light image to the corresponding B channel on the blank image, to obtain the third white light image; wherein the blank image has the same size as the first white light image and the second white light image;

[0061] The second image processing module is used to adjust the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel respectively to obtain a target white light image.

[0062] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the method disclosed in the first aspect when executing the instructions stored in the memory.

[0063] In a fourth aspect, an embodiment of the present invention provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the method disclosed in the first aspect when executed by a processor.

[0064] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code, wherein when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the method disclosed in the first aspect.

[0065] (3) Beneficial effects

[0066] In summary, the pixel values ​​of all pixels on the white light image on the G channel and the B channel are gain processed by the gamma index algorithm, and the pixel values ​​of all pixels on the R channel of the white light image after the gain processing are removed, and the pixel values ​​of all pixels on the B channel of the white light image before the gain processing are fused to the R channel of the white light image after the gain processing. The gamma index algorithm is then used to gain the pixel values ​​on the fused image to enhance the vascular features on the white light image and improve the clarity of the vascular details, thereby improving the accuracy of detecting early cancerous sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 It is a flow chart of an image processing method of the present invention.

[0069] Figure 2 is another flow chart of an image processing method of the present invention.

[0070] Figure 3 is another flow chart of an image processing method of the present invention.

[0071] Figure 4 is another flow chart of an image processing method of the present invention.

[0072] Figure 5 is another flow chart of an image processing method of the present invention.

[0073] Figure 6 is another flow chart of an image processing method of the present invention.

[0074] Figure 7 is another flow chart of an image processing method of the present invention.

[0075] Figure 8 is another flow chart of an image processing method of the present invention.

[0076] Fig. 9 is another flow chart of an image processing method of the present invention.

[0077] Fig.10 It is a structural schematic diagram of an invented image processing device.

[0078] Fig.11 It is a display effect diagram of the white light image to be processed of the present invention.

[0079] Fig.12 It is a display effect diagram of the target white light image of the present invention.

[0080] In the figure:

[0081] 10 - image acquisition module; 20 - first image processing module; 30 - image fusion module; 40 - second image processing module. DETAILED DESCRIPTION

[0082] The following is a further detailed description of the embodiments of the present invention in conjunction with the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the described embodiments, and covers any modification, replacement and improvement of parts, components and connection modes without departing from the spirit of the present invention.

[0083] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0084] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0085] The image processing method of the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster of multiple server groups. Any electronic device can implement the image processing method of the embodiment of the present disclosure by a processor calling a computer-readable instruction stored in a memory.

[0086] Figure 1 A flow chart of a method for blood vessel enhancement of an endoscopic white light image according to the present invention is shown. Figure 1 As shown, the image processing method of the disclosed embodiment of the present invention may include the following steps S10-S50.

[0087] Step S10: Acquire a white light image to be processed.

[0088] In a possible implementation, the white light image to be processed may be an original image directly obtained by controlling a white light camera device of a medical endoscope to capture an image of a target object through an electronic device, or may be an original image processed by an image processing device. For example, the white light image to be processed may be an original image after image processing after a real-time image data stream captured by a camera of a medical endoscope, wherein the target object may be the inner wall of a digestive tract (such as a stomach, rectum, esophagus, and colon, etc.) photographed in a white light mode. Therefore, the white light image to be processed is an image of the inner wall of the digestive tract with RGB features (such as Fig.11 As shown), of course, due to the different parameters of the medical endoscope camera, the data format of the original image directly obtained will also be different. For example, some existing medical endoscope camera devices are based on COMS image sensors to collect images of target objects, resulting in the image data of the original image obtained by the COMS image sensor being a BAYER array image, i.e., a RAW image, which cannot be directly visualized on the terminal display. Therefore, the format of the RAW image needs to be processed accordingly to obtain a white light image to be processed with RGB features, so as to meet the visualization requirements.

[0089] like Figure 2 As shown, the method for acquiring a white light image to be processed according to an embodiment of the present invention includes the following steps S110 - S120 .

[0090] S110: Acquire an original image.

[0091] In a possible implementation, the CMOS image sensor in the camera of the medical endoscope is used to capture the image of the target object to obtain an original image. The image data of the original image is a BAYER array image, that is, a RAW image, as shown in the following table:

[0092]

[0093]

[0094] The letter in each box in the table represents a pixel on the RAW image, where B represents the pixel value of the blue channel, Gr represents the pixel value of the green channel in odd rows, Gb represents the pixel value of the green channel in even rows, and R represents the pixel value of the red channel. Therefore, the pixels on the RAW image only have the pixel value of one of the R, G, and B channels, that is, there is no pixel that has the pixel value of the RGB channels at the same time, that is, the RGB feature.

[0095] S120: Based on the original image, obtain a white light image to be processed having RGB features.

[0096] In a possible implementation, since the original image cannot be directly displayed due to its own format and cannot meet the needs of subsequent image processing, it is necessary to use a color interpolation algorithm to process the pixel values ​​of all pixels on the R, G, and B channels of the original image to obtain a white light image to be processed with RGB characteristics, that is, all pixels on the white light image to be processed have pixel values ​​of RGB channels at the same time. Specifically, the color interpolation algorithm includes a bilinear interpolation method or a neighboring pixel mean method. Exemplarily, this embodiment uses the neighboring pixel mean method to process the pixel values ​​of all pixels on the original image, and arranges the pixel values ​​according to the distribution of the pixel values ​​in the above table, and uses the corresponding calculation formula to process the pixels with R characteristics respectively. Pixels with Gr features, pixels with Gb features, and pixels with B features are processed so that the pixels have RGB features. Specifically, except for the pixels in the 1st row and the 1st column and the K+1th row and the K+1th column, the pixels in other rows or columns have 8 adjacent pixels, where the adjacent pixel average method is similar for the pixels with R features and the pixels with B features. For example, the pixel with R features in the 3rd row and the 3rd column is represented as R(3,3), and the pixel value of the R channel of the pixel at this position is the pixel value of R(3,3), and the pixel value of the G channel of the pixel is the average of the values ​​of the 4 adjacent pixels with G channel features, that is, G(3,3)=1 / 4. *[Gr(3,2)+Gr(3,4)+Gb(2,3)+Gb(4,3)]. Similarly, the pixel value of the B channel of this pixel is the average of the values ​​of the four adjacent pixels with B channel characteristics, that is, B(3,3)=1 / 4*[B(2,2)+B(2,4)+B(4,2)+B(4,4)]. Similarly, taking the pixel with B characteristics in the 4th row and 4th column as an example, it is represented as B(4,4). The pixel value of the B channel of the pixel at this position is the pixel value of B(4,4), and the pixel value of the R channel of this pixel is the average of the values ​​of the four adjacent pixels with R channel characteristics, R(4,4)=1 / 4*[R(3,3)+R(3,5)+R(5,3)+R( 5,5)], the pixel value of the G channel of this pixel is the average of the values ​​of its four neighboring pixels with G channel characteristics, that is, G(4,4)=1 / 4*[Gb(4,3)+Gb(4,5)+Gr(3,4)+Gr(5,4)], and the neighboring pixel average method is similar for the pixel points with Gb characteristics and the pixel points with Gr characteristics. For example, the pixel point with Gr characteristics in the 3rd row and 4th column is represented as Gr(3,4), and the pixel value of the G channel of the pixel at this position is the pixel value of Gr(3,4), and the pixel value of the R channel of this pixel is the average of the values ​​of its two neighboring pixels with R channel characteristics, that is, R(3,4)=1 / 2*[R(3,3)+R(3,5)], the pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(3,4)=1 / 2*[B(2,4)+B(4,4)]. Taking the pixel with Gb characteristics at the 4th row and the 3rd column as an example, it is characterized as Gb(4,3). The pixel value of the G channel of the pixel at this position is the pixel value of Gb(4,3), and the pixel value of the R channel of the pixel is the average of the values ​​of the two adjacent pixels with R channel characteristics, that is, R(4,3)=1 / 2*[R(3,3)+R(5,3)]. The pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(4,3)=1 / 2*[B(4,2)+B(4,4)]. Therefore, the pixel values ​​of all pixels on the white light image to be processed have corresponding RGB characteristics. ,

[0097] S20. Based on the white-light image to be processed, adjust the pixel values ​​of all pixels on the white-light image to be processed on the R channel, the G channel, and the B channel to obtain a first white-light image.

[0098] In a possible implementation, since the G channel data on the to-be-processed white light image with RGB features obtained after processing by the color interpolation algorithm is large, there is a problem of white misalignment, that is, the overall display effect of the to-be-processed image is greenish, causing a color imbalance problem. In order to meet the subsequent display effect of the target image, that is, to ensure that the color display on the target image is balanced, it is necessary to use a white balance algorithm to adjust the pixel values ​​on the R channel, the pixel values ​​on the G channel, and the pixel values ​​on the B channel of all pixels on the to-be-processed white light image, that is, to perform gain processing on the pixel values ​​on the RGB channels to obtain the first white light image. Similarly, the first white light image also has RGB features.

[0099] like Figure 3 As shown, the method of obtaining a first white light image by adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, G channel, and B channel respectively based on the white light image to be processed in an embodiment disclosed in the present invention includes the following steps S210-S220.

[0100] S210 , obtaining white balance gain coefficients, where the white balance gain coefficients include a first gain coefficient, a second gain coefficient, and a third gain coefficient.

[0101] In one possible implementation, in order to ensure the balance of pixel values ​​of RGB channels on the white light image to be processed and to restore a color effect close to the real thing, the pixel values ​​on the R channel, G channel, and B channel need to match corresponding gain coefficients, so as to achieve different degrees of gain for the pixel values ​​on each color channel, thereby ensuring that the pixel values ​​on the R channel, G channel, and B channel after gain can maintain the balance of image color.

[0102] like Figure 4 As shown, the method for obtaining the white balance gain coefficient in the embodiment disclosed by the present invention includes steps S2101-S2103.

[0103] S2101. Acquire a white light test image.

[0104] In one possible implementation, in order to ensure the accuracy of the obtained white balance gain coefficient, a COMS image sensor is used to shoot a standard white paper or other white background (such as white wallpaper or floor, etc.) in white light mode to obtain a white light test image required for subsequent white balance algorithm processing.

[0105] S2102 . Based on the white light test image, obtain a first average pixel value in an R channel, a second average pixel value in a G channel, and a third average pixel value in a B channel of the white light test image.

[0106] In a possible implementation, the method for obtaining the white balance gain coefficient mainly includes obtaining it by using a peak method or a mean method. In this embodiment, the white balance gain coefficient is obtained by using the mean method for the white light test image to reduce the complexity of the calculation. Specifically, the process of obtaining the first average pixel value, the second average pixel value and the third average pixel value includes: counting the sum of the pixel values ​​of all pixels on the white light test image on the R channel, the G channel and the B channel respectively; and obtaining the corresponding first average pixel value, second average pixel value and third average pixel value based on the ratio of the sum of the pixel values ​​to the number of all pixels. Exemplarily, counting the sum of the pixel values ​​of all pixels on the white light test image The number of pixels count and the sum Rsum of the pixel values ​​of all pixels in the white light test image in the R channel, the sum Gsum of the pixel values ​​in the G channel, and the sum Bsum of the pixel values ​​in the B channel are calculated using the average value calculation formulas Ravg=Rsum / count, Gavg=Gsum / count, and Bavg=Bsum / count to obtain the corresponding first average pixel value, second average pixel value, and third average pixel value, where Ravg, Gavg, and Bavg are respectively used to represent the first average pixel value of the R channel, the second average pixel value of the G channel, and the third average pixel value of the B channel on the white light test image.

[0107] S2103. Obtain the first gain coefficient based on the ratio of the second average pixel value to the first average pixel value; obtain the second gain coefficient based on the ratio of the second average pixel value to the second average pixel value; obtain the third gain coefficient based on the ratio of the second average pixel value to the third average pixel value.

[0108] In a possible implementation, since the white light test image is acquired by a COMS image sensor, the pixel values ​​of the G channel on the white light test image are too many. After the pixel values ​​of the G channel are subjected to difference processing in step S120, the pixel values ​​of the G channel on the white light test image are larger than the pixel values ​​of the B channel and the R channel. In order to ensure that the pixel values ​​on the RGB channels of the white light test image are amplified and the true colors are restored, the value range of the gain coefficient is determined to be greater than or equal to 1, so that the second average pixel value on the G channel is selected as the dividend, and the first average pixel value on the R channel, the second average pixel value on the G channel, and the third average pixel value on the B channel are selected as the dividend. The values ​​are used as divisors respectively, and the corresponding quotients are calculated as the first gain coefficient of the corresponding R channel, the second gain coefficient of the G channel and the third gain coefficient of the B channel. Exemplarily, the corresponding ratio formulas Rgain=Gavg / Ravg, Ggain=Gavg / Gavg, Bgain=Gavg / Bavg are used to calculate the first gain coefficient of the corresponding R channel, the second gain coefficient of the G channel and the third gain coefficient of the B channel respectively, where Rgain, Ggain, Bgain represent the first gain coefficient of the R channel, the second gain coefficient of the G channel and the third gain coefficient of the B channel respectively, and the value range of Rgain, Ggain, Bgain is 1-10.

[0109] S220 , based on the first gain coefficient, the second gain coefficient, and the third gain coefficient, respectively adjust the pixel values ​​of all pixels on the to-be-processed white-light image on the R channel, the G channel, and the B channel to obtain the first white-light image.

[0110] In a possible implementation, based on the pixel values ​​on different channels, corresponding different gain coefficients are used for adjustment, so that a first white light image with balanced image color and close to the real color can be obtained. Specifically, the first gain coefficient is multiplied by the pixel values ​​of all pixels on the R channel of the white light image to be processed to obtain a first gain pixel value; the second gain coefficient is multiplied by the pixel values ​​of all pixels on the G channel of the white light image to be processed to obtain a second gain pixel value; the third gain coefficient is multiplied by the pixel values ​​of all pixels on the B channel of the white light image to be processed to obtain a third gain pixel value. Exemplarily, the corresponding pixel gain formulas R1=R*Rgain, G1=G*Ggain, and B1=B*Bgain are used to perform gain processing on the pixel values ​​on the R channel, G channel, and B channel, respectively, where R, G, and B are the pixel values ​​of the R channel, the pixel values ​​of the G channel, and the pixel values ​​of the B channel corresponding to all pixels on the white light image to be processed, respectively, and R1, G1, and B1 are the pixel values ​​of the R channel, the pixel values ​​of the G channel, and the pixel values ​​of the B channel corresponding to all pixels on the first white light image, respectively.

[0111] In a possible implementation, since the pixel value of the white light image is usually an integer ranging from 0 to 255, and since the first gain coefficient, the second gain coefficient, and the third gain coefficient are all greater than or equal to 1, the pixel values ​​of the R channel, the G channel, and the B channel corresponding to all pixels on the first white light image after gain adjustment may exceed the preset pixel threshold, that is, the maximum pixel value of 255. In order to avoid the situation where the pixel values ​​on the first white light image cannot be displayed due to exceeding the maximum pixel threshold, that is, data is lost, the pixel values ​​of the R channel, the G channel, and the B channel corresponding to all pixels on the first white light image after gain are compared with the preset pixel threshold, and the pixel values ​​of the pixel on the R channel, the G channel, and the B channel are assigned values ​​according to the comparison results to ensure that the pixel values ​​of the R channel, the G channel, and the B channel corresponding to all pixels on the first white light image after gain adjustment do not exceed the maximum pixel value. Specifically, the first gain pixel value is compared with the preset pixel threshold. The pixel threshold is determined based on the first gain pixel value being less than the preset pixel threshold, and based on the first gain pixel value being less than the preset pixel threshold, the first gain pixel value is assigned as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed; otherwise, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed, and at the same time, the second gain pixel value is compared with the preset pixel threshold, and based on the second gain pixel value being less than the preset pixel threshold, the second gain pixel value is assigned as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed; otherwise, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed, and at the same time, the third gain pixel value is compared with the preset pixel threshold, and based on the third gain pixel value being less than the preset pixel threshold, the third gain pixel value is assigned as the pixel value of the corresponding pixel point on the B channel of the white light image to be processed; otherwise, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the B channel of the white light image to be processed.

[0112] like Figure 5 As shown, the method disclosed in the embodiment of the present invention adjusts the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel respectively based on the white light image to be processed to obtain a first white light image, including the following steps S2100-S2300

[0113] S2100 , performing RGB channel decomposition on the white light image to be processed to obtain corresponding R channel image, G channel image and B channel image.

[0114] In a possible implementation, since the pixel value of the R channel on the white light image to be processed is relatively large while the pixel values ​​on the G channel and the B channel are generally small, it is difficult to clearly display the vascular features or vascular details on the white light image to be processed. Therefore, the white light image to be processed is decomposed into channel images corresponding to the R channel, the G channel, and the B channel components to perform gain adjustment suitable for the corresponding channel images.

[0115] S2200 , performing white balance gain processing on the R channel image, the G channel image, and the B channel image to obtain an R channel gain image, a G channel gain image, and a B channel gain image.

[0116] In one possible implementation, the pixel values ​​of all pixels on different channels are gain processed using white balance gain coefficients corresponding to different channels in the same manner as step S220, and the method for obtaining the white balance gain coefficients is the same as steps S2101-S2103, which will not be repeated here.

[0117] S2300 , performing RGB channel inverse decomposition on the R channel gain image, the G channel gain image, and the B channel gain image to obtain the first white light image.

[0118] S30. Based on the first white light image, adjust the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain a second white light image.

[0119] In one possible implementation, after obtaining a first white light image with balanced image color and close to the true color, the image color displayed by the first white light image obtained in the white light mode is reddish, and because the digestive tract is narrow, the entire image is insufficient in light or depth of field, resulting in overall or partial darkening, which makes it difficult to clearly highlight the vascular features. Therefore, it is necessary to perform a secondary adjustment on the pixel values ​​of all pixels on the first white light image on the R channel, G channel, and B channel, respectively, to solve the problem of unclear image caused by the overall or partial darkening of the first white light image.

[0120] like Figure 6 As shown, the embodiment disclosed in the present invention adjusts the pixel values ​​of all pixels on the first white light image on the R channel, G channel, and B channel respectively based on the first white light image to obtain a second white light image, including the following steps S310-S340.

[0121] S310 , obtaining a first gamma index coefficient, where the first gamma index coefficient includes a first R channel gamma index coefficient, a first G channel gamma index coefficient, and a first B channel gamma index coefficient.

[0122] In a possible implementation, since the second white light image is dark in its entirety or in a local area, in the process of adjusting the pixel values ​​of all pixels on the first white light image on the R channel, G channel, and B channel, respectively, in order to be able to achieve a larger adjustment to the pixel values ​​of areas with smaller pixel values, i.e., darker areas, and a smaller adjustment to the pixel values ​​of areas with larger pixel values, i.e., brighter areas, a gamma index algorithm is used to adjust the pixel values ​​of all pixels on the first white light image on the R channel, G channel, and B channel, respectively. Based on the large differences in the pixel values ​​on the R channel, G channel, and B channel, the gains of the first gamma index coefficient on the pixel values ​​on the R channel, G channel, and B channel are also different, and therefore, there are also differences in the values ​​of the first R channel gamma index coefficient, the first G channel gamma index coefficient, and the first B channel gamma index coefficient in the first gamma index coefficient.

[0123] In a possible implementation, the first gamma index coefficient is an empirical value. Exemplarily, the gamma index coefficient can be assigned by selecting a plurality of parameter values ​​within a coefficient parameter range of 0.5-2 for a limited number of times, and then performing gain processing on the pixel values ​​on the R channel, G channel, and B channel of the image to be processed through a gamma index algorithm, and recording relevant parameter values ​​that meet the requirements of the target image until the optimal values ​​of the first R channel gamma index coefficient Rgamma, the first G channel gamma index coefficient Ggamma, and the first B channel gamma index coefficient Bgamma corresponding to the R channel, G channel, and B channel are obtained. In this embodiment, Rgamma takes a value of 0.8, and Ggamma and Bgamma both take values ​​of 0.9.

[0124] S320, normalizing the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel respectively to obtain a first normalized image, wherein the first normalized image includes a first R channel normalized image, a first G channel normalized image, and a first B channel normalized image.

[0125] In a possible implementation, in order to reduce the complexity of calculation and improve the speed of pixel value processing, the pixel values ​​of all pixels on the first white light image on the R channel, G channel, and B channel are normalized to obtain a corresponding first normalized image. The pixel values ​​of all pixels on the first normalized image are between 0 and 1, that is, the pixel values ​​of all pixels on the first R channel normalized image, the first G channel normalized image, and the first B channel normalized image are also between 0 and 1, which is convenient for subsequent exponential operations using a gamma exponent algorithm. Exemplarily, the normalization formula K=N / 255 is used for calculation, where K is the normalization coefficient, including normalization on the R channel, G channel, and B channel, and N is the pixel value of all pixels on the first white light image on the R channel, G channel, and B channel, respectively.

[0126] S330: Based on the first gamma index coefficient, process the first normalized image using a gamma index algorithm to obtain a first normalized gain image.

[0127] In a possible implementation, the gamma index algorithm is used in conjunction with the normalized coefficient after normalization processing to further reduce the complexity of the calculation. For example, the gamma index mathematical formula M=(K) is used. gamma Calculation is performed, where M is the normalized coefficient after gamma index processing, including the normalized coefficient after gamma index processing on the R channel, G channel, and B channel, and gamma is the first gamma index coefficient, including the first R channel gamma index coefficient Rgamma, the first G channel gamma index coefficient Ggamma, and the first B channel gamma index coefficient Bgamma.

[0128] S340: Perform a denormalization process on the first normalized gain image to obtain the second white light image.

[0129] In a possible implementation, the pixel values ​​on the normalized first white light image need to be denormalized to obtain the first white light image after pixel value gain, that is, the second white light image with clear overall image display. Exemplarily, the denormalization formula F=M*255 is used for calculation, where F is the pixel value of all pixels on the second white light image on the R channel, G channel, and B channel respectively.

[0130] S40: performing image fusion on the first white-light image and the second white-light image to obtain a third white-light image.

[0131] In a possible implementation, after the pixel values ​​of the entire or local area of ​​the second white light image are gain processed, the clarity of the entire image will be improved, but only the surface features of the blood vessels can be displayed, but the details of the blood vessels under the mucosa cannot be observed. Therefore, the image fusion method is used to superimpose or assign values ​​to the pixel points at corresponding positions on the first white light image and the second white light image for fusion, which can improve the clarity of the blood vessel features and enhance the detailed texture of the blood vessels.

[0132] like Figure 7 As shown, the method for fusing the first white-light image and the second white-light image to obtain a third white-light image according to the embodiment disclosed in the present invention includes the following steps S410-S430.

[0133] S410 , removing pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the R channel and the B channel to obtain a first channel image.

[0134] In one possible implementation, in order to eliminate the effect of the reddish color on the first white light image that makes it difficult to clearly highlight the vascular features, and based on the absorption characteristics of hemoglobin in the blood vessels to blue light wavelengths and green light wavelengths, the purpose of discovering the presence of vascular lesions in early cancerous sites is to observe the mid-level vascular features or deep-level vascular features. Therefore, the pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the pixel values ​​on the R channel and the B channel are removed, thereby retaining the corresponding pixel values ​​on the B channel or the G channel, and obtaining a first channel image with monochrome channel pixel values. It should be noted that the methods for removing the pixel values ​​of the monochrome channel are all existing technologies and will not be repeated here.

[0135] S420: Remove the pixel values ​​of all pixels on the second white light image on the R channel to obtain a second channel image.

[0136] In one possible implementation, in order to eliminate the influence of the pixel value of the R channel on the second white light image on the clarity of the vascular features, the pixel values ​​of all pixels on the second white light image on the R channel are removed, and then the corresponding pixel values ​​on the B channel and the G channel are retained to obtain a second channel image with only two color channels.

[0137] S430 , fusing the pixel values ​​on the B channel or the G channel of the first channel image to the R channel of the second channel image to obtain the third white light image.

[0138] In a possible implementation, in order to observe mid-level vascular features or deep-level vascular features to discover the presence of vascular lesions in early cancerous sites, it is necessary to fuse the pixel points on the monochrome channel of the first channel image with the blank pixel points of the R channel pixel values ​​missing on the second channel image to obtain a third white light image with RGB features. Specifically, by replacing or assigning the pixel values ​​on the B channel and the G channel of the first channel image to the blank pixel points on the second channel image, i.e., the R channel, the pixel value on the R channel of the third white light image is the pixel value on the B channel or the G channel of the first channel image, thereby reducing the influence of the red part of the image on the clarity of the vascular features, and using the pixel values ​​on the B channel or the G channel of the first channel image to obtain mid-level vascular features or deep-level vascular features according to actual purposes.

[0139] like Figure 8 As shown, the method for fusing the first white-light image and the second white-light image to obtain a third white-light image according to the embodiment disclosed in the present invention includes the following steps S440-S450.

[0140] S440: Acquire a blank image, wherein the blank image has the same size as the first white light image and the second white light image.

[0141] In one possible implementation, .

[0142] S450, fusing the pixel values ​​of the B channel on the first white light image to the corresponding R channel on the blank image; fusing the pixel values ​​of the G channel on the second white light image to the corresponding G channel on the blank image; and fusing the pixel values ​​of the B channel on the second white light image to the corresponding B channel on the blank image, to obtain the third white light image.

[0143] In one possible implementation, .

[0144] S50 . Based on the third white light image, adjust the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a target white light image.

[0145] In a possible implementation, based on the fact that the middle-level vascular features or deep-level vascular features can be highlighted on the third white light image, since the white light image to be processed is acquired in the white light mode, even if the pixel values ​​on the B channel or the G channel on the first white light image replace the pixel values ​​on the R channel on the second white light image, a large amount of red components still remain in the overall area of ​​the third white light image. Therefore, the pixel values ​​on the R channel, the G channel, and the B channel on the third white light image need to be adjusted again. By increasing the pixel values ​​on the G channel and the B channel and reducing the pixel value on the R channel, the target white light image is obtained to enhance the vascular features (such as Fig.12 shown).

[0146] like Fig. 9 As shown, the method of obtaining a target white light image by adjusting the pixel values ​​of all pixels on the third white light image on the R channel, G channel, and B channel respectively based on the third white light image in the embodiment disclosed in the present invention includes the following steps S510-S540.

[0147] S510 , obtaining a second gamma index coefficient, where the second gamma index coefficient includes a second R channel gamma index coefficient, a second G channel gamma index coefficient, and a second B channel gamma index coefficient.

[0148] In a possible implementation, since the red component on the overall area of ​​the third white light image is relatively large, that is, the pixel value on the R channel is relatively high, which makes the pixel values ​​on the G channel and the B channel relatively low, it is necessary to increase the pixel values ​​on the G channel and the B channel and reduce the pixel value on the R channel. In order to achieve a larger adjustment to the area with a smaller pixel value on the third white light image and a smaller adjustment to the area with a larger pixel value, the gamma index algorithm is still used to adjust the pixel values ​​of all pixels on the third white light image on the R channel, G channel, and B channel, respectively. Based on the large difference in the pixel values ​​on the R channel, G channel, and B channel, the gain of the second gamma index coefficient on the pixel values ​​on the R channel, G channel, and B channel is also different, so the values ​​of the second R channel gamma index coefficient, the second G channel gamma index coefficient, and the second B channel gamma index coefficient in the second gamma index coefficient are also different. In a possible implementation, the second gamma index coefficient is an empirical value. For example, the gamma index coefficient can be assigned by selecting a plurality of parameter values ​​within the coefficient parameter range of 0.5-2 for a limited number of times, and then performing gain processing on the pixel values ​​on the R channel, G channel, and B channel of the image to be processed by the gamma index algorithm, and recording the relevant parameter values ​​that meet the requirements of the target image until the optimal values ​​of the second R channel gamma index coefficient Rgamma', the second G channel gamma index coefficient Ggamma', and the second B channel gamma index coefficient Bgamma' corresponding to the R channel, G channel, and B channel are obtained. In this embodiment, Rgamma' takes a value of 1.1, and Ggamma' and Bgamma' both take a value of 0.8. By setting the second R channel gamma index coefficient Rgamma' to be greater than the second G channel gamma index coefficient Ggamma' and the second B channel gamma index coefficient Bgamma' and greater than 1, it is possible to increase the pixel values ​​on the G channel and the B channel on the third white light image and reduce the pixel value on the R channel at the same time.

[0149] S520, normalizing the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel respectively to obtain a second normalized image, wherein the second normalized image includes a second R channel normalized image, a second G channel normalized image, and a second B channel normalized image.

[0150] In one possible implementation, in order to reduce the complexity of calculation and improve the speed of pixel value processing, the pixel values ​​of all pixels on the third white light image on the R channel, G channel, and B channel are normalized to obtain a corresponding second normalized image. The pixel values ​​of all pixels on the second normalized image are between 0 and 1, that is, the pixel values ​​of all pixels on the second R channel normalized image, the second G channel normalized image, and the second B channel normalized image are also between 0 and 1, which is convenient for subsequent exponential operations using a gamma exponent algorithm. Exemplarily, the normalization formula k=n / 255 is used for calculation, where k is the normalization coefficient, including normalization on the R channel, G channel, and B channel, and n is the pixel value of all pixels on the third white light image on the R channel, G channel, and B channel, respectively.

[0151] S530: Based on the second gamma index coefficient, process the second normalized image using a gamma index algorithm to obtain a second normalized gain image.

[0152] In a possible implementation, the gamma index algorithm is used in conjunction with the normalized coefficient after normalization to further reduce the complexity of the calculation. For example, the gamma index mathematical formula m=(k) is used. gamma’ Calculation is performed, where m is the normalized coefficient after gamma index processing, including the normalized coefficient after gamma index processing on the R channel, the G channel, and the B channel, and gamma' is the second gamma index coefficient, including the second R channel gamma index coefficient Rgamma', the second G channel gamma index coefficient Ggamma' and the second B channel gamma index coefficient Bgamma'.

[0153] S540: Perform a denormalization process on the second normalized gain image to obtain the target white light image.

[0154] In a possible implementation, the pixel values ​​on the normalized third white light image need to be denormalized to obtain the third white light image after pixel value adjustment, that is, the target white light image with clear vascular features. Exemplarily, the denormalization formula f=m*255 is used for calculation, where f is the pixel value of all pixels on the target white light image on the R channel, G channel, and B channel respectively.

[0155] Fig.10 FIG. 4 shows a schematic structural diagram of a blood vessel enhancement device of the present invention. Fig.10 As shown, a blood vessel enhancement device according to a disclosed embodiment of the present invention comprises

[0156] An image acquisition module 10, used to acquire a white light image to be processed;

[0157] The first image processing module 20 is used to adjust the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel to obtain a first white light image; and to adjust the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain a second white light image;

[0158] An image fusion module 30, configured to fuse the first white light image and the second white light image to obtain a third white light image;

[0159] The second image processing module 40 is used to adjust the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a target white light image;

[0160] The image fusion module is further configured to: remove the pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the pixel values ​​on the R channel and the B channel to obtain the first channel image, and remove the pixel values ​​of all pixels on the second white light image on the R channel to obtain the second channel image; and fuse the pixel values ​​on the B channel or the G channel of the first channel image to the R channel of the second channel image to obtain the target white light image; or

[0161] Used to fuse the pixel values ​​on the B channel or the G channel on the first white light image to the corresponding R channel on the blank image; and used to fuse the pixel values ​​on the G channel on the second white light image to the corresponding G channel on the blank image; and used to fuse the pixel values ​​on the B channel on the second white light image to the corresponding B channel on the blank image, to obtain the third white light image; wherein the blank image has the same size as the first white light image and the second white light image.

[0162] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0163] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0164] The embodiment of the present disclosure also proposes an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory. The electronic device can be provided as a server or a terminal device. For example, the electronic device includes a processing component, which further includes one or more processors, and a memory resource represented by a memory, which is used to store instructions executable by the processing component, such as an application. The application stored in the memory may include one or more modules, each corresponding to a set of instructions. In addition, the processing component is configured to execute instructions to perform the above method, and the electronic device may also include a power supply component configured to perform power management of the electronic device, a wired or wireless network interface configured to connect the electronic device to a network, and an input / output interface (I / O interface). The electronic device can operate based on an operating system stored in the memory, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0165] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0166] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0167] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0168] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0169] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0170] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0171] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0172] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0173] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0174] It should be clear that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present invention is not limited to the specific steps and structures described above and shown in the figures. In addition, for the sake of brevity, a detailed description of known methods and technologies is omitted here.

[0175] The above is only an embodiment of the present application and is not limited to the present application. For those skilled in the art, the present application may have various changes and variations without departing from the scope of the present invention. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for blood vessel enhancement in an endoscopic white light image, characterized in that: include Acquire a white light image to be processed; Based on the white-light image to be processed, adjusting the pixel values ​​of all pixels on the white-light image to be processed on the R channel, the G channel, and the B channel, respectively, to obtain a first white-light image; Based on the first white light image, adjusting the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel, respectively, to obtain a second white light image; performing image fusion on the first white-light image and the second white-light image to obtain a third white-light image; Based on the third white light image, adjusting the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel, respectively, to obtain a target white light image; The first white light image and the second white light image are fused to obtain a third white light image, which includes: Removing pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the R channel and the B channel to obtain a first channel image; Removing pixel values ​​of all pixels on the second white light image on the R channel to obtain a second channel image; Merging the pixel values ​​on the B channel or the G channel of the first channel image into the R channel of the second channel image to obtain the third white light image; or Acquire a blank image, wherein the blank image has the same size as the first white light image and the second white light image; Merging the pixel values ​​of the B channel or the G channel on the first white light image to the corresponding R channel on the blank image; Merging the pixel values ​​of the G channel on the second white light image to the corresponding G channel on the blank image; And the pixel values ​​of the B channel on the second white light image are merged into the corresponding B channel on the blank image to obtain the third white light image.

2. The blood vessel enhancement method of endoscopic white light image according to claim 1, characterized in that: The method comprises adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel to obtain a first white light image based on the white light image to be processed, including Acquire a white balance gain coefficient, wherein the white balance gain coefficient includes a first gain coefficient, a second gain coefficient, and a third gain coefficient; Based on the first gain coefficient, the second gain coefficient and the third gain coefficient, the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel and the B channel are adjusted respectively to obtain the first white light image.

3. The blood vessel enhancement method of endoscopic white light image according to claim 2, characterized in that: The obtaining of the white balance gain coefficient comprises: Acquire a white light test image; Based on the white light test image, obtaining a first average pixel value in the R channel, a second average pixel value in the G channel, and a third average pixel value in the B channel of the white light test image; Obtaining the first gain coefficient based on a ratio of the second average pixel value to the first average pixel value; obtaining the second gain coefficient based on a ratio of the second average pixel value to the second average pixel value; The third gain coefficient is obtained based on a ratio of the second average pixel value to the third average pixel value.

4. The blood vessel enhancement method of endoscopic white light image according to claim 2, characterized in that: The pixel values ​​of all pixels on the to-be-processed white light image on the R channel, the G channel, and the B channel are adjusted based on the first gain coefficient, the second gain coefficient, and the third gain coefficient, respectively, including Multiplying the first gain coefficient by the pixel values ​​of all pixels on the R channel of the white light image to be processed to obtain a first gain pixel value; Multiplying the second gain coefficient by the pixel values ​​of all pixels on the G channel of the white light image to be processed to obtain a second gain pixel value; Multiplying the third gain coefficient by the pixel values ​​of all pixels on the B channel of the white light image to be processed to obtain a third gain pixel value; Comparing the first gain pixel value with the preset pixel threshold, and assigning the first gain pixel value as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed based on the first gain pixel value being less than the preset pixel threshold; otherwise, assigning the preset pixel threshold as the pixel value of the corresponding pixel point on the R channel of the white light image to be processed; Comparing the second gain pixel value with the preset pixel threshold, and assigning the second gain pixel value as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed based on the second gain pixel value being less than the preset pixel threshold; On the contrary, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the G channel of the white light image to be processed; Comparing the third gain pixel value with a preset pixel threshold, and assigning the third gain pixel value as a pixel value of a corresponding pixel point on the B channel of the white light image to be processed based on the fact that the third gain pixel value is less than the preset pixel threshold; On the contrary, the preset pixel threshold is assigned as the pixel value of the corresponding pixel point on the B channel of the white light image to be processed.

5. The blood vessel enhancement method of endoscopic white light image according to claim 1, characterized in that: The method comprises: adjusting the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain a second white light image based on the first white light image, including: Acquire a first gamma index coefficient, where the first gamma index coefficient includes a first R channel gamma index coefficient, a first G channel gamma index coefficient, and a first B channel gamma index coefficient; Normalizing the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel to obtain a first normalized image, wherein the first normalized image includes a first R channel normalized image, a second G channel normalized image, and a third B channel normalized image; Based on the first gamma index coefficient, processing the first normalized image using a gamma index algorithm to obtain a first normalized gain image; The first normalized gain image is denormalized to obtain the second white light image.

6. The blood vessel enhancement method of endoscopic white light image according to claim 1, characterized in that: Acquire the white light image to be processed, including Get the original image; Based on the original image, a white light image to be processed having RGB features is obtained.

7. The blood vessel enhancement method of endoscopic white light image according to claim 1, characterized in that: Based on the third white light image, the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel are adjusted to obtain a target white light image, including Acquire a second gamma index coefficient, wherein the second gamma index coefficient includes a second R channel gamma index coefficient, a second G channel gamma index coefficient, and a second B channel gamma index coefficient; Normalizing the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a second normalized image, wherein the second normalized image includes a second R channel normalized image, a second G channel normalized image, and a second B channel normalized image; Based on the second gamma index coefficient, processing the second normalized image by using a gamma index algorithm to obtain a second normalized gain image; The second normalized gain image is subjected to denormalization processing to obtain the target white light image.

8. A blood vessel enhancement device, characterized in that: include An image acquisition module, used for acquiring a white light image to be processed; A first image processing module, used for adjusting the pixel values ​​of all pixels on the white light image to be processed on the R channel, the G channel, and the B channel respectively, to obtain a first white light image; and used to adjust the pixel values ​​of all pixels on the first white light image on the R channel, the G channel, and the B channel respectively, to obtain a second white light image; an image fusion module, configured to fuse the first white light image with the second white light image to obtain a third white light image; A second image processing module is used to adjust the pixel values ​​of all pixels on the third white light image on the R channel, the G channel, and the B channel to obtain a target white light image; The image fusion module is further configured as follows: Used to remove the pixel values ​​of all pixels on the first white light image on the R channel and the G channel or the pixel values ​​of the R channel and the B channel to obtain a first channel image, and used to remove the pixel values ​​of all pixels on the second white light image on the R channel to obtain a second channel image; and for fusing the pixel values ​​on the B channel or the G channel of the first channel image to the R channel of the second channel image to obtain the target white light image; or Used to merge the pixel values ​​of the B channel or the G channel on the first white light image into the corresponding R channel on the blank image; and for fusing the pixel values ​​of the G channel on the second white light image to the corresponding G channel on the blank image; And used to merge the pixel values ​​of the B channel on the second white light image with the corresponding B channel on the blank image to obtain the third white light image; wherein the blank image has the same size as the first white light image and the second white light image.

9. An electronic device, characterized in that: comprising a processor; and a memory for storing processor executable instructions, wherein the processor is configured to implement the method of any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images