Image processing method and device, endoscope system and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, an endoscope system and a storage medium, and the method comprises the steps: obtaining a Y-channel image, a U-channel image and a V-channel image according to a YUV image of an original RGB blood vessel image; performing equalization processing on the Y channel image and the U channel image to obtain a Y channel target image and a U channel target image; performing channel splicing on the Y channel target image, the U channel target image and the V channel image to obtain a first RGB image; and fusing the first RGB image and the original RGB blood vessel image to obtain a blood enhanced image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, endoscope system and storage medium. Background Art

[0002] In the medical field, in order to understand the patient's vascular morphology and function, vascular images of relevant parts of the human body are usually obtained through endoscopy, so that doctors can make accurate clinical decisions based on the vascular images.

[0003] However, due to the limited light environment inside the human body, the clarity of the blood vessels in the collected vascular images may be low, which may affect the accuracy of the doctor's decision-making. Therefore, to solve this technical problem, related technologies use guided graph filtering technology to process vascular images to enhance the vascular blood features in the vascular images, that is, vascular enhancement or blood enhancement, thereby obtaining a blood-enhanced image. However, guided graph filtering technology has problems such as large computational complexity, slow processing speed, and high requirements for processor performance. It cannot meet the image processing needs of endoscopes with high timeliness requirements, such as endoscopes used in surgical scenarios. It also cannot meet the image processing needs of endoscopes with limited processor performance, especially portable endoscopes. On the one hand, the computing power of portable endoscopes is limited due to size and heat dissipation limitations. On the other hand, due to the instability of network transmission and the long time required to upload and download images, portable endoscopes may not be able to use servers for image processing. Summary of the Invention

[0004] In view of this, in order to at least solve the technical problems of large computational complexity, slow processing speed, and high processor performance requirements in the related art of vascular image enhancement technology, the purpose of the present invention is to provide an image processing method, device, endoscope system and storage medium.

[0005] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] According to a first aspect of an embodiment of the present invention, there is provided an image processing method, comprising:

[0007] According to the YUV image of the original RGB blood vessel image, a Y channel image, a U channel image and a V channel image are obtained;

[0008] Performing equalization processing on the Y channel image and the U channel image respectively to obtain a Y channel target image and a U channel target image;

[0009] Perform channel splicing on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image;

[0010] The first RGB image and the original RGB blood vessel image are fused to obtain a blood enhancement image.

[0011] In an alternative embodiment,

[0012] The equalization processing process of any one of the Y channel image and the U channel image to be processed includes:

[0013] Dividing the image to be processed into a plurality of image blocks according to a set first block grid size;

[0014] Perform local equalization processing on each image block to obtain a local mapping table corresponding to each image block;

[0015] For each edge pixel of each image block, adjusting the pixel of the edge pixel by bilinear interpolation based on the local mapping table of the four image blocks closest to the edge pixel to obtain an updated image block;

[0016] All updated image blocks are merged to obtain the target image after blood enhancement.

[0017] In an optional embodiment, the method further comprises:

[0018] Obtaining an R channel image, a G channel image, and a B channel image according to the original RGB blood vessel image;

[0019] Performing equalization processing and correction processing on each of the R channel image, the G channel image, and the B channel image in sequence to obtain an R channel target image, a G channel target image, and a B channel target image;

[0020] Perform channel stitching on the R channel target image, the G channel target image, and the B channel target image to obtain a second RGB image;

[0021] The step of fusing the first RGB image with the original RGB blood vessel image to obtain the blood-enhanced image is adjusted to: fusing the first RGB image with the second RGB image to obtain the blood-enhanced image.

[0022] In an optional embodiment, in the process of performing equalization processing on each to-be-processed image in the Y channel image and the U channel image, each channel image is subjected to block processing using a first block network size, wherein the first block network size is (2 n , 2 n ), where n∈[0,5] and n is an integer; and / or

[0023] In the process of performing equalization processing on each channel image of the R channel image, the G channel image and the B channel image, each channel image is subjected to block processing using a second block network size, wherein the second block network size is (2 m , 2 m ), where m∈[1,6] and m is an integer; and / or

[0024] In the process of correcting the R channel image, the G channel image and the B channel image respectively, the first exponent value, the second exponent value and the third exponent value are respectively used to correct the R channel image, the G channel image and the B channel image; the value range of the first exponent value is [0.8, 1.5], and the value range of the second exponent value and the value range of the third exponent value are both [1.1, 1.8].

[0025] In an optional embodiment, the first block grid size is smaller than the second block grid size.

[0026] In an optional embodiment, the step of fusing the first RGB image and the second RGB image to obtain a blood-enhanced image includes:

[0027] calculating a first mean square error between the first RGB image and the original RGB blood vessel image;

[0028] calculating a second mean square error between the second RGB image and the original RGB blood vessel image;

[0029] determining a first weight of the first RGB image and a second weight of the second RGB image according to the first mean square error and the second mean square error;

[0030] The first RGB image and the second RGB image are fused according to the first weight and the second weight to obtain a blood-enhanced image.

[0031] In an alternative embodiment,

[0032] The step of determining a first weight of the first RGB image and a second weight of the second RGB image according to the first mean square error and the second mean square error includes:

[0033] When the first mean square error is greater than the second mean square error, configuring the first weight to be a value smaller than the second weight;

[0034] When the first mean square error is equal to the second mean square error, configuring the first weight and the second weight to be the same value;

[0035] When the first mean square error is less than the second mean square error, configuring the first weight to be a value greater than the second weight;

[0036] The sum of the first weight and the second weight is 1.

[0037] In an optional embodiment, when the first mean square error is greater than the second mean square error, the value range of the first weight is [0.3, 0.5], and the value range of the second weight is (0.5, 0.7];

[0038] When the first mean square error is smaller than the second mean square error, the value range of the first weight is (0.5, 0.7], and the value range of the second weight is [0.3, 0.5].

[0039] According to a second aspect of the embodiments of the present invention, there is provided an image processing apparatus, comprising:

[0040] The disassembly module is configured to: obtain a Y channel image, a U channel image, and a V channel image according to the YUV image of the original RGB blood vessel image;

[0041] The equalization processing module is configured to: perform equalization processing on the Y channel image and the U channel image respectively to obtain a Y channel target image and a U channel target image;

[0042] a splicing module configured to: perform channel splicing on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image;

[0043] The fusion module is configured to fuse the first RGB image with the original RGB blood vessel image to obtain a blood enhancement image.

[0044] According to a third aspect of an embodiment of the present invention, an endoscope system is provided, comprising:

[0045] An image acquisition device, used for acquiring original RGB blood vessel images;

[0046] An image processing device, configured to execute the image processing method provided in any one of the first aspects above, so as to obtain a blood-enhanced image based on the original RGB blood vessel image;

[0047] and a display device for displaying the blood enhancement image.

[0048] According to a fourth aspect of an embodiment of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the image processing method provided in any one of the first aspects above.

[0049] According to a fifth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method provided in any one of the first aspects is implemented.

[0050] The image processing method, device, endoscope system, and storage medium provided by any of the above embodiments of the present invention have at least the following beneficial technical effects:

[0051] During the processing of the original RGB vascular image, the original RGB vascular image is converted into a YUV image to obtain a Y-channel image, a U-channel image, and a V-channel image. Next, the Y-channel and U-channel images are equalized, while the V-channel image remains unchanged. This not only reduces the amount of image processing computation required compared to guided graph filtering techniques, but also enhances vascular color and preserves the light and dark structure and details in the image, avoiding overexposure, underexposure, or loss of detail caused by brightness adjustment, thereby further improving the contrast between vascular and non-vascular areas. Subsequently, the V-channel image, along with the equalized Y-channel and U-channel target images, is combined to create a first RGB image, which is then fused with the original RGB vascular image to produce a blood-enhanced image with distinct vascular blood features and high blood color reproduction. This image processing process, compared to guided graph filtering techniques, has a reduced computational workload and high image processing efficiency. It also achieves excellent vascular blood feature enhancement and high blood color reproduction, resulting in improved visual quality. Therefore, it is suitable for endoscopes with moderate processor performance and meets the timeliness requirements of endoscopic image output in surgical scenarios, making it particularly suitable for portable endoscopes.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 A structural block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0055] Figure 2 A flowchart of an image processing method provided by an embodiment of the present invention is shown;

[0056] Figure 3 A schematic diagram showing a comparison between an original RGB blood vessel image and a blood enhancement image provided by an embodiment of the present invention is shown;

[0057] Figure 4 A schematic diagram of an endoscope system provided by an embodiment of the present invention is shown;

[0058] Figure 5 A functional module diagram of an image processing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0061] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0062] To at least address the technical issues of high computational complexity, slow processing speed, and high processor performance requirements associated with vascular image enhancement techniques in the related art, the present invention provides an image processing method. During processing of an original RGB vascular image, the original RGB vascular image is converted into a YUV image to obtain a Y-channel image, a U-channel image, and a V-channel image. Subsequently, the Y-channel image and the U-channel image are equalized, while the V-channel image remains unchanged. This method not only reduces the computational complexity required for image processing compared to guided graph filtering techniques, but also enhances vascular color and protects the light and dark structures and details in the image, avoiding overexposure, underexposure, or loss of detail due to brightness adjustment, thereby further improving the contrast between vascular and non-vascular areas. Subsequently, a first RGB image, obtained by splicing the V-channel image, the Y-channel target image obtained after equalization, and the U-channel target image, is fused with the original RGB vascular image to produce a blood-enhanced image with distinct vascular blood features and high vascular blood color reproduction. It can be seen that the above-mentioned image processing process has a small amount of computation compared to the guided graph filtering technology, has high image processing efficiency, and has excellent technical effects such as vascular blood feature enhancement and high vascular blood color restoration, which can optimize the visual effect. Therefore, it can be applied to endoscopes with general processor performance, and can meet the timeliness requirements of endoscopic image output in surgical scenarios, and is particularly suitable for portable endoscopes.

[0063] The image processing method provided by the present invention can be applied to electronic devices. Figure 1 , is a block diagram of the structure of an electronic device. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0064] The memory is used to store programs or data. The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0065] The processor is used to read / write data or programs stored in the memory and execute corresponding functions.

[0066] The communication module is used to establish a communication connection between the electronic device and other communication terminals through a network, and to send and receive data through the network.

[0067] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device, and the electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0068] In some embodiments, the electronic device can be an image processing device, which can be sold as a separate product or as a processing module in an image device that has both image acquisition and image processing functions to execute the image processing method provided in an embodiment of the present invention, thereby reducing the amount of computation in image processing and achieving a blood enhancement effect.

[0069] The following combination Figure 2 The image processing method provided by the embodiment of the present invention is described. Figure 2 : is a flowchart of an image processing method provided by an embodiment of the present invention, the image processing method comprising:

[0070] In step S100, a Y channel image, a U channel image, and a V channel image are obtained according to the YUV image of the original RGB blood vessel image;

[0071] In step S200, the Y channel image and the U channel image are respectively equalized to obtain a Y channel target image and a U channel target image;

[0072] In step S300, channel splicing is performed on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image;

[0073] In step S400, the first RGB image and the original RGB blood vessel image are fused to obtain a blood-enhanced image.

[0074] In scenarios where vascular image enhancement is required, the image processing method provided by an embodiment of the present invention can be used to perform blood enhancement processing on the original RGB vascular image captured by the image acquisition device in the endoscope system. After the processing is completed, the processed blood-enhanced image can be displayed on a display device for the user to view. For example, in an endoscope application scenario, the image processing method provided by an embodiment of the present invention can be used to process the images captured in real time by the endoscope system, and the processed blood-enhanced image can be output to the display device in real time, thereby achieving a smooth display of the image sequence captured by the endoscope system.

[0075] The execution subject of the image processing method provided in the embodiment of the present invention may start executing the above steps S100 to S400 after receiving the blood enhancement instruction. In this case, all received original RGB blood vessel images may be processed by default so that the processed original RGB blood vessel images can have a better blood enhancement effect.

[0076] Alternatively, if blood enhancement processing is determined to be necessary based on the original RGB vascular image, for example, upon receiving the original RGB vascular image, the contrast between the vascular and non-vascular regions in the original RGB vascular image can be first obtained. If the contrast is found to be lower than a set contrast threshold, steps S100 to S400 can be resumed. Conversely, if the contrast is found to be higher than or equal to the set contrast threshold, steps S100 to S400 can be skipped. In this case, it can be assumed that the vascular blood features in the original RGB vascular image are relatively clear and at a good level, and therefore further blood enhancement can be omitted. This can reduce the number of images required for processing to a certain extent, and can enhance the immediacy of image output.

[0077] In the above description, the original RGB vascular image can be acquired by an image acquisition device and sent to the execution entity, or can be sent by a server to the execution entity, or can be obtained by a user selecting from the execution entity's image library. The present embodiment does not restrict the source of the original RGB vascular image. Furthermore, the contrast threshold can be set based on experience or experimentation, and will not be further explained here.

[0078] After obtaining the original RGB vascular image, step S100 is executed to convert the original RGB vascular image into a YUV image. The conversion principle can be found in related art. The YUV image can then be channel-decomposed to obtain a Y channel image, a U channel image, and a V channel image. The decomposition principle can also be found in related art. It should be understood that image data is typically stored in a matrix format at the underlying layer.

[0079] After obtaining the Y channel image and the U channel image, step S200 is executed to perform equalization processing on the Y channel image and the U channel image respectively to improve the contrast between the vascular area and the non-vascular area, as well as the clarity of the vascular details, improve the blood enhancement effect, and help optimize the visual effect.

[0080] In some examples, the equalization process may be a histogram equalization process, but is not limited thereto.

[0081] During step S200, to achieve a better blood enhancement effect, the contrast-limited adaptive histogram equalization technique (CLAHE) may be used to process the Y-channel image and the U-channel image. Based on this, in some embodiments, the image processing method provided by the embodiments of the present invention further provides another solution for performing equalization processing on any one of the Y-channel image and the U-channel image to be processed. That is, in step S200, the equalization processing process for any one of the Y-channel image and the U-channel image to be processed may include:

[0082] In step S210, the image to be processed is divided into a plurality of image blocks according to a set first block grid size;

[0083] In step S220, local equalization processing is performed on each image block to obtain a local mapping table corresponding to each image block;

[0084] In step S230, for each edge pixel of each image block, the pixel of the edge pixel is adjusted by bilinear interpolation based on the local mapping tables of the four image blocks closest to the edge pixel to obtain an updated image block;

[0085] In step S240, all updated image blocks are merged to obtain a target image after blood enhancement.

[0086] The process of obtaining the corresponding target image after blood enhancement based on the image to be processed through steps S210 to S240 is as follows:

[0087] First, in step S210, the image to be processed is divided into a plurality of image blocks according to the first block grid size. The first block grid size may refer to the number of rows and columns into which the image to be processed is evenly divided. For example, assuming the first block grid size is (8, 8), it means that the image to be processed is evenly divided into image blocks of 8 rows × 8 columns, for a total of 64 image blocks. The size of each image block can be automatically calculated based on the image size, which will not be explained here.

[0088] In some embodiments, the size of the first block grid can be positively correlated with the resolution of the image to be processed, that is, if the resolution of the image to be processed is higher, the first block grid size can also be larger. This can make the total number of image blocks adapt to the image resolution, and can reduce the amount of block division calculation to a certain extent.

[0089] In some embodiments, to obtain a better blood strengthening effect, the first block grid size is (2 n , 2 n ), where n∈[0,5], and n is an integer. Thus, by adopting the first block grid size within the above value range, a better balance can be achieved between the local details, local contrast, and computational efficiency of the resulting blocks, enabling a more global image enhancement effect with less detail loss.

[0090] After obtaining multiple image blocks, step S220 is executed to perform local equalization on each image block to obtain a local mapping table corresponding to each image block. Each local mapping table records the conversion relationship between the original grayscale level of each element in the corresponding image block and the new grayscale level, which is used to define how to stretch or compress the grayscale values in the corresponding image block to enhance local contrast. The original grayscale level can be obtained from the image blocks obtained by the division in step S210, and the new grayscale level can be calculated by performing local histogram equalization on the image blocks.

[0091] After obtaining the local mapping table corresponding to each image block, in order to eliminate boundary artifacts between image blocks and ensure the continuity of blood vessels, step S230 is executed to adjust the pixel of each edge pixel point of each image block by bilinear interpolation based on the local mapping tables of the four image blocks closest to the edge pixel point. After the pixel of each edge pixel point is adjusted, the corresponding updated image block can be obtained.

[0092] As an example of adjustment, let's take the pixel adjustment of an edge pixel as an example. During the pixel adjustment process, if there are image blocks above, to the left, below, and to the right of the edge pixel, the image blocks closest to the edge pixel in these four directions can be used as the basis for adjustment. Then, based on the distance between the edge pixel and the center points of these four image blocks and the new grayscale recorded in the local mapping table, a weighted calculation is performed to obtain the final grayscale value of the edge pixel. Based on this principle, all edge pixels are updated to obtain the updated image blocks, so that there are no artifacts between the boundaries of the image blocks, ensuring the continuity of the blood vessels.

[0093] After obtaining the updated image blocks, step S240 is executed to merge all the updated image blocks to obtain the target image after blood enhancement.

[0094] Therefore, the Y channel image and the U channel image can be processed respectively through steps S210 to S240 to obtain the Y channel target image and the U channel target image after blood enhancement.

[0095] After obtaining the Y-channel target image and the U-channel target image, step S300 is executed to perform channel splicing on the Y-channel target image, the U-channel target image, and the V-channel image to obtain a first RGB image.

[0096] Next, step S400 is performed to fuse the first RGB image with the original RGB vascular image to obtain a blood-enhanced image. As an example, a weighted fusion of the first RGB image and the original RGB image can be performed using a set weight. To maintain a good blood-enhancing effect, the weight of the first RGB image is greater than the weight of the original RGB vascular image. For example, the fusion weight of the first RGB image can be 0.7, and the fusion weight of the original RGB vascular image can be 0.3, but this is not limited to these. This balances the blood-enhancing effect and the color restoration effect while reducing the computational complexity required for image processing.

[0097] However, in some cases, the original RGB blood vessel image may be underexposed or overexposed due to insufficient or strong light in the acquisition environment. Therefore, in step S400, if the first RGB image and the original RGB blood vessel image are fused, the fused blood enhancement image may introduce issues from the original RGB blood vessel image, thereby affecting the blood enhancement effect or visual effect. To address this technical problem, in some embodiments, the image processing method provided by the embodiments of the present invention also provides another image fusion solution. The image processing method provided by the embodiments of the present invention may also include:

[0098] In step S10, an R channel image, a G channel image, and a B channel image are obtained according to the original RGB blood vessel image;

[0099] In step S20, each of the R channel image, the G channel image, and the B channel image is subjected to equalization processing and correction processing in sequence to obtain an R channel target image, a G channel target image, and a B channel target image;

[0100] In step S30, channel splicing is performed on the R channel target image, the G channel target image, and the B channel target image to obtain a second RGB image.

[0101] Accordingly, the above step S400 is adjusted, that is, the step of fusing the first RGB image and the original RGB blood vessel image to obtain the blood-enhanced image is adjusted to: fusing the first RGB image and the second RGB image to obtain the blood-enhanced image.

[0102] It is understandable that after the original RGB blood vessel image is obtained, steps S10 to S30 are further executed. The execution order of step S10 and step S100 is not limited and they can also be executed in parallel.

[0103] During step S10 , the original RGB blood vessel image is decomposed into an R channel image, a G channel image, and a B channel image.

[0104] Next, step S20 is executed to sequentially perform equalization and correction processing on each of the R, G, and B channel images to obtain the R, G, and B channel target images. By first equalizing each channel image, global contrast and color contrast can be enhanced. Further correction processing is then performed on each equalized channel image to achieve brightness and color balance, enhance image detail, and improve the visual quality. It can be seen that combining equalization and correction can significantly improve the quality of the original RGB vascular image, thereby effectively enhancing the blood enhancement and visual optimization effects of the subsequent blood-enhanced image.

[0105] In some examples, the equalization process in step S20 may be a histogram equalization process, and the correction process may be a gamma correction process, but is not limited thereto.

[0106] In some embodiments, in order to obtain better global contrast and color contrast enhancement effects, in step S20, the limited contrast adaptive histogram equalization technology can also be used to process each channel image in the R channel image, the G channel image and the B channel image.

[0107] Based on the previous embodiment, in some embodiments, in order to make the total number of image blocks adaptable to the image resolution, to reduce the amount of block division operations to a certain extent, and to ensure the processing quality of the above-mentioned channel images, in the above-mentioned step S20, in the process of performing equalization processing on each channel image in the R channel image, the G channel image and the B channel image, the second block network size is used to perform block processing on each channel image, wherein the second block network size is (2 m , 2 m ), where m∈[0, 6], and m is an integer.

[0108] Based on the previous embodiment, in some embodiments, to achieve a better blood enhancement effect by fusing the first RGB image and the second RGB image, and to better restore the color and characteristic clarity of the blood vessels themselves, the first block grid size is smaller than the second block grid size. Thus, dividing the Y-channel image and the U-channel image into fewer image blocks can, to a certain extent, ensure blood vessel continuity and achieve a more global blood enhancement effect. Combined with dividing the R-channel image, the G-channel image, and the B-channel image into more image blocks, the local contrast of the blood vessels can be better enhanced. Consequently, the blood enhancement image obtained by fusing the first RGB image and the second RGB image not only has a local blood enhancement effect, but also a global blood enhancement effect.

[0109] In some embodiments, to provide a better visual effect for the second RGB image and thereby enhance the visual effect of the blood-enhanced image, in step S20, during the correction processing of the R-channel image, the G-channel image, and the B-channel image, respectively, a first exponent value, a second exponent value, and a third exponent value are used to correct the R-channel image, the G-channel image, and the B-channel image. The first exponent value has a value range of [0.8, 1.5], and the second exponent value and the third exponent value both have a value range of [1.1, 1.8]. Thus, by limiting the exponent value corresponding to each channel image during the correction processing, the visual effect of the image can be further enhanced.

[0110] After obtaining the R channel target image, the G channel target image, and the B channel target image through any of the above embodiments, step S30 is executed to perform channel splicing on the R channel target image, the G channel target image, and the B channel target image to obtain a second RGB image.

[0111] After obtaining the first RGB image and the second RGB image, the adjusted step S400 can be performed to fuse the first RGB image and the second RGB image to obtain a blood-enhanced image. The fusion processing method can be found in the relevant description above and will not be described in detail here.

[0112] However, to achieve a better blood enhancement effect, in some embodiments, the image processing method provided by the embodiment of the present invention further provides another image fusion solution. That is, in the above-adjusted step S400, the step of fusing the first RGB image and the second RGB image to obtain the blood enhancement image may include:

[0113] In step S410, a first mean square error between the first RGB image and the original RGB blood vessel image is calculated;

[0114] In step S420, a second mean square error between the second RGB image and the original RGB blood vessel image is calculated;

[0115] In step S430, a first weight of the first RGB image and a second weight of the second RGB image are determined according to the first mean square error and the second mean square error;

[0116] In step S440 , the first RGB image and the second RGB image are fused according to the first weight and the second weight to obtain a blood-enhanced image.

[0117] It can be understood that after obtaining the first RGB image and the second RGB image, steps S410 to S440 are performed to obtain a blood enhancement image with a better enhancement effect.

[0118] The above steps S410 and S420 can be executed in parallel or in series. In the case of serial execution, the execution order of steps S410 and S420 is not limited. In addition, the calculation principle of the first mean square error and the second mean square error can be referred to in related art.

[0119] After obtaining the first mean square error and the second mean square error, step S430 is performed to determine a first weight of the first RGB image and a second weight of the second RGB image based on the first mean square error and the second mean square error. In some examples, embodiments of the present invention provide a method for determining the first weight and the second weight. That is, in step S430, determining the first weight of the first RGB image and the second weight of the second RGB image based on the first mean square error and the second mean square error may include:

[0120] In step S431, when the first mean square error is greater than the second mean square error, the first weight is configured to be a value smaller than the second weight;

[0121] In step S432, when the first mean square error is equal to the second mean square error, the first weight and the second weight are configured to be the same value;

[0122] In step S433, when the first mean square error is smaller than the second mean square error, the first weight is configured to be a value greater than the second weight;

[0123] The sum of the first weight and the second weight is 1.

[0124] Therefore, by assigning a larger weight to the RGB image with a smaller mean square error with the original RGB blood vessel image, and assigning a smaller weight to the RGB image with a larger mean square error with the original RGB blood vessel image, the mean square error between the blood enhancement image obtained by fusion of the first RGB image and the second RGB image and the original RGB blood vessel image can be guaranteed or reduced to a certain extent, thereby better improving the blood enhancement effect and avoiding image distortion. Figure 3 As shown, Figure 3 FIG2 is a schematic diagram comparing an original RGB vascular image and a blood-enhanced image provided by an embodiment of the present invention. It can be seen that the obtained blood-enhanced image B can enhance the vascular blood features in the original RGB vascular image A while retaining them, making them clearly visible while also achieving excellent color reproduction and visual optimization effects.

[0125] In some embodiments, in order to obtain a better blood enhancement effect, when the first mean square error is greater than the second mean square error, the value range of the first weight is [0.3, 0.5), and the value range of the second weight is (0.5, 0.7]; when the first mean square error is less than the second mean square error, the value range of the first weight is (0.5, 0.7], and the value range of the second weight is [0.3, 0.5].

[0126] After obtaining the blood enhancement image, in some embodiments, the blood enhancement image can also be output to a display device for the user to view. For example, in the field of endoscopy applications, the processed blood enhancement image sequence can be output to a display device for the doctor to observe and perform relevant diagnosis.

[0127] It is worth noting that the technical features or technical solutions in any of the above embodiments of the present invention can be combined with each other as long as there is no combination contradiction.

[0128] In addition, an embodiment of the present invention also provides an endoscope system, see Figure 4 , Figure 4 FIG. 4 is a schematic diagram of an endoscope system provided by an embodiment of the present invention. The endoscope system 400 includes:

[0129] The image acquisition device 410 is used to acquire original RGB blood vessel images; the image acquisition device 410 can be set at the front end of the insertion tube 440 connected to the endoscope operating handle 450.

[0130] An image processing device 420 is configured to execute the image processing method provided in any of the above embodiments to obtain a blood-enhanced image based on the original RGB blood vessel image;

[0131] and a display device 430 for displaying the blood enhancement image.

[0132] The endoscope system provided in the embodiment of the present invention is only an example. In other examples, the endoscope system may also include Figure 4 More or fewer components than shown, or with Figure 4 Different configurations shown.

[0133] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of an image processing device is given below. Optionally, the image processing device can adopt the above Figure 1 For further information, please refer to Figure 5 , Figure 5 This is a functional block diagram of an image processing device provided by an embodiment of the present invention. It should be noted that the basic principles and technical effects of the image processing device provided by this embodiment are the same as those of the above-mentioned embodiments. For the sake of simplicity, any parts not mentioned in this embodiment can be referred to the corresponding contents of the above-mentioned embodiments. The image processing device 500 includes:

[0134] The disassembly module 510 is configured to obtain a Y channel image, a U channel image, and a V channel image according to the YUV image of the original RGB blood vessel image;

[0135] The equalization processing module 520 is configured to: perform equalization processing on the Y channel image and the U channel image respectively to obtain a Y channel target image and a U channel target image;

[0136] The stitching module 530 is configured to: perform channel stitching on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image;

[0137] The fusion module 540 is configured to fuse the first RGB image with the original RGB blood vessel image to obtain a blood enhancement image.

[0138] In some embodiments, the equalization processing module 520 performs equalization processing on any one of the Y channel image and the U channel image to be processed, and is configured as follows:

[0139] Dividing the image to be processed into a plurality of image blocks according to a set first block grid size;

[0140] Perform local equalization processing on each image block to obtain a local mapping table corresponding to each image block;

[0141] For each edge pixel of each image block, adjusting the pixel of the edge pixel by bilinear interpolation based on the local mapping table of the four image blocks closest to the edge pixel to obtain an updated image block;

[0142] All updated image blocks are merged to obtain the target image after blood enhancement.

[0143] In some embodiments, the disassembly module 510 is further configured to: obtain an R channel image, a G channel image, and a B channel image according to the original RGB blood vessel image;

[0144] Accordingly, the equalization processing module 520 is further configured to: perform equalization processing and correction processing on each channel image of the R channel image, the G channel image and the B channel image in sequence to obtain the R channel target image, the G channel target image and the B channel target image;

[0145] Correspondingly, the stitching module 530 is further configured to: perform channel stitching on the R channel target image, the G channel target image, and the B channel target image to obtain a second RGB image;

[0146] Accordingly, the step of fusing the first RGB image and the original RGB blood vessel image to obtain the blood-enhanced image by the fusion module 540 is adjusted and configured to: fuse the first RGB image and the second RGB image to obtain the blood-enhanced image.

[0147] In some embodiments, in the process of performing equalization processing on each to-be-processed image in the Y channel image and the U channel image, each channel image is processed in blocks using a first block network size, wherein the first block network size is (2 n , 2 n ), where n∈[0,5], and n is an integer.

[0148] In some embodiments, in the process of performing equalization processing on each channel image of the R channel image, the G channel image, and the B channel image, each channel image is subjected to block processing using a second block network size, wherein the second block network size is (2 m , 2 m ), where m∈[0, 6], and m is an integer.

[0149] In some embodiments, during the process of correcting the R channel image, G channel image and B channel image respectively, the first exponent value, the second exponent value and the third exponent value are respectively used to correct the R channel image, the G channel image and the B channel image; the value range of the first exponent value is [0.8, 1.5], and the value range of the second exponent value and the value range of the third exponent value are both [1.1, 1.8].

[0150] In some embodiments, the first block grid size is smaller than the second block grid size.

[0151] In some embodiments, the process of the fusion module 540 fusing the first RGB image and the second RGB image to obtain the blood enhancement image is configured as follows:

[0152] calculating a first mean square error between the first RGB image and the original RGB blood vessel image;

[0153] calculating a second mean square error between the second RGB image and the original RGB blood vessel image;

[0154] determining a first weight of the first RGB image and a second weight of the second RGB image according to the first mean square error and the second mean square error;

[0155] The first RGB image and the second RGB image are fused according to the first weight and the second weight to obtain a blood-enhanced image.

[0156] In some embodiments, the process of determining the first weight of the first RGB image and the second weight of the second RGB image according to the first mean square error and the second mean square error by the fusion module 540 is configured as follows:

[0157] When the first mean square error is greater than the second mean square error, configuring the first weight to be a value smaller than the second weight;

[0158] When the first mean square error is equal to the second mean square error, configuring the first weight and the second weight to be the same value;

[0159] When the first mean square error is less than the second mean square error, configuring the first weight to be a value greater than the second weight;

[0160] The sum of the first weight and the second weight is 1.

[0161] In some embodiments, when the first mean square error is greater than the second mean square error, the value range of the first weight is [0.3, 0.5), and the value range of the second weight is (0.5, 0.7]; when the first mean square error is less than the second mean square error, the value range of the first weight is (0.5, 0.7], and the value range of the second weight is [0.3, 0.5].

[0162] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown in FIG. 1 or the operating system (OS) of the electronic device may be fixed and may be Figure 1 Meanwhile, the data and program codes required to execute the above modules may be stored in the memory.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0164] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0165] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0166] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: According to the YUV image of the original RGB blood vessel image, a Y channel image, a U channel image and a V channel image are obtained; Performing equalization processing on the Y channel image and the U channel image respectively to obtain a Y channel target image and a U channel target image; Perform channel splicing on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image; The first RGB image and the original RGB blood vessel image are fused to obtain a blood enhancement image.

2. The method according to claim 1, characterized in that The equalization processing process of any one of the Y channel image and the U channel image to be processed includes: Dividing the image to be processed into a plurality of image blocks according to a set first block grid size; Perform local equalization processing on each image block to obtain a local mapping table corresponding to each image block; For each edge pixel of each image block, adjusting the pixel of the edge pixel by bilinear interpolation based on the local mapping table of the four image blocks closest to the edge pixel to obtain an updated image block; All updated image blocks are merged to obtain the target image after blood enhancement.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Obtaining an R channel image, a G channel image, and a B channel image according to the original RGB blood vessel image; Performing equalization processing and correction processing on each of the R channel image, the G channel image, and the B channel image in sequence to obtain an R channel target image, a G channel target image, and a B channel target image; Perform channel stitching on the R channel target image, the G channel target image, and the B channel target image to obtain a second RGB image; The step of fusing the first RGB image with the original RGB blood vessel image to obtain the blood-enhanced image is adjusted to: fusing the first RGB image with the second RGB image to obtain the blood-enhanced image.

4. The method according to claim 3, characterized in that In the process of performing equalization processing on each of the Y channel image and the U channel image, each channel image is processed in blocks using a first block network size, wherein the first block network size is (2 n , 2 n ), where n∈[0,5] and n is an integer; and / or In the process of performing equalization processing on each channel image of the R channel image, the G channel image and the B channel image, each channel image is subjected to block processing using a second block network size, wherein the second block network size is (2 m , 2 m ), where m∈[0,6] and m is an integer; and / or In the process of correcting the R channel image, the G channel image and the B channel image respectively, the first exponent value, the second exponent value and the third exponent value are respectively used to correct the R channel image, the G channel image and the B channel image; the value range of the first exponent value is [0.8, 1.5], and the value range of the second exponent value and the value range of the third exponent value are both [1.1, 1.8].

5. The method according to claim 3, characterized in that The step of fusing the first RGB image and the second RGB image to obtain a blood-enhanced image includes: calculating a first mean square error between the first RGB image and the original RGB blood vessel image; calculating a second mean square error between the second RGB image and the original RGB blood vessel image; determining a first weight of the first RGB image and a second weight of the second RGB image according to the first mean square error and the second mean square error; The first RGB image and the second RGB image are fused according to the first weight and the second weight to obtain a blood-enhanced image.

6. The method according to claim 5, characterized in that The step of determining a first weight of the first RGB image and a second weight of the second RGB image according to the first mean square error and the second mean square error includes: When the first mean square error is greater than the second mean square error, configuring the first weight to be a value smaller than the second weight; When the first mean square error is equal to the second mean square error, configuring the first weight and the second weight to be the same value; When the first mean square error is less than the second mean square error, configuring the first weight to be a value greater than the second weight; The sum of the first weight and the second weight is 1.

7. The method according to claim 6, characterized in that When the first mean square error is greater than the second mean square error, the value range of the first weight is [0.3, 0.5], and the value range of the second weight is (0.5, 0.7); When the first mean square error is smaller than the second mean square error, the value range of the first weight is (0.5, 0.7], and the value range of the second weight is [0.3, 0.5].

8. An image processing device, characterized in that: include: The disassembly module is configured to: obtain a Y channel image, a U channel image, and a V channel image according to the YUV image of the original RGB blood vessel image; The equalization processing module is configured to: perform equalization processing on the Y channel image and the U channel image respectively to obtain a Y channel target image and a U channel target image; a splicing module configured to: perform channel splicing on the Y channel target image, the U channel target image, and the V channel image to obtain a first RGB image; The fusion module is configured to fuse the first RGB image with the original RGB blood vessel image to obtain a blood enhancement image.

9. An endoscope system, characterized in that: include: An image acquisition device, used for acquiring original RGB blood vessel images; An image processing device, configured to execute the method according to any one of claims 1 to 7, so as to obtain a blood-enhanced image based on the original RGB blood vessel image; and a display device for displaying the blood enhancement image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images