Image processing methods, devices, equipment, and endoscopes for endoscopes

Abstract

This application relates to an image processing method, apparatus, image processing device, endoscope, and storage medium for an endoscope. The method includes: acquiring red narrowband images, green narrowband images, and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope; fusing the red, green, and blue narrowband images captured by each narrowband camera module to obtain a color fused image corresponding to each narrowband camera module; performing image enhancement processing on each color fused image; and outputting a standard display image corresponding to each narrowband camera module. This method can improve image display quality, facilitating disease detection by doctors using standard display images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, endoscope, and storage medium for an endoscope. Background Technology

[0002] With the development of medical technology, minimally invasive surgery has been widely used in clinical practice due to its advantages such as small incisions, less blood loss, less postoperative pain, and shorter hospitalization and recovery period. Endoscopes are responsible for displaying and guiding doctors in minimally invasive surgery.

[0003] In traditional technology, most endoscopes use white light camera modules to acquire images, process them using image processing equipment, and finally display the processed images on a display device. However, the quality of the processed images displayed by the display devices in traditional endoscopes is relatively low, which is not conducive to doctors' disease detection. Summary of the Invention

[0004] Therefore, it is necessary to provide an image processing method, apparatus, image processing device, endoscope, and storage medium for an endoscope that can improve the quality of displayed images, in order to address the aforementioned technical problems.

[0005] Firstly, an image processing method for an endoscope is provided, the image processing method comprising:

[0006] Acquire red narrowband images, green narrowband images, and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope;

[0007] The red, green, and blue narrowband images captured by each narrowband camera module are fused to obtain the color fused image corresponding to each narrowband camera module.

[0008] Image enhancement processing is performed on each color fusion image to output the standard display image corresponding to each narrowband camera module.

[0009] In one embodiment, the step of fusing the red narrowband image, green narrowband image, and blue narrowband image acquired by each narrowband camera module to obtain a color fused image corresponding to each narrowband camera module includes: performing image alignment processing on the red narrowband image, green narrowband image, and blue narrowband image acquired by each narrowband camera module to obtain each image to be fused corresponding to each narrowband camera module; and performing fusion processing on each image to be fused corresponding to each narrowband camera module to obtain a color fused image corresponding to each narrowband camera module.

[0010] In one embodiment, the step of performing image alignment processing on the red narrowband image, green narrowband image, and blue narrowband image acquired by each narrowband camera module to obtain each image to be fused corresponding to each narrowband camera module includes: performing low-pass filtering processing on the red narrowband image acquired by each narrowband camera module to obtain a first filtered image corresponding to each narrowband camera module; performing low-pass filtering processing on the green narrowband image acquired by each narrowband camera module to obtain a second filtered image corresponding to each narrowband camera module; performing low-pass filtering processing on the blue narrowband image acquired by each narrowband camera module to obtain a third filtered image corresponding to each narrowband camera module; and performing image alignment processing on the first filtered image, second filtered image, and third filtered image corresponding to each narrowband camera module to obtain each image to be fused corresponding to each narrowband camera module.

[0011] In one embodiment, the step of performing image enhancement processing on each color fusion image and outputting a standard display image corresponding to each narrowband camera module includes: performing single-frame denoising processing on the color fusion image corresponding to each narrowband camera module to obtain a denoised image corresponding to each narrowband camera module; performing image alignment processing on the first denoised image and the second denoised image in each denoised image set to obtain an alignment region for each dual-path image; the denoised image set includes denoised images corresponding to any two narrowband camera modules; performing image enhancement processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain a standard display image corresponding to each narrowband camera module; wherein, the first alignment region image is the image of the first denoised image within the corresponding alignment region; and the second alignment region image is the image of the second denoised image within the corresponding alignment region.

[0012] In one embodiment, the step of performing image enhancement processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain a standard display image corresponding to each narrowband camera module includes: determining a basic denoised image corresponding to each first alignment region based on each first alignment region image and the second alignment region image corresponding to the first alignment region image; and performing image enhancement processing on the corresponding first alignment region image and the corresponding second alignment region image using each basic denoised image based on an anti-sharpening mask algorithm to obtain a standard display image corresponding to each narrowband camera module.

[0013] In one embodiment, the step of determining the image base denoising map corresponding to each first alignment region based on each first alignment region image and the second alignment region image corresponding to the first alignment region image includes: performing weighted average processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain the image base denoising map corresponding to each first alignment region.

[0014] In one embodiment, the step of performing image enhancement processing on the corresponding first aligned region image and the corresponding second aligned region image using each basic denoised image to obtain the standard display image corresponding to each narrowband camera module includes: performing difference processing on each basic denoised image and the corresponding first aligned region image to obtain each first high-frequency component; superimposing each first high-frequency component onto the corresponding first aligned region image to obtain each first enhanced image; and performing stitching and restoration processing on each first enhanced image and the corresponding first residual image to obtain the standard display image corresponding to each narrowband camera module; wherein, the first residual image is the image of the first denoised image outside the corresponding aligned region.

[0015] In one embodiment, the step of performing single-frame denoising processing on the color fusion image corresponding to each narrowband camera module to obtain a denoised image corresponding to each narrowband camera module includes: performing discrete cosine transform on the color fusion image corresponding to each narrowband camera module to obtain each first frequency domain image; performing denoising processing on each first frequency domain image based on a frequency domain sparsity algorithm to obtain each second frequency domain image; and performing inverse discrete cosine transform on each second frequency domain image to obtain a denoised image corresponding to each narrowband camera module.

[0016] In one embodiment, the step of performing image alignment processing on the first and second denoised images in each set of denoised images to obtain the alignment region of each dual-path image includes: calculating the target displacement difference corresponding to each set of denoised images based on the first and second denoised images in each set of denoised images; the target displacement difference refers to the displacement difference between the first denoised image and the corresponding second denoised image; and determining the alignment region of the corresponding dual-path image based on the target displacement difference corresponding to each set of denoised images.

[0017] In one embodiment, the step of calculating the target displacement difference corresponding to each denoised image set based on the first and second denoised images in each denoised image set includes: determining reference points in each reference image; the reference image being either the first or second denoised image in each denoised image set; finding corresponding matching points in the corresponding target image based on each reference point; wherein, when the reference image is the first denoised image in the corresponding denoised image set, the target image is the second denoised image in the corresponding denoised image set; when the reference image is the second denoised image in the corresponding denoised image set, the target image is the first denoised image in the corresponding denoised image set; and calculating the target displacement difference corresponding to each denoised image set based on each reference point and the corresponding matching point.

[0018] Secondly, an image processing apparatus for an endoscope is provided, the image processing apparatus comprising:

[0019] The image acquisition module is used to acquire red narrowband images, green narrowband images and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope;

[0020] The image fusion module is used to fuse the red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module to obtain the color fused image corresponding to each narrowband camera module;

[0021] The image enhancement module is used to perform image enhancement processing on each color fusion image and output the standard display image corresponding to each narrowband camera module.

[0022] Thirdly, an image processing apparatus is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the above method embodiments.

[0023] Fourthly, an endoscope is provided, which includes at least two narrowband camera modules and an image processing device in the above-described image processing device embodiment; the narrowband camera modules are electrically connected to the image processing device and are used to acquire red narrowband images, green narrowband images and blue narrowband images.

[0024] In one embodiment, the endoscope further includes: a light source device for providing illumination to each narrowband camera module via a beam guide; and / or a display device electrically connected to the image processing device and for displaying a standard display image corresponding to each narrowband camera module.

[0025] In one embodiment, the narrowband camera module includes a first camera with a red filter, a second camera with a green filter, and a third camera with a blue filter; wherein the first camera is electrically connected to an image processing device and is used to acquire red narrowband images; the second camera is electrically connected to an image processing device and is used to acquire green narrowband images; and the third camera is electrically connected to an image processing device and is used to acquire blue narrowband images.

[0026] In one embodiment, the endoscope further includes a housing; each narrow-band camera module is symmetrically arranged on both sides of the center of the end face of the housing.

[0027] Fifthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described in the above method embodiments.

[0028] The aforementioned image processing method, apparatus, image processing device, endoscope, and storage medium for the endoscope acquire high-quality red, green, and blue narrowband images respectively through at least two narrowband camera modules. Then, the red, green, and blue narrowband images acquired by each narrowband camera module are fused to obtain a color fused image corresponding to each narrowband camera module. This avoids the color difference and purple fringing problems commonly found in white light images caused by white light camera modules and improves the image signal-to-noise ratio. Finally, image enhancement processing is performed on each color fused image to output a standard display image corresponding to each narrowband camera module, effectively distinguishing image noise from details, improving the clarity of the standard display image, and further improving image quality. This facilitates disease detection by doctors using the standard display image. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating an image processing method for an endoscope in one embodiment;

[0030] Figure 2 This is a flowchart illustrating the fusion processing steps in one embodiment;

[0031] Figure 3 This is a flowchart illustrating the steps of image alignment processing for a narrowband image in one embodiment.

[0032] Figure 4 This is a flowchart illustrating the steps of image enhancement processing for a color-fused image in one embodiment;

[0033] Figure 5 This is a flowchart illustrating the single-frame noise reduction process in one embodiment.

[0034] Figure 6 This is a flowchart illustrating the steps of image alignment processing for a denoised image in one embodiment.

[0035] Figure 7 This is a flowchart illustrating the target displacement difference calculation steps in one embodiment;

[0036] Figure 8 This is a flowchart illustrating the steps of image enhancement processing for an aligned region image in one embodiment;

[0037] Figure 9 This is a flowchart illustrating the steps of image enhancement processing of the aligned region image using each base denoised image in one embodiment.

[0038] Figure 10 This is a structural block diagram of an image processing device for an endoscope in one embodiment;

[0039] Figure 11This is an internal structural diagram of an image processing device in one embodiment;

[0040] Figure 12 This is a schematic diagram of the endoscope structure in one embodiment;

[0041] Figure 13 This is a schematic diagram of the structure of a narrowband camera module in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0044] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first resistor may be referred to as a second resistor, and similarly, a second resistor may be referred to as a first resistor. Both the first resistor and the second resistor are resistors, but they are not the same resistor.

[0045] It is understood that the term "connection" in the following embodiments should be understood as "electrical connection," "communication connection," etc., if the connected circuits, modules, units, etc., have electrical signal or data transmission with each other.

[0046] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” or “having,” etc., specify the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof.

[0047] This application provides an image processing method, apparatus, image processing device, endoscope, and storage medium for an endoscope. It avoids the common color difference and purple fringing problems in white light images caused by white light camera modules and improves the image signal-to-noise ratio. At the same time, it can effectively distinguish image noise from details, improve the clarity of the standard display image, and thus further improve the image quality, making it easier for doctors to perform disease detection using the standard display image.

[0048] In one embodiment, such as Figure 1 As shown, an image processing method for an endoscope is provided. This embodiment illustrates the application of this method to an image processing device. In this embodiment, the method includes steps 102 to 106.

[0049] Step 102: Acquire red narrowband image, green narrowband image and blue narrowband image respectively captured by at least two narrowband camera modules of the endoscope.

[0050] The endoscope is equipped with at least two narrowband camera modules, and any one of the narrowband camera modules in the endoscope will acquire the corresponding red narrowband image, green narrowband image and blue narrowband image; then, the image processing device can acquire the high-quality red narrowband image, green narrowband image and blue narrowband image acquired by the at least two narrowband camera modules of the endoscope respectively.

[0051] Step 104: The red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module are fused to obtain the color fused image corresponding to each narrowband camera module.

[0052] In this process, after the image processing device acquires the red, green, and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope, it can perform fusion processing on the red, green, and blue narrowband images captured by each narrowband camera module to obtain the color fused image corresponding to each narrowband camera module. This avoids the color difference and purple fringing problems commonly found in white light images caused by white light camera modules, and improves the image signal-to-noise ratio.

[0053] In a specific example, the endoscope is equipped with two narrowband camera modules: a first narrowband camera module and a second narrowband camera module. The image processing device acquires red, green, and blue narrowband images captured by the first narrowband camera module, and simultaneously acquires the same images from the second narrowband camera module. The image processing device then fuses the red, green, and blue narrowband images from the first narrowband camera module to obtain a color-fused image corresponding to the first narrowband camera module; similarly, it fuses the red, green, and blue narrowband images from the second narrowband camera module to obtain a color-fused image corresponding to the second narrowband camera module. This is merely a specific example; in actual applications, the settings can be flexibly configured according to user needs, and no limitations are imposed here.

[0054] In one embodiment, such as Figure 2 As shown, the steps of fusing the red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module to obtain the color fused image corresponding to each narrowband camera module include steps 201 and 202.

[0055] Step 201: Perform image alignment processing on the red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module to obtain the images to be fused corresponding to each narrowband camera module.

[0056] Since the positions of the red, green, and blue narrowband filters in each narrowband camera module are slightly different, the red, green, and blue narrowband images acquired by each narrowband camera module in the endoscope are aligned to obtain the images to be fused for each narrowband camera module.

[0057] In a specific example, the positions of the red, green, and blue narrowband filters in each narrowband camera module are slightly different. Therefore, the center positions of the red, green, and blue narrowband images captured by each narrowband camera module form an equilateral triangle. Based on the overlapping area of ​​the red, green, and blue narrowband images captured by each narrowband camera module, the corresponding red, green, and blue narrowband images are cropped to complete the image alignment process and obtain the images to be fused for each narrowband camera module. The above is only a specific example; in actual applications, the settings can be flexibly configured according to user needs, and no restrictions are imposed here.

[0058] In one embodiment, such as Figure 3As shown, the steps of performing image alignment processing on the red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module to obtain the images to be fused corresponding to each narrowband camera module include steps 301 to 304.

[0059] Step 301: Perform low-pass filtering on the red narrowband image acquired by each narrowband camera module to obtain the first filtered image corresponding to each narrowband camera module.

[0060] Step 302: Perform low-pass filtering on the green narrowband image acquired by each narrowband camera module to obtain the second filtered image corresponding to each narrowband camera module.

[0061] Step 303: Perform low-pass filtering on the blue narrowband image acquired by each narrowband camera module to obtain the third filtered image corresponding to each narrowband camera module.

[0062] Step 304: Perform image alignment processing on the first filtered image, the second filtered image, and the third filtered image corresponding to each narrowband camera module to obtain each image to be fused corresponding to each narrowband camera module.

[0063] The image processing device performs low-pass filtering on the red narrowband image acquired by each narrowband camera module to obtain a first filtered image corresponding to each narrowband camera module. Simultaneously, it performs low-pass filtering on the green narrowband image acquired by each narrowband camera module to obtain a second filtered image corresponding to each narrowband camera module. Furthermore, it performs low-pass filtering on the blue narrowband image acquired by each narrowband camera module to obtain a third filtered image corresponding to each narrowband camera module. Then, it performs image alignment processing on the first, second, and third filtered images corresponding to each narrowband camera module to obtain the images to be fused for each narrowband camera module. It is understood that the execution order of steps 301, 302, and 303 is not strictly limited; they can be executed simultaneously or in a certain order. In practical applications, this can be flexibly set according to user needs, and no restrictions are imposed here.

[0064] In this embodiment, the image quality of each image to be fused is further improved by performing low-pass filtering on the red narrowband image, green narrowband image and blue narrowband image acquired by each narrowband camera module.

[0065] Step 202: Perform fusion processing on each image to be fused corresponding to each narrowband camera module to obtain a color fused image corresponding to each narrowband camera module.

[0066] The image processing device uses an image fusion processing algorithm to perform fusion processing on each image to be fused corresponding to each narrowband camera module, thereby obtaining a color fused image corresponding to each narrowband camera module.

[0067] In this embodiment, by performing image alignment processing on the red, green, and blue narrowband images acquired by each narrowband camera module, images to be fused corresponding to each narrowband camera module are obtained. Then, the images to be fused corresponding to each narrowband camera module are fused to obtain color fused images corresponding to each narrowband camera module. This improves the fusion processing efficiency of the images to be fused and avoids the color difference and purple fringing problems commonly found in white light images caused by white light camera modules, thereby improving the image signal-to-noise ratio.

[0068] Step 106: Perform image enhancement processing on each color fusion image and output the standard display image corresponding to each narrowband camera module.

[0069] The image processing device enhances the fused color images obtained after fusion processing, outputting standard display images corresponding to each narrowband camera module. This effectively distinguishes image noise from details, improving the clarity of the standard display images. Furthermore, the display device can display high-quality 3D images based on the standard display images output by the image processing device.

[0070] Based on this, the image processing method for the endoscope described above acquires high-quality red, green, and blue narrowband images respectively from at least two narrowband camera modules of the endoscope. Then, the red, green, and blue narrowband images acquired by each narrowband camera module are fused to obtain a color fused image corresponding to each narrowband camera module. This avoids the color difference and purple fringing problems commonly found in white light images caused by white light camera modules and improves the image signal-to-noise ratio. Finally, image enhancement processing is performed on each color fused image to output a standard display image corresponding to each narrowband camera module, thereby effectively distinguishing image noise from details, improving the clarity of the standard display image, and further improving image quality. This makes it easier for doctors to perform disease detection using the standard display image.

[0071] In one embodiment, such as Figure 4 As shown, the steps of performing image enhancement processing on each color fusion image and outputting the standard display image corresponding to each narrowband camera module include steps 401 to 403.

[0072] Step 401: Perform single-frame denoising processing on the color fusion image corresponding to each narrowband camera module to obtain the denoised image corresponding to each narrowband camera module.

[0073] The image processing device can perform single-frame denoising on the color fusion image corresponding to each narrowband camera module based on a single-frame denoising algorithm to obtain the denoised image corresponding to each narrowband camera module.

[0074] In one embodiment, such as Figure 5 As shown, the steps of performing single-frame denoising processing on the color fusion image corresponding to each narrowband camera module to obtain the denoised image corresponding to each narrowband camera module include steps 501 to 503.

[0075] Step 501: Perform discrete cosine transform on the color fusion image corresponding to each narrowband camera module to obtain each first frequency domain image.

[0076] Step 502: Based on the frequency domain sparsity algorithm, denoise the first frequency domain images to obtain the second frequency domain images.

[0077] Step 503: Perform discrete cosine inverse transform on each second frequency domain image to obtain the denoised image corresponding to each narrowband camera module.

[0078] In this embodiment, the image processing device performs a discrete cosine transform on the color fusion image corresponding to each narrowband camera module in the spatial domain to obtain first frequency domain images. Then, based on a frequency domain sparsity algorithm, denoising is applied to each first frequency domain image to obtain second frequency domain images. Finally, an inverse discrete cosine transform is performed on each second frequency domain image to obtain the denoised image corresponding to each narrowband camera module. In this embodiment, the above steps further reduce the impact of noise on the color fusion image.

[0079] In a specific example, a discrete cosine transform is performed on the color fusion image corresponding to any narrowband camera module of the endoscope, thereby transforming the color fusion image corresponding to the narrowband camera module from the spatial domain to the frequency domain, obtaining the corresponding first frequency domain image. Then, the absolute value of the frequency domain coefficients corresponding to each pixel is determined based on the first frequency domain image. Next, iteratively checking whether the absolute value of the frequency domain coefficients corresponding to each pixel in the first frequency domain image is greater than a preset threshold; and, if the absolute value of the frequency domain coefficients corresponding to a pixel is greater than the preset threshold, the value and sign of the frequency domain coefficients corresponding to that pixel are maintained; simultaneously, if the absolute value of the frequency domain coefficients corresponding to a pixel is less than or equal to the preset threshold, the frequency domain coefficients corresponding to that pixel are set to 0; then, after comparing the absolute values ​​of the frequency domain coefficients corresponding to all pixels in the first frequency domain image with the preset threshold, the frequency domain sparsity process is completed, obtaining the corresponding second frequency domain image; finally, a discrete cosine inverse transform is performed on the second frequency domain image, that is, the second frequency domain image is transformed from the frequency domain to the spatial domain, thus obtaining the denoised image corresponding to the narrowband camera module, further reducing the impact of noise on the color fusion image. The above are just specific examples. In actual applications, the settings should be flexibly adjusted according to user needs, and no restrictions are imposed here.

[0080] Step 402: Perform image alignment processing on the first and second denoised images in each set of denoised images to obtain the alignment region of each dual-path image.

[0081] The denoised image set includes denoised images corresponding to any two narrowband camera modules, namely the first denoised image and the second denoised image. The image processing device performs image alignment processing on the first and second denoised images in each denoised image set based on an image alignment algorithm, thereby obtaining the alignment region of each dual-channel image.

[0082] In one embodiment, such as Figure 6 As shown, the steps of performing image alignment processing on the first and second denoised images in each set of denoised images to obtain the alignment region of each dual-path image include steps 601 and 602.

[0083] Step 601: Calculate the target displacement difference corresponding to each denoised image set based on the first and second denoised images in each denoised image set.

[0084] Step 602: Determine the alignment region of the corresponding dual-path image based on the target displacement difference corresponding to each set of denoised images.

[0085] Here, the target displacement difference refers to the displacement difference between the first denoised image and the corresponding second denoised image. Since there is a displacement difference between any two narrowband camera modules, the image processing device needs to calculate the target displacement difference for each denoised image set based on the first and second denoised images in each set. Then, the alignment region of the corresponding dual-channel image is determined based on the target displacement difference for each denoised image set. In this embodiment, by calculating the target displacement difference for each denoised image set, the alignment region of the corresponding dual-channel image is determined, thus improving the efficiency of image enhancement processing.

[0086] In one embodiment, such as Figure 7 As shown, the steps for calculating the target displacement difference corresponding to each denoised image set based on the first and second denoised images in each denoised image set include steps 701 to 703.

[0087] Step 701: Determine the reference points in each reference image.

[0088] The reference image is either the first denoised image or the second denoised image in each set of denoised images. The set of denoised images includes the first denoised image and the second denoised image. The image processing device can select the corresponding first denoised image or the second denoised image in each set of denoised images as the reference image and determine the reference points in each reference image.

[0089] Step 702: Find the corresponding matching point in the corresponding target image based on each reference point.

[0090] It is understandable that when the selected reference image is the first denoised image in the corresponding denoised image set, the image processing device will use the second denoised image in the corresponding denoised image set as the target image; and when the selected reference image is the second denoised image in the corresponding denoised image set, the image processing device will use the first denoised image in the corresponding denoised image set as the target image. Based on the determined reference points of the reference image, the image processing device can find matching points in the target image corresponding to the reference points of the reference image.

[0091] Step 703: Calculate the target displacement difference corresponding to each denoised image set based on each reference point and the corresponding matching point.

[0092] The image processing device calculates the target displacement difference corresponding to each set of denoised images by using the reference points of each reference image and the matching points on the corresponding target image.

[0093] In a specific example, the image processing device identifies a reference point (i1,j1) in the reference image and searches for a candidate matching point (i2,j2) in the corresponding target image based on the reference point (i1,j1). Then, it determines the neighborhood of the candidate matching point (i2,j2) centered on the candidate matching point (i2,j2) as target(i2,j2) and the neighborhood of the reference point (i1,j1) centered on the reference point (i1,j1) as reference(i1,j1), thereby calculating the SAD values ​​of target(i2,j2) and reference(i1,j1).

[0094] The expression for calculating SAD is:

[0095]

[0096] To reduce computation, the epipolar theorem can be used to determine that the candidate matching point (i2,j2) and the reference point (i1,j1) are on the same horizontal line, and the matching point (i3,j3) is within the range to the left of the corresponding position of the reference point (i1,j1). This defines the search range, and the candidate matching point (i2,j2) with the smallest SAD value within the search range is selected as the matching point (i3,j3). Finally, the target displacement difference corresponding to each denoised image set can be obtained by calculating the absolute value of the difference in the x-coordinates between the matching point (i3,j3) and the reference point (i1,j1). The above is only a specific example; in actual applications, the settings can be flexibly configured according to user needs, and no restrictions are imposed here.

[0097] In this embodiment, reference points in each reference image are determined; then, corresponding matching points are found in the corresponding target image based on each reference point; next, the target displacement difference corresponding to each set of denoised images can be accurately obtained by calculating based on each reference point and the corresponding matching point, which improves the convenience of image enhancement processing.

[0098] Step 403: Perform image enhancement processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain the standard display image corresponding to each narrowband camera module.

[0099] The first aligned region image is the image of the first denoised image within its corresponding aligned region; the second aligned region image is the image of the second denoised image within its corresponding aligned region. Based on an image enhancement algorithm, the image processing device can perform image enhancement processing on each of the first aligned region images and the corresponding second aligned region images to obtain the standard display image corresponding to each narrowband camera module.

[0100] In this embodiment, single-frame denoising processing is performed on the color fusion image corresponding to each narrowband camera module to obtain a denoised image for each narrowband camera module. Then, image alignment processing is performed on the first and second denoised images in each set of denoised images to obtain the alignment region of each dual-channel image. Finally, image enhancement processing is performed on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain the standard display image corresponding to each narrowband camera module. This effectively distinguishes image noise from details and improves the clarity of the standard display image.

[0101] In one embodiment, such as Figure 8 As shown, the steps of performing image enhancement processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain the standard display image corresponding to each narrowband camera module include steps 801 and 802.

[0102] Step 801: Determine the basic denoised image corresponding to each first alignment region based on each first alignment region image and the second alignment region image corresponding to the first alignment region image.

[0103] The image processing device can determine the basic denoised image corresponding to each first alignment region based on each first alignment region image and the second alignment region image corresponding to the first alignment region image.

[0104] In one embodiment, the step of determining the basic denoised image corresponding to each first aligned region based on each first aligned region image and the second aligned region image corresponding to the first aligned region image includes: performing a weighted average processing on each first aligned region image and the second aligned region image corresponding to the first aligned region image to obtain the basic denoised image corresponding to each first aligned region. Therefore, the convenience and efficiency of image enhancement processing are improved.

[0105] Step 802: Based on the anti-sharpening mask algorithm, image enhancement processing is performed on the corresponding first alignment region image and the corresponding second alignment region image using each basic denoised image to obtain the standard display image corresponding to each narrowband camera module.

[0106] The image processing device, based on the anti-sharpening mask algorithm, can use the basic denoised images to perform image enhancement processing on the corresponding first aligned region image and the corresponding second aligned region image, thereby obtaining the standard display image corresponding to each narrowband camera module.

[0107] It is understandable that the desharpening mask algorithm refers to performing point-by-point difference operations between the image to be processed and the image to be processed after low-pass filtering, then multiplying by a correction factor and summing with the image to be processed, in order to improve the high-frequency components in the image and enhance the image contour.

[0108] In this embodiment, a base denoised image corresponding to each first alignment region is determined based on each first alignment region image and a second alignment region image corresponding to the first alignment region image. Then, based on the anti-sharpening mask algorithm, the corresponding first alignment region image and the corresponding second alignment region image are processed by each base denoised image to obtain the standard display image corresponding to each narrowband camera module, thereby achieving the purpose of improving the high-frequency components in the image and enhancing the image contour.

[0109] In one embodiment, such as Figure 9 As shown, the steps of performing image enhancement processing on the corresponding first alignment region image and the corresponding second alignment region image using each basic denoised image to obtain the standard display image corresponding to each narrowband camera module include steps 901 to 903.

[0110] Step 901: Perform interpolation processing on each base denoised image and the corresponding first aligned region image to obtain each first high-frequency component.

[0111] Step 902: Superimpose each first high-frequency component onto the corresponding first aligned region image to obtain each first enhanced image.

[0112] Step 903: Perform stitching and restoration processing on each first enhanced image and the corresponding first remaining image to obtain the standard display image corresponding to each narrowband camera module.

[0113] The first residual image is the image of the first denoised image outside the corresponding alignment region. The image processing device performs difference processing on each basic denoised image and the corresponding first alignment region image to obtain each first high-frequency component, thus realizing high-frequency extraction of the image of the corresponding first alignment region; then, each first high-frequency component is superimposed on the corresponding first alignment region image to obtain each first enhanced image; next, stitching and restoration processing is performed on each first enhanced image and the corresponding first residual image to obtain the standard display image corresponding to each narrowband camera module.

[0114] In this embodiment, the standard display images corresponding to each narrowband camera module are obtained through the above steps, thereby achieving the purpose of improving image sharpness and enhancing image contours.

[0115] In one embodiment, such as Figure 9 As shown, the step of performing image enhancement processing on the corresponding first alignment region image and the corresponding second alignment region image using each basic denoised image to obtain the standard display image corresponding to each narrowband camera module also includes steps 904 to 906.

[0116] Step 904: Perform interpolation processing on each base denoised image and the corresponding second aligned region image to obtain each second high-frequency component.

[0117] Step 905: Superimpose each second high-frequency component onto the corresponding second aligned region image to obtain each second enhanced image.

[0118] Step 906: Perform stitching and restoration processing on each second enhanced image and the corresponding second remaining image to obtain the standard display image corresponding to each narrowband camera module.

[0119] The second residual image is the image of the second denoised image outside the corresponding alignment region. The image processing device performs difference processing on each basic denoised image and the corresponding second alignment region image to obtain each second high-frequency component, thus realizing high-frequency extraction of the image of the corresponding first alignment region; then, each second high-frequency component is superimposed on the corresponding second alignment region image to obtain each second enhanced image; next, stitching and restoration processing is performed on each second enhanced image and the corresponding second residual image to obtain the standard display image corresponding to each narrowband camera module.

[0120] In this embodiment, the standard display images corresponding to each narrowband camera module are obtained through the above steps, thereby achieving the purpose of improving the high-frequency components in the image and enhancing the image contour.

[0121] It should be understood that, although Figure 1-9 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1-9 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0122] In one embodiment, such as Figure 10 As shown, an image processing device for an endoscope is provided, comprising: an image acquisition module 1010, an image fusion module 1020, and an image enhancement module 1030, wherein:

[0123] The image acquisition module 1010 is used to acquire red narrowband images, green narrowband images, and blue narrowband images respectively acquired by at least two narrowband camera modules of the endoscope; the image fusion module 1020 is used to fuse the red narrowband images, green narrowband images, and blue narrowband images acquired by each narrowband camera module to obtain a color fused image corresponding to each narrowband camera module; the image enhancement module 1030 is used to perform image enhancement processing on each color fused image and output a standard display image corresponding to each narrowband camera module.

[0124] Specific limitations regarding the image processing device for endoscopes can be found in the limitations of the image processing methods for endoscopes described above, and will not be repeated here. Each module in the aforementioned image processing device for endoscopes can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0125] In one embodiment, an image processing device 1100 is provided. The image processing device 1100 may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown. The image processing device 1100 includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor of the image processing device 1100 provides computing and control capabilities. The memory of the image processing device 1100 includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the image processing device 1100 is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an endoscopic image processing method. The display screen of the image processing device 1100 can be a liquid crystal display screen or an e-ink display screen. The input device of the image processing device 1100 can be a touch layer covering the display screen, buttons, a trackball, or a touchpad located on the casing of the image processing device 1100, or an external keyboard, touchpad, or mouse, etc.

[0126] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the image processing device 1100 to which the present application is applied. The specific image processing device 1100 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0127] In one embodiment, an image processing device 1100 is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the above method embodiments.

[0128] In one embodiment, such as Figure 12 As shown, an endoscope is provided, which includes at least two narrowband camera modules 1210 and an image processing device 1100 in the above-described image processing device embodiment.

[0129] The narrowband camera module 1210 is electrically connected to the image processing device 1100 and is used to acquire red narrowband images, green narrowband images, and blue narrowband images. This improves the convenience of the endoscope.

[0130] In one embodiment, such as Figure 12 As shown, the narrowband camera module 1210 includes a first camera 1211 with a red filter, a second camera 1212 with a green filter, and a third camera 1213 with a blue filter. The first camera 1211 is electrically connected to an image processing device and is used to acquire red narrowband images; the second camera 1212 is electrically connected to the image processing device and is used to acquire green narrowband images; and the third camera 1213 is electrically connected to the image processing device and is used to acquire blue narrowband images. This improves the convenience of endoscopy.

[0131] In a specific example, the first camera 1211 includes a lens, a red filter, a prism, a CMOS sensor, and an AD converter. Optical fiber enters the first camera 1211 through the lens. After processing by the red filter and prism, the effective light signal is transmitted to the CMOS sensor. The CMOS sensor converts the light signal into an analog electrical signal, which is then converted into a digital electrical signal by the AD converter and transmitted to an image processing device. This is merely a specific example; in actual applications, the configuration can be flexibly adjusted according to user needs, and no limitations are imposed here.

[0132] In a specific example, such as Figure 13As shown, the endoscope body includes a narrowband camera module 1210, which comprises a first objective lens 12101, a second objective lens 12102, a third objective lens 102103, a red narrowband filter 12104, a blue narrowband filter 12105, a green narrowband filter 12106, a first prism 12107, a second prism 12108, a third prism 102109, a first image sensor 12110, a second image sensor 12111, a third image sensor 12112, and a beam guide 2000. In this configuration, the first objective lens 12101, the red narrowband filter 12104, the first prism 12107, and the first image sensor 12110 are arranged in a horizontal sequence; the second objective lens 12102, the blue narrowband filter 12105, the second prism 12108, and the second image sensor 12111 are arranged in a horizontal sequence; and the third objective lens 12103, the green narrowband filter 12106, the third prism 12109, and the second image sensor 12112 are arranged in a horizontal sequence. This is merely a specific example; in actual applications, the configuration can be flexibly adjusted according to user needs, and no restrictions are imposed here.

[0133] In one embodiment, such as Figure 12 As shown, the endoscope also includes a light source device 1220 and / or a display device 230.

[0134] The light source device 1220 provides illumination to each narrowband camera module 1210 via the beam guide 2000; the display device 1230 is electrically connected to the image processing device 1100 and displays the standard display image corresponding to each narrowband camera module 1210. It is understood that the display device 1230 can display the standard display image corresponding to each narrowband camera module 1210, i.e., a three-dimensional image, based on the standard display image output by the image processing device 1100. Therefore, the convenience of the endoscope is improved. In a specific example, the display device can be a three-dimensional display or a two-dimensional display, depending on the user's needs, and is not limited here.

[0135] In one embodiment, such as Figure 12 As shown, the endoscope also includes a housing 1240; each narrow-band camera module 1210 is symmetrically arranged on both sides of the center of the end face of the housing 1240. This improves the convenience of the endoscope. In a specific example, the housing 1240 may be, but is not limited to, a cylinder, and can be flexibly configured according to user needs; no limitation is imposed here.

[0136] In one embodiment, such as Figure 12As shown, at least two light-transmitting holes 1241 are symmetrically arranged on both sides of the center of the end face of the housing 1240; the light-transmitting holes 1241 are used to set the light guide 2000; wherein, the light guide 2000 is used to connect the light source device 1220. Therefore, the convenience of the endoscope is improved.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described in the above method embodiments.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image processing method for an endoscope, characterized in that, The image processing method includes: Acquire red narrowband images, green narrowband images, and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope; The red narrowband image, the green narrowband image, and the blue narrowband image acquired by each of the narrowband camera modules are fused to obtain a color fused image corresponding to each of the narrowband camera modules; Image enhancement processing is performed on each of the color fusion images to output the standard display image corresponding to each of the narrowband camera modules; The step of performing image enhancement processing on each of the color fusion images and outputting the standard display image corresponding to each of the narrowband camera modules includes: Perform single-frame denoising processing on the color fusion image corresponding to each of the narrowband camera modules to obtain the denoised image corresponding to each of the narrowband camera modules; The step of performing single-frame denoising processing on the color fusion image corresponding to each of the narrowband camera modules to obtain the denoised image corresponding to each narrowband camera module includes: Discrete cosine transform is performed on the color fusion image belonging to the spatial domain corresponding to each of the narrowband camera modules to obtain each first frequency domain image; Based on the frequency domain sparsity algorithm, each of the first frequency domain images is denoised to obtain each of the second frequency domain images. Perform a discrete cosine inverse transform on each of the second frequency domain images to obtain a denoised image belonging to the spatial domain corresponding to each of the narrowband camera modules.

2. The image processing method according to claim 1, characterized in that, The step of fusing the red narrowband image, the green narrowband image, and the blue narrowband image acquired by each of the narrowband camera modules to obtain a color fused image corresponding to each narrowband camera module includes: Image alignment processing is performed on the red narrowband image, the green narrowband image, and the blue narrowband image acquired by each of the narrowband camera modules to obtain each image to be fused corresponding to each narrowband camera module; The images to be fused corresponding to each of the narrowband camera modules are fused to obtain the color fused images corresponding to each narrowband camera module.

3. The image processing method according to claim 2, characterized in that, The step of performing image alignment processing on the red narrowband image, green narrowband image, and blue narrowband image acquired by each of the narrowband camera modules to obtain the images to be fused corresponding to each narrowband camera module includes: The red narrowband image acquired by each of the narrowband camera modules is subjected to low-pass filtering to obtain the first filtered image corresponding to each narrowband camera module; The green narrowband image acquired by each of the narrowband camera modules is subjected to low-pass filtering to obtain the second filtered image corresponding to each narrowband camera module; The blue narrowband image acquired by each of the narrowband camera modules is subjected to low-pass filtering to obtain the third filtered image corresponding to each narrowband camera module; Image alignment processing is performed on the first filtered image, the second filtered image, and the third filtered image corresponding to each narrowband camera module to obtain each image to be fused corresponding to each narrowband camera module.

4. The image processing method according to claim 1, characterized in that, The step of performing image enhancement processing on each of the color fusion images and outputting the standard display image corresponding to each of the narrowband camera modules further includes: Image alignment processing is performed on the first and second denoised images in each denoised image set to obtain the alignment region of each dual-channel image; the denoised image set includes the denoised images corresponding to any two of the narrowband camera modules; Image enhancement processing is performed on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain a standard display image corresponding to each narrowband camera module; wherein, the first alignment region image is the image of the first denoised image within the corresponding alignment region; and the second alignment region image is the image of the second denoised image within the corresponding alignment region.

5. The image processing method according to claim 4, characterized in that, The step of performing image enhancement processing on each first alignment region image and the second alignment region image corresponding to the first alignment region image to obtain the standard display image corresponding to each narrowband camera module includes: Based on each of the first alignment region images and the second alignment region images corresponding to the first alignment region images, determine the image base denoising map corresponding to each of the first alignment regions; Based on the anti-sharpening mask algorithm, the corresponding first alignment region image and the corresponding second alignment region image are enhanced using the base denoised images to obtain the standard display image corresponding to each narrowband camera module.

6. The image processing method according to claim 5, characterized in that, The step of determining the image base denoised map corresponding to each first alignment region based on each first alignment region image and the second alignment region image corresponding to the first alignment region image includes: A weighted average is performed on each of the first aligned region images and the second aligned region images corresponding to the first aligned region images to obtain the basic denoised image of each image corresponding to the first aligned region.

7. The image processing method according to claim 5, characterized in that, The step of performing image enhancement processing on the corresponding first aligned region image and the corresponding second aligned region image using the base denoised images to obtain the standard display image corresponding to each narrowband camera module includes: The difference processing is performed on each of the base denoised images and the corresponding first aligned region images to obtain each first high frequency component; Each of the first high-frequency components is superimposed onto the corresponding first aligned region image to obtain each first enhanced image; Based on each of the first enhanced images and the corresponding first remaining images, a stitching and restoration process is performed to obtain the standard display image corresponding to each of the narrowband camera modules; wherein, the first remaining image is the image of the first denoised image outside the corresponding alignment area.

8. The image processing method according to claim 4, characterized in that, The step of performing image alignment processing on the first denoised image and the second denoised image in each denoised image set to obtain the aligned region of each dual-path image includes: The target displacement difference is calculated based on the first denoised image and the second denoised image in each of the denoised image sets; the target displacement difference refers to the displacement difference between the first denoised image and the corresponding second denoised image. The alignment region of the corresponding dual-path image is determined based on the target displacement difference corresponding to each of the denoised image sets.

9. The image processing method according to claim 8, characterized in that, The step of calculating the target displacement difference corresponding to each denoised image set based on the first denoised image and the second denoised image in each denoised image set includes: Determine reference points in each reference image; the reference image is either the first denoised image or the second denoised image in each set of denoised images. Find corresponding matching points in the corresponding target image based on each of the reference points; wherein, when the reference image is the first denoised image in the corresponding denoised image set, the target image is the second denoised image in the corresponding denoised image set; when the reference image is the second denoised image in the corresponding denoised image set, the target image is the first denoised image in the corresponding denoised image set. The target displacement difference corresponding to each set of denoised images is obtained by calculating based on each reference point and the corresponding matching point.

10. An image processing device for an endoscope, characterized in that, The image processing device includes: The image acquisition module is used to acquire red narrowband images, green narrowband images and blue narrowband images respectively captured by at least two narrowband camera modules of the endoscope; The image fusion module is used to perform fusion processing on the red narrowband image, the green narrowband image and the blue narrowband image acquired by each of the narrowband camera modules to obtain a color fused image corresponding to each of the narrowband camera modules; The image enhancement module is used to perform image enhancement processing on each of the color fusion images and output the standard display image corresponding to each of the narrowband camera modules; The image enhancement module is further used for: Perform single-frame denoising processing on the color fusion image corresponding to each of the narrowband camera modules to obtain the denoised image corresponding to each of the narrowband camera modules; The step of performing single-frame denoising processing on the color fusion image corresponding to each of the narrowband camera modules to obtain the denoised image corresponding to each narrowband camera module includes: Discrete cosine transform is performed on the color fusion image belonging to the spatial domain corresponding to each of the narrowband camera modules to obtain each first frequency domain image; Based on the frequency domain sparsity algorithm, each of the first frequency domain images is denoised to obtain each of the second frequency domain images. Perform a discrete cosine inverse transform on each of the second frequency domain images to obtain a denoised image belonging to the spatial domain corresponding to each of the narrowband camera modules.

11. An image processing device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. An endoscope, characterized in that, The endoscope includes at least two narrowband camera modules and the image processing device as described in claim 11; the narrowband camera modules are electrically connected to the image processing device and are used to acquire the red narrowband image, the green narrowband image and the blue narrowband image.

13. The endoscope according to claim 12, characterized in that, The endoscope also includes: A light source device for providing illumination to each of the narrowband camera modules via a beam guide; and / or, The display device is electrically connected to the image processing device and is used to display the standard display image corresponding to each of the narrowband camera modules.

14. The endoscope according to claim 12, characterized in that, The narrowband camera module includes a first camera with a red filter, a second camera with a green filter, and a third camera with a blue filter; wherein, the first camera is electrically connected to the image processing device and is used to acquire red narrowband images; the second camera is electrically connected to the image processing device and is used to acquire green narrowband images; and the third camera is electrically connected to the image processing device and is used to acquire blue narrowband images.

15. The endoscope according to claim 14, characterized in that, The endoscope also includes a housing; each of the narrow-band camera modules is symmetrically arranged on both sides of the center of the end face of the housing.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images