Endoscope color crosstalk suppression method and endoscope

Abstract

The invention belongs to the technical field of endoscope imaging, and provides an endoscope color crosstalk suppression method and an endoscope, and the method comprises the steps: obtaining an original endoscope image collected by an image sensor; identifying target object parameters in the original endoscope image, wherein the target object parameters comprise a target color channel and a target size parameter corresponding to a target object; determining a target fuzzy convolution kernel corresponding to the target object parameter according to a mapping relation between a pre-calibrated calibration target object parameter and a calibration fuzzy convolution kernel, the target fuzzy convolution kernel represents a combination of an optical diffusion parameter corresponding to a target size parameter of the target object under the target color channel and a sensor crosstalk parameter under the target channel; and performing deconvolution processing on the original endoscope image based on the target fuzzy convolution kernel to obtain an endoscope imaging image. The problems of tissue boundary blurring, tissue color distortion and the like caused by optical diffusion and image sensor crosstalk during narrow-band imaging are solved, so that the tissue color change is displayed more accurately.

Description

Technical Field

[0001] The present application relates to the technical field of medical endoscopic imaging, and in particular to a method for suppressing endoscopic color crosstalk and an endoscope. Background Art

[0002] Endoscopic narrowband imaging utilizes a narrowband wavelength as the illumination source, combining the absorption characteristics of hemoglobin and the ability of light to penetrate tissue. This improves the contrast between blood vessels and tissue, enhancing image visibility. CMOS image sensors offer the advantages of low power consumption, low heat generation, high integration, and miniaturization, making them the mainstream in endoscopic imaging systems. However, the wide spectral range of the light-sensing elements of CMOS image sensors results in a single wavelength of light being detected simultaneously in multiple color channels (R, G, and B). Therefore, when using CMOS image sensors for multispectral narrowband imaging, the simultaneous illumination of multiple narrowband lights causes unintended interference between sensor pixels, resulting in unintended signal overlap in the images of each channel. Narrowband images manifest as color bleeding at high-contrast edges and blurred tissue boundaries. In actual endoscopic imaging, edge blurring is the result of two factors: optical diffusion and CMOS image sensor crosstalk. These blurred tissue boundaries and color distortion caused by optical diffusion and image sensor crosstalk reduce tissue differentiation in narrowband images, making it difficult to accurately display tissue color variations.

[0003] Therefore, how to improve the tissue differentiation of narrow-band light images based on COMS image sensors and accurately display tissue color changes is an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present application provide an endoscope color crosstalk suppression method and an endoscope, which are used to solve the following technical problem in the prior art: how to improve the tissue differentiation of narrow-band light images based on COMS image sensors.

[0005] According to a first aspect, the present application provides a method for suppressing color crosstalk in an endoscope, comprising: Acquiring an original endoscopic image captured by an image sensor; Identifying target object parameters in the original endoscopic image, the target object parameters including target color channels and target size parameters corresponding to the target object; Determining a target blur convolution kernel corresponding to the target object parameter according to a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target color channel; Deconvolution processing is performed on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

[0006] In one embodiment, performing crosstalk suppression processing on the original endoscopic image based on the crosstalk parameter to obtain an endoscopic imaging image includes: determining a target area where the target object is located in the original endoscopic image; Performing a deconvolution operation on the target region color channels using the target blurred convolution kernel to obtain a crosstalk suppressed image; The crosstalk suppression image is fused with the original endoscopic image to obtain the endoscopic imaging image.

[0007] In one embodiment, the calibration method of the mapping relationship includes: Acquire a crosstalk calibration image, wherein the crosstalk calibration image has a calibration target object; Constructing a line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image; The calibration target object in each calibration color channel in the crosstalk calibration image is fitted using a Gaussian distribution model to obtain optical diffusion parameters corresponding to different calibration size parameters of the calibration target object in different calibration color channels and sensor crosstalk parameters in different calibration color channels; The optical diffusion parameter and the sensor crosstalk parameter are combined to construct a mapping relationship between the calibration target object parameter and the fuzzy convolution kernel.

[0008] In one embodiment, the crosstalk calibration image includes an optical crosstalk calibration image, which is a bionic target image of multiple calibration color channels collected by a calibration image sensor under illumination of a single narrow-band light source corresponding to the calibration color channel; Constructing the line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image includes: Sampling calibration objects of different calibration size parameters in the optical crosstalk image of each calibration color channel respectively to obtain first sampling points of the calibration objects of different calibration size parameters in each calibration color channel, wherein the number of the first sampling points is inversely correlated with the value of the calibration size parameter; Extracting a first point spread function of the calibration target object corresponding to each calibration size parameter in each calibration color channel based on the first sampling point corresponding to each calibration size parameter; A first line spread function is constructed based on the first point spread function.

[0009] In one embodiment, the crosstalk calibration image includes sensor crosstalk calibration images under multiple calibration color channels; Constructing the line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image includes: Sampling the calibration target in the sensor crosstalk calibration image under each calibration color channel respectively to obtain a second sampling point of the calibration target under each calibration color channel; extracting a second point spread function of the calibration target object in each calibration color channel for the second sampling point; A second line spread function is constructed based on the second point spread function.

[0010] In one embodiment, fitting the calibration target object in each calibration color channel in the crosstalk calibration image using a Gaussian distribution model includes: Fitting the first line spread functions using a first Gaussian distribution model to obtain optical diffusion parameters corresponding to different size parameters under a calibration color channel; Establishing a first mapping table of optical diffusion parameters related to calibrated color channels and calibrated size parameters; Fitting the second line spread functions using a second Gaussian distribution model to obtain sensor crosstalk parameters corresponding to the calibration color channels; A second mapping table for calibrating sensor crosstalk parameters related to the color channels is established.

[0011] In one embodiment, the method for acquiring the sensor crosstalk calibration image includes: Acquire a mixed calibration color channel bionic target image of the calibration image sensor set under simultaneous illumination by all narrow-band light sources corresponding to all the calibration color channels; merging all the optical crosstalk calibration images into a composite image; Subtracting the mixed calibration color channel bionic target image from the synthesized image to obtain a mixed calibration color channel sensor crosstalk image; Channel separation is performed on the mixed calibration color channel sensor crosstalk image to obtain the sensor crosstalk calibration images under multiple calibration color channels.

[0012] In one embodiment, the deconvolution method includes but is not limited to one of direct inverse filtering, Wiener filtering, iterative optimization, regularization, and blind deconvolution.

[0013] In one embodiment, identifying target object parameters in the original endoscopic image includes: The target object is identified in the original endoscopic image by the spectral absorption difference between the target biological tissue corresponding to the target object and the surrounding tissue.

[0014] According to a second aspect, an embodiment of the present application provides an endoscope color crosstalk suppression device, comprising: An acquisition module, used for acquiring the original endoscopic image collected by the image sensor; an identification module, configured to identify target object parameters in the original endoscopic image, wherein the target object parameters include a target color channel and a target size parameter corresponding to the target object; a determination module, configured to determine a target blur convolution kernel corresponding to the target object parameter based on a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target color channel; The crosstalk suppression module is used to perform deconvolution processing on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

[0015] According to a third aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for suppressing endoscopic color crosstalk as described in any one of the first aspects above.

[0016] According to a fourth aspect, the present application provides an endoscope, characterized in that it includes an image sensor, an illumination light source and a controller, wherein the controller includes one or more processors, one or more memories, and one or more computer program instructions, and when the computer program instructions are executed by the processor, the endoscope color crosstalk suppression method described in any one of the above-mentioned first aspects is implemented.

[0017] The present invention proposes a method for suppressing endoscopic color crosstalk, which obtains an original endoscopic image collected by an image sensor; identifies target object parameters in the original endoscopic image, wherein the target object parameters include a target color channel and a target size parameter corresponding to the target object; determines a target fuzzy convolution kernel corresponding to the target object parameters based on a mapping relationship between pre-calibrated calibrated target object parameters and a calibrated fuzzy convolution kernel, wherein the target fuzzy convolution kernel characterizes a combination of an optical diffusion parameter corresponding to the target size parameter of the target object under the target color channel and a sensor crosstalk parameter under the target color channel; and deconvolution is performed on the original endoscopic image based on the target fuzzy convolution kernel to obtain an endoscopic imaging image. After acquiring the original endoscopic image, the target object and target object parameters are identified in the original endoscopic image. The target fuzzy convolution kernel corresponding to the combination of the optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and the sensor crosstalk parameter in the target channel is determined through the mapping relationship between the pre-calibrated calibration target object parameters and the calibration fuzzy convolution kernel. The target fuzzy convolution kernel is used to perform a deconvolution operation on the original endoscopic image to eliminate crosstalk in the image by tissue and region. This solves the problems of blurred tissue boundaries and tissue color distortion caused by optical diffusion and image sensor crosstalk in narrowband imaging, thereby more accurately displaying tissue color changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for suppressing color crosstalk in an endoscope provided in some embodiments of the present application; Figure 2 A mapping relationship calibration method provided for some embodiments of the present application; Figure 3 A schematic diagram of a blood vessel sampling method provided in some embodiments of the present application; Figure 4 A schematic structural diagram of an endoscope color crosstalk suppression device provided in some embodiments of the present application; Figure 5 Schematic diagram of a controller in an endoscope provided for some embodiments of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] Explanation of terms: Sensor crosstalk: The band crosstalk of a CMOS image sensor refers to the phenomenon in which light signals of different color channels (such as red, green, and blue) overlap due to the filter transmission range overlap, charge leakage of adjacent pixels, or oblique incidence of light penetrating adjacent pixels, resulting in color aliasing, blurred details, or a decrease in signal-to-noise ratio. This can be suppressed by optimizing the filter design, adding microlenses, or improving the pixel isolation process.

[0021] Endoscope: An endoscope is an optical instrument consisting of a cold light source lens, fiber optic conductors, an image transmission system, and a screen display system. It is usually inserted into the body through natural orifices or small incisions. During use, the doctor uses external manipulation to guide the endoscope into the area to be examined, allowing direct detection of changes in the relevant area and, in conjunction with surgical instruments, necessary diagnosis and treatment. The quality of endoscopic imaging directly affects the doctor's user experience, the accuracy and efficiency of exploration, and also marks the level of development of endoscopic technology.

[0022] Spectral properties: Biological tissue exhibits distinct absorption and reflection characteristics at different wavelengths, known as spectral properties. In medicine, the optical properties of biological tissue (absorption, scattering, and reflection) are used for imaging, primarily through the following approaches: absorption-based techniques (such as near-infrared spectral imaging for monitoring blood oxygenation and photoacoustic imaging for tumor detection); scattering / reflection-based techniques (such as optical coherence tomography (OCT) for imaging micron-scale structures and confocal microscopy for observing cell morphology); and multimodal fusion (such as combined photoacoustic and ultrasound imaging). These methods analyze the characteristic responses of tissue at specific wavelengths (such as hemoglobin absorption peaks and collagen scattering signals) to enable macroscopic to microscopic disease diagnosis (cancer detection, vascular imaging), surgical navigation, and physiological monitoring, combining both functional and structural information.

[0023] Narrowband imaging: Narrowband imaging is an optical imaging technique that uses one or more narrowband light sources as illumination to enhance the visibility of mucosal vessels, lesions, and inflammatory areas. Common narrowband imaging modalities include NBI (Narrowband Imaging), BLI (Blue Laser Imaging), and LCI (Linked Color Imaging). Because narrowband imaging only covers a portion of the visible spectrum or alters the proportion of white light, the resulting image color can differ significantly from white light imaging.

[0024] Point spread function: The point spread function (PSF) describes the response of an imaging system to a point source (object). A general term for PSF is system response, and the PSF is the impulse response of a focusing optical system. In most cases, the PSF can be thought of as an extended patch in the image that represents the unresolved object. Functionally, the PSF is a spatial expression of the transfer function of the imaging system. The PSF is an important concept that is used in Fourier optics, astronomical imaging, medical imaging, electron microscopy, and other imaging techniques such as 3D microscopy and fluorescence microscopy. The degree of spread (blurring) of a point object is a measure of the quality of an imaging system.

[0025] The present application proposes a method for suppressing color crosstalk in an endoscope, which is applied to an endoscope. The endoscope includes a CMOS image sensor for imaging; an illumination light source, which can specifically be composed of 5 LEDs such as a blue-violet light source, a green light source, an amber light source, and a red light source. The 5 LEDs can synthesize white light or other multi-spectral mixed light of any other combination. The light source illumination combination adopted in the present application is a narrow-band blue-violet light source, a narrow-band blue light source, a narrow-band green light source, a narrow-band amber light source, and a narrow-band red light source. Specifically, the narrow-band blue-violet light source has a center wavelength of 415±10nm, the narrow-band blue light source has a center wavelength of 460±10nm, the narrow-band green light source has a center wavelength of 540±15nm, the narrow-band amber light source has a center wavelength of 600±15nm, and the narrow-band red light source has a center wavelength of 630±10nm. The controller is used to execute the method for suppressing color crosstalk in an endoscope. For details, see Figure 1 As shown, the following steps are included: S10. Obtain the original endoscopic image captured by the image sensor. Under the illumination of the illumination light source, the image sensor of the endoscope captures the image of human tissue containing the target object as the original endoscopic image. The target object can be tissues such as blood vessels, mucosa, tumors, etc. In the following embodiments, blood vessels are used as an example. The image sensor adopts a CMOS image sensor. The sensory unit in the CMOS image sensor can accept a wide spectral range, resulting in a single wavelength light signal being detected simultaneously in multiple color channels. Therefore, when using a CMOS image sensor camera for multi-spectral narrowband light imaging, multiple narrowband lights are irradiated at the same time, and the reflected light causes unexpected interference between the sensor pixels, resulting in unexpected signal overlap in the images of each channel. There is image sensor crosstalk in the original endoscopic image. At the same time, in the original endoscopic image, there is also optical crosstalk caused by optical diffusion on the target object.

[0026] S20. Identify target object parameters in the raw endoscopic image, where the target object parameters include a target color channel and target size parameters corresponding to the target object. In this embodiment, after acquiring the raw endoscopic image, the target object in the raw endoscopic image is first identified. For example, the target object can be identified in the raw endoscopic image based on the spectral absorption difference between the target biological tissue corresponding to the target object and surrounding tissue.

[0027] For example, if the target object is a blood vessel, the blood vessel can be identified based on the hemoglobin content. For example, hemoglobin is an important protein in human red blood cells. Blood vessels are rich in hemoglobin, so hemoglobin can be used as a reference factor to identify blood vessels.

[0028] Hemoglobin has strong absorption peaks near 415nm and 540nm, and the absorption difference is greater than that in mucosal tissue, that is, Therefore, the above information can be used to mark the location of blood vessels in the image. When the illumination light source has the above two light sources, the main channel images of the above two wavelengths can be used for processing, which are recorded as 、 . Preset vessel marking parameters for: ; in, and To adjust the parameters, and The value of When presetting the vessel marking parameters The pixel value in is greater than the set threshold When , the area is considered to be the vascular part.

[0029] In one embodiment, the target object in the original endoscopic image can also be identified through morphology. For example, edge detection method, region growing method, threshold method, fuzzy clustering method, mathematical morphology-based method and the like can be used.

[0030] In one embodiment, target object recognition can also be performed in the original endoscopic image based on the deep learning model. In this embodiment, taking blood vessels as an example, the image content attributes of the target endoscopic application scenario are first defined, such as bronchial mucosa, mucus, superficial blood vessels, deep blood vessels, bleeding points, inflammation, etc., and then the image is segmented into regions, and each region represents similar image attributes.

[0031] Image region segmentation can be achieved using both traditional image processing methods and deep learning methods. Traditional image processing methods include, but are not limited to, threshold segmentation, edge detection segmentation, and genetic algorithm segmentation. Deep learning processing methods include, but are not limited to, feature encoding-based segmentation methods (such as VGGNet and ResNet), region selection-based methods (such as Fast RCNN), and image segmentation methods based on U-Net or DeepLab networks.

[0032] S30. Determine the target fuzzy convolution kernel corresponding to the target object parameter based on the mapping relationship between the pre-calibrated calibration target object parameter and the calibration fuzzy convolution kernel, wherein the target fuzzy convolution kernel represents a combination of the optical diffusion parameter corresponding to the target size parameter of the target object under the target color channel and the sensor crosstalk parameter under the target channel. In one embodiment, the mapping relationship is constructed based on a first mapping relationship between calibration targets of different size parameters and optical diffusion coefficients under the calibration color channel and a second mapping relationship between different calibration color channels of the calibration target and sensor crosstalk coefficients. Exemplarily, the first mapping relationship and the second mapping relationship can be a first mapping table of optical diffusion parameters related to the calibration color channel of the calibration target and blood vessel diameter, and a second mapping table of sensor crosstalk parameters related to the calibration color channel of the calibration target.

[0033] The first mapping table may include an optical diffusion parameter mapping table related to the red channel and blood vessel diameter, an optical diffusion parameter mapping table related to the green channel and blood vessel diameter, and an optical diffusion parameter mapping table related to the blue channel and blood vessel diameter. The second mapping table may include a sensor crosstalk parameter mapping table related to the red channel, a sensor crosstalk parameter mapping table related to the green channel and blood vessel diameter, and a sensor crosstalk parameter mapping table related to the blue channel.

[0034] Color diffusion at high-contrast edges and blurred tissue boundaries in raw endoscopic images are caused by both optical diffusion and sensor crosstalk. Total image crosstalk is a combination of these two factors. Therefore, in this embodiment, after obtaining the first and second mapping tables, they are used to construct a global blur convolution kernel. In this embodiment, since optical diffusion is related to the color channel and vessel diameter, while sensor crosstalk is only related to the color channel, a two-dimensional lookup table for the blur convolution kernel can be established based on the first and second mapping tables. This means that the blur convolution kernel corresponding to the total crosstalk of the current target in a particular color channel can be obtained using the color channel and the target's size parameters.

[0035] In another embodiment, the above mapping relationship may also be a mapping function obtained by fitting, for example, the above mapping relationship may be fitted by a univariate polynomial.

[0036] S40. Deconvolution processing is performed on the original endoscopic image based on the target fuzzy convolution kernel to obtain an endoscopic imaging image. The target fuzzy convolution kernel corresponding to the color channel and size parameters of the target object in the original endoscopic image is determined through a pre-calibrated mapping relationship, i.e., the total crosstalk of the target object in the current original endoscopic image. After obtaining the target fuzzy convolution kernel, a corresponding deconvolution operation is performed on the image in the vascular region, thereby restoring an image with clear vascular boundaries.

[0037] In this embodiment, the deconvolution method includes but is not limited to direct inverse filtering, Wiener filtering, iterative optimization, regularization, blind deconvolution, and the like.

[0038] For example, the original endoscopic image is Fourier transformed channel by channel to obtain a spectrum for each color channel. The spectrum is divided by the spectrum of the point spread function corresponding to the blood vessel diameter, that is, the spectrum obtained by Fourier transforming the convolution kernel of each color channel. The spectrum is then inversely Fourier transformed to obtain the restored image.

[0039] ; Where F is the Fourier transform, F -1 is the inverse Fourier transform, is the red channel image of the original endoscopic image, is the green channel image of the original endoscopic image, is the blue channel image of the original endoscope image, δ (R, d) is the convolution kernel of the red channel corresponding to the blood vessel diameter d, δ (G, d) is the convolution kernel of the green channel corresponding to the blood vessel diameter d, δ (B, d) is the convolution kernel of the blue channel corresponding to the blood vessel diameter d. Restore the image for the red channel, Restore the image for the green channel, Restore the image for the blue channel.

[0040] Finally, to avoid boundary mutation, the processed image is fused with the original endoscopic image, and the fused image is: I final (x,y)=ωI recovered (x,y)+(1-ω)I blurred (x,y); Among them, I final (x,y) is the fused image, I recovered (x,y) is the restored image, I blurred (x,y) is the original endoscopic image, and ω is the fusion coefficient.

[0041] In the present application, after acquiring the original endoscopic image, the target object and target object parameters are identified in the original endoscopic image, and the target fuzzy convolution kernel corresponding to the combination of the optical diffusion parameters corresponding to the target size parameters of the target object under the target color channel and the sensor crosstalk parameters under the target channel is determined by the mapping relationship between the pre-calibrated calibration target object parameters and the calibration fuzzy convolution kernel. The target fuzzy convolution kernel is used to perform a deconvolution operation on the original endoscopic image, and the image is crosstalk-eliminated by tissue and region. This solves the problems of blurred tissue boundaries and tissue color distortion caused by optical diffusion and image sensor crosstalk during narrowband imaging, thereby more accurately displaying the color changes of the tissue.

[0042] In one embodiment, the mapping relationship can be calibrated by using an image sensor to capture a bionic target to obtain a crosstalk calibration image, and then determining the optical diffusion parameters corresponding to different calibration color channels and different size parameters in the calibration crosstalk image and the sensor crosstalk parameters corresponding to different calibration color channels to obtain the mapping relationship.

[0043] Specifically, such as Figure 2 As shown, the calibration method of the mapping relationship includes the following steps: S100. Acquire a crosstalk calibration image, wherein the crosstalk calibration image includes the calibration target. In this embodiment, the crosstalk calibration image may include an optical crosstalk calibration image under a single calibration color channel and a sensor crosstalk calibration image under multiple calibration color channels.

[0044] Among them, the optical crosstalk image is the bionic target image of multiple calibration color channels captured by the calibration image sensor under the illumination of a single narrow-band light source corresponding to the calibration color channel. For example, a single narrow-band light source illuminates the bionic target separately in sequence, and the calibration camera is used to collect images separately, thereby obtaining multiple narrow-band images. They are respectively denoted as 、 、 、 、 The bionic target includes calibration objects of various size parameters, for example, a blood vessel structure of 50-500 μm.

[0045] Extract the main spectrum response color channel image of each narrowband image as the optical crosstalk calibration image. For example, when only a narrowband light source with a main spectrum of 415nm is used as the illumination light source, the main response is on the B channel, so the main response image of the B channel with a wavelength of 415nm is extracted and recorded as Similarly, the main response image of the 460nm wavelength light on channel B can be obtained: , the main response image of 540nm wavelength light on the G channel , the main response images of wavelength 600nm and wavelength 620nm on the R channel and .

[0046] The main response image of 415nm wavelength light and the main response image of 460nm wavelength light are synthesized into the blue channel optical crosstalk calibration image according to the following formula ; The main response image of 540nm wavelength light is used as the green channel optical crosstalk calibration image ; The main response image at a wavelength of 600nm and the main response image at a wavelength of 620nm are synthesized into the red channel optical crosstalk calibration image , you can get the optical crosstalk calibration image of each calibration color channel:

[0047] The acquisition of sensor crosstalk calibration images under multiple calibration color channels can be determined by the optical crosstalk calibration image of a single calibration color channel in the above embodiment.

[0048] Combine the optical crosstalk calibration images of each single calibration color channel into an RGB image to obtain a composite image . The mixed calibration color channel bionic target image of the calibration image sensor set is obtained under the simultaneous illumination of all narrow-band light sources corresponding to all the calibration color channels. ; Mixed calibration color channel bionic target image An image that includes both optical diffusion crosstalk and sensor crosstalk.

[0049] The mixed calibration color channel bionic target image With the composite image Subtract and get the mixed calibration color channel sensor crosstalk imageI cmos , see formula: .

[0050] The mixed calibration color channel sensor crosstalk image is channel-separated to obtain the sensor crosstalk calibration images under multiple calibration color channels. The sensor crosstalk calibration images are respectively recorded as red channel sensor crosstalk calibration images , Green channel sensor crosstalk calibration image , blue channel sensor crosstalk calibration image Sensor crosstalk is not related to blood vessel size but only to the characteristics of the CMOS image sensor itself. Therefore, when calibrating CMOS image sensor crosstalk, there is no need to distinguish blood vessel diameter.

[0051] Since optical diffusion crosstalk is related to the color channel and the size parameters of the target object, while sensor crosstalk is only related to the color channel, in this embodiment, two Gaussian kernels of different widths are used to quantify the optical diffusion parameters and sensor crosstalk parameters respectively, and finally the kernels are combined to complete the calibration of the total crosstalk characteristics. For details, see the description of steps S200 to S400.

[0052] S200. Construct a line spread function of the calibration target under each calibration color channel in the crosstalk calibration image. For the optical crosstalk calibration image, construct a line spread function under each calibration color channel. Specifically, sample calibration targets with different calibration size parameters in the optical crosstalk image of each calibration color channel to obtain first sampling points of the calibration targets with different calibration size parameters under each calibration color channel, wherein the number of the first sampling points is inversely correlated with the value of the calibration size parameter. Extract a first point spread function of the calibration target corresponding to each calibration size parameter under each calibration color channel based on the first sampling point corresponding to each calibration size parameter; and construct a first line spread function based on the first point spread function.

[0053] For example, taking the red channel as an example, see Figure 3 As shown, the image is calibrated for optical crosstalk in the red channel. For example, in Vessels of different diameters are marked in the image. Normals are generated perpendicular to the vessel centerline. The normal length extends beyond the diffusion boundary and covers the entire diffusion transition region (for example, the diffusion boundary is ±3 times the vessel diameter). Since small vessels are close to the optical diffraction limit, dense sampling is required to capture subtle optical diffusion. Therefore, for small vessels (50-100μm), the normal interval is , the number of sampling points (number of normals) is Medium vessels (100-300 μm), normal interval is , the number of sampling points is Large blood vessels (300-500 μm), normal interval is , the number of sampling points is . The following relationship is satisfied: < < , > > .

[0054] Use the first sampling point to perform the first point spread function PSF d (x) is fitted as:

[0055] Where d is the diameter of the blood vessel, N is the number of normals under this diameter, (x,y i ) is the coordinate of the first sampling point.

[0056] The first point spread function is fitted to the first line spread function LSF(x) according to the following formula: .

[0057] Based on the sensor crosstalk parameters, the calibration target in the sensor crosstalk calibration image under each calibration color channel is sampled to obtain a second sampling point of the calibration target under each calibration color channel; a second point spread function of the calibration target under each calibration color channel is extracted based on the second sampling point; and a second line spread function is constructed based on the second point spread function.

[0058] Taking the red channel as an example, the image is calibrated for the red channel sensor crosstalk ,right Processing is performed to mark M large blood vessels, medium blood vessels, and small blood vessels. The number of the second sampling points is N, and the normal interval is Perform superposition and averaging to obtain the second point spread function PSF(x) corresponding to the red channel sensor crosstalk calibration image: .

[0059] The second point spread function is fitted to the second line spread function LSF(x) according to the following formula: .

[0060] S300. Use the Gaussian distribution model to fit the calibration target object in each calibration color channel in the crosstalk calibration image respectively, and obtain the optical diffusion parameters corresponding to the different calibration size parameters of the calibration target object in different calibration color channels and the sensor crosstalk parameters in the different calibration color channels.

[0061] Specifically, the first line spread function is fitted using a first Gaussian distribution model to obtain optical diffusion parameters corresponding to different size parameters under the calibrated color channel. A first mapping table of optical diffusion parameters related to the calibrated color channel and the calibrated size parameter is established. The first mapping table includes an optical diffusion parameter mapping table related to the red channel and the calibrated size parameter. , Optical diffusion parameter mapping table related to green channel and calibration size parameters , Optical diffusion parameter mapping table related to blue channel and calibration size parameters .

[0062] The second line spread function is fitted using a second Gaussian distribution model to obtain sensor crosstalk parameters corresponding to the calibration color channel. A second mapping table of sensor crosstalk parameters related to the calibration color channel is established, and the second mapping table includes a mapping table of sensor crosstalk parameters related to the red channel. , mapping table of sensor crosstalk parameters related to the green channel , mapping table of sensor crosstalk parameters related to the blue channel The first Gaussian distribution model and the second Gaussian distribution model have different Gaussian kernel widths.

[0063] S400. Combine the optical diffusion parameters and the sensor crosstalk parameters to construct a mapping relationship between the calibration target parameters and the fuzzy convolution kernel. Since optical crosstalk is related to the color channel and blood vessel diameter, while sensor crosstalk is only related to the color channel, a two-dimensional lookup table of the fuzzy convolution kernel is established. , is the color channel, The diameter of the blood vessel. Among them, the two-dimensional lookup table of the blurred convolution kernel δ(channal,d)=δ channal *δ d , δ channal is a combination of the optical diffusion parameter associated with the color channel in the first mapping table and the sensor crosstalk parameter associated with the color channel in the second mapping table, is the optical diffusion parameter associated with the size parameter in the first mapping table. Using two Gaussian kernels of different widths, we quantify edge color diffusion caused by lens blur and sensor crosstalk, respectively, completing multi-dimensional color diffusion feature calibration. This then eliminates color diffusion in the image by tissue and region. This solves issues such as blurred tissue boundaries and tissue color distortion caused by color crosstalk in narrowband imaging, allowing for more accurate visualization of tissue color changes.

[0064] like Figure 4 As shown, the embodiment of the present application also provides an endoscope color crosstalk suppression device, comprising: An acquisition module 401 is configured to acquire an original endoscopic image captured by an image sensor; an identification module 402 for identifying target parameters in the original endoscopic image, wherein the target parameters include a target color channel and a target size parameter corresponding to the target; a determination module 403 for determining a target blur convolution kernel corresponding to the target object parameter based on a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target channel; The crosstalk suppression module 404 is configured to perform deconvolution processing on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

[0065] The embodiment of the present application also provides an endoscope, including an image sensor, an illumination light source and a controller, wherein, as Figure 5 The controller includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, the communication interface 502, and the memory 503 communicate with each other via the communication bus 504. Memory 503, used to store computer programs; The processor 501 is configured to execute the computer program stored in the memory 503, and implement the following steps: Acquiring a raw endoscopic image captured by an image sensor; Identifying target object parameters in the original endoscopic image, the target object parameters including target color channels and target size parameters corresponding to the target object; Determining a target blur convolution kernel corresponding to the target object parameter according to a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target channel; Deconvolution processing is performed on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

[0066] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0067] The communication interface is used for communication between the above controller and other devices.

[0068] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Alternatively, the memory may also be at least one storage device located away from the aforementioned processor.

[0069] The above-mentioned processor can be a general-purpose processor, which can include but is not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0070] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0071] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0072] According to another aspect of the embodiment of the present application, a storage medium is further provided. Optionally, in this embodiment, the storage medium can be used to execute program code of the endoscope color crosstalk suppression method.

[0073] Optionally, in this embodiment, the above-mentioned storage medium may be located on at least one network device among the multiple network devices in the network shown in the above-mentioned embodiment.

[0074] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: Acquiring a raw endoscopic image captured by an image sensor; Identifying target object parameters in the original endoscopic image, the target object parameters including target color channels and target size parameters corresponding to the target object; Determining a target blur convolution kernel corresponding to the target object parameter according to a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target channel; Deconvolution processing is performed on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

[0075] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, which will not be described in detail in this embodiment.

[0076] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0077] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0078] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application.

[0079] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the purpose of the solution provided in this embodiment.

[0082] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0083] Anything not described in this application can be achieved by adopting or drawing on existing technologies.

[0084] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0085] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for suppressing color crosstalk in an endoscope, characterized in that: Acquiring a raw endoscopic image captured by an image sensor; Identifying target object parameters in the original endoscopic image, the target object parameters including target color channels and target size parameters corresponding to the target object; Determining a target blur convolution kernel corresponding to the target object parameter according to a mapping relationship between a pre-calibrated calibration target object parameter and a calibration blur convolution kernel, wherein the target blur convolution kernel represents a combination of an optical diffusion parameter corresponding to the target size parameter of the target object in the target color channel and a sensor crosstalk parameter in the target color channel; Deconvolution processing is performed on the original endoscopic image based on the target blurred convolution kernel to obtain an endoscopic imaging image.

2. The method for suppressing color crosstalk in an endoscope according to claim 1, wherein: The performing crosstalk suppression processing on the original endoscopic image based on the crosstalk parameter to obtain an endoscopic imaging image includes: determining a target area where the target object is located in the original endoscopic image; Performing a deconvolution operation on the target region color channels using the target blurred convolution kernel to obtain a crosstalk suppressed image; The crosstalk suppression image is fused with the original endoscopic image to obtain the endoscopic imaging image.

3. The method for suppressing color crosstalk in an endoscope according to claim 1, wherein: The calibration method of the mapping relationship includes: Acquire a crosstalk calibration image, wherein the crosstalk calibration image has a calibration target object; Constructing a line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image; The calibration target object in each calibration color channel in the crosstalk calibration image is fitted using a Gaussian distribution model to obtain optical diffusion parameters corresponding to different calibration size parameters of the calibration target object in different calibration color channels and sensor crosstalk parameters in different calibration color channels; The optical diffusion parameter and the sensor crosstalk parameter are combined to construct a mapping relationship between the calibration target object parameter and the fuzzy convolution kernel.

4. The method for suppressing color crosstalk in an endoscope according to claim 3, wherein: The crosstalk calibration image includes an optical crosstalk calibration image, which is a bionic target image of multiple calibration color channels collected by a calibration image sensor under illumination of a single narrow-band light source corresponding to the calibration color channel; Constructing the line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image includes: Sampling calibration objects of different calibration size parameters in the optical crosstalk image of each calibration color channel respectively to obtain first sampling points of the calibration objects of different calibration size parameters in each calibration color channel, wherein the number of the first sampling points is inversely correlated with the value of the calibration size parameter; Extracting a first point spread function of the calibration target object corresponding to each calibration size parameter in each calibration color channel based on the first sampling point corresponding to each calibration size parameter; A first line spread function is constructed based on the first point spread function.

5. The method for suppressing color crosstalk in an endoscope according to claim 4, wherein: The crosstalk calibration image includes sensor crosstalk calibration images under multiple calibration color channels; Constructing the line spread function of the calibration target object under each calibration color channel in the crosstalk calibration image includes: Sampling the calibration target in the sensor crosstalk calibration image under each calibration color channel respectively to obtain a second sampling point of the calibration target under each calibration color channel; extracting a second point spread function of the calibration target object in each calibration color channel for the second sampling point; A second line spread function is constructed based on the second point spread function.

6. The method for suppressing color crosstalk in an endoscope according to claim 5, wherein: The step of fitting the calibration target object in each calibration color channel in the crosstalk calibration image using a Gaussian distribution model comprises: Fitting the first line spread functions using a first Gaussian distribution model to obtain optical diffusion parameters corresponding to different size parameters under a calibration color channel; Establishing a first mapping table of optical diffusion parameters related to calibrated color channels and calibrated size parameters; Fitting the second line spread functions using a second Gaussian distribution model to obtain sensor crosstalk parameters corresponding to the calibration color channels; A second mapping table for calibrating sensor crosstalk parameters related to the color channels is established.

7. The method for suppressing color crosstalk in an endoscope according to claim 5, wherein: The method for obtaining the sensor crosstalk calibration image includes: Acquire a mixed calibration color channel bionic target image of the calibration image sensor set under simultaneous illumination by all narrow-band light sources corresponding to all the calibration color channels; merging all the optical crosstalk calibration images into a composite image; Subtracting the mixed calibration color channel bionic target image from the synthesized image to obtain a mixed calibration color channel sensor crosstalk image; Channel separation is performed on the mixed calibration color channel sensor crosstalk image to obtain the sensor crosstalk calibration images under multiple calibration color channels.

8. The method for suppressing color crosstalk in an endoscope according to claim 2, wherein: The deconvolution method includes but is not limited to one of direct inverse filtering, Wiener filtering, iterative optimization, regularization, and blind deconvolution.

9. The method for suppressing color crosstalk in an endoscope according to claim 1, wherein: The identifying target object parameters in the original endoscopic image includes: The target object is identified in the original endoscopic image by the spectral absorption difference between the target biological tissue corresponding to the target object and the surrounding tissue.

10. An endoscope, characterized in that: The invention comprises an image sensor, an illumination light source and a controller, wherein the controller comprises one or more processors, one or more memories, and one or more computer program instructions, and when the computer program instructions are executed by the processor, the endoscope color crosstalk suppression method according to any one of claims 1 to 9 is implemented.

Images