Defogging method based on endoscope image and endoscope system

Abstract

The invention discloses a defogging method based on an endoscope image and an endoscope system.The method is applied to a processor of the endoscope system.The method comprises the steps that the endoscope image is obtained; extracting a texture feature, at least one color feature and a global contrast feature of the endoscope image; determining that fog exists in the endoscope image according to the texture feature, the at least one color feature and the global contrast feature; determining the fog concentration of the endoscope image; and according to the fog concentration, carrying out defogging processing on the endoscope image so as to obtain a defogged image. The accuracy of fog detection and fog removal can be improved, automatic fog monitoring and removal are achieved, and the automation degree of the endoscope is improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a defogging method based on endoscopic images and an endoscopic system. Background Art

[0002] Endoscopes are important devices used in minimally invasive medical treatment of the human body. By inserting the endoscope into the lumen of the object being observed, the doctor is provided with images of the lumen for observation, thereby facilitating the diagnosis of diseases and the implementation of surgery.

[0003] However, when doctors use endoscopes for surgery, temperature differences or tissue cutting can generate varying degrees of fog, obscuring the image and interfering with the surgeon's field of view, thus affecting diagnostic accuracy and treatment effectiveness. Therefore, identifying and removing fog from surgical images is an urgent problem that needs to be addressed. Summary of the Invention

[0004] The embodiments of the present application provide a defogging method and an endoscope system based on endoscopic images to improve the accuracy of fog detection and fog removal, realize automatic monitoring and removal of fog, and improve the degree of automation of the endoscope.

[0005] In a first aspect, an embodiment of the present application provides a defogging method based on an endoscopic image, which is applied to a processor of an endoscope system, comprising:

[0006] Acquiring endoscopic images;

[0007] extracting a texture feature, at least one color feature, and a global contrast feature of the endoscopic image;

[0008] determining, based on the texture feature, the at least one color feature, and the global contrast feature, that fog exists in the endoscopic image;

[0009] determining a fog concentration of the endoscopic image;

[0010] Defogging is performed on the endoscopic image according to the fog concentration to obtain a defogging image.

[0011] In a second aspect, an embodiment of the present application provides an endoscope system, which includes: an image acquisition device and a processor, wherein the image acquisition device is used to acquire endoscopic images, and the processor executes the method described in the first aspect.

[0012] In a third aspect, an embodiment of the present application provides a defogging device based on an endoscopic image, which is applied to a processor of an endoscope system, and the device includes:

[0013] an acquisition unit, configured to acquire an endoscopic image;

[0014] an extraction unit, configured to extract a texture feature, at least one color feature, and a global contrast feature of the endoscopic image;

[0015] a first determining unit, configured to determine whether fog exists in the endoscopic image based on the texture feature, the at least one color feature, and the global contrast feature;

[0016] a second determining unit, configured to determine a fog concentration of the endoscope image;

[0017] The defogging unit is used to perform defogging on the endoscopic image according to the fog concentration to obtain a defogged image.

[0018] In a fourth aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and an executable program code stored in the memory and runnable on the processor, wherein the processor executes the steps of the method described in the first aspect when executing the executable program code.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which executable program code is stored. The executable program code includes execution instructions, and the execution instructions are used to execute the steps of the method described in the first aspect.

[0020] In a sixth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0021] It can be seen that in the embodiment of the present application, an endoscopic image is first acquired; then the texture features, at least one color feature and the global contrast features of the endoscopic image are extracted; then, based on the texture features, the at least one color feature and the global contrast features, it is determined that there is fog in the endoscopic image; then the fog concentration of the endoscopic image is determined; finally, based on the fog concentration, the endoscopic image is defogged to obtain a defogged image.

[0022] The present application performs multi-dimensional feature extraction on the acquired endoscopic image, and determines whether there is fog in the endoscopic image in combination with the multi-dimensional features, thereby improving the accuracy of fog detection. When it is determined that fog exists, the fog is defogged in combination with the fog concentration, thereby improving the accuracy of fog removal, realizing automatic monitoring and removal of fog, improving the work efficiency of doctors, reducing work intensity, making surgery more efficient and safer, and improving the degree of automation of endoscopy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 This is a system architecture diagram of an endoscope system provided in an embodiment of the present application;

[0025] Figure 2 This is a schematic diagram of an extended architecture of an endoscope system provided in an embodiment of the present application;

[0026] Figure 3 1 is a flow chart of a defogging method based on an endoscopic image provided in an embodiment of the present application;

[0027] Figure 4 This is a functional control interface of an endoscope provided in an embodiment of the present application;

[0028] Figure 5 1 is a flow chart of another defogging method based on endoscopic images provided in an embodiment of the present application;

[0029] Figure 6 This is a block diagram of the functional units of an endoscopic image defogging device provided in an embodiment of the present application;

[0030] Figure 7 This is a block diagram of the functional units of another endoscopic image-based defogging device provided in an embodiment of the present application;

[0031] Figure 8 This is a structural diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0034] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] Currently, during surgery, the endoscope tip is prone to fogging due to the temperature difference between the human body and the external environment, or the high temperature, body fluid changes and gas products generated during tissue cutting. This causes fog to appear in varying degrees and forms in the image, obscuring the image and interfering with the doctor's surgical field of view. Therefore, real-time detection and removal of fog is necessary.

[0036] Regarding the fog detection step, due to the various forms of fog in endoscopic images, it is currently impossible to accurately identify fog in endoscopic images.

[0037] For the fog removal step, the fog is usually sucked out of the body by suction, or the image is defogged based on a dark channel algorithm. However, the suction operation requires the suspension of the current observation or surgical operation, interrupting the doctor's operating process, resulting in a significant reduction in diagnostic efficiency; and because the fog is unevenly distributed, it is difficult for the suction device to accurately and quickly completely remove all the fog, and multiple suctions are often required, further extending the diagnosis or treatment time. In addition, the dark channel algorithm has problems such as darkening the image after defogging, not being able to adapt the defogging intensity to different fog concentrations, color distortion due to overly strong defogging, and incomplete fog removal due to weak defogging. At the same time, in endoscopic images, fog-free images also have large dark channel values. Therefore, the size of the fog cannot be effectively estimated by the dark channel value, and the image cannot be accurately defogged.

[0038] In response to the above problems, an embodiment of the present application provides a defogging method and an endoscope system based on an endoscopic image. The embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0039] See also Figure 1 , Figure 1This is a system architecture diagram of an endoscope system provided by an embodiment of the present application. Figure 1 As shown, the endoscope system 100 includes an image acquisition device 101 and a first processor 102, wherein the image acquisition device 101 and the first processor 102 are communicatively connected, and the image acquisition device 101 includes a high-resolution camera and a high-quality optical system, which can clearly capture image information of the internal tissue of an observed object such as a human body or an animal in its working state, and transmit it to the first processor 102 for processing.

[0040] Among them, the first processor 102 is the control core of the system, and is at least used to receive image information from the image acquisition device 101, identify and detect fog based on the image information, and defog the fog in the image when it is determined that fog exists.

[0041] In one possible embodiment, fog recognition and detection and fog removal processing can be performed by an external processor that is in communication with the endoscope system 100, thereby facilitating the release of computing resources of the endoscope host. Figure 2 , Figure 2 This is a schematic diagram of an extended architecture of an endoscope system provided in an embodiment of the present application. Figure 2 As shown, the second processor 103 is communicatively connected to the endoscope system 100 and replaces the first processor 102 to perform fog detection and removal tasks.

[0042] Among them, the second processor 103 is an external processor, which is a computing device or platform independent of the endoscope hardware body. It is connected to the image acquisition device 101 through a wired or wireless communication link, and is used to obtain the original data or pre-processed image transmitted by the image acquisition device, and then perform fog detection and defogging processing tasks.

[0043] Among them, the external processor can be deployed locally, such as a workstation in an operating room; or deployed remotely, such as a cloud server; or deployed on a mobile terminal device, such as a smartphone.

[0044] Based on this, the present application provides a defogging method and related devices based on endoscopic images, and the present application is described in detail below in conjunction with the accompanying drawings.

[0045] See also Figure 3 , Figure 3 This is a flow chart of a defogging method based on endoscopic images provided in an embodiment of the present application. Figure 3 As shown, the method includes the following steps:

[0046] S210, acquiring an endoscopic image.

[0047] An endoscope is a slender, flexible medical device typically equipped with a light source and a camera. It can penetrate the natural cavities of a human or animal subject, or enter the body cavity through a small incision, to observe internal structures. An endoscopic image refers to a visual image of the internal organs or tissues of a human or animal subject, obtained through an endoscopic device.

[0048] In one possible embodiment, the endoscopic image can be a historical image, that is, a stored image of a past examination, which can be read through local storage, such as retrieving image data of a past examination or surgery from the built-in hard disk of the endoscope host or a workstation; it can also be retrieved through a cloud database, such as querying historical images through a medical cloud platform, and supporting keyword retrieval.

[0049] In one possible embodiment, the endoscopic image may be real-time, ie, an image directly acquired during the current examination or surgery.

[0050] S220: Extract texture features, at least one color feature, and a global contrast feature of the endoscopic image.

[0051] Among them, foggy images have low clarity, the overall picture is white and has low contrast. Therefore, we can judge whether an image is foggy by image texture features, color features and contrast features.

[0052] Among them, texture features can be represented by the edge strength represented by the variance or gradient operator of the image, and the edge strength refers to the sum of the edge strengths of each pixel; color features can be represented by hue and / or saturation; global contrast features can be represented by the global brightness ratio.

[0053] The variance of an image measures the degree of discreteness of its pixel values, reflecting the underlying characteristics of a texture, such as roughness or smoothness. Fog brightens the image overall and averages pixel values, reducing the variance. For example, a fog-free image has clear textures, large pixel value fluctuations, and a high variance. A foggy image, on the other hand, has a whitish appearance, with pixel values ​​concentrated in the middle grayscale, resulting in a significantly reduced variance.

[0054] To calculate the variance, we first traverse all pixels in the image, calculate the mean of all pixels in the image, then calculate the square of the difference between each pixel and the mean, and finally normalize the image. The calculation formula (1) of the image variance is as follows:

[0055]

[0056] Where var is the image variance; H refers to the height of the endoscopic image, that is, the number of pixels in the image in the vertical direction; W refers to the width of the endoscopic image, that is, the number of pixels in the image in the horizontal direction; H*W is the total number of pixels in the image, that is, the height multiplied by the width, which is used as the denominator for averaging and normalization to make the variance result unaffected by the image size; pix(i,j) represents the pixel value at the i-th row and j-th column position of the endoscopic image; pix_mean represents the mean of the pixel values ​​of the entire image, which is obtained by calculating the average value of all pix(i,j) and is used to measure the average level of image pixels and to calculate the difference between each pixel and the mean.

[0057] Edge strength is used to capture the edges of sudden pixel changes in an image and reflects the complexity of texture contours. Fog blurs edges, causing them to decrease. For example, in a fog-free image, the blood vessels and boundaries of normal tissue are clear, resulting in high edge strength. In a foggy image, fog blurs edges, reduces gradient amplitude, and significantly reduces edge strength.

[0058] In one possible embodiment, the edge strength is calculated by first using an edge detection operator to calculate the gradient magnitude of each pixel, and then summing all the gradient magnitudes to obtain the overall edge strength. The edge strength calculation formula (2) is as follows:

[0059]

[0060] Among them, edge_str is the sum of the edge strength of each pixel, edge(i,j) represents the edge strength at position (i,j), which indicates the obviousness of the edge at a certain point in the image.

[0061] The edge detection operator is used to calculate the gradient or difference of the neighborhood around each pixel, thereby highlighting the edge information in the image. Exemplary edge detection operators include the Sobel operator, the Prewitt operator, the Canny operator, etc.

[0062] For example, the edge strength is calculated by the Sobel operator. Two 3x3 convolution kernels are used, one for detecting the horizontal gradient and the other for detecting the vertical gradient. For each pixel in the image, the horizontal gradient and the vertical gradient are calculated, and then the edge strength is determined based on the horizontal gradient and the vertical gradient. The calculation formula (3) is as follows:

[0063]

[0064] Among them, Gx is the horizontal gradient and Gy is the vertical gradient.

[0065] For example, the Canny operator is used to calculate the edge strength. First, Gaussian filtering is performed to blur the image to reduce noise. Then, gradient calculation is performed to calculate the gradient strength and direction of the image. After that, non-maximum suppression is performed to retain the points with the largest local gradient and suppress other points. Finally, double threshold processing is performed to use two thresholds, namely a high threshold and a low threshold, to connect the edges to form the final edge image.

[0066] In one possible embodiment, extracting the texture features of the endoscopic image includes: determining the pixel value variance of the endoscopic image; performing edge detection on the endoscopic image to obtain the corresponding edge intensity; obtaining a first weight corresponding to the pixel value variance and a second weight corresponding to the edge intensity; and fusing the pixel value variance and the edge intensity according to the first weight and the second weight to obtain the texture features.

[0067] The texture feature is represented by the variance and edge strength of the image, that is, the texture is characterized by the dual dimensions of variance and edge strength, and then the final feature is obtained by weight fusion. The specific formula (4) is as follows:

[0068] texture_total=(w1*var+w2*edge_str) / (w1+w2),

[0069] Among them, texture_total is the texture feature, w1 is the first weight, and w2 is the second weight.

[0070] The first and second weights are determined based on the degree of fog's impact on different features. The first and second weights can also be adaptively adjusted based on actual conditions. For example, thresholds can be set for variance and edge strength, and the weights can be adjusted in stages based on the thresholds. For example, when the variance is below a certain threshold, the fog is considered severe, blurring the edges and increasing edge detection errors. Therefore, the edge strength weight can be reduced. By increasing the variance weight, the reliance on edges is reduced, making the features more robust.

[0071] It can be seen that in the embodiment of the present application, the contribution of variance and edge strength can be dynamically adjusted through the weight fusion strategy, so that texture features can effectively characterize characteristics and improve the accuracy of feature extraction.

[0072] In a possible embodiment, other values ​​that can represent texture features, such as entropy, mean, etc., can be calculated to further enrich the texture features.

[0073] Hue and saturation are key dimensions for describing color in color space. Hue, which uses an angle between 0° and 359° to represent color type, represents red, 120° to green, and 240° to blue. Hue captures color categorization and is the most intuitive dimension for distinguishing colors.

[0074] Fog can vary in color tone, which can be used to aid in fog identification. For example, in a fog-free image, the color of normal gastric mucosa may tend toward the inherent color of healthy tissue, corresponding to a color close to the red region. In a foggy image, the color of normal gastric mucosa may tend toward grayish white due to fog scattering and attenuating light. Alternatively, the original color of the tissue may become lighter or grayer due to light scattering. Alternatively, if the fog itself appears pale yellow due to ambient light, the color may shift toward the yellow range.

[0075] Saturation measures the vividness of a color, specifically the proportion of gray in the color. Higher values ​​indicate brighter and richer colors, while lower values ​​indicate closer to gray. 0% is gray and unsaturated, while 100% is pure color and extremely vivid. Fog, essentially tiny particles that scatter light, mixes color components and reduces saturation. Therefore, saturation is an important indicator for determining whether an image is affected by fog. For example, normal intestinal mucosa appears light red in fog-free areas, indicating high saturation. However, in foggy areas, the saturation of normal intestinal mucosa decreases significantly due to the graying and fading of colors caused by fog.

[0076] Endoscopic images are in RGB format, representing color by mixing the three primary colors of red, green, and blue. The intensity of each color is typically represented by a value from 0 to 255, corresponding to the brightness of the three primary colors. If a pixel is pure red, its channel value is (255, 0, 0); if a pixel is pure green, its channel value is (0, 255, 0); if a pixel is pure blue, its channel value is (0, 0, 255); and if a pixel is white, its channel value is (255, 255, 255).

[0077] To calculate hue or saturation, the RGB image must be converted to HSV format. Normalize the three color channel values ​​by dividing them by 255 to [0, 1], i.e., r = R / 255, g = G / 255, and b = B / 255. Then, calculate the maximum value Gmax = (r, g, b) and the minimum value Gmin = (r, g, b), and then calculate the difference Δ = Gmax - Gmin.

[0078] In one possible embodiment, when determining the hue, first determine whether the difference Δ is 0. If Δ is 0, the hue is also 0, which is grayscale and has no color tendency. If Δ is not 0, the judgment is made based on the maximum channel value of the three color channels.

[0079] Among them, if the maximum channel value of Gmax is r, the hue of a single pixel i The calculation formula (5) is as follows:

[0080]

[0081] Among them, if the maximum channel value of Gmax is g, the hue of a single pixel i The calculation formula (6) is as follows:

[0082]

[0083] Among them, if the maximum channel value of Gmax is b, the hue of a single pixel i The calculation formula (7) is as follows:

[0084]

[0085] If the hue H is less than 0, then hue i =hue i +360.

[0086] The hue value of each pixel is calculated by traversing each pixel of the image. The hue value is mapped to 0°-359°, covering all color categories, and different weights are assigned to different hues, with red having the highest weight. Based on the weights, the hue value of each pixel is weighted and summed to obtain the hue of the image. The specific calculation formula (8) is as follows:

[0087]

[0088] Among them, hue_str is the hue of the image, w i Indicates the weight of different hues.

[0089] In one possible embodiment, when determining saturation, the difference is normalized by Gmax, which represents the ratio of the non-gray component to the maximum value. The larger the ratio, the purer the color; the smaller the ratio, the grayer the color. The normalized data is then converted to a percentage to obtain the percentage value of saturation. The specific calculation formula (9) for saturation Sat is as follows:

[0090]

[0091] Regarding the global contrast feature, the global brightness-darkness ratio is used to measure the distribution range of brightness or the degree of color difference in the image, reflecting the sharp contrast between the bright and dark parts and different color areas in the image. The higher the contrast, the more obvious the difference between different areas in the image, and the picture looks clearer and more layered; the lower the contrast, the grayer the image looks and the details are difficult to distinguish.

[0092] For example, in a fog-free image of the intestinal mucosa, the folds of the mucosa are relatively darker, while the smooth areas are brighter. There are also color differences between the vascular distribution area and the surrounding mucosa. These distinct brightness and color differences between these different areas result in a higher calculated contrast value. However, in a foggy image of the intestinal mucosa, the fog scatters and absorbs light, making the image hazy overall. This weakens the brightness differences in previously clear details such as the mucosal texture and blood vessels, causing the brightness of different areas to gradually approach the same level. Consequently, when calculating contrast, the contrast value is significantly reduced.

[0093] In one possible embodiment, extracting the global contrast feature of the endoscopic image includes: determining the brightness value of each pixel in the endoscopic image; sorting the brightness value of each pixel to obtain a first sequence; extracting at least one first brightness value at the head of the first sequence according to a first preset ratio; and extracting at least one second brightness value at the tail of the first sequence according to a second preset ratio; and determining the global contrast feature based on the at least one first brightness value and the at least one second brightness value.

[0094] In one possible embodiment, the brightness value of a pixel can be calculated using the RGB space, such as by directly averaging the values ​​of the three color channels of a single pixel to obtain the brightness value of the single pixel; or by setting weights for the three color channels and performing weighted summation to obtain the brightness value of the single pixel.

[0095] In a possible embodiment, the brightness value of a pixel may also be calculated using the HSV space, where V refers to brightness, corresponding to the brightness of the pixel, and the maximum value of the three color channels is determined as the brightness value V.

[0096] In one possible embodiment, the brightness value of the pixel can also be calculated using the YUV space. The Y channel is the brightness. According to the standard conversion formula, Y is obtained. The conversion coefficients under different standards are slightly different. For example, the calculation formula (10) of the standard conversion formula is as follows:

[0097] Y=0.257R+0.504G+0.098B+16.

[0098] The method traverses each pixel of the image to obtain a brightness value for each pixel. The brightness values ​​are sorted from largest to smallest to obtain a brightness value sequence. A first preset ratio of brightness values ​​is extracted from the head of the brightness value sequence, and a second preset ratio of brightness values ​​is extracted from the tail of the brightness value sequence. A first mean of the brightness values ​​extracted from the head is calculated, as is a second mean of the brightness values ​​extracted from the tail. The difference between the first mean and the second mean is then calculated, as is the sum of the first and second mean values. The ratio of the difference to the sum is then determined as the global brightness ratio, i.e., the global contrast feature.

[0099] The first preset ratio and the second preset ratio are the same.

[0100] Preferably, the first preset ratio and the second preset ratio are 10%.

[0101] The calculation formula (11) of the global contrast feature is as follows:

[0102]

[0103] Among them, Contrast is the global contrast feature, Luma bright is the first mean, Luma dark represents the second mean.

[0104] It can be seen that in the embodiment of the present application, the brightness distribution of the image can be quantified, which is conducive to evaluating the visual effect and quality of the image and providing an important basis for subsequent fog detection tasks.

[0105] S230: Determine whether fog exists in the endoscopic image based on the texture feature, the at least one color feature, and the global contrast feature.

[0106] In a possible embodiment, determining the presence of fog in the endoscopic image based on the texture feature, the at least one color feature and the global contrast feature includes: determining a corresponding first confidence level based on the texture feature; determining at least one corresponding second confidence level based on the at least one color feature; determining a corresponding third confidence level based on the global contrast feature; and determining the presence of fog in the endoscopic image based on the first confidence level, the at least one second confidence level and the third confidence level.

[0107] Among them, the image features can be mapped through linear functions or exponential functions to calculate the corresponding confidence level. The first confidence level is based on the quantitative judgment of the probability of the image being fog-free based on the texture feature. The closer the texture feature value is to 1, the clearer the texture is and the higher the possibility of the image being fog-free. The second confidence level is based on the quantitative judgment of the probability of the image being fog-free based on the color feature. The closer the color feature value is to 1, the brighter the color, the normal the saturation, and the higher the possibility of the image being fog-free. The third confidence level is based on the quantitative judgment of the probability of the image being fog-free based on the global contrast feature. The closer the global contrast feature value is to 1, the more obvious the difference in brightness and color is, and the higher the possibility of being fog-free.

[0108] If a linear function is used to calculate the first confidence level corresponding to the texture feature, the specific formula (12) is as follows:

[0109] confidence texture =k texture *texture_total+b texture ,

[0110] Among them, confidence texture is the first confidence level, k texture Used to measure the correlation between texture features and fog, b texture is the baseline confidence, used to adjust the offset of the model, k texture and b texture Determined by experiment or data fitting.

[0111] If the exponential function is used to calculate the first confidence level corresponding to the texture feature, the specific formula (13) is as follows:

[0112]

[0113] Here, σ controls the rate of exponential decay and can be set based on extensive experimentation and business experience. For example, when processing the texture confidence of endoscopic images, we can select an appropriate σ by testing different σ values ​​to see whether the confidence output meets the actual diagnostic requirements.

[0114] In a possible embodiment, a labeled data set may be used to find σ that optimizes the confidence level differentiation effect through an optimization algorithm, such as the least squares method or maximum likelihood estimation fitting.

[0115] Among them, if at least one color feature includes hue, the second confidence corresponding to the hue is calculated; if at least one color feature includes saturation, the second confidence corresponding to the saturation is calculated; if at least one color feature includes hue and saturation, the second confidence corresponding to the hue and the second confidence corresponding to the saturation are calculated.

[0116] If a linear function is used to calculate the second confidence level corresponding to the hue, the specific formula (14) is as follows:

[0117] confidence hue =k hue *hue_str+b hue ,

[0118] Among them, confidence hue is the second confidence level corresponding to the hue, k hue Used to measure the degree of correlation between hue and fog, b hue Used to provide benchmark confidence and to adjust the offset of the model, k hue and b hue Determined by experiment or data fitting.

[0119] If the exponential function is used to calculate the second confidence level corresponding to the hue, the specific formula (15) is as follows:

[0120]

[0121] Here, σ is used to control the rate of exponential decay and is determined by experiments or data fitting.

[0122] If a linear function is used to calculate the third confidence level corresponding to the global contrast feature, the specific formula (16) is as follows:

[0123] confidence sat =k sat *sat+b sat ,

[0124] Among them, confidence sat is the third confidence level corresponding to the global contrast feature, k sat Used to measure the correlation between contrast and fog, b sat Used to provide benchmark confidence and to adjust the offset of the model, k sat and b sat Determined by experiment or data fitting.

[0125] If the exponential function is used to calculate the third confidence level corresponding to the global contrast feature, the specific formula (17) is as follows:

[0126]

[0127] Here, σ is used to control the rate of exponential decay and is determined by experiments or data fitting.

[0128] After obtaining the first confidence level, at least one second confidence level, and the third confidence level, a confidence weight corresponding to each feature is determined. The confidence weight can be preset or determined based on historical data fitting or model optimization. A weighted summation is performed on the confidence weights to determine whether the image is foggy. The confidence level is then determined based on a preset threshold.

[0129] If at least one color feature includes hue and saturation, the specific formula (18) for the total confidence is as follows:

[0130] confidence total =(w3*confidence texture +w4*confidence hue +w5*confidence sat +w6*confidence contrast ) / (w3+w4+w5+w6),

[0131] Among them, confidence total is the total confidence level, used to determine whether the image is foggy, w3 is the confidence weight of the first confidence level, w4 is the confidence weight of the second confidence level corresponding to the hue, w5 is the confidence weight of the second confidence level corresponding to the saturation, and w6 is the confidence weight of the third confidence level.

[0132] If at least one color feature includes hue, the specific formula (19) for the total confidence is as follows:

[0133] confidence total =(w3*confidence texture +w4*confidence hue +w6*

[0134] confidence contrast ) / (w3+w4+w6).

[0135] If at least one color feature includes saturation, the specific formula (20) for the total confidence is as follows:

[0136] confidence total =(w3*confidence texture +w5*confidence sat +w6*

[0137] confidence contrast ) / (w3+w5+w6).

[0138] For example, if the total confidence is detected to be less than a preset threshold, then there is fog in the endoscopic image, and the fog removal operation is performed; if the total confidence is detected to be greater than or equal to the preset threshold, then there is no fog in the endoscopic image, and the fog removal operation is not performed.

[0139] As can be seen, in this embodiment, multi-dimensional feature fusion is used to determine whether an image is foggy, covering the multiple optical properties of fog, which helps improve the accuracy of fog detection. Furthermore, fog detection can accurately distinguish between foggy and non-fog images, and the defogging operation does not affect the normal image quality. Therefore, the defogging function can be always enabled, eliminating the need for doctors to activate the defogging function during surgery, reducing the operator's operational burden and improving user convenience.

[0140] S240: Determine the fog concentration of the endoscope image.

[0141] In one possible embodiment, determining the fog concentration of the endoscopic image includes: determining a first dark channel map of the endoscopic image; obtaining multiple image blocks based on a preset pixel window with each pixel in the endoscopic image as the center; determining a second dark channel map corresponding to each of the image blocks; and determining the fog concentration of the endoscopic image based on the first dark channel map and the second dark channel map.

[0142] Among them, according to the dark channel algorithm, the minimum value of each pixel in the three color channels is calculated to obtain the first dark channel image, and the interference of overexposed areas and white areas on the dark channel image is eliminated, so that the dark channel image can more accurately represent the size of the fog.

[0143] When extracting a preset pixel window centered on a pixel, if the window exceeds the image boundary and the complete window cannot be obtained, this can be addressed through border padding, border truncation, and adaptive window adjustment. For example, edge pixels can be mirrored along the image boundary, or the area outside the boundary can be filled with edge pixel values ​​to make the filled area symmetrical with the edge content, thereby obtaining a complete window.

[0144] Exemplarily, the preset pixel window may be a 5*5 window, a 7*7 window or a window of other sizes.

[0145] The endoscopic image is split into multiple image blocks according to a preset pixel window, and then the second dark channel map of each image block is calculated according to the dark channel algorithm.

[0146] In a possible embodiment, determining the fog concentration of the endoscopic image based on the first dark channel map and the second dark channel map includes: determining a first statistical eigenvalue of the first dark channel map based on the first dark channel map; determining a second statistical eigenvalue of the second dark channel map based on the second dark channel map; and determining the fog concentration of the endoscopic image based on the first statistical eigenvalue and the second statistical eigenvalue.

[0147] The statistical characteristic value may be the mean value of the dark channel image, the median value of the dark channel image, or a value weighted by the brightness value.

[0148] The first statistical eigenvalue is used to characterize the size of the global fog, and the second statistical eigenvalue is used to characterize the size of the local fog.

[0149] The size of the global fog or the size of the local fog is used to calculate the fog density through a linear function or an exponential function. If the size of the global fog or the size of the local fog is used to calculate the global fog density or the local fog density through a linear function, the specific formula (21) is as follows:

[0150] fog_str=k fog *M+b fog ,

[0151] Among them, fog_str is the global fog concentration or the local fog concentration, k fog and b fog To influence the parameters, it is determined according to experiments or data fitting, and M is the size of the global fog or the size of the local fog.

[0152] If the size of global fog or the size of local fog is used to calculate the concentration of global fog or the concentration of local fog through an exponential function, the specific formula (22) is as follows:

[0153]

[0154] Here, σ is used to control the rate of exponential decay and is determined by experiments or data fitting.

[0155] Among them, the global fog concentration and the local fog concentration are obtained, and the corresponding weights are combined to perform weighted summation to obtain the final fog concentration. The specific formula (23) for the final fog concentration is as follows:

[0156] fog_str_total=(w7*fog_str_global+w8*fog_str_local) / (w7+w8),

[0157] Among them, fog_str_global is the concentration of global fog, fog_str_local is the concentration of local fog, fog_str_total is the final fog concentration, w7 is the weight corresponding to the global fog concentration, and w8 is the weight corresponding to the local fog concentration.

[0158] It can be seen that in the embodiment of the present application, combining the size of the global fog and the size of the local fog to determine the fog concentration of the image can fully capture the distribution characteristics of the fog in the image, which is conducive to improving the accuracy of fog concentration judgment.

[0159] S250 , performing defogging processing on the endoscopic image according to the fog concentration to obtain a defogged image.

[0160] In a possible embodiment, the defogging process is performed on the endoscopic image according to the fog concentration to obtain a defogged image, including: determining a defogging intensity according to the fog concentration; and performing dark channel defogging on the endoscopic image according to the defogging intensity to obtain a defogged image.

[0161] The fog concentration is mapped through a linear function or an exponential function to calculate the defogging intensity. The higher the fog concentration, the greater the defogging intensity, and the lower the fog concentration, the lower the fog intensity.

[0162] If the fog concentration is calculated using a linear function to calculate the defogging intensity, the specific formula (24) is as follows:

[0163] ω=k ω *fog_str_total+b ω ,

[0164] Where ω is the defogging strength, k ω and b ω It is an influencing parameter determined by experiments or data fitting.

[0165] If the fog concentration is calculated using an exponential function to calculate the defogging intensity, the specific formula (25) is as follows:

[0166]

[0167] Here, σ is used to control the rate of exponential decay and is determined by experiments or data fitting.

[0168] In a possible embodiment, the method also includes: determining the global atmospheric light intensity of the endoscopic image in each color channel; performing dark channel defogging on the endoscopic image according to the defogging intensity to obtain a defogged image, including: normalizing the pixel values ​​of each color channel in the endoscopic image according to the global atmospheric light intensity of each color channel to obtain a normalized image; determining a third dark channel map of the normalized image; determining the transmittance of the endoscopic image according to the defogging intensity and the third dark channel map; and defogging the endoscopic image according to the global atmospheric light intensity of each color channel and the transmittance to obtain a defogged image.

[0169] First, a foggy physical model is constructed. The specific formula (26) is as follows:

[0170] I(x)=J(x)t(x)+A(1-t(x)),

[0171] Where I(x) is the foggy image, J(x) is the fog-free image to be restored, A is the global atmospheric light intensity, and t(x) represents the transmittance.

[0172] Next, the transmittance is evaluated, and for each pixel, the ratio of the pixel value in each channel to the global atmospheric light intensity is calculated, a process known as normalization. The dark channel map of the normalized image is then calculated using a dark channel algorithm. This involves taking the minimum of the three channels and then the minimum within a window to generate a third dark channel map.

[0173] In real life, even on a sunny day with white clouds, there are still some particles in the air. Therefore, the influence of fog can still be felt when looking at distant objects. In addition, the presence of fog makes humans feel the existence of depth of field. Therefore, it is necessary to retain a certain degree of fog when defogging. Therefore, when calculating the transmittance, the defogging intensity is added to adjust the degree of fog removal to avoid image distortion caused by insufficient or excessive defogging.

[0174] The specific formula (27) for transmittance is as follows:

[0175]

[0176] Where c represents any color channel among the three color channels r, g, and b, Ω(x) is the local window centered on pixel x, and y is the pixel in the local window centered on pixel x.

[0177] Finally, the image is dehazed based on the transmittance and global atmospheric light intensity to obtain the dehazed image. The specific formula (28) for image restoration is as follows:

[0178]

[0179] It can be seen that in the embodiments of the present application, the present application can effectively remove interfering fog on the endoscopic image, improve the problem of darkening or color distortion of the image after defogging, and display clear tissue images to facilitate clinicians to conduct further diagnosis and treatment.

[0180] In one possible embodiment, determining the global atmospheric light intensity of the endoscopic image in each color channel includes: sorting the pixel values ​​of each pixel in the first dark channel image to obtain a second sequence; extracting at least one pixel value at the head of the second sequence according to a third preset ratio; determining the pixel position corresponding to the at least one pixel value in the endoscopic image; obtaining the channel value of the pixel corresponding to the pixel position in each color channel; and determining the global atmospheric light intensity of each color channel based on the channel value.

[0181] The method involves sorting all pixel values ​​in the dark channel image in ascending order to obtain a second sequence. A certain percentage of pixel values, such as the darkest 0.1%, are extracted from the head of the sorted second sequence (i.e., the lowest pixel value portion). The locations of these pixels are then located in the original endoscopic image. The raw pixel values ​​of the three RGB channels corresponding to these locations are obtained, forming candidate pixels. The channel values ​​for each color channel in the candidate pixels are then counted, and the maximum or average of these channel values ​​is taken as the global atmospheric light intensity for that channel.

[0182] In a possible embodiment, a certain proportion of pixel values ​​extracted may be used as an average of these values ​​as the global atmospheric light intensity.

[0183] In a possible embodiment, among a certain proportion of pixel values ​​extracted, the pixel with the highest brightness is found, and the channel values ​​of the three color channels of the pixel are taken as the global atmospheric light intensity.

[0184] It can be seen that in the embodiments of the present application, the present application performs multi-dimensional feature extraction on the acquired endoscopic image, and determines whether there is fog in the endoscopic image in combination with the multi-dimensional features, thereby improving the accuracy of fog detection, and when it is determined that there is fog, the fog is defogged in combination with the fog concentration, thereby improving the accuracy of fog removal, realizing automatic monitoring and removal of fog, improving the work efficiency of doctors, reducing work intensity, making surgery more efficient and safer, and improving the degree of automation of endoscopy.

[0185] In one possible embodiment, the electronic defogging function can be turned on through a touch screen connected to the endoscope, thereby automatically removing the fog that interferes with the image. Figure 4 , Figure 4 This is a functional control interface of an endoscope provided in an embodiment of the present application, such as Figure 4As shown, the function control interface includes multiple operating functions, such as electronic magnification, color enhancement, video recording, noise reduction and electronic defogging, etc. Electronic magnification is used for real-time digital zoom to observe tiny blood vessels or tissue details, including 1x, 1.2x and 1.6x, etc., and the current example is 1.2x, which means 1.2 times magnification; color enhancement is used to configure different color optimization schemes, including C3, C6 and C9, etc., and the color optimization scheme of C6 is currently used as an example; video recording is used to start or stop video recording; noise reduction is used to turn on or off image noise reduction; electronic defogging is used to turn on or off image defogging.

[0186] For example, each operation function may correspond to an icon, and the running state of the current function may be set below the icon. The running state may be on or off. Clicking the function control may switch the running state.

[0187] In one possible embodiment, see Figure 5 , Figure 5 This is a flow chart of another defogging method based on endoscopic images provided in an embodiment of the present application. Figure 5 As shown in the figure, the defogging algorithm process includes image input, image feature statistics, fog judgment, fog concentration calculation, defogging intensity, dark channel defogging and image output.

[0188] The system receives an input image and then extracts texture, color, and contrast features from the input image to determine whether it is foggy. If the image is fog-free, the image is output; if it is foggy, the dark channel map of the input image is calculated, and the mean, median, or brightness-weighted value of this dark channel map is used to determine the global fog size. The image is then split using a 3x3 window, a 6x6 window, or other window size, and the dark channel maps of the split images are calculated. The mean, median, or brightness-weighted value of the corresponding dark channel maps is used to determine the local fog size. The fog concentration is then calculated based on the global or local fog size using a linear or exponential function.

[0189] The defogging intensity is then calculated based on the fog concentration using a linear or exponential function. Combined with the defogging intensity, the input image is defogged using a dark channel based on a physical model of foggy weather. The resulting defogging image is then output for real-time display.

[0190] For the same example as above, please refer to Figure 6 , Figure 6 This is a functional unit block diagram of a defogging device based on endoscopic images provided in an embodiment of the present application, such as Figure 6As shown, the defogging device 60 based on the endoscopic image includes: an acquisition unit 61, used to acquire the endoscopic image; an extraction unit 62, used to extract the texture feature, at least one color feature and the global contrast feature of the endoscopic image; a first determination unit 63, used to determine the presence of fog in the endoscopic image based on the texture feature, the at least one color feature and the global contrast feature; a second determination unit 64, used to determine the fog concentration of the endoscopic image; and a defogging unit 65, used to defog the endoscopic image based on the fog concentration to obtain a defogged image.

[0191] In one possible embodiment, in terms of determining the presence of fog in the endoscopic image based on the texture feature, the at least one color feature and the global contrast feature, the first determination unit 63 is specifically used to: determine a corresponding first confidence level based on the texture feature; determine a corresponding at least one second confidence level based on the at least one color feature; determine a corresponding third confidence level based on the global contrast feature; and determine the presence of fog in the endoscopic image based on the first confidence level, the at least one second confidence level and the third confidence level.

[0192] In one possible embodiment, in terms of determining the fog concentration of the endoscopic image, the second determination unit 64 is specifically used to: determine a first dark channel map of the endoscopic image; obtain multiple image blocks based on a preset pixel window with each pixel in the endoscopic image as the center; determine a second dark channel map corresponding to each of the image blocks; and determine the fog concentration of the endoscopic image based on the first dark channel map and the second dark channel map.

[0193] In a possible embodiment, in terms of determining the fog concentration of the endoscopic image based on the first dark channel map and the second dark channel map, the second determination unit 64 is specifically further used to: determine the first statistical eigenvalue of the first dark channel map based on the first dark channel map; determine the second statistical eigenvalue of the second dark channel map based on the second dark channel map; and determine the fog concentration of the endoscopic image based on the first statistical eigenvalue and the second statistical eigenvalue.

[0194] In one possible embodiment, in terms of extracting the global contrast feature of the endoscopic image, the extraction unit 62 is specifically used to: determine the brightness value of each pixel in the endoscopic image; sort the brightness value of each pixel to obtain a first sequence; extract at least one first brightness value at the head of the first sequence according to a first preset ratio; and extract at least one second brightness value at the tail of the first sequence according to a second preset ratio; and determine the global contrast feature based on the at least one first brightness value and the at least one second brightness value.

[0195] In one possible embodiment, in terms of extracting the texture features of the endoscopic image, the extraction unit 62 is further specifically used to: determine the pixel value variance of the endoscopic image; perform edge detection on the endoscopic image to obtain the corresponding edge intensity; obtain a first weight corresponding to the pixel value variance and a second weight corresponding to the edge intensity; and fuse the pixel value variance and the edge intensity according to the first weight and the second weight to obtain the texture features.

[0196] In one possible embodiment, in terms of defogging the endoscopic image according to the fog concentration to obtain a defogged image, the defogging unit 65 is specifically used to: determine the defogging intensity according to the fog concentration; and perform dark channel defogging on the endoscopic image according to the defogging intensity to obtain a defogged image.

[0197] In one possible embodiment, the endoscopic image-based defogging device 60 is further specifically used to: determine the global atmospheric light intensity of the endoscopic image in each color channel; perform dark channel defogging on the endoscopic image according to the defogging intensity to obtain a defogged image, including: normalizing the pixel values ​​of each color channel in the endoscopic image according to the global atmospheric light intensity of each color channel to obtain a normalized image; determining a third dark channel map of the normalized image; determining the transmittance of the endoscopic image according to the defogging intensity and the third dark channel map; and defogging the endoscopic image according to the global atmospheric light intensity of each color channel and the transmittance to obtain a defogged image.

[0198] In one possible embodiment, in terms of determining the global atmospheric light intensity of the endoscopic image in each color channel, the endoscopic image-based defogging device 60 is specifically further used to: determine the global atmospheric light intensity of the endoscopic image in each color channel, including: sorting the pixel values ​​of each pixel in the first dark channel image to obtain a second sequence; extracting at least one pixel value at the head of the second sequence according to a third preset ratio; determining the pixel position corresponding to the at least one pixel value in the endoscopic image; obtaining the channel value of the pixel corresponding to the pixel position in each color channel; and determining the global atmospheric light intensity of each color channel based on the channel value.

[0199] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0200] In the case of integrated units, see Figure 7 , Figure 7This is a block diagram of the functional units of another endoscopic image-based defogging device provided in an embodiment of the present application, such as Figure 7 As shown, the defogging device 60 based on endoscopic images includes: a processing module 602 and a communication module 601. The processing module 602 is used to control and manage the actions of the defogging device 60 based on endoscopic images, for example, executing the steps of the acquisition unit 61, the extraction unit 62, the first determination unit 63, the second determination unit 64 and the defogging unit 65, and / or other processes for executing the technology described herein. The communication module 601 is used for interaction between the defogging device 60 based on endoscopic images and other devices. Figure 7 As shown, the endoscopic image-based defogging device 60 may further include a storage module 603 , which is used to store program codes and data of the endoscopic image-based defogging device 60 .

[0201] Among them, the processing module 602 can be a processor or controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 601 can be a transceiver, an RF circuit or a communication interface, etc. The storage module 603 can be a memory.

[0202] Among them, all relevant contents of each scene involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. The above-mentioned defogging device 60 based on endoscopic image can perform the above-mentioned Figure 3 The dehazing method based on endoscopic images is shown.

[0203] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of the present application. Figure 8As shown, the electronic device 800 includes a processor 810, a memory 820, a communication interface 830 and one or more programs 821. The one or more programs 821 are stored in the memory and are configured to be executed by the processor. When the program is executed, it includes part or all of the steps of any one of the endoscopic image-based defogging methods recorded in the above method embodiments. The processor, memory and communication interface are interconnected and complete communication with each other.

[0204] The memory may be a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a mechanical hard disk. The memory is used to store a set of executable program codes, and the processor is used to call the executable program codes stored in the memory to execute some or all of the steps of any endoscopic image-based defogging method described in the above-mentioned embodiment of the endoscopic image-based defogging method.

[0205] It can be seen that the electronic device 800 described in the embodiment of the present application first acquires an endoscopic image; then extracts the texture features, at least one color feature and the global contrast feature of the endoscopic image; then determines the presence of fog in the endoscopic image based on the texture features, the at least one color feature and the global contrast feature; then determines the fog concentration of the endoscopic image; finally, defogging is performed on the endoscopic image based on the fog concentration to obtain a defogged image.

[0206] The present application performs multi-dimensional feature extraction on the acquired endoscopic image, and determines whether there is fog in the endoscopic image in combination with the multi-dimensional features, thereby improving the accuracy of fog detection. When it is determined that fog exists, the fog is defogged in combination with the fog concentration, thereby improving the accuracy of fog removal, realizing automatic monitoring and removal of fog, improving the work efficiency of doctors, reducing work intensity, making surgery more efficient and safer, and improving the degree of automation of endoscopy.

[0207] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0208] The present application also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may comprise an electronic device.

[0209] It should be noted that for the aforementioned method implementations, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the implementations described in the specification are all optional implementations, and the actions and modules involved are not necessarily required for this application.

[0210] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0212] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0213] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0214] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0215] A person skilled in the art will understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0216] The above is a detailed introduction to the implementation methods of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above implementation methods is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A defogging method based on endoscopic images, characterized in that: Processors for endoscopy systems include: Acquiring endoscopic images; extracting a texture feature, at least one color feature, and a global contrast feature of the endoscopic image; determining, based on the texture feature, the at least one color feature, and the global contrast feature, that fog exists in the endoscopic image; determining a fog concentration of the endoscopic image; Defogging is performed on the endoscopic image according to the fog concentration to obtain a defogging image.

2. The method according to claim 1, characterized in that The determining, based on the texture feature, the at least one color feature, and the global contrast feature, that fog exists in the endoscopic image includes: Determining a corresponding first confidence level according to the texture feature; Determining at least one corresponding second confidence level based on the at least one color feature; Determining a corresponding third confidence level according to the global contrast feature; It is determined that fog exists in the endoscopic image based on the first confidence level, the at least one second confidence level, and the third confidence level.

3. The method according to claim 1, characterized in that Determining the fog concentration of the endoscopic image includes: determining a first dark channel image of the endoscopic image; Taking each pixel in the endoscopic image as the center, a plurality of image blocks are obtained according to a preset pixel window; Determine a second dark channel image corresponding to each of the image blocks; The fog concentration of the endoscope image is determined according to the first dark channel image and the second dark channel image.

4. The method according to claim 3, characterized in that The determining, according to the first dark channel map and the second dark channel map, the fog concentration of the endoscope image includes: Determining a first statistical eigenvalue of the first dark channel image according to the first dark channel image; Determining a second statistical eigenvalue of the second dark channel image according to the second dark channel image; The fog concentration of the endoscope image is determined according to the first statistical characteristic value and the second statistical characteristic value.

5. The method according to any one of claims 1 to 4, characterized in that The extracting of the global contrast feature of the endoscopic image comprises: determining a brightness value for each pixel in the endoscopic image; Sorting the brightness value of each pixel to obtain a first sequence; Extracting at least one first brightness value at the head of the first sequence according to a first preset ratio; and extracting at least one second brightness value at the tail of the first sequence according to a second preset ratio; The global contrast feature is determined based on the at least one first luminance value and the at least one second luminance value.

6. The method according to any one of claims 1 to 4, characterized in that The extracting of texture features of the endoscopic image comprises: determining a pixel value variance of the endoscopic image; Performing edge detection on the endoscopic image to obtain corresponding edge strength; Obtaining a first weight corresponding to the pixel value variance and a second weight corresponding to the edge strength; The pixel value variance and the edge strength are fused according to the first weight and the second weight to obtain the texture feature.

7. The method according to claim 3, characterized in that The defogging process is performed on the endoscopic image according to the fog concentration to obtain a defogged image, comprising: determining a defogging intensity according to the fog concentration; Dark channel defogging is performed on the endoscopic image according to the defogging strength to obtain a defogging image.

8. The method according to claim 7, characterized in that The method further comprises: determining a global atmospheric light intensity of the endoscopic image in each color channel; The step of performing dark channel defogging on the endoscopic image according to the defogging intensity to obtain a defogged image includes: Normalizing the pixel values ​​of each color channel in the endoscopic image according to the global atmospheric light intensity of each color channel to obtain a normalized image; Determining a third dark channel image of the normalized image; determining a transmittance of the endoscopic image according to the defogging intensity and the third dark channel map; The endoscopic image is defogged according to the global atmospheric light intensity of each color channel and the transmittance to obtain a defogged image.

9. The method according to claim 8, characterized in that Determining the global atmospheric light intensity of the endoscopic image in each color channel includes: sorting the pixel value of each pixel in the first dark channel image to obtain a second sequence; extracting at least one pixel value at the head of the second sequence according to a third preset ratio; In the endoscopic image, determining a pixel position corresponding to the at least one pixel value; Obtain the channel value of the pixel corresponding to the pixel position in each color channel; The global atmospheric light intensity of each color channel is determined according to the channel value.

10. An endoscope system, characterized in that: The endoscope system includes: an image acquisition device and a processor, the image acquisition device is used to acquire endoscopic images, and the processor executes the method according to any one of claims 1 to 9.

Images