Endoscope video image filtering enhancement method

Abstract

This invention belongs to the field of medical image processing technology, specifically relating to a method for filtering and enhancing endoscopic video images. This invention achieves reliable restoration of tissue texture beneath specular highlight-occluded areas in endoscopic videos through five processing stages. The first stage uses a three-factor joint decision to adaptively detect highlight regions in video frames and generate a time-varying highlight mask, effectively distinguishing between moving highlights and fixed bright spots. The second stage uses hierarchical sparse Lucas-Kanade optical flow tracing and RANSAC affine transformation estimation to accurately map highlight-occluded pixels to corresponding non-highlight tissue locations in historical frames, constructing a candidate texture set. The third stage uses a two-color reflectance model to extract tissue intrinsic colors and perform illumination normalization migration on the candidate textures, eliminating inter-frame illumination differences. The fourth stage constructs confidence scores using the product of three-dimensional weights and obtains the restored color through weighted median aggregation. The fifth stage applies guided filtering and inter-frame exponential smoothing to output a temporally consistent enhanced video.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more particularly to a method for filtering and enhancing endoscopic video images. Background Technology

[0002] Rigid optical endoscopes are core diagnostic and treatment equipment for minimally invasive surgery. Their rigid bodies and fixed optical paths offer advantages such as stable imaging and low optical distortion, making them a core tool for intraoperative clinical observation. Meanwhile, fluorescence imaging modalities (such as ICG fluorescence angiography, autofluorescence imaging, and narrow-band fluorescence imaging) have become core functions for lesion identification, blood supply assessment, and boundary delineation during rigid endoscopy, with significant clinical professional demand.

[0003] During endoscopic procedures, reflections from the optical end face of the rigid endoscope, metallic reflections from surgical instruments, specular reflections from moist tissue, and direct reflections from fluorescent excitation light can all cause strong highlight areas in the video image. The pixel values ​​in these areas approach saturation, completely obscuring the underlying tissue texture and fluorescence signals. This information is precisely the core basis for determining lesion boundaries and tissue blood supply during the procedure. At the same time, fluorescence channel images have the characteristics of low signal-to-noise ratio, narrow dynamic range of weak fluorescence signals, and susceptibility to being obscured by highlights. Current technology cannot simultaneously meet the dual professional requirements of reliable restoration of white light mode highlight texture and fidelity preservation of weak fluorescence mode signals.

[0004] Existing technologies for addressing the specular interference problem in endoscopic images can be broadly categorized into two types. The first type comprises specular suppression methods based on single-frame images, including specular separation algorithms based on dual-color reflectance models and image inpainting algorithms based on deep learning. These methods utilize only the spatial pixel information of the current frame and cannot recover any real tissue texture and fluorescence signals from pixel-level saturated regions. They rely solely on spatial interpolation of surrounding pixels, leading to blurred and distorted textures in the restored area. Deep learning inpainting methods, due to the highly diverse distribution of tissue textures and fluorescence signals, are prone to producing illusory textures that do not match real tissue, lacking clinical interpretability. The second type focuses on processing methods for endoscopic videos. Existing video processing technologies primarily concentrate on denoising and motion blur removal for flexible endoscopes. There are currently no methods systematically adapted to the rigid body motion characteristics of rigid endoscopes and compatible with fluorescence modality image processing. Direct multi-frame fusion without precise rigid body registration can produce severe ghosting artifacts in the restored area and easily cause excessive smoothing loss of weak fluorescence signals, making it unsuitable for clinical scenarios involving minimally invasive rigid endoscope surgery. Summary of the Invention

[0005] The purpose of this invention is to provide an endoscopic video image filtering and enhancement method to solve the problems existing in the prior art.

[0006] This invention provides a method for filtering and enhancing endoscopic video images, comprising the following steps:

[0007] S100, Get the current frame And its previous N frames of historical frame sequence, converting each frame from RGB color space to HSV color space, for each pixel position Calculate the saturation drop term Brightness peak item and time series variation terms Three binary quantities, according to the formula Calculate the overall specular score. When the score exceeds the decision threshold, the pixel is identified as a specular pixel. Perform morphological dilation on the decision result to obtain the specular mask. ;

[0008] S200, in the specular mask K feature anchor points are extracted from a ring-shaped region extending 10 pixels outward from the boundary. The sparse Lucas-Kanade optical flow algorithm is used on a 3-layer image pyramid to trace these K anchor points from the current frame to each historical frame. For each historical frame interval, the RANSAC algorithm is used to estimate a 6-parameter affine transformation matrix from the anchor point displacements. ,pass The inverse transform maps the coordinates of each pixel to be restored within the specular mask to the corresponding position in the historical frame, and selects pixels in the historical frame whose coordinates are in the non-spectral region to form a candidate set. ;

[0009] S300 uses the mean chromaticity distribution of highlight pixels within the first 10 frames of a video sequence to establish a light source chromaticity vector. For each candidate pixel in the candidate set, a two-color reflectance line is fitted within its 7×7 neighborhood to extract the tissue intrinsic color. According to the formula The candidate pixels are subjected to illumination normalization migration to obtain the candidate texture color after chromaticity correction;

[0010] S400, for each candidate frame interval in the candidate set Calculate motion confidence separately , ... and time confidence According to the formula Calculate the normalized comprehensive confidence weight, sort the chroma correction color values ​​of each candidate frame by channel and accumulate the weight, and take the candidate value corresponding to the first time the accumulated weight exceeds 0.50 as the restored color of that channel;

[0011] S500, using the original pixels outside the mask in the current frame as the guiding image, applies a guiding filter to the restored pixels inside the mask, and then follows the formula Temporal smoothing is applied to the restored region, and the restored pixels inside the mask are merged with the original pixels outside the mask to output an enhanced frame. .

[0012] Furthermore, in S100, The determination is based on the global saturation average of the current frame. 0.30 times is the adaptive threshold. When the pixel saturation component is lower than this threshold, the value is 1; otherwise, the value is 0. The determination is based on the average brightness of the current frame. Add 3 times the standard deviation of brightness This is a threshold value; the value is 1 when the pixel luminance component exceeds this threshold, and 0 otherwise. The determination is based on the condition that the temporal variance of the pixel's brightness within the past N frames exceeds 1.5 times the mean temporal variance of the entire image. If this condition is met, the value is 1; otherwise, the value is 0. The three weighting coefficients are set to... , , The decision threshold is set to 0.60, the morphological dilation radius is 3 pixels, and N is set to 5.

[0013] Furthermore, in S200, feature anchors are extracted using the Shi-Tomasi corner detection operator, with the number K set to 50. Lucas-Kanade optical flow tracing employs a 3-layer image pyramid with a scaling factor of 0.5 for each layer, a maximum of 20 iterations, and a convergence threshold of 0.03 pixels. When estimating the affine transformation matrix using RANSAC, the maximum number of iterations is 500, the inlier determination distance threshold is 2 pixels, and if the inlier ratio is less than 0.40, the historical frame is marked as an invalid frame and excluded from the candidate set.

[0014] Furthermore, in S200, the average displacement amplitude of all valid anchor points in the current frame is calculated. Determine motion state: when When the number of pixels per frame is less than 2, dense optical flow is added to the affine estimation for refined registration; when When the pixel coordinate mapping is between 2 pixels per frame and 15 pixels per frame, the affine transformation matrix is ​​used directly for pixel coordinate mapping; when... If the number of pixels per frame exceeds 15, the historical frame is marked as an unregisterable frame and removed from the candidate set.

[0015] Furthermore, in S300, the light source chromaticity vector After the initial 10 frames of the video sequence, an exponential average dynamic update is performed with an update weight of 0.05; in the formula... This represents the average brightness value of a 15-pixel annular neighborhood surrounding the boundary of the specular mask in the current frame. For historical frames The average brightness value of a local area of ​​the same size as the candidate pixel.

[0016] Furthermore, in S400, motion confidence... The value is the proportion of RANSAC interior points. ; Confidence level of sharpness The value is the normalized result of the mean square value of the Laplacian response within the 7×7 neighborhood of the candidate pixel; time confidence. According to the formula Calculate the attenuation coefficient. The value is 0.30.

[0017] Furthermore, in S400, when the candidate set When the number of valid candidate frames is less than 2, anisotropic diffusion filling is performed on all specular pixels in the mask along the gradient vector field of the pixels outside the mask boundary. The filling direction is determined by interpolation of the gradient vector field of the pixels outside the mask boundary, and a low confidence marker is written in this area. The low confidence marker area is not subjected to exponential moving average processing in the temporal smoothing step of S500.

[0018] Furthermore, in S500, the radius of the guided filter's filtering window... The value is 8 pixels, and the regularization parameter is... Values Smoothing intensity coefficient in time series smoothing formula The value is 0.25, and time-series smoothing is applied only to the recovery region where the confidence flag is high.

[0019] The beneficial effects of this invention are:

[0020] First, the present invention introduces a temporal variance decision term in the highlight detection stage. By analyzing the brightness change characteristics of pixels in N consecutive frames, it distinguishes between real highlights that drift with the movement of the lens and static bright spots in fixed positions. Compared with existing detection methods that rely solely on the HSV threshold of a single frame, this effectively reduces the false detection rate of normal bright tissue areas and improves the accuracy of the highlight mask.

[0021] Second, this invention proposes a registration strategy that extracts feature anchor points and estimates local affine transformations within the annular buffer region surrounding the specular mask boundary. This avoids the problem of optical flow tracing failure caused by directly extracting feature points within the pixel-saturated specular region. At the same time, it combines a motion state classification mechanism to identify and remove rapidly moving frames. Even under the condition of large displacement and rapid movement of the endoscope, reliable inter-frame organization block registration can still be achieved, providing accurate spatial correspondence for subsequent multi-frame texture aggregation.

[0022] Third, this invention combines illumination normalization migration constrained by the dual-color reflectance model with multi-dimensional confidence weighted median aggregation. On the basis of eliminating the inter-frame brightness difference caused by the change in the distance between the endoscope and the tissue, it effectively suppresses the influence of abnormal pixels introduced by registration error by using a weighted median rather than a weighted mean fusion method. After guided filtering and temporal exponential smoothing post-processing, the color and texture of the restored area maintain structural consistency with the surrounding real tissue in space and maintain inter-frame continuity stability in time. The output result has the credibility required for clinical diagnosis. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the overall method of the present invention;

[0025] Figure 2 This is a flowchart of the highlight detection process of the present invention;

[0026] Figure 3 This is a schematic diagram of optical flow tracing and registration according to the present invention;

[0027] Figure 4 This is a flowchart of the color correction and weighted aggregation process of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0029] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0030] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0031] See Figures 1 to 4 As shown

[0032] This invention provides an endoscope video image filtering enhancement method. Based on the temporal redundancy information of the endoscope video sequence, the method achieves reliable restoration of tissue texture below the specular highlight masking area in the endoscope video through five stages: time-varying highlight detection, mask-guided optical flow tracing, dual-color reflection constrained chromaticity correction, multi-frame confidence weighted aggregation, and temporal consistency post-processing. This eliminates highlight interference and maintains the consistency between video frames.

[0033] S100 Time-Varying Highlight Area Detection

[0034] S101: Get the current frame and its previous N historical frame sequences With N set to 5, each frame is converted from the RGB color space to the HSV color space, resulting in the hue channel H, saturation channel S, and luminance channel V.

[0035] S102: Calculate the global saturation mean of the current frame. Set the adaptive saturation threshold to For each pixel position in the current frame Determine its saturation component If the saturation level is below a certain threshold, the result is recorded as 1; otherwise, it is recorded as 0, thus generating a saturation drop plot. .

[0036] S103: Calculate the average value of the current frame's luminance channel. with standard deviation For each pixel location, determine its luminance component. Does it exceed If it exceeds, a brightness spike will appear. The position is recorded as 1, otherwise it is recorded as 0.

[0037] S104: For each pixel location, calculate its luminance value sequence over the past N frames. Time variance Threshold the time series variance Set to 1.5 times the mean temporal variance of the entire image; if the temporal variance of this pixel exceeds... Then the time series change term Record it as 1 if it is not, otherwise record it as 0. This item is used to distinguish between real highlights that drift with the movement of the lens and bright spots in a fixed position.

[0038] S105: Calculate the overall highlight score for each pixel using the following formula:

[0039]

[0040] in These are the weighting coefficients for the saturation drop term, the brightness peak term, and the time-series variation term, respectively, and the three satisfy the following: The values ​​are respectively , All three values ​​are binary values ​​of 0 or 1. When the weighted score of the three values ​​exceeds the decision threshold of 0.60, the pixel is determined to be a highlight pixel. The decision results are then combined into a binary mask. The value is 1 for highlight areas and 0 for non-highlight areas.

[0041] S106: Mask A morphological dilation process with a radius of 3 pixels is performed to eliminate residual specular edge gradients, resulting in the final specular mask. The mask is then stored in the temporal mask history library for use in subsequent frame processing.

[0042] S200 mask-guided hierarchical sparse optical flow tracing

[0043] S201: Based on the current frame specular mask Within the annular buffer region formed by extending 10 pixels outward from the mask boundary, K feature anchors are extracted using the Shi-Tomasi corner detection operator. With K set to 50, the pixels within this annular region are located in the specular neighborhood, have rich texture gradients, and are not subject to pixel saturation, thus providing a reliable basis for optical flow calculation.

[0044] S202: For each historical frame interval The sparse Lucas-Kanade optical flow algorithm is used on a 3-layer image pyramid to move K anchor points from the current frame. Tracking historical frames This yields the displacement vector for each anchor point. The scaling factor for each layer of the pyramid is 0.5, the maximum number of iterations is 20, and the convergence threshold is set to 0.03 pixels.

[0045] S203: For each historical frame interval The RANSAC algorithm is used to displace K anchor points. Robust estimation of 6-parameter local affine transformation matrix The maximum number of RANSAC iterations is 500, the inlier determination distance threshold is 2 pixels, and the inlier ratio is recorded. Used for subsequent confidence calculations; when the proportion of interior points is less than 0.40, it is considered that the first... If frame registration is unreliable, mark it as an invalid frame and exclude it.

[0046] S204: Calculate the average displacement amplitude of all valid anchor points in the current frame. The motion state is determined according to the following rules: when Each pixel is determined to be in a slow motion state per frame, and dense optical flow is added to refine the registration based on affine estimation; when When the motion is between 2 pixels per frame and 15 pixels per frame, it is considered a normal motion state, and an affine transformation matrix is ​​directly applied; when... If a pixel is determined to be in a rapid jitter state at each frame, the frame is marked as unregisterable and removed from the candidate frame set.

[0047] S205: For each pixel position to be restored within the specular mask of the current frame. Through affine transformation matrix The inverse transform is used to calculate its value in historical frames. Corresponding tissue block coordinates To check if the coordinate is located in a non-highlight area in historical frames, i.e. The historical frame positions that meet the conditions are recorded as candidate texture sources, forming a candidate set. .

[0048] Chromaticity correction constrained by the S300 dual-color reflectance model

[0049] S301: Establish a normalized chromaticity vector for the endoscope light source using the average chromaticity values ​​of confirmed highlight pixels in the current frame and historical frames. The vector is established by statistically analyzing the mean chromaticity distribution of highlight pixels in the first 10 frames of the video sequence, and is dynamically updated in subsequent frame processing using an exponential average method with an update weight of 0.05.

[0050] S302: For the candidate set Each candidate pixel position Within its 7×7 pixel neighborhood, a two-color reflection line is fitted. The color vectors of all non-spectral pixels in the neighborhood are decomposed into diffuse and specular components, and the tissue eigencolor of the candidate pixel is extracted. The intrinsic color of a tissue is the diffuse color after removing specular components, representing the tissue's inherent biological color information.

[0051] S303: Perform illumination normalization migration for each candidate texture pixel, normalizing the brightness level of the candidate pixel to the local illumination level of the current frame, and then correcting the color of the candidate texture pixel. The calculation is as follows:

[0052]

[0053] in The organization's intrinsic color vector is obtained by two-color decomposition of candidate pixels. This represents the average brightness value of a 15-pixel annular neighborhood surrounding the boundary of the highlight region in the current frame. For historical frames The average brightness value of a local area of ​​the same size as the candidate pixel is used to scale the illumination intensity of the candidate pixel to match the local lighting environment of the current frame, thus eliminating brightness differences between different frames caused by changes in the distance between the endoscope and the tissue.

[0054] S400 Multi-Frame Confidence Weighted Median Aggregation

[0055] S401: For the candidate set Interval between each candidate frame Calculate the three-dimensional confidence sub-weights, where the motion confidence... The value is the proportion of RANSAC interior points. Clarity and confidence The value is the mean squared normalized result of the Laplacian response within the 7×7 neighborhood of the candidate pixel, and the time confidence. According to the exponential decay formula Calculate the attenuation coefficient. The value is 0.30.

[0056] S402: Multiply and combine the three sub-weights to obtain the comprehensive confidence weight for each candidate frame, and normalize the weights of all valid candidate frames. The sum of the normalized weights is 1. The formula for the comprehensive confidence weight is as follows:

[0057]

[0058] in For the first Intra-point ratio for optical flow registration of candidate frames For the first The normalized value of local image sharpness at the candidate position in the candidate frame. For exponentially decaying weights based on time distance, The current pixel to be restored The set of all valid candidate frames is given, and the denominator is the sum of the weight products of all candidate frames in the set, which is used for normalization.

[0059] S403: Correct the color values ​​of each frame in the candidate set. Sort the candidate values ​​by color channel separately, and use a weighted median method to determine the final restored color value: for each channel, arrange the candidate values ​​in ascending order and accumulate the weights corresponding to each candidate value. The candidate value corresponding to the first time the cumulative weight exceeds 0.50 is taken as the restoration result of the channel. Compared with the weighted mean, the weighted median method has a significant suppression effect on abnormal pixels introduced by registration error.

[0060] S404: When the number of valid candidate frames is less than 2, start the fallback space filling strategy: perform anisotropic diffusion filling along the local gradient direction of the specular mask boundary, the filling range is all specular pixels within the mask, the filling direction is determined by the gradient vector field interpolation of the pixels outside the mask boundary, and write a low confidence flag bit in the metadata of this area. This flag bit can be used as a reference by the diagnostic auxiliary module of the endoscope image processing system.

[0061] S500 Timing Consistency Edge Preservation Post-Processing

[0062] S501: Using the original, unmodified pixels outside the mask region of the current frame as the guide image, apply guide filtering to the restored pixels within the mask. The filter window radius... The value is 8 pixels, and the regularization parameter is... Values Guided filtering uses the structural information of the real tissue edges in the guided image to constrain the smooth direction of the restored region, so that the edge response of the restored texture maintains structural consistency with the surrounding real tissue and avoids blurring at the restoration boundary.

[0063] S502: Apply an exponential moving average constraint to the restored regions between consecutive frames, and perform weighted fusion based on the high-confidence restoration result of the current frame as the primary factor and the final output result of the previous frame as a secondary factor. This eliminates the slight color jumps caused by independent processing between frames, and finally outputs the color of the restored region of the frame. The calculation is as follows:

[0064]

[0065] in The restored color vector is obtained after weighted median aggregation of the current frame in step S400. This is the final color vector output after processing the previous frame in this step. The temporal smoothing intensity coefficient is set to 0.25. This step is applied only to the restored region where the confidence level is high. Temporal smoothing is not applied to the low-confidence region to prevent the accumulation and propagation of erroneous textures.

[0066] S503: Merges the original pixels outside the mask with the restored pixels inside the mask after processing by S501 and S502, and outputs the final enhanced video frame of the current frame. The process iterates through each frame of the video sequence to complete the highlight filtering and enhancement of the entire endoscopic video.

[0067] Example 1

[0068] In this embodiment, the above method is applied to a white light colonoscopy video. The video was captured by an Olympus CF-H290I endoscope with a resolution of 1920×1080 pixels, a frame rate of 30 frames per second, and a video duration of 120 seconds. The video content includes the process of intestinal mucosal inspection and polyp observation. There are obvious specular highlight areas in both scenarios.

[0069] In the S101 processing stage, each frame in the video sequence is converted from the RGB color space to the HSV color space, with the current frame as the starting point. Global saturation mean Based on this, the adaptive threshold calculation result has a dynamic range of 0.08 to 0.21 throughout the entire video, ensuring that the threshold is automatically adjusted according to changes in endoscope exposure conditions, without relying on a fixed threshold.

[0070] In processing stages S102 to S105, time-varying highlight scores are calculated for each frame of the image. Taking a typical frame from the inspection segment as an example: there is a highlight area of ​​approximately 2400 square pixels in the center-right region of this frame. The average saturation of this region is 0.06, which is lower than the adaptive threshold of 0.17. The value is 1; the mean brightness is 247, which exceeds the mean brightness of the entire image plus three standard deviations, a value of 239. The value is 1; due to the continuous camera movement, the location of this area has shifted by approximately 8 pixels over the past 5 frames, with a temporal brightness variance of 486, exceeding 1.5 times the mean temporal variance of the entire image. The item score is 1; the weighted score of the three items is 1.00, which far exceeds the judgment threshold of 0.60, and the area is correctly identified as a highlight area.

[0071] In the S106 processing stage, the above judgment results are subjected to morphological dilation processing. After dilation, the area of ​​the specular mask is expanded to 2784 square pixels, and the width of the boundary region is increased by about 3 pixels, effectively covering the transition gradient region at the edge of the specular highlight.

[0072] In processing stages S201 to S202, a total of 47 Shi-Tomasi corner points were detected within a ring-shaped region 10 pixels outside the specular mask boundary, satisfying the approximation requirement of K=50. These 47 anchor points were then traced forward on the 3-layer pyramid to... to A total of 5 frames were captured, and the tracking success rate was [missing information]. to The frame rate reached 93.6%, in and The cumulative error in the frame decreased to 81.2%, but it was still higher than the effective threshold.

[0073] In the S203 processing stage, with The frame serves as an illustration: RANSAC identified 39 inliers from 43 successfully tracked anchor points, representing an inlier ratio of 0.907, and estimated the affine transformation matrix. The matrix parameters show that the lens shifts to the right by about 5.3 pixels, down by about 2.1 pixels, and rotates by about 0.4 degrees, which matches the actual lens motion characteristics.

[0074] During the S204 processing stage, the average displacement amplitude of the anchor point is 5.7 pixels per frame, which is within the normal motion range of 2 to 15 pixels per frame. All 5 candidate frames enter the subsequent processing flow, and no frame is judged to be in a rapid shaking state.

[0075] In the S205 processing stage, an inverse affine transformation is performed on each of the 2784 pixels to be restored within the specular mask, and the corresponding coordinates in each candidate frame are checked to see if they are located in a non-spectral region. In the frame, 2651 out of 2784 pixel coordinates were mapped to non-highlight areas, achieving an effectiveness rate of 95.2%; to Within the frames, due to the continuous movement of the lens during this period, the position of the highlight area shifted, with effectiveness rates of 93.7%, 91.4%, 88.9%, and 83.6%, respectively. All valid candidates were included in the candidate set.

[0076] During processing stages S301 to S303, the chromaticity distribution of highlight pixels is statistically analyzed within the first 10 frames of the video to determine the light source chromaticity vector. The normalized RGB ratio is It has characteristics close to white light source and is consistent with the color temperature characteristics of the xenon lamp light source of the Olympus CF-H290I endoscope. After extracting the tissue intrinsic color for each candidate pixel, the chromaticity is scaled by the ratio of the local average brightness of the current frame to the local average brightness of the historical frames. The correction coefficient ranges from 0.84 to 1.17 in this video segment, covering the range of light intensity fluctuations caused by changes in the depth of entry.

[0077] In the S401 to S402 processing stages, the central pixel of the highlight area to be restored is used as an example: the motion confidence of this pixel in the five candidate frames is 0.907, 0.893, 0.871, 0.844 and 0.819, the sharpness confidence is 0.85, 0.81, 0.78, 0.72 and 0.65, the temporal confidence is 0.741, 0.549, 0.407, 0.301 and 0.223, and the normalized comprehensive weights are 0.312, 0.256, 0.207, 0.151 and 0.074, respectively. The candidate weight of the first frame is the highest, which is consistent with the expectation that the closer the time distance, the greater the reference value.

[0078] In the S403 processing stage, the candidate values ​​of the R channel for this pixel are sorted in ascending order. The corresponding cumulative weight is The cumulative weight first exceeded 0.50 at the fourth candidate value of 146. Therefore, the restored value of the R channel of this pixel is 146. The G channel and B channel were restored to 124 and 108 respectively using the same method. After restoration, the color of this pixel is a normal mucosal pink tone, which is visually consistent with the color of the surrounding tissue that is not obscured by highlights.

[0079] In the S501 processing stage, guided filtering is applied to the restored area inside the mask using the real pixels outside the mask as the guide image. After filtering, the absolute error of the boundary gradient between the restored area and the surrounding real tissue decreases from the mean of 11.3 gray levels before filtering to 3.7 gray levels, and the naturalness of the boundary transition is significantly improved.

[0080] In the S502 processing stage, temporal smoothing is applied to the highlight restoration results of 10 consecutive frames in the polyp observation segment. Before smoothing, the mean color standard deviation of the restoration area between adjacent frames is 8.2 gray levels. After smoothing, it drops to 3.1 gray levels, effectively suppressing the inter-frame flickering phenomenon. The output video presents stable and continuous tissue color visually, providing clinicians with clear and reliable mucosal texture information.

[0081] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0082] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for filtering and enhancing endoscopic video images, characterized in that, Includes the following steps: S100, Get the current frame And its previous N frames of historical frame sequence, converting each frame from RGB color space to HSV color space, for each pixel position Calculate the saturation drop term Brightness peak item and time series variation terms Three binary quantities, according to the formula Calculate the overall specular score. When the score exceeds the decision threshold, the pixel is identified as a specular pixel. Perform morphological dilation on the decision result to obtain the specular mask. ; S200, in the specular mask K feature anchor points are extracted from a ring-shaped region extending 10 pixels outward from the boundary. The sparse Lucas-Kanade optical flow algorithm is used on a 3-layer image pyramid to trace these K anchor points from the current frame to each historical frame. For each historical frame interval, the RANSAC algorithm is used to estimate a 6-parameter affine transformation matrix from the anchor point displacements. ,pass The inverse transform maps the coordinates of each pixel to be restored within the specular mask to the corresponding position in the historical frame, and selects pixels in the historical frame whose coordinates are in the non-spectral region to form a candidate set. ; S300 uses the mean chromaticity distribution of highlight pixels within the first 10 frames of a video sequence to establish a light source chromaticity vector. For each candidate pixel in the candidate set, a two-color reflectance line is fitted within its 7×7 neighborhood to extract the tissue intrinsic color. According to the formula The candidate pixels are subjected to illumination normalization migration to obtain the candidate texture color after chromaticity correction; S400, for each candidate frame interval in the candidate set Calculate motion confidence separately , ... and time confidence According to the formula Calculate the normalized comprehensive confidence weight, sort the chroma correction color values ​​of each candidate frame by channel and accumulate the weight, and take the candidate value corresponding to the first time the accumulated weight exceeds 0.50 as the restored color of that channel; S500, using the original pixels outside the mask in the current frame as the guiding image, applies a guiding filter to the restored pixels inside the mask, and then follows the formula Temporal smoothing is applied to the restored region, and the restored pixels inside the mask are merged with the original pixels outside the mask to output an enhanced frame. .

2. The method according to claim 1, characterized in that, In S100, The determination is based on the global saturation average of the current frame. 0.30 times is the adaptive threshold. When the pixel saturation component is lower than this threshold, the value is 1; otherwise, the value is 0. The determination is based on the average brightness of the current frame. Add 3 times the standard deviation of brightness This is a threshold value; the value is 1 when the pixel luminance component exceeds this threshold, and 0 otherwise. The determination is based on the condition that the temporal variance of the pixel's brightness within the past N frames exceeds 1.5 times the mean temporal variance of the entire image. If this condition is met, the value is 1; otherwise, the value is 0. The three weighting coefficients are set to... , , The decision threshold is set to 0.60, the morphological dilation radius is 3 pixels, and N is set to 5.

3. The method according to claim 1, characterized in that, In S200, feature anchors are extracted using the Shi-Tomasi corner detection operator, with a quantity K of 50. Lucas-Kanade optical flow tracing uses a 3-layer image pyramid with a scaling factor of 0.5 for each layer, a maximum number of iterations of 20, and a convergence threshold of 0.03 pixels. When estimating the affine transformation matrix using RANSAC, the maximum number of iterations is 500, the inlier determination distance threshold is 2 pixels, and if the inlier ratio is less than 0.40, the historical frame is marked as an invalid frame and excluded from the candidate set.

4. The method according to claim 1, characterized in that, In S200, the average displacement amplitude of all valid anchor points in the current frame is calculated. Determine motion state: when When the number of pixels per frame is less than 2, dense optical flow is added to the affine estimation for refined registration; when When the pixel coordinate mapping is between 2 pixels per frame and 15 pixels per frame, the affine transformation matrix is ​​used directly for pixel coordinate mapping; when... If the number of pixels per frame exceeds 15, the historical frame is marked as an unregisterable frame and removed from the candidate set.

5. The method according to claim 1, characterized in that, In S300, the light source chromaticity vector After the initial 10 frames of the video sequence, an exponential average dynamic update is performed with an update weight of 0.05; in the formula... This represents the average brightness value of a 15-pixel annular neighborhood surrounding the boundary of the specular mask in the current frame. For historical frames The average brightness value of a local area of ​​the same size as the candidate pixel.

6. The method according to claim 1, characterized in that, In S400, motion confidence The value is the proportion of RANSAC interior points. ; Confidence level of sharpness The value is the normalized result of the mean square value of the Laplacian response within the 7×7 neighborhood of the candidate pixel; time confidence. According to the formula Calculate the attenuation coefficient. The value is 0.

30.

7. The method according to claim 1, characterized in that, In S400, when the candidate set When the number of valid candidate frames is less than 2, anisotropic diffusion filling is performed on all specular pixels in the mask along the gradient vector field of the pixels outside the mask boundary. The filling direction is determined by interpolation of the gradient vector field of the pixels outside the mask boundary, and a low confidence marker is written in this area. The low confidence marker area is not subjected to exponential moving average processing in the temporal smoothing step of S500.

8. The method according to claim 1, characterized in that, In S500, the radius of the guided filter window The value is 8 pixels, and the regularization parameter is... Values Smoothing intensity coefficient in time series smoothing formula The value is 0.25, and time-series smoothing is applied only to the recovery region where the confidence flag is high.

Images