Urological endoscope image restoration method based on scattering physical model

Abstract

The application relates to the technical field of image processing, and particularly discloses a urinary surgery endoscope image restoration method based on a scattering physical model, which comprises the following steps: acquiring an original image collected by an endoscope and performing pretreatment to generate a standardized image; determining a scattering parameter constraint domain of a total attenuation coefficient based on the optical scattering characteristics of a surgical liquid medium; extracting image statistical features from the standardized image, wherein the image statistical features comprise a dark area pixel statistical quantity and a contrast attenuation quantity, and extracting ambient scattering light brightness based on the dark area pixel statistical quantity. The method provided by the application establishes a constraint domain based on the optical characteristics of a specific surgical liquid medium, and combines the image dark area pixel statistical quantity and the contrast attenuation quantity to online calculate a comprehensive scattering parameter in the constraint domain, so that the model parameter can be adaptively updated according to the fluctuation of the turbidity state of the liquid medium in surgery.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for restoring urological endoscopic images based on a scattering physics model. Background Technology

[0002] In urological endoscopic surgery, it is usually necessary to continuously infuse the surgical cavity with a surgical fluid medium to maintain the field of vision. This medium contains suspended particles such as microbubbles, tissue debris, and blood. These suspended particles scatter and absorb the endoscopic illumination, causing a decrease in contrast and blurring of details in the acquired images. Existing image restoration methods are mostly based on atmospheric defogging models or natural underwater imaging models, using relevant statistical priors to estimate model parameters, thereby performing image inversion.

[0003] However, the optical scattering characteristics of surgical fluids differ from those of natural atmospheres or open water bodies. Directly applying a general scattering model prior to the surgical environment can lead to a mismatch in the range of scattering parameter values. Furthermore, the concentration of suspended particles within the surgical fluid changes dynamically during the procedure. Existing methods typically use fixed parameters or offline calibration parameters, which cannot achieve online adaptive adjustment within the corresponding constraint domain as intraoperative conditions change.

[0004] The above situation causes existing methods to have biases in estimating scattering parameters when processing endoscopic images, making it difficult to maintain stable recovery accuracy in dynamic surgical environments. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the objective of this invention is to propose a method for restoring urological endoscopic images based on a scattering physics model, so as to dynamically restore the surface texture features of the tissue in the obscured area according to real-time surgical conditions.

[0006] To achieve the above objectives, a first aspect of the present invention proposes a method for restoring urological endoscopic images based on a scattering physics model, comprising:

[0007] The raw images acquired by the endoscope are acquired and preprocessed to generate standardized images; and the scattering parameter constraint domain of the total attenuation coefficient is determined based on the optical scattering characteristics of the surgical fluid medium.

[0008] Image statistical features are extracted from the standardized image, including pixel statistics in dark areas and contrast attenuation. Ambient scattered light intensity is extracted based on the pixel statistics in dark areas. An analytical mapping relationship between the image statistical features and the total attenuation coefficient is established, and the comprehensive scattering parameter value is determined online within the scattering parameter constraint domain.

[0009] Based on the ambient scattered light brightness, the initial transmittance map of the standardized image is calculated using the dark channel prior, and a refined transmittance map is generated through edge-aware refinement.

[0010] The refined transmittance map is truncated by a lower limit threshold to generate a safe transmittance map. The standardized image, ambient scattered light brightness, and safe transmittance map are substituted into the forward scattering degradation model for algebraic inversion to obtain the restored image.

[0011] The restored image is converted to a luminance-chrominance separated color space, and a global mean shift correction is performed on the chrominance channel to eliminate color shift between channels, generating the final output image.

[0012] To achieve the above objectives, a second aspect of the present invention provides a urological endoscopic image restoration system based on a scattering physics model, comprising:

[0013] The image preprocessing and constraint domain determination module is used to acquire the original images obtained by the endoscope and preprocess them to generate standardized images; and to determine the scattering parameter constraint domain of the total attenuation coefficient based on the optical scattering characteristics of the surgical fluid medium.

[0014] The online scattering parameter determination module is used to extract image statistical features from the standardized image, the image statistical features including dark area pixel statistics and contrast attenuation, and extract ambient scattered light brightness based on the dark area pixel statistics; establish an analytical mapping relationship between the image statistical features and the total attenuation coefficient, and determine the comprehensive scattering parameter value online within the scattering parameter constraint domain;

[0015] The transmittance calculation and refinement module is used to calculate the initial transmittance map of the standardized image based on the ambient scattered light brightness using the dark channel prior, and generate a refined transmittance map through edge-aware refinement.

[0016] The model inversion and restoration module is used to perform lower limit threshold truncation on the refined transmittance map to generate a safe transmittance map. The standardized image, ambient scattered light brightness and safe transmittance map are substituted into the forward scattering degradation model for algebraic inversion to obtain the restored image.

[0017] The color fidelity correction module is used to convert the restored image to a luminance-chrominance separated color space, perform global mean shift correction on the chrominance channel, eliminate color shift between channels, and generate the final output image.

[0018] To achieve the above objectives, a third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described method for restoring urological endoscopic images based on a scattering physics model.

[0019] The method provided by this invention establishes a constraint domain based on the optical properties of the specific surgical fluid medium. Within this constraint domain, it combines pixel statistics in dark areas of the image with contrast attenuation to calculate comprehensive scattering parameters online, enabling the model parameters to adaptively update according to fluctuations in the turbidity of the fluid medium during surgery. In the actual restoration process, a safe transmittance map is generated by truncating the refined transmittance map with a lower limit threshold. This effectively suppresses the amplification of far-field noise caused by transmittance approaching zero during algebraic inversion. Furthermore, after inversion, a global mean shift of the chromaticity channel is performed using a luminance-chromaticity separation space, eliminating relative color shift interference caused by independent channel compensation. The overall solution can dynamically restore the surface texture features of the tissue in the occluded area according to the real-time surgical conditions, while maintaining the objective authenticity of the tissue's appearance and color. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the urological endoscopic image restoration method based on a scattering physics model provided by the present invention.

[0021] Figure 2 This is a comparison curve of the peak signal-to-noise ratio of the urological endoscopic image restoration method based on the scattering physics model provided by this invention under different scattering coefficients;

[0022] Figure 3 This is a comparison chart of the peak signal-to-noise ratio and color difference index CIEDE2000 of the urological endoscopic image restoration method based on the scattering physics model provided by this invention under different scattering coefficients.

[0023] Figure 4 This is a waterfall plot showing the contribution of each technical module to the total peak signal-to-noise ratio gain in the urological endoscopic image restoration method based on the scattering physics model provided by this invention.

[0024] Figure 5 This is a radar chart comparing the normalized performance of the urological endoscopic image restoration method based on the scattering physics model provided by this invention across five dimensions.

[0025] Figure 6 This is a schematic diagram illustrating the implementation of the urological endoscopic image restoration system based on a scattering physics model provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the electronic device provided by the present invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] The following description, with reference to the accompanying drawings, describes an embodiment of a urological endoscopic image restoration system, method, and electronic device based on a scattering physics model.

[0029] Example 1:

[0030] like Figure 1 As shown, this embodiment provides a method for restoring urological endoscopic images based on a scattering physics model. The underlying hardware architecture of this method typically includes a urological endoscope optical lens, a built-in near-field point light source illumination system, an image acquisition sensor, and a surgical fluid irrigation pump that provides continuous fluid flushing.

[0031] In routine minimally invasive urological surgeries such as transurethral resection of the prostate (TURP) or percutaneous nephrolithotomy, the surgical cavity must be continuously filled with irrigation fluid to expand the operating space and remove loose tissue. However, surgical resection and electrocoagulation inevitably generate a large number of microbubbles, sloughed tissue debris, and spilled red blood cells in the fluid. These dynamically changing suspended particles constitute a complex optical scattering medium, causing strong multiple scattering and absorption of light during propagation. Ultimately, this results in a degraded image at the image acquisition sensor, characterized by severely reduced contrast, obscured details, and distorted colors.

[0032] To address the aforementioned physical degradation problem, this embodiment constructs a full-link image restoration technology solution, specifically including the following steps:

[0033] Step 1: Image preprocessing and constraint domain determination.

[0034] For example, the first core step in this embodiment is to acquire the raw images obtained by the endoscope and preprocess them to generate standardized images; and to determine the scattering parameter constraint domain of the total attenuation coefficient based on the optical scattering characteristics of the surgical fluid medium. Since the endoscope acquisition system has inherent optical distortion and color temperature deviation, directly physical modeling the raw data will lead to huge errors. Therefore, the pre-processing standardization is indispensable.

[0035] Optionally, during actual system operation, a radial-tangential distortion model is used to perform anti-distortion mapping on the original image, and each color channel is normalized to a unified reference based on the grayscale world algorithm to generate the standardized image with normalized pixel values. For the severe barrel distortion caused by wide-angle endoscope lenses, this solution uses pre-calibrated camera intrinsic parameters to extract radial and tangential distortion coefficients, and recovers the geometric space that conforms to the true physical perspective relationship through coordinate resampling.

[0036] Subsequently, because the color temperature of the LED or xenon light source used at the endoscope tip deviates from standard white light, this solution introduces a grayscale world algorithm to calibrate the illumination color. The calculation logic of the grayscale world algorithm can be described by the following formula:

[0037] ;

[0038] In the formula, This indicates the color channel after white balance calibration. and pixel position Pixel intensity value at; These represent color channel identifiers, corresponding to the red, green, and blue channels respectively; This indicates the intensity value of the corresponding pixel that has undergone anti-distortion mapping processing; This represents the set baseline mean constant for the entire map; This indicates an uncalibrated image in a single color channel. The global pixel average value.

[0039] This formula can eliminate systematic color shift caused by the color temperature of the light source, and strictly constrain the dynamic range of the output standardized image within the preset legal space.

[0040] It is also important to note that after generating the standardized image, the upper and lower bounds of the total attenuation coefficient must be determined based on the type of the surgical fluid medium. Since surgical fluids with different chemical compositions exhibit significant differences in refractive index and particulate matter containment characteristics, this difference directly determines the physical limit of light attenuation. When the medium is physiological saline, a first constraint interval is determined; when the medium is a glycine solution, a second constraint interval is determined; the finite interval formed by these upper and lower bounds serves as the scattering parameter constraint domain that defines the online parameter estimation search space. Physiological saline, in its pure state, exhibits almost no visible light scattering; its turbidity depends entirely on the inclusion of blood and debris, therefore its corresponding first constraint interval is generally lower.

[0041] Glycine solution, as a non-electrolyte irrigation fluid, is widely used in high-frequency electrosurgical resection. The large number of charred protein particles generated during the electrosurgical process further worsens the scattering background of the glycine solution. Therefore, the determined second constraint interval not only has a wider range but also a higher overall upper limit. Restricting the parameter search to a physically meaningful finite interval can effectively prevent the subsequent online inversion process from falling into an ill-posed state of numerical divergence due to extreme noise interference.

[0042] Step 2: Extraction of image statistical features and estimation of brightness.

[0043] Specifically, to achieve online and adaptive parameter estimation, this step extracts image statistical features from the standardized image. These features include pixel statistics in dark areas and contrast attenuation, and the ambient scattered light intensity is extracted based on the pixel statistics in dark areas. These three parameters quantify the degree of scattering degradation in the current fluid medium from different data dimensions.

[0044] For example, to extract the pixel statistics of dark areas, the minimum value of each pixel in the three color channels of the standardized image is taken to generate a dark channel map, and the minimum value is taken within a local window to obtain a local dark channel map; the preset percentile value of the pixel value distribution in the local dark channel map is extracted as the pixel statistics of the dark area.

[0045] The physical basis of this operation lies in the fact that in clear and non-scattering tissue surface imaging, due to shadows or the color saturation of the tissue itself, most areas will always have at least one color channel with reflectance approaching zero. When a scattering medium is involved, the superposition effect of light will significantly increase the pixel value that should originally be extremely dark. This extraction process can be formalized into the following formula:

[0046] ;

[0047] In the formula, Indicates the pixel position The calculated local dark channel intensity value; Represented by pixel position A local rectangular window centered on a specific area; Represents a rectangular window located in local space. Internal traversal pixel coordinates; Represents the normalized image in pixel coordinates and color channels The pixel intensity value; This represents a mathematical operator for finding the minimum value.

[0048] By setting a large preset percentile value, from the entire A scalar is extracted from the numerical distribution of the matrix as a statistical measure of pixels in the dark area. The higher the scalar, the more severe the overall scattering and fogging within the field of view.

[0049] Optionally, this example also calculates the global standard deviation of the grayscale image corresponding to the standardized image, and normalizes the global standard deviation with a preset reference contrast value to obtain the contrast attenuation amount, which reflects the degree of global contrast reduction. Scattering not only increases the brightness of dark areas but also compresses the dynamic range of highlights and shadows in the image. To address this, the following formula is constructed for feature extraction:

[0050] ;

[0051] In the formula, This represents the calculated contrast attenuation scalar value. This represents the global pixel standard deviation variable calculated after the current standardized image is converted to a single-channel grayscale image; This represents a reference contrast value constant that has been pre-measured and solidified under ideal conditions in a clear medium.

[0052] As the medium becomes increasingly turbid, the image tends to become flatter and more uniform. Reduced, thus leading to It approaches the maximum limit value.

[0053] Specifically, the process of extracting ambient scattered light brightness based on the pixel statistics of the dark area includes: selecting the position corresponding to the pixel with the highest brightness in the dark channel image at a predetermined ratio; extracting the maximum pixel value of each color channel in the normalized image at the corresponding position as the ambient scattered light brightness of each color channel. From a physical perspective, the brightest area in the dark channel image often corresponds to the area in the scene furthest from the lens, with the most severe attenuation, and mainly dominated by backscattered light. Therefore, by locating the coordinates of these pixels and tracing back to the original three-channel normalized image to find the peak value, the brightness basis variable purely contributed by the environmental medium can be separated.

[0054] Step 3: Analytical mapping construction and online determination of scattering parameters.

[0055] Specifically, after feature extraction, it is necessary to establish an analytical mapping relationship between the image statistical features and the total attenuation coefficient, and determine the comprehensive scattering parameter value online within the scattering parameter constraint domain. This step is the core link in breaking the limitations of fixed parameters and achieving real-time adaptation during surgery.

[0056] For example, the process of constructing the mapping includes: determining the equivalent distance corresponding to a preset percentile based on the physical range of the endoscopic field of view depth, and constructing the first scattering parameter estimate driven by dark channel features. The calculation formula is as follows:

[0057] ;

[0058] In the formula, This represents the first estimated scattering parameter value derived solely from dark region statistics; This represents the equivalent distance constant mapped based on the standard working distance range for clinical endoscopy and the established cumulative probability distribution. Represents the logarithmic function with the natural constant as the base; This represents a scalar of the dark region pixel statistics obtained in the preceding steps; This represents the estimated ambient scattered light brightness value extracted in the preceding steps. This formula achieves a deep decoupling between the two-dimensional statistical results of the image plane and the three-dimensional attenuation physical law along the optical axis.

[0059] Optionally, to prevent a single feature from failing in specific lesion scenarios, this scheme simultaneously determines the mean distance within the endoscopic field of view based on the global mean of transmittance approximation assumption, constructing a second scattering parameter estimate driven by the contrast feature. The calculation formula is as follows:

[0060] ;

[0061] In the formula, This represents the estimated value of the second scattering parameter derived solely from contrast information; This represents the mean distance constant within the endoscopic field of view, determined based on the depth-of-field characteristics of the endoscope. This represents the scalar value of contrast attenuation calculated in the preceding steps. This formula provides an independent estimation path that does not depend on local color characteristics.

[0062] It should also be noted that determining the comprehensive scattering parameter value online within the scattering parameter constraint domain includes: obtaining preset dark channel fusion weights and contrast fusion weights; and weighting and summing the first scattering parameter estimate and the second scattering parameter estimate based on the dark channel fusion weights and contrast fusion weights to obtain an initial comprehensive estimate. Through the fusion strategy, the system can be biased towards high-sensitivity dark channel estimation in cases of mild turbidity and rely on highly robust contrast estimation in cases of severe turbidity. Its mathematical expression is as follows:

[0063] ;

[0064] In the formula, This represents the initial comprehensive estimate variable obtained through the weighted fusion algorithm; This represents the preset dark channel fusion weight constant; This represents the preset contrast blending weight constant, and the sum of the two weight constants is strictly set to natural number units.

[0065] Subsequently, a truncation function is used to restrict the initial comprehensive estimate to the scattering parameter constraint domain, and the comprehensive scattering parameter value is output. Once the weighted result exceeds the preset physical extreme value of the medium, the truncation function restricts it to the boundary value. This design ensures that even if the image is dominated by a large amount of pure white tissue, such as fat or highly reflective mucous membranes, causing statistical distortion, the attenuation coefficient output by the system will not exceed the physical carrying capacity limit of the current irrigation fluid.

[0066] Step 4: Transmittance map calculation and edge-sensing refinement.

[0067] Specifically, after obtaining the precise parameters, an initial transmittance map of the standardized image needs to be calculated using the dark channel prior based on the ambient scattered light brightness, and a refined transmittance map is generated through edge-aware refinement. The transmittance map characterizes the proportion matrix of photons that penetrate the medium without scattering, directly determining the spatial distribution of compensation intensity at each location.

[0068] For example, the pixel values ​​of each channel of the normalized image are divided by the corresponding ambient scattered light intensity, and the minimum value is taken within a local window. The initial transmittance map is then calculated based on the dark channel prior. This involves a core normalization process:

[0069] ;

[0070] In the formula, Indicates the pixel position The initial transmittance scalar obtained from the preliminary estimate; Represents a spatial local calculation window; Represents the normalized image at pixel locations passage The strength; Indicates the ambient scattered light brightness in the channel Independent components on.

[0071] Because the formula uses an extremum search operator within a local block, the generated initial transmittance map exhibits a stepped mosaic-like ghosting phenomenon at the boundaries of different depths, such as the edge between the surgical forceps and the posterior tissue, which severely disrupts the high-frequency continuity of the structure.

[0072] Optionally, to correct this defect, the standardized image is used as a guide image to perform a guided filtering operation on the initial transmittance map, eliminating the block effect introduced by the local window minimization operation and generating the refined transmittance map. The guided filtering technique uses the texture gradient of the high-resolution standardized image as a reference anchor point, assuming that the transmittance in local regions is linearly correlated with the original image structure. Through this filtering mechanism, the algorithm can accurately fit smooth low-frequency transmittance values ​​to the edges of complex vascular networks or tissue textures, thus preventing edge halo leakage in subsequent compensation.

[0073] Step 5: Threshold truncation mechanism and model algebraic inversion.

[0074] Specifically, once the refined transmittance map is constructed, directly applying division inversion will cause system calculation anomalies. Therefore, this step requires performing lower limit threshold truncation on the refined transmittance map to generate a safe transmittance map. The standardized image, ambient scattered light brightness, and safe transmittance map are then substituted into the forward scattering degradation model for algebraic inversion to obtain the restored image.

[0075] For example, a lower threshold for transmittance is set, and the refined transmittance map is processed pixel-by-pixel by taking the maximum value of the lower threshold to generate the safe transmittance map. In the depths of an endoscopic image, transmittance may approach zero infinitely. If a minimum value is directly introduced into the denominator at this point, the inherent dark current noise or quantization particles of the sensor will be amplified hundreds or thousands of times, causing a large amount of high-frequency noise. To avoid this risk, the following forced truncation mechanism is adopted:

[0076] ;

[0077] In the formula, This represents the safe transmittance scalar output after threshold protection processing. This represents the refined transmittance scalar input. This represents a pre-set lower limit threshold constant for transmittance. This represents the operator that extracts the larger of the two values.

[0078] This mechanism allows some dehazing depth to be sacrificed in extremely distant, invisible areas in exchange for the stability of the image's signal-to-noise ratio.

[0079] It should be noted that the standardized image, ambient scattered light intensity, and safety transmittance map are substituted into the algebraic inversion formula to obtain the restored image. The restored image is then truncated within a preset pixel value range. The derivation formula for the inversion is as follows:

[0080] ;

[0081] In the formula, This represents the matrix components at the corresponding pixels of the restored image obtained through algebraic inversion calculation; Represents the normalized image matrix components of the input; This represents the global ambient scattered light brightness constant; This represents the scalar value of the safe transmittance corresponding to each pixel.

[0082] The actual operating mechanism of this formula is very clear: the molecular part removes the background noise of diffuse reflection in the medium by subtraction, then performs inverse geometric compensation on the attenuated tissue reflection signal by dividing by transmittance, and finally replenishes the basic lighting environment. After the inversion is completed, a legal numerical domain limitation mechanism (such as truncating floating-point numbers to a specified dynamic display range) is used to prevent display overflow patches.

[0083] Step 6: Iterative refinement mechanism for spatial non-uniform lighting models.

[0084] Specifically, the above steps are all based on the classical assumption of spatially uniform ambient light. However, after obtaining the restored image, it is also necessary to perform iterative refinement through algebraic inversion based on a spatially non-uniform illumination model of the endoscopic light source characteristics. Within the urological cavity, the only light source is a point source at the very front of the lens, which completely contradicts the assumption in atmospheric models that sunlight is a parallel source at infinity. This extreme illumination non-uniformity caused by the close proximity of the light sources must be eliminated through nonlinear iteration.

[0085] For example, based on the inverse square attenuation characteristic of the illuminance of a near-range point light source in an endoscope, a pixel-wise distance-dependent ambient scattered light brightness model is constructed, the mathematical form of which is:

[0086] ;

[0087] In the formula, This represents the ambient scattered light brightness matrix variable introduced in the iterative refinement stage, which attenuates with spatial location. It represents an equivalent constant that integrates the luminous power of the light source and the background scattering coefficient of the medium; This represents the three-dimensional spatial distance variable from a scene point to the endoscope's light source, derived through reverse derivation; that is, the pixel-by-pixel scene distance. Directly solving... An unknown distance is required. However, distance depends on transmittance, thus forming a cyclic dependency, so a multi-round iterative strategy must be adopted.

[0088] Optionally, the restored image and the secure transmittance map obtained from the solution are used as the initial inversion result and initial transmittance map for the 0th iteration. In the current iteration, the pixel-by-pixel scene distance is calculated using the transmittance map from the previous iteration and the integrated scattering parameter value.

[0089] Subsequently, the back-calculated pixel-by-pixel scene distance is substituted into the ambient scattering light brightness model to update the spatially adaptive ambient scattering light brightness matrix. Using the updated ambient scattering light brightness matrix, the transmittance map for the current round is recalculated, and model algebraic inversion is performed to generate the restored image for the current round. This series of steps, by substituting the previously obtained first-order approximate depth map into the light source attenuation model, yields a more accurate non-uniform illumination background map.

[0090] It is important to note that the iteration must have a rigorous termination and convergence mechanism. The change between the restored image results of two adjacent iterations is calculated; it is then determined whether the change is less than a preset convergence threshold; if it is less than the threshold, the iteration loop terminates, and the restored image of the current iteration is used as the final restored image for the step of converting the restored image to the luminance-chrominance separation color space; otherwise, the next iteration continues. This feedback loop not only suppresses overexposure caused by strong light at the near end but also eliminates overestimation caused by light attenuation at the far end, significantly enhancing the scientific rationality of the three-dimensional light distribution.

[0091] Step 7: Color fidelity correction and output.

[0092] Specifically, after iteratively removing the brightness degradation caused by physical scattering, the final step of the method is to convert the restored image to a brightness-chroma separation color space, perform global mean shift correction on the chroma channel to eliminate color shift between channels, and generate the final output image. Because hemoglobin in blood and rinsing fluid have drastically different absorption coefficients for long wavelengths (red light) and short wavelengths (blue-green light) in the visible spectrum, the inversion operation of independent channels accumulates spectral color differences, leading to color distortion in the final image.

[0093] For example, the restored image is converted from the RGB color space to the CIELab color space, separating the luminance and chrominance channels. The global mean of the chrominance channels across the entire image is calculated; based on the gray-world assumption, the global mean of the chrominance channels is shifted to preset reference chrominance neutral values ​​to obtain the corrected chrominance channels. This operation deliberately avoids the luminance channel, which is responsible for the overall image contrast and detail sharpness, and only performs spatial displacement on the two-dimensional phasor of color, ensuring that the anatomical texture is not destroyed.

[0094] It should also be noted that the calibrated chroma channel and the luminance channel are matrix-merged; the merged channel data is inversely converted from the CIELab color space to the RGB color space; and a preset pixel value range is truncated on each channel data after inverse conversion to generate the final output image.

[0095] In summary, this method demonstrates significant advancements and practicality compared to existing general image dehazing algorithms that rely on fixed parameter priors or blind training on large-scale datasets. Existing techniques often suffer from large-area black hole effects, color distortion, or severe noise in distant fields of view when dealing with dynamically changing perfusion fluids and unique near-field point light source illumination systems during endoscopic surgery due to discrepancies between the model's priors and actual conditions.

[0096] This embodiment, through fundamental innovation in the physical optics dimension, combines an online parameter dynamic adaptive estimation mechanism based on the inherent limitations of the fluid medium, and further supplements it with a non-uniform illumination iterative correction model specifically designed for endoscopic optical structures, constructing a logically self-consistent recovery loop. In actual clinical applications, this solution not only maintains robust dehazing output under harsh transient conditions such as electrocautery or lithotripsy generating a large amount of high-frequency scattering debris, accurately removing visual barriers obscuring the field of vision, but also, through decoupling correction of brightness and chromaticity, realistically reconstructs the microvascular color texture on the mucosal surface. While ensuring a high signal-to-noise ratio, it provides the operator with a high-quality surgical field image of great medical diagnostic value.

[0097] Example 2:

[0098] This embodiment provides a color fidelity correction processing scheme in a urological endoscopic image restoration method based on a scattering physics model. After performing the algebraic inversion solution of the forward scattering degradation model, although the spatial structural details and contrast of the image are significantly restored, the restored image obtained at this stage often faces a serious risk of color shift. The physical root cause of this color shift is that the irrigation fluid filling the urological surgical cavity is not an ideal uniform scattering medium.

[0099] During surgical procedures, the irrigation fluid inevitably mixes with a large number of red blood cells, sloughed tissue debris, and microbubbles. Among these, the hemoglobin abundant in red blood cells exhibits an extremely unbalanced optical response to visible light of different wavelengths.

[0100] Specifically, hemoglobin exhibits extremely strong absorption of short-wavelength blue light and medium-wavelength green light, while it mainly scatters long-wavelength red light with very weak absorption. This typical wavelength-dependent absorption and scattering characteristic results in significantly different actual attenuation rates of the red, green, and blue color channels as they propagate in the irrigation fluid.

[0101] However, in the preceding model inversion process, due to the lack of physical information in monocular vision, the algorithm can usually only use a uniform safety transmittance map to perform the same degree of inverse compensation calculation for all color channels. This uniform compensation strategy cannot accurately match the real nonlinear attenuation differences of each color channel, thus inevitably introducing a relative numerical imbalance between channels while restoring the structure. This is typically manifested visually as an overall reddish tint to the image, or an abnormal lack of blue-green tones.

[0102] In order to correct this optical distortion without destroying the restored structural texture, this embodiment details the specific implementation path of converting the restored image to a luminance-chrominance separated color space, performing global mean shift correction on the chrominance channel, eliminating color shift between channels, and generating the final output image.

[0103] For example, the technical implementation of this embodiment begins with a non-linear conversion of the color space. The restored image is converted from the RGB color space to the CIELab color space, separating the luminance channel and the chrominance channel. , In conventional digital image processing and endoscopic video streaming, image data is stored by default in the RGB color space. The RGB space is a highly coupled, device-dependent color space where the values ​​of the red, green, and blue channels collectively determine not only the pixel's color attributes (hue and saturation) but also its brightness attributes (luminance). If channel values ​​in a color-biased image are directly added, subtracted, multiplied, or divided within the RGB space, any adjustment to a single color will irreversibly alter the brightness distribution of that pixel, thereby destroying the tissue edge contrast restored by the previous inversion algorithm.

[0104] To decouple luminance and chrominance, this step projects the image onto the CIELab color space. The CIELab color space is a uniform color model built upon the physiological characteristics of human visual perception. Its greatest technological advantage lies in achieving a strict orthogonal separation of luminance and chrominance information. In this space, the separated luminance channels... It centrally carries the scene's geometry, shadow contrast, and edge textures, while the chroma channels... and chroma channels It simply records the color distribution. Among them, the chroma channels... This represents the opposing bias of a pixel between green and red; a negative value indicates a green bias, and a positive value indicates a red bias; chroma channel This characterizes the pixel's bias between blue and yellow, with negative values ​​indicating a blue bias and positive values ​​indicating a yellow bias. Through this separation and extraction, all subsequent color correction operations will be strictly limited to... and These two low-frequency chromaticity two-dimensional planes are executed, thus mathematically ensuring that high-frequency organizational structure information is preserved in the luminance channel. It is not subject to any interference or damage.

[0105] Optionally, after separating luminance and chrominance, the algorithm needs to perform quantitative statistical analysis on the degree of color deviation within the current field of view. This involves calculating the chrominance channels. , The global mean of the entire image. The calculation logic of this step is to traverse all valid pixel positions in the two-dimensional matrix of the chroma channels of the current restored image, accumulate the chroma value of each pixel, and finally divide by the total number of pixels involved in the calculation, thereby extracting a scalar value that can represent the macroscopic color tendency of the entire image.

[0106] In clinical practice in urology, while healthy bladder mucosa or renal pelvis walls may appear slightly reddish, the overall color distribution within the macroscopic field of view should be within a relatively flat range. When blood mixes with the surgical irrigation fluid, causing wavelength compensation errors in the inversion algorithm, the color of the entire image will undergo a systematic unidirectional shift. This systematic shift is like adding a filter with a specific color to a lens; it is evenly superimposed on the true color of every pixel in the image.

[0107] Therefore, by calculating the chromaticity channels across the entire image... and The global mean can effectively capture and quantify the systematic color bias caused by errors in the physical attenuation model. This global mean not only filters out high-frequency color fluctuations in local tissues, such as local bleeding points or specific vascular networks, but also accurately extracts low-frequency color interference components that need to be eliminated.

[0108] Specifically, after extracting the global mean of the quantized color shift, this embodiment proceeds to the core numerical correction stage. Based on the gray-world assumption, the global mean of the chromaticity channels is shifted to preset reference chromaticity neutral values ​​to obtain the corrected chromaticity channels. The gray-world assumption is an important physical prior in the field of calculating color constancy. Its core theory states that for a complex natural scene with rich colors, under ideal neutral white light illumination, the average color of light reflected from the surfaces of all objects in the scene should approach a neutral gray. Mapped to the CIELab color space, the theoretical origin of neutral gray is the point with a chromaticity value of zero.

[0109] Within the closed surgical cavity of a urological endoscope, although the overall tissue skews towards warm red and yellow hues, statistical analysis of a vast amount of normal, non-scattering endoscopic images in clinical practice shows that, under standardized illumination, the global average chromaticity of the surgical field remains stable around a fixed neutral reference coordinate. When inversion color shift occurs, it is equivalent to a global translation of the entire color distribution in the chromaticity coordinate system. Therefore, this embodiment constructs a linear translation formula to forcibly pull the shifted chromaticity distribution center back to a preset physical baseline. The relevant core correction formula is as follows:

[0110] ;

[0111] ;

[0112] In the formula: Represents the two-dimensional spatial pixel position coordinates within the image plane; This indicates the pixel position coordinates after color fidelity translation correction. The numerical variable of the green to red chroma channel at that location; When indicating the input color correction module, the pixel position coordinates The numerical variable of the original green to red chroma channel; Represents the original green to red chroma channel of the input image. The global mean scalar across all valid pixels in the entire image; This represents a pre-defined and fixed reference neutral value constant for green to red chromaticity. This indicates the pixel position coordinates after color fidelity translation correction. The numerical variable of the blue to yellow chroma channel at that location; When indicating the input color correction module, the pixel position coordinates The numerical variable of the original blue to yellow chroma channel at that location; Represents the original blue to yellow chroma channel of the input image. The global mean scalar across all valid pixels in the entire image; This represents a pre-defined and fixed reference neutral value constant for blue to yellow chromaticity.

[0113] It should be noted that the reference neutral value constant in the above formula... and This has significant clinical calibration implications. In the simplest configuration lacking external references, based on the standard gray-world assumption, it is possible to directly... and The values ​​are all set to 0. At this time, the algorithm forces the average color of the image to be reduced to 0, which can eliminate most of the serious red shift or blue shift caused by uneven absorption of the medium.

[0114] However, considering the specific characteristics of urological tissues, such as the slight warm spectral reflectance of prostate tissue and normal bladder mucosa, a preset reference neutral color value can be derived from the preoperative calibration of a standard color chart to achieve higher precision in medical color fidelity. Before the surgery begins, the endoscopic probe is aligned with a standard medical color chart placed in clear saline to acquire a reference image. The mean color value of a specific area or the entire image of the color chart under scattering-free conditions is calculated, and this measured value is entered into the system as a reference. and .

[0115] Through this directional translation, the above formula not only eliminates the distortion increment caused by scattering and absorption (i.e., subtracting from the formula) and The operation also accurately anchors the color reference of the image to the actual tissue color state as determined before the operation (i.e., adding to the formula). and (Operation).

[0116] Through linear matrix operations of the two formulas mentioned above, the algorithm efficiently generates the corrected chroma channels. The entire process ensures global consistency adjustment of hue and saturation in terms of mathematical mechanism, avoiding color banding and artifact patches that may be caused by local nonlinear color mapping.

[0117] Specifically, after completing the steps detailed in this embodiment, those skilled in the art will understand that by using the original brightness channel without any numerical modification... With the new chromaticity channel obtained after translation correction , The final image generation process can be completed by re-stitching the matrix and applying a standard color space inverse conversion matrix to map it back to the commonly used RGB video stream format.

[0118] The overall effectiveness of the solution disclosed in this embodiment is explained in conjunction with existing technologies: Many existing conventional dehazing techniques based on dark channel priors typically perform transmittance division compensation independently on the three RGB color components when processing industrial videos or outdoor images. However, when these existing technologies are directly applied to the complex medical endoscopic environment filled with blood and irrigation fluid, the independent compensation inevitably leads to color imbalance because it fails to recognize the asymmetric absorption of different wavelengths of light by suspended particles in the liquid. For example, since red light attenuates less than green light, using a uniform transmittance in existing technologies can easily lead to excessive amplification of the red channel, resulting in large areas of red channel oversaturation in the image. This can easily mislead clinicians, who may misinterpret the redness induced by this algorithm as acute tissue congestion or severe bleeding, leading to unnecessary medical interventions.

[0119] In contrast, the color fidelity correction technology proposed in this embodiment deeply integrates the dehazing model of physical scattering with the color constancy theory of human visual perception. Through a series of operations including separating brightness and chromaticity, global mean statistics, and neutral reference shifting, this solution effectively eliminates the nonlinear errors accumulated between various color channels while fully preserving the high-contrast tissue structure information brought about by the physical model inversion. The final image presented on the surgical monitor not only overcomes the physical obstruction of turbid liquid, making deep tissue textures clearly visible, but more importantly, it maintains the original objective physiological color of the tissue surface.

[0120] This restoration effect, which unifies structural and color fidelity, fundamentally eliminates the interference of algorithmic artifacts on visual judgment, providing reliable visual evidence for surgeons to accurately identify tumor boundaries and promptly detect minute bleeding points, significantly improving the safety and accuracy of endoscopic minimally invasive surgery in harsh environments.

[0121] Example 3:

[0122] This embodiment provides an image data merging, color space inverse conversion, and secure output processing scheme in a urological endoscopic image restoration method based on a scattering physics model. After performing the color fidelity correction processing described in the aforementioned embodiment, the system has effectively eliminated inter-channel color shifts caused by wavelength-dependent scattering and absorption of suspended particulate matter in the chromaticity two-dimensional plane of the CIELab color space, which is unaffected by brightness interference.

[0123] At this point, the luminance channel, which carries the high-frequency tissue texture and edge contrast information recovered through physical model inversion, and the chroma channel, which has recovered the objective physiological color after global mean shift correction, remain separate and independent matrix structures in terms of data structure. Furthermore, the CIELab color space, as an abstract visual perception space, does not conform to the physical level driving requirements of clinical standard medical display devices in its data representation format.

[0124] To provide surgeons with visual images directly usable for clinical observation and diagnosis, these feature matrices distributed across different dimensions must be reintegrated, and precise color space inverse mapping and numerical boundary safety protection must be performed. This embodiment illustrates the specific implementation path for merging the calibrated chroma and luminance channels into matrices, inversely converting the merged channel data from the CIELab color space to the RGB color space, and performing a preset pixel value range truncation operation on each channel data after inverse conversion.

[0125] For example, the primary operation in this embodiment involves the spatial alignment and structured recombination of data. The system receives data output from the previous processing module and performs matrix merging of the corrected chroma channel and the luminance channel without any numerical modification. In the digital image processing architecture, the luminance channel is a two-dimensional data matrix whose spatial resolution is strictly consistent with the resolution of the original acquired endoscope sensor, recording the brightness and structural contour of each pixel position in the surgical field of view. The corrected chroma channel is also represented as two two-dimensional data matrices with completely equal spatial resolution, recording the chroma channel values ​​of the green-red opposing dimension and the blue-yellow opposing dimension, respectively. The matrix merging operation is essentially performing a deep concatenation and assembly of data in the channel dimension of the tensor. The algorithm strictly follows the two-dimensional spatial coordinate index of the pixel, strongly binding the luminance value located at the same coordinate position with the chroma values ​​of the two dimensions, thereby merging three independent two-dimensional single-channel matrices into a three-dimensional tensor data structure with three depth channel dimensions.

[0126] In the reconstruction calculations for minimally invasive urological surgeries, this merging operation requires ensuring absolute data alignment at the system memory level. Because minute capillary networks and tiny lesion edges may occupy only a few pixels of physical area, even a single pixel's spatial misalignment during the merging stage can lead to artifacts of hue and brightness separation in the final display. Therefore, strict coordinate consistency matrix merging is a fundamental prerequisite for ensuring precise alignment of the anatomical structure and surface physiological color in the reconstructed image.

[0127] Optionally, after completing the matrix merging of multidimensional tensors, the algorithm enters the core color space inverse mapping stage. The system calls the standard color conversion module to inversely convert the merged channel data from the CIELab color space to the RGB color space. The CIELab color space is a theoretical perception space based on human visual physiology, while modern operating room medical monitors or endoscope main unit display systems rely on applying specific physical driving signals to red, green, and blue light-emitting devices to mix and generate visual colors, i.e., using the standard RGB additive color model. Therefore, it is necessary to inversely map the merged Lab abstract data back to the RGB physical color space.

[0128] This inverse conversion process typically follows standard colorimetric conversion protocols (such as Lab2RGB function processing logic). Internally, it performs a nonlinear power-law inverse operation and a composite mapping of linear matrices to accurately restore the visually uniform perceived data into the primary color driving values ​​that the display hardware can understand and excite. This step ensures that the pure tissue signal, after being precisely stripped of interference from irrigated fluid environment scattering light in the preceding scattering physical model inversion, can be converted into a standard video stream format without loss. This provides an image data foundation that conforms to general medical display standards for intraoperative assessment of tissue blood supply and differentiation of benign and malignant boundaries.

[0129] It is also important to note that although the image restoration calculation process is theoretically closed after the inverse color space conversion, a final data constraint step must be performed from the perspective of safety and stability in medical system engineering. The system performs a truncation operation on the data of each channel after inverse conversion within a preset pixel value range to limit numerical overflow and generate the final output image. The necessity of this operation stems from the physical limits of the previous algebraic inversion and the cumulative effect of multiple color space conversions.

[0130] When performing scattering compensation, especially when dealing with deep cavity regions far from the endoscope lens, the transmittance value is often extremely small. In the division operation of algebraic inversion, such a small transmittance can cause the recovered signal in that region to be significantly amplified. Although the preceding steps introduce a lower limit protection for safe transmittance, after subsequent Lab space translation and nonlinear mapping to RGB space, the theoretically calculated values ​​of some highlight areas (such as strong specular reflections on tissue surfaces or direct illumination of the center by the endoscope point light source) or areas amplified by noise may still exceed the legal storage boundaries of standard digital images. For example, they may exceed 1.0 in normalized floating-point representation or 255 in 8-bit integer representation.

[0131] Without truncation, the image signal processor will experience severe numerical overflow when outputting to peripherals or performing type conversions: pixels exceeding their maximum brightness will directly drop to zero, abruptly appearing as pure black abnormal patches on the screen. In urological surgery, this pure black artifact can easily be mistaken by surgeons for tissue perforation or necrotic cavities, posing a potential medical safety hazard. By setting strict upper and lower limits for pixel values ​​(e.g., [0,1]), the algorithm clamps all negative values ​​below the lower limit to the lower limit and all overflow values ​​above the upper limit to the upper limit, thus establishing an effective numerical boundary constraint mechanism. After this processing, the system ultimately generates a final output image where all pixel values ​​are within the legal and valid range and do not contain any hardware parsing overflow errors.

[0132] The effectiveness of this embodiment and the overall solution disclosed in this invention will be explained in conjunction with existing technologies: Currently, most conventional medical image dehazing or enhancement technologies disclosed in the industry rely solely on single-dimensional contrast stretching or use unverified general atmospheric dehazing models when dealing with turbid media scenes. These existing technologies generally suffer from insufficient restoration accuracy due to prior mismatch when dealing with the degradation of complex endoscopic irrigation fluid composed of red blood cells and tissue debris, as well as the common problems of inducing severe noise amplification and color distortion in the far field of view.

[0133] The urological endoscopic image restoration method proposed in this embodiment starts from the underlying physical optical degradation mechanism, and sequentially constructs a constraint domain that matches the characteristics of the irrigation fluid, realizes online adaptive estimation of parameters based on image statistical features, performs model algebraic inversion protected by a lower limit threshold, and finally completes the closed loop of the whole chain through precise color space separation, correction, merging and safe inverse mapping truncation mechanism.

[0134] This approach not only effectively avoids the intraoperative maladaptation issues caused by fixed parameters in traditional techniques, but also eliminates potential overflow artifacts from medical display devices through a safe numerical truncation mechanism. The final image presented to the surgeon effectively overcomes scattering degradation caused by irrigation fluid, significantly improves the contrast of tissue structure edges, and maintains a high degree of objective realism and visual safety regarding the physiological color of the tissue mucosa. This overall approach significantly improves the upper limit of real-time imaging quality of existing endoscopic imaging systems under harsh working conditions, providing stable, clear, and realistic visual support for complex minimally invasive urological surgeries.

[0135] To verify the effectiveness of the proposed urological endoscopic image restoration method based on a scattering physics model, this embodiment also provides corresponding simulation experiments and data analysis. The simulation dataset is generated based on the Beer-Lambert scattering model: 50 standard endoscopic images are selected as non-degradable references, and for each image, the scattering coefficient β is calculated to be between 0.05 and 0.50 mm. -1 Within the range (step size 0.01mm) -1 The study simulated degraded images under different concentrations of scattering media, totaling 1380 degradation-reference image pairs. Evaluation metrics included peak signal-to-noise ratio (PSNR), structural similarity (SSIM), CIEDE2000 chromatic aberration, contrast gain ratio, and single-frame processing time.

[0136] For example, the following comparison schemes are set up: Scheme A is the complete scheme of the present invention; Scheme B (standard dark channel prior) is a representative physical model method in the current field of endoscopic image dehazing; Scheme C (ablation version without dual feature fusion) uses only a single dark channel feature for parameter estimation; Scheme D (adaptive histogram equalization) is a non-physical model baseline method widely deployed in clinical endoscopic systems; Scheme E (ablation version without iterative refinement) performs only single scattering parameter estimation and image inversion.

[0137] like Figure 2 The figure shows a comparison of peak signal-to-noise ratio (PSNR) curves under different scattering coefficients. Each curve exhibits a monotonically decreasing trend in PSNR as the scattering coefficient increases. Throughout the entire scattering coefficient range, the PSNR curve of scheme (A) of this invention consistently lies above the other four curves. At a scattering coefficient β = 0.30 mm... -1 At this point, the peak signal-to-noise ratio (PSNR) of the proposed solution is 31.2 dB, which is 1.4 dB higher than the best existing method (Solution E) of 29.8 dB. The PSNR of ablation schemes C and E are both lower than that of scheme A, respectively verifying the core improvement role of the dual-feature fusion module and the spatial adaptive iterative refinement module in enhancing the restoration accuracy. Furthermore, the PSNR decay rate of scheme A with increasing scattering coefficient is significantly lower than that of schemes B and D, significantly enhancing the robustness of the restoration algorithm to changes in scattering intensity.

[0138] like Figure 3 The figure shows a comparison of peak signal-to-noise ratio (PSNR) and color difference index (CIEDE2000) under different scattering coefficients. In traditional methods, improving structural fidelity often leads to increased color shift; for example, scheme B improves PSNR but significantly worsens color difference index than scheme D. However, the present invention (A) successfully overcomes this trade-off: at β=0.30mm... -1At this point, the color difference of CIEDE2000 in this invention is only 2.14, an improvement of 60.1% compared to scheme B. This is due to the fact that the dual-feature fusion module of this invention reduces the imbalance error of each channel during the inversion process, while the color fidelity correction module further constrains the balance between channels through the channel ratio preservation strategy, enabling this invention to significantly reduce color shift without sacrificing structural fidelity. (At β>0.35mm) -1 In areas with strong scattering, the color fidelity robustness advantage of this invention is more pronounced.

[0139] like Figure 4 The figure shows a waterfall plot decomposing the contribution of each technical module to the total peak signal-to-noise ratio gain. Using scheme D as the baseline, the present invention brings a total gain of 7.20 dB. Specifically: the dual-feature fusion module contributes the most (+2.87 dB, 39.9%), improving inversion accuracy from the source of parameter estimation; the physical model framework contributes the second most (+2.63 dB, 36.5%); the iterative refinement module contributes the third most (+1.18 dB, 16.4%), improving the non-uniform restoration problem in image edge regions; and the color fidelity correction module provides fine-tuning gain (+0.52 dB, 7.2%) while ensuring color reproduction. This waterfall plot demonstrates a progressively increasing and synergistic positive contribution relationship between the modules.

[0140] like Figure 5 As shown, this is a radar chart comparing the normalized performance across five dimensions. The proposed solution (A) exhibits the most robust polygonal outline in the five-dimensional radar chart, with a normalized coverage area of ​​0.519, clearly demonstrating its comprehensive superiority in multiple performance indicators. In the four quality dimensions of peak signal-to-noise ratio (0.860), structural similarity (0.728), color fidelity (0.786), and contrast-to-gain ratio (0.747), this invention ranks highest. In terms of processing speed, the single-frame processing time is approximately 247 milliseconds, fully meeting the basic requirements for real-time assisted enhancement in clinical endoscopy (approximately 4 frames / second).

[0141] In summary, simulation verification shows that this invention effectively overcomes the color shift problem caused by inconsistent estimation between channels in traditional methods, and achieves simultaneous improvement in structural fidelity and color fidelity in the restoration of endoscopic image scattering degradation, thus possessing extremely high clinical application value.

[0142] Example 4:

[0143] like Figure 6 As shown, this embodiment provides a urological endoscopic image restoration system based on a scattering physics model. This system is typically deployed in a clinical operating room environment as an embedded hardware chipset, an external digital signal processing box, or a standalone medical image processing workstation.

[0144] In practical minimally invasive surgeries, such as transurethral resection of the prostate (TURP) and percutaneous nephrolithotomy, this system is physically connected in series between the endoscope camera connector and the medical high-definition monitor, or directly integrated as a core algorithm component within the endoscope image signal processor (ISP) and the endoscope's main control console. The overall hardware chain typically consists of an endoscope optical lens, a near-field point light source illumination device, an image acquisition sensor, a core image processing processor (such as an FPGA, GPU, or dedicated DSP), and a medical digital video output interface. During surgery, a fluid perfusion pump continuously injects saline or glycine solution into the patient's surgical cavity as a flushing medium. The dynamically mixed microbubbles, red blood cells, and tissue debris within this medium can cause optical scattering and absorption degradation. This system, through a dedicated modular hardware architecture and underlying logic circuitry, eliminates this physical degradation in real time, ensuring the quality of the image output.

[0145] Specifically, the urological endoscopic image restoration system based on a scattering physics model includes:

[0146] The image preprocessing and constraint domain determination module acquires and preprocesses the raw images obtained by the endoscope, generating standardized images. It also determines the scattering parameter constraint domain of the total attenuation coefficient based on the optical scattering characteristics of the surgical fluid medium. During operation, the module's input is connected to the digital video bus of the image acquisition sensor, receiving high-resolution raw images in real time. Because the endoscope's front end is equipped with a wide-angle optical lens, the raw images often carry significant barrel distortion. Furthermore, due to the non-standard color temperature of the near-field light source, the image exhibits systematic color shift. The image preprocessing and constraint domain determination module integrates a distortion correction calculation unit and a white balance correction unit. The distortion correction calculation unit utilizes pre-calibrated geometric parameters stored in read-only memory, employing a radial-tangential distortion model to perform inverse stretching mapping and bilinear interpolation on the pixel coordinates of the raw image, thereby repairing edge tissue deformation. The white balance correction unit calculates the global mean of the red, green, and blue channels using a gray-world algorithm, shifting the pixel gain of each channel to the same target reference, and outputting a standardized image with its dynamic range normalized to the standard range.

[0147] Meanwhile, the module has a media selection register, which can retrieve the corresponding physical limit from a preset lookup table based on the liquid type information entered on the control panel or by automatically obtaining the physical status of the injection pump through the sensor bus.

[0148] When the surgical irrigation medium is physiological saline, the module restricts the range of the total attenuation coefficient to the first constraint interval. When the medium is glycine solution, considering the large number of charred particles generated by the electrocautery operation, the module automatically switches and locks to the second constraint interval with a higher upper limit. This finite interval constitutes the scattering parameter constraint domain for subsequent online parameter estimation, preventing the risk of parameter calculations exceeding the physically feasible range due to sudden changes in scene content.

[0149] The online scattering parameter determination module is used to extract image statistical features from the standardized image. These features include dark area pixel statistics and contrast attenuation. Based on the dark area pixel statistics, the module extracts the ambient scattered light brightness. It establishes an analytical mapping relationship between the image statistical features and the total attenuation coefficient, and determines the comprehensive scattering parameter value online within the scattering parameter constraint domain. In terms of hardware resource allocation, this module contains two parallel feature extraction hardware pipelines. The first pipeline performs a pixel-by-pixel minimization operation on the three color channels of the standardized image using a comparator matrix. Within a set local spatial sliding window, it statistically analyzes the preset percentile value of the pixel distribution to quantify the increase in dark area pixel values ​​caused by backscattering from impurities in the fluid, outputting the dark area pixel statistics. The second pipeline uses a variance calculation unit to obtain the global standard deviation of the corresponding grayscale image in real time, and performs a normalized division operation with the stored ideal clear state reference contrast value, outputting the contrast attenuation, which reflects the degree of compression of the global high and low light dynamic range.

[0150] Simultaneously, the ambient scattered light brightness extraction unit locates the highest-brightness pixel in the dark channel image, prior to a predetermined ratio, and backtracks to the standardized image to extract the maximum pixel value of each channel as the ambient scattered light brightness. Based on this, the mapping calculation logic unit substitutes these extracted scalar features into two independent analytical equations derived from the Beer-Lambert law and the global mean approximation assumption of transmittance, respectively, to calculate the first scattering parameter estimate driven by dark channel features and the second scattering parameter estimate driven by contrast features. The weighted fusion logic unit then calls the weight parameters stored in the register to perform a weighted summation of these two independent estimates to obtain an initial comprehensive estimate. Finally, the truncation logic circuit forces this initial comprehensive estimate to be projected into the aforementioned scattering parameter constraint domain, outputting online the comprehensive scattering parameter value that conforms to the actual turbidity physical state of the intraoperative irrigation fluid.

[0151] The transmittance calculation and refinement module is used to calculate the initial transmittance map of the normalized image based on the ambient scattered light brightness using dark channel priors, and to generate a refined transmittance map through edge-aware refinement. In the workflow, the initial transmittance calculation unit uses a divider to normalize each channel matrix of the normalized image pixel-by-pixel with the corresponding ambient scattered light brightness. Based on the duality principle of dark channel priors, it estimates the initial proportion of scene light that penetrates the medium without scattering, i.e., the initial transmittance map, through an extremum search operator within a local block.

[0152] However, due to the coarse-grained block effect inherent in local extremum search, the initial transmittance map exhibits discontinuous mosaic steps at the edges of surgical instruments or abrupt changes in tissue folds. To address this, the thinning unit uses the high-resolution normalized image as a guide image to perform real-time guided filtering on the initial transmittance map. The coefficient matrix solver within the thinning unit utilizes the high-frequency gradient texture of the original image as boundary constraints, forcing the output transmittance gradient to maintain precise spatial alignment with the actual anatomical structure edges, thereby eliminating the block effect and generating a sharp and physically smooth thinned transmittance map. This lays the foundation for accurate signal recovery in subsequent stages.

[0153] The model inversion and restoration module is used to perform lower limit threshold truncation on the refined transmittance map to generate a safe transmittance map. The standardized image, ambient scattered light intensity, and safe transmittance map are substituted into the forward scattering degradation model for algebraic inversion to obtain the restored image. In the deep cavity region of minimally invasive urological surgery, the scene point is far from the endoscope lens, and the refined transmittance often exhibits exponential decay, approaching zero. If the hardware inversion directly uses the minimum value as the denominator for division, the inherent thermal noise or dark current of the sensor will be amplified hundreds or thousands of times, causing severe snow-like noise artifacts.

[0154] To suppress this degradation, the model inversion and restoration module includes a lower threshold truncation unit. This unit introduces a positive real number as a safety margin and uses a comparator to perform a maximum value operation on each pixel of the refined transmittance map. Pixels below the threshold are forcibly raised to the safety lower threshold, thereby outputting a safe transmittance map and controlling the noise amplification factor at the far end within a preset range.

[0155] Subsequently, the three-channel algebraic inversion calculation core performs cascaded subtraction and division operations: first, the ambient scattered light brightness is subtracted from the standardized image to remove diffuse stray light generated by suspended particles, then it is divided by the corresponding safe transmittance to compensate for transmission loss, and finally the basic lighting environment is added back to generate the initial version of the restored image.

[0156] It is also worth noting that this module includes a spatially non-uniform illumination iterative refinement unit. This unit addresses the inverse square attenuation characteristic of near-field point light source illumination unique to endoscopes. Based on the initial restored image, it calculates the pixel-by-pixel scene distance, dynamically updates the non-uniform ambient scattered light brightness matrix, and re-introduces it into the inversion process until the difference between two adjacent iterations is less than a preset convergence threshold. Finally, it outputs the restored image with high signal-to-noise ratio and balanced near-far illumination.

[0157] The color fidelity correction module is used to convert the restored image to a luminance-chrominance separated color space, perform global mean shift correction on the chrominance channel, eliminate inter-channel color shift, and generate the final output image. Due to the asymmetric absorption and scattering of visible light at different wavelengths by components such as hemoglobin in the surgical irrigation fluid (e.g., strong absorption of blue-green light and weak absorption of red light), cumulative color shift inevitably occurs during multi-channel independent inversion, which visually manifests as a reddish or off-color appearance in the surgical field image.

[0158] To address this issue, the color fidelity correction module incorporates a non-linear color space conversion engine that inversely projects the restored RGB image to the CIELab color space. This space orthogonally separates the luminance channel, which records tissue spatial details and geometric contrast, from the two chroma channels, which record color information. A chroma mean statistical unit traverses the entire image, calculating the global mean of the two chroma channels representing the systematic color bias. Based on the grayscale world assumption, the translation correction circuit uses linear subtraction to remove this global mean from the current coordinates and adds a reference neutral value calibrated and solidified by a pre-operative standard color chart. This pulls the deviated color distribution back to the objectively true physical baseline, resulting in the corrected chroma channels.

[0159] Subsequently, the matrix merging and inverse conversion unit re-performs spatial alignment and splicing of the unmodified luminance channel and the corrected chrominance channel, and converts them back to RGB physical space. Finally, the limiting and truncation output unit performs pixel-based legal display domain truncation on the RGB three-channel data through a hard threshold limiting circuit, eliminating numerical anomalies that may cause hardware display overflow, generating and streaming the final output image with high fidelity and clear details to the medical display device.

[0160] To illustrate this with existing technology: Existing defogging or underwater restoration equipment, due to the lack of constraints on irrigation fluids specific to urological surgery, often results in blackened images, overexposure, or large-area color distortion when facing transient conditions such as dynamic bleeding and drastic fluctuations in debris concentration during surgery, due to mismatch in offline parameter calibration.

[0161] The integrated restoration system constructed in this embodiment achieves a closed-loop operation of the entire chain, including image standardization preprocessing, scattering constraint domain definition, online parameter estimation through dual feature fusion, edge-aware transmittance refinement, spatial non-uniform iterative inversion of point light sources, and linear translation correction of color space, through modular pipeline hardware design.

[0162] In practical clinical applications, this system's hardware can efficiently process high-resolution video streams in parallel. It not only physically penetrates the fog-like barrier caused by suspended particulate matter, clearly restoring the edges of hidden lesions and capillary networks, but also visually restores the original objective color of human tissue, eliminating interference from the algorithm itself on the surgeon's judgment. The system's comprehensive numerical truncation protection design further ensures the data security of medical displays. The entire system overcomes the shortcomings of existing technologies in adapting to dynamic changes in intraoperative conditions, significantly improving the surgeon's visual field quality and the overall safety of minimally invasive surgery in harsh imaging media environments.

[0163] Example 5:

[0164] To further enrich and support the technical solution of this invention, this embodiment systematically formulates the underlying optical scattering constraints, feature mapping derivation process, and boundary processing function, which were not fully elaborated in the previous embodiments. The specific content is as follows:

[0165] 1. Mathematical model for geometric distortion correction in the preprocessing stage.

[0166] During the preprocessing to generate standardized images, the aforementioned embodiments employed a radial-tangential distortion model to perform anti-distortion mapping to address the severe distortion introduced by the wide-angle lens of the endoscope. To ensure the accuracy of spatial coordinate mapping, this solution specifically uses the Brown-Conrady radial-tangential distortion model, which maps the actual pixel coordinates of the image to... With ideal distortion-free physical coordinates The mapping relationship between them is strictly defined as follows:

[0167] ;

[0168] ;

[0169] in, It represents the squared Euclidean distance from the pixel to the optical center. , It is the radial distortion coefficient, responsible for correcting barrel or pincushion distortion caused by lens curvature; , This is the tangential distortion coefficient, used to compensate for distortion caused by the lens not being parallel to the imaging plane.

[0170] 2. Physical model of optical attenuation and forward scattering in surgical fluid media.

[0171] Within the scattering parameter constraint domain defined in the foregoing embodiments, the attenuation process of light propagating in the rinsing fluid follows Beer-Lambert's law. The total attenuation coefficient describes this attenuation process. (unit: ) by scattering coefficient and absorption coefficient Together they constitute:

[0172] ;

[0173] Considering that the scattering of microbubbles and tissue debris in the irrigation fluid is mainly characterized by strong forward scattering (anisotropy factor) In practical applications, it has already been Implicit inclusion reduces the equivalent scattered light intensity loss rate, and the effective attenuation coefficient is defined as follows: Based on this physical model, the aforementioned defined constraint intervals have strict physical boundaries: the first constraint interval of the physiological saline environment is... The second constraint region of the glycine solution environment is .

[0174] Based on the above constraints, the transmittance of scene reflected light reaching the sensor Rigorously defined as varying with distance An exponentially decaying function:

[0175] ;

[0176] Incorporating the ambient scattered light component, the fundamental forward model of a complete endoscopic degraded image can be mathematically expressed as:

[0177] ;

[0178] in, For standardized images containing degraded observation information, This represents the clear scene radiation to be restored in the physical model.

[0179] 3. The derivation mechanism of the analytical mapping from image statistical features to scattering parameters.

[0180] The aforementioned embodiments calculate the estimated values ​​of the first and second scattering parameters online by establishing analytical equations. The underlying mathematical derivation mechanism is as follows:

[0181] (1) Derivation based on pixel statistics in dark areas:

[0182] By applying dark channel operations to the forward model and approximating it using dark channel priors, the local dark channels of the degraded image can be obtained. Distance from the scene The parsing relationship:

[0183] ;

[0184] Substituting into the definition of exponential decay of transmittance, we expand to:

[0185] ;

[0186] To map this pixel-wise function to a scalar statistical relation for the entire image, take... The distribution of the first percentile It exactly corresponds to the first of the scene distance distributions. percentile :

[0187] ;

[0188] Among them, equivalent distance The independent determination method is based on the physical range of the field of view depth. The uniform distribution assumption:

[0189] ;

[0190] Solving the above scalar equation yields the estimated value of the first scattering parameter. .

[0191] (2) Derivation based on contrast attenuation:

[0192] Based on the forward model, the global variance of the degraded image is calculated:

[0193] ;

[0194] Under the condition that the depth of the endoscopic field of view is limited, a global mean approximation of transmittance is introduced. The above formula can be simplified to:

[0195] ;

[0196] Take the standard deviation and substitute it into the contrast attenuation. The definition of . In this theoretical derivation, a preset reference contrast value constant is set. Equivalent to the standard deviation of a sharp image under ideal scatter-free conditions Thus we get:

[0197] ;

[0198] Substitute into the transmittance definition (using the mean distance) The parsing relation is then obtained:

[0199] ;

[0200] This allows for the direct derivation of the estimated value of the second scattering parameter. .

[0201] (3) Mathematical expression of truncation operation within the constraint domain: To ensure that the initial comprehensive estimate obtained by weighted summation is physically reasonable, the truncation function is defined as a piecewise function projection mechanism:

[0202] ;

[0203] in, The final online determined comprehensive scattering parameter values, This is the initial comprehensive estimate after fusion. and This represents the upper and lower bounds of the current medium constraint domain.

[0204] 4. Transmittance refinement and matrix operation protection logic for color space.

[0205] In the refinement stage of eliminating transmittance patch effects, the mathematical essence of the guided filtering operation is to solve a local linear regression problem, the mapping relationship of which is expressed as:

[0206] ;

[0207] in To guide the image, The initial transmittance, The radius of the filtering window. Regularization parameters to prevent edge overfitting.

[0208] Color fidelity correction and final data output involve non-linear color space transformation. (Restored image) The operation of separating the luminance and chrominance channels is defined by the standard colorimetric conversion protocol: ;

[0209] In color channels and After performing a global mean shift, the corrected value is obtained. and The expression for matrix merging and inverse transformation is as follows: ;

[0210] In addition, to completely eliminate any wraparound artifacts that may occur at the hardware display end, hard thresholding is configured at the output end after algebraic inversion and color inverse conversion: .

[0211] This mechanism forces all abnormal underflow negative values ​​to be clamped to 0 and abnormal overflow bright spots to be clamped to 1, ensuring the generation of an absolutely safe display video stream signal.

[0212] Example 6:

[0213] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0214] like Figure 7 The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0215] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0216] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0217] The memory 103 stores a computer program corresponding to the urological endoscopic image restoration method based on a scattering physics model according to the above embodiments of the present invention. This computer program is executed under the control of the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0218] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 7 The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0219] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for restoring urological endoscopic images based on a scattering physics model, characterized in that, include: Acquire raw images from the endoscope and preprocess them to generate standardized images; The scattering parameter constraint domain for the total attenuation coefficient was determined based on the optical scattering characteristics of the surgical fluid medium. Image statistical features are extracted from the standardized image, including pixel statistics in dark areas and contrast attenuation. Ambient scattered light intensity is extracted based on the pixel statistics in dark areas. An analytical mapping relationship between the image statistical features and the total attenuation coefficient is established, and the comprehensive scattering parameter value is determined online within the scattering parameter constraint domain. Based on the ambient scattered light brightness, the initial transmittance map of the standardized image is calculated using the dark channel prior, and a refined transmittance map is generated through edge-aware refinement. The refined transmittance map is truncated by a lower limit threshold to generate a safe transmittance map. The standardized image, ambient scattered light brightness, and safe transmittance map are substituted into the forward scattering degradation model for algebraic inversion to obtain the restored image. The restored image is converted to a luminance-chrominance separated color space, and a global mean shift correction is performed on the chrominance channel to eliminate color shift between channels and generate the final output image. The step of establishing the analytical mapping relationship between the image statistical features and the total attenuation coefficient includes: The equivalent distance corresponding to the preset percentile is determined based on the physical range of the endoscopic field of view depth. Construct the first scattering parameter estimate driven by dark channel features. The calculation formula is as follows: ； The mean distance within the endoscopic field of view is determined based on the approximate assumption of the global mean of transmittance. Construct a second scattering parameter estimate driven by contrast features. The calculation formula is as follows: ； in, This refers to the pixel statistics of the dark area. The ambient scattered light brightness, This refers to the contrast attenuation amount; The online determination of the comprehensive scattering parameter value within the scattering parameter constraint domain includes: Obtain the preset dark channel blending weight and contrast blending weight; The first scattering parameter estimate and the second scattering parameter estimate are weighted and summed based on the dark channel fusion weight and the contrast fusion weight to obtain the initial comprehensive estimate. The initial comprehensive estimate is constrained within the scattering parameter constraint domain by using a truncation function, and the comprehensive scattering parameter value is output.

2. The method according to claim 1, characterized in that, The process involves acquiring the original images obtained by the endoscope and preprocessing them to generate standardized images; The scattering parameter constraint domain for the total attenuation coefficient is determined based on the optical scattering characteristics of the surgical fluid medium, including: The original image is subjected to anti-distortion mapping using a radial-tangential distortion model, and each color channel is normalized to a unified reference based on the gray world algorithm to generate the normalized image with normalized pixel values. Based on the type information of the surgical fluid medium, the upper and lower bounds of the total attenuation coefficient are determined; when the medium is physiological saline, a first constraint interval is determined; when the medium is glycine solution, a second constraint interval is determined; the finite interval formed by the upper and lower bounds is used as the scattering parameter constraint domain that limits the online parameter estimation search space.

3. The method according to claim 1, characterized in that, The step of extracting image statistical features from the standardized image includes: The minimum value of each pixel in the three color channels of the standardized image is taken to generate a dark channel image, and the minimum value within a local window is taken to obtain a local dark channel image; the preset percentile value of the pixel value distribution in the local dark channel image is extracted as the pixel statistics of the dark area. Calculate the global standard deviation of the grayscale image corresponding to the standardized image, and normalize the global standard deviation with a preset reference contrast value to obtain the contrast attenuation amount that reflects the degree of global contrast reduction.

4. The method according to claim 3, characterized in that, The process of extracting ambient scattered light brightness based on the pixel statistics of the dark area includes: Select the position corresponding to the pixel with the highest brightness in the dark channel image at a preset ratio; The maximum pixel value of each color channel at the corresponding position in the standardized image is extracted as the ambient scattered light brightness of each color channel.

5. The method according to claim 1, characterized in that, The step of calculating the initial transmittance map of the standardized image based on the ambient scattered light brightness using dark channel priors, and generating a refined transmittance map through edge-aware refinement, includes: The pixel values ​​of each channel of the standardized image are divided by the corresponding ambient scattered light brightness, and the minimum value is taken within a local window. The initial transmittance map is then calculated based on the dark channel prior. Using the standardized image as the guide image, a guided filtering operation is performed on the initial transmittance map to eliminate the block effect introduced by the local window minimum operation, thereby generating the refined transmittance map.

6. The method according to claim 1, characterized in that, The process involves performing a lower threshold truncation on the refined transmittance map to generate a safe transmittance map. Then, the standardized image, ambient scattered light intensity, and safe transmittance map are substituted into a forward scattering degradation model for algebraic inversion to obtain the restored image. This includes: Set a lower limit threshold for transmittance The refined transmittance map Perform pixel-by-pixel execution with the lower threshold. The operation of taking the maximum value generates the security transmittance map. ; The standardized image Ambient scattered light brightness and safety transmittance diagram Substituting into the algebraic inversion formula, the restored image is obtained. Its algebraic inversion formula is: ； The restored image obtained by the solution is truncated within a preset pixel value range.

7. The method according to claim 6, characterized in that, After obtaining the restored image, the process further includes iterative refinement through algebraic inversion based on a spatially non-uniform illumination model of the endoscopic light source characteristics. Specifically, this includes: Based on the inverse square attenuation characteristic of near-range point light source illumination in endoscopes, a pixel-by-pixel distance-dependent ambient scattered light brightness model is constructed. ； in For equivalent constants, For pixel-by-pixel scene distance; The restored image and the secure transmittance map obtained by solving are used as the initial inversion result and initial transmittance map of the 0th iteration. In the current iteration, the pixel-by-pixel scene distance is calculated using the transmittance map from the previous iteration and the comprehensive scattering parameter values. ; Substitute the inversely calculated pixel-by-pixel scene distance into the ambient scattered light brightness model to update the spatially adaptive ambient scattered light brightness matrix; The transmittance map for the current round is recalculated using the updated ambient scattering light brightness matrix, and model algebraic inversion is performed to generate the restored image for the current round.

8. The method according to claim 7, characterized in that, The iterative refinement of the spatial non-uniform illumination model based on the characteristics of endoscopic light sources through algebraic inversion also includes: Calculate the change between the restored image results of two adjacent rounds; Determine whether the change is less than a preset convergence threshold; If the value is less than the convergence threshold, the iteration loop is terminated, and the restored image of the current round is used as the restored image for the step of converting the restored image to the luminance-chrominance separated color space; otherwise, the next iteration round is continued.

Images