Medical grade image processing method, system, device, and computer program product

Abstract

The application relates to a medical image processing method, system, device and computer program product, the method comprising: calculating a mask image of a to-be-processed original image; from the maximum time step to the minimum time step, iteratively: inputting the to-be-processed original image, the mask image and a residual into a residual diffusion bridge model to output predicted noise of a current time step; calculating a denoising residual based on the predicted noise and the residual, calculating an intermediate residual of the current time step based on the denoising residual and the mask image, and in the case that the current time step is the minimum time step, terminating the iteration and taking the intermediate residual of the current time step as a reconstruction residual; and obtaining a target clear image based on the to-be-processed original image and the reconstruction residual. The application uses the reconstruction residual to represent image degradation caused by bubble atomization, non-uniform illumination, light scattering and selective absorption effect in a liquid medium, and solves the problem of non-uniform and instantaneous burst single-frame image quality degradation, so that the quality of the reconstructed image can meet the requirements of medical-level surgical applications.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, specifically to a medical-grade image processing method, system, device, and computer program product. Background Technology

[0002] Transurethral surgeries, such as laser enucleation of the prostate and transurethral resection of bladder tumors, are typically performed in a fluid environment with continuous saline irrigation. The localized high temperatures generated during holmium laser cutting or high-frequency electrocoagulation hemostasis cause intense vaporization of the fluid, forming dense, multi-scale bubble clusters. This results in severe surgical bubble atomization, which causes strong multi-level scattering and refraction of the endoscopic light source, visually manifesting as localized non-uniform blurring, indistinct anatomical edges, and even complete obscuring of core tissue structures. Simultaneously, due to the extremely narrow transurethral surgical passage, endoscopic systems inherently suffer from physical bottlenecks such as a small field of view, non-uniform illumination distribution with near-strong and far-weak characteristics, and highly repetitive mucosal textures. These physical bottlenecks, combined with the light scattering and selective absorption effects of the fluid medium, result in extremely complex non-uniformity, spatiotemporal locality, and transient abrupt changes during image degradation. Existing techniques often employ global low-order digital image processing operators such as histogram equalization or classical dark channel priors, or conventional hardware illumination optimization. However, images suitable for this type of globally uniform processing method need to have globally consistent degradation. However, when processing images of transurethral surgery, the degradation caused by local blurring or occlusion due to bubble atomization, as well as non-uniform and transient degradation caused by non-uniform illumination, light scattering in liquid media, and selective absorption effects, makes it difficult for the reconstructed images obtained by using the globally uniform processing method to meet the requirements of medical-grade surgical applications.

[0003] Therefore, how to avoid local blurring or occlusion caused by bubble atomization during transurethral surgery, and the non-uniform and transient degradation caused by non-uniform illumination, light scattering in liquid media, and selective absorption effects, which makes it difficult for the reconstructed image quality to meet the requirements of medical-grade surgical applications, are technical problems that need to be solved. Summary of the Invention

[0004] The technical problem solved by this application is to provide a medical-grade image processing method, system, device, and computer program product that can effectively deal with local blurring or occlusion caused by bubble atomization, as well as degradation caused by non-uniform illumination and light scattering and selective absorption effects in liquid media, which exhibit non-uniformity and transient abrupt changes, ensuring that the reconstructed image can meet the requirements of medical-grade surgical applications.

[0005] According to a first aspect of the embodiments of this application, a medical-grade image processing method is provided, comprising: Calculate the local variance map of the original image to be processed, and construct a mask image based on the local variance map; the original image to be processed is an image acquired under bubble atomization, non-uniform lighting and liquid medium scenarios; Based on the original image to be processed, the initial residual, and the mask image, wherein the initial residual is an all-zero matrix, the following steps are iteratively executed from the maximum time step to the minimum time step: The original image to be processed, the masked image, and the residual are input into the residual diffusion bridge model, which outputs the prediction noise at the current time step. When the current time step is the maximum time step, the residual is the initial residual; when the current time step is the maximum time step, the residual is the minimum time step. When, the residual is the time step. The obtained intermediate residuals; these residuals are used to characterize image degradation caused by bubble atomization, non-uniform illumination, light scattering in liquid media, and selective absorption effects; the residual diffusion bridge model is obtained based on the diffusion model training. The denoising residual is calculated based on the predicted noise and residual. The intermediate residual at the current time step is calculated based on the denoising residual and the mask image. If the current time step is the minimum time step, the iteration is terminated, and the intermediate residual at the current time step is used as the reconstruction residual. A clear image of the target is obtained based on the original image to be processed and the reconstruction residual.

[0006] Compared to existing technologies, this application no longer performs global indiscriminate enhancement and restoration on the acquired images. Instead, it introduces a spatial mask to model the residual that characterizes the degradation between the original image and the ideal clear image. Specifically, this application constructs an adaptive mask image based on the local variance map to characterize the non-uniform degradation of the image that is prone to transient changes. This dynamically distinguishes between degraded regions and regions with clear tissue structure. Furthermore, during the diffusion process, the introduction of the mask image enables selective processing of degraded regions, avoiding the participation of irrelevant regions in the calculation. This simplifies the complex end-to-end pixel generation problem of the entire image into a problem of continuous probability distribution evolution within the residual space, focusing only on degradation information and avoiding interference with clear tissue structure. Finally, this application performs a weighted fusion of the reconstructed residual image and the original image based on the corresponding mask image, ensuring a balance between degradation region restoration and clear tissue structure, thereby improving image realism and reliability. In summary, this application can effectively address the problems of local blurring or occlusion caused by bubble atomization, as well as the non-uniformity and transient abrupt changes in degradation caused by non-uniform illumination, light scattering in liquid media, and selective absorption effects. It optimizes the processing of degraded areas while minimizing processing of clear tissue structures in the original image, thereby improving the quality of the reconstructed image and ensuring that the final reconstructed image meets the requirements of medical-grade surgical applications.

[0007] In one implementation, constructing the mask image based on the local variance map includes: The degradation region determination threshold is calculated based on the local variance map; the degradation region determination threshold is the product of the maximum variance scalar value of all pixel spatial locations in the local variance map and the adjustment factor; the local variance map is used to characterize the sharpness distribution of the original image to be processed; Based on the aforementioned degradation region determination threshold, the local variance map is converted into an initial binary mask; The initial binary mask is Gaussian smoothed to obtain the mask image.

[0008] In one embodiment, calculating the intermediate residual at the current time step based on the denoised residual and the mask image includes: performing pixel-by-pixel weighted fusion of the denoised residual and the initial residual using the mask image, wherein the value of each pixel in the fused intermediate residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the denoised residual, plus... The sum of the products of the pixel values ​​corresponding to the initial residual; where, This represents the mask image corresponding to the original image to be processed. In this embodiment, by introducing the mask image to impose a forced truncation constraint on the denoising residual, it is possible to effectively prevent the generated residual from including the clear tissue structure region in the construction range, thus avoiding contamination of the clear tissue structure region.

[0009] In one embodiment, obtaining a target clear image based on the original image to be processed and the reconstruction residual includes: weighted fusing the original image to be processed and the reconstruction residual using a mask image corresponding to the original image to be processed to obtain the target clear image.

[0010] In one embodiment, when processing multiple consecutive frames of images, the mask image is smoothly updated based on the local variance map of adjacent frames, thereby ensuring that visual flickering of the bubble fogging area can be effectively prevented when the frames change.

[0011] In one embodiment, calculating the denoised residual based on the predicted noise and the residual includes: ; in, Indicates the first Denoising residuals at time steps; This represents the amplitude adjustment parameter, used to control the energy ratio of the residual and noise; Indicates the first The residual of the time step; Indicates prediction noise. This represents the original image to be processed. This represents the mask image corresponding to the original image to be processed. This represents the residual diffusion bridge model.

[0012] In one embodiment, the training process of the residual diffusion bridge model includes: Randomly select an original image and its corresponding clear image, calculate the true residual between the original image and the clear image, and construct a mask image of the original image; the original image is an image including blurred areas obtained under the scene of bubble atomization, non-uniform lighting and liquid medium, and the clear image is an ideal reference image that is completely consistent with the content of the original image and the blurred areas have been repaired; A time step and random noise are randomly sampled. At that time step: a noisy term is obtained by diffusion based on the real residual and random noise. The noisy residual is calculated based on the noisy term, the mask image and the real residual. The original image, the noise-added residual, the time step, and the mask image are input into the diffusion model to obtain the predicted noise for that time step. The loss function value is calculated based on the predicted noise and random noise. If the loss function value does not meet the iteration termination condition, the parameters of the diffusion model are updated, and another original image and its corresponding clear image are selected. Otherwise, the current diffusion model is used as the residual diffusion bridge model.

[0013] In one embodiment, the diffusion process is as follows: ; in, Indicates the first The noise term obtained by time-step diffusion; This represents the amplitude adjustment parameter, used to control the energy ratio of the residual and noise; Represents the true residual; Indicates the first Random noise at the time step.

[0014] In one embodiment, the method for calculating the noise-added residual is as follows: The noise-added term and the true residual are weighted and fused pixel-by-pixel using the mask image. After fusion, the value of each pixel in the noise-added residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the noise-added term, plus... The sum of the products of the pixel values ​​corresponding to the true residual; where, This represents the mask image corresponding to the original image.

[0015] According to a second aspect of the embodiments of this application, an image processing system is provided, comprising: The image acquisition module is used to acquire raw images to be processed in real time; the raw images to be processed are images acquired under scenarios of bubble atomization, non-uniform lighting, and liquid media. A mask image construction module is used to calculate the local variance map of the original image to be processed and construct a mask image based on the local variance map; the original image to be processed is an image acquired under bubble atomization, non-uniform lighting, and liquid medium scenarios; a noise prediction module is used to iteratively execute the following steps from the maximum time step to the minimum time step, taking the original image to be processed, the initial residual, and the mask image, where the initial residual is an all-zero matrix: inputting the original image to be processed, the mask image, and the residual into a residual diffusion bridge model, and outputting the predicted noise at the current time step; when the current time step is the maximum time step, the residual is the initial residual; when the current time step is the minimum time step, the noise prediction module is used to construct a mask image based on the local variance map of the original image to be processed, the mask image, and the mask image, where the initial residual is an all-zero matrix; when the current time step is the minimum time step, the noise prediction module is used to construct a mask image based on the local variance map of the original image to be processed, the mask image, and the mask image, where the initial residual is an all-zero matrix; when the current time step is the minimum time step, the noise prediction module is used to construct a mask image based on the local variance map of the original image to be processed, the mask image, and the mask image, where the initial residual is an all-zero matrix; when the current time step is the maximum ... When, the residual is the time step. The obtained intermediate residual; the residual is used to characterize the image degradation caused by bubble atomization, non-uniform illumination and light scattering in liquid medium and selective absorption effect; the residual diffusion bridge model is obtained by training based on the diffusion model; the denoising residual is calculated based on the predicted noise and residual, the intermediate residual of the current time step is calculated based on the denoising residual and the mask image, and the iteration is terminated when the current time step is the minimum time step, and the intermediate residual of the current time step is used as the reconstruction residual; A clear image output module is used to obtain a target clear image based on the original image to be processed and the reconstruction residual.

[0016] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0017] According to a fourth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a medical-grade image processing method according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a diffusion model training method according to an exemplary embodiment; Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0019] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. Specific embodiments of this application will be described below in conjunction with the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. Without departing from the spirit and scope of this application, those skilled in the art can modify and substitute the embodiments of this application, and the resulting embodiments are also within the protection scope of this application.

[0020] As mentioned above, existing methods for processing images from urethral surgery suffer from problems such as local blurring or occlusion caused by bubble atomization, and non-uniform and transient degradation caused by non-uniform illumination, light scattering in liquid media, and selective absorption effects. Furthermore, because existing technologies often perform global uniform processing on both the originally clear tissue structure areas and the degraded areas, the quality of the reconstructed images is difficult to meet the requirements for medical-grade surgical applications.

[0021] To address the aforementioned technical issues, this application proposes a medical-grade image processing method, system, device, and computer program product. This method reconstructs only the degraded regions (i.e., the residual portions) where the image variance is below the degradation region determination threshold. After obtaining the residual image, a mask image is introduced for global fusion processing. This achieves the goal of not processing clear tissue structure regions in the image, while enhancing and repairing the degraded regions.

[0022] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] One embodiment of this application provides a medical-grade image processing method. This medical-grade image processing method can be applied to electronic devices such as image processors and display chips. Please refer to... Figure 1 The medical-grade image processing method may include the following steps 101 to 105: Step 101: Calculate the local variance map of the original image to be processed, and construct a mask image based on the local variance map.

[0024] In this step, the original image to be processed is an image acquired under conditions of bubble atomization, non-uniform lighting, and liquid medium. At that time, local degradation analysis is used to identify the degradation regions corresponding to local blurring or occlusion caused by bubble atomization, and a spatial weight mask is constructed to determine the algorithm's intensity in subsequent steps; among which, Indicates the image height. This represents the image width; preferably, in this embodiment, the image size is... This ensures the clarity of sub-millimeter-level microvessels and anatomical edges in the surgical field while minimizing computational overhead and avoiding the risk of memory overflow caused by high resolution.

[0025] Specifically, this application introduces an averaging pooling method to process the original image to obtain a local variance map. This local variance map can quantitatively describe the sharpness distribution of the image. This process can be represented as follows: ; in, Represented in pixels Centered on the size of the local sliding window The local variance scalar value is calculated for the range and satisfies... and ; This indicates averaging pool processing.

[0026] In some embodiments, the size of the sliding window is set to... This window size provides sufficient statistical bandwidth to smooth out intraoperative electronic noise, ensuring mask continuity, while avoiding smoothing out delicate blood vessels and instrument edges due to an excessively large receptive field. This ensures a closed-loop unity between precise local repair and global fidelity at the underlying mechanism level. It's also important to note that if the window size is too small, insufficient local statistical samples will significantly reduce the resistance to high-frequency electronic noise and cellular debris during surgery, leading to initial mask fragmentation and isolated cavity artifacts. Conversely, if the window size is too large, an excessively large receptive field will trigger a strong low-pass smoothing effect, causing a degradation in the resolution of clear blood vessel and tissue boundary features in the variance map. This results in the spatial mask spreading beyond the boundaries of clear tissue areas, failing to achieve precise protection of healthy tissue structures.

[0027] In some embodiments, constructing a mask image based on a local variance map includes: Step 1011: Calculate the degradation region determination threshold based on the local variance map, where the local variance map is used to characterize the sharpness distribution of the original image to be processed. The degradation region determination threshold is the product of the maximum variance scalar value of all pixel spatial locations in the local variance map and the adjustment factor, expressed as: ; in, This indicates the threshold for determining a degraded region; Indicates the regulating factor; The maximum variance scalar value represents the spatial location of all pixels in the local variance map. This scalar value serves as the statistical reference benchmark for the clearest and most textured area in the entire image, typically the edge of healthy tissue without fogging, the high-sharp outline of microvessels, or surgical instruments.

[0028] In some embodiments, the range of values ​​for the above-mentioned adjustment factor is: Ideally, an adjustment factor of 0.2 is optimal for achieving adaptive dynamic segmentation of degradation boundaries in complex and variable surgical environments. It's important to note that setting the adjustment factor below 0.15 results in an extremely low threshold for determining degradation regions. This means only severely degraded core pixels with absolutely smooth, textureless surfaces meet the criteria, while numerous peripheral areas with low-concentration fogging or slight bubble occlusion will be incorrectly identified as clear tissue structures due to residual weak variance. This leads to significant missed detections of degradation regions and prevents them from undergoing residual repair. Conversely, a adjustment factor above 0.30 results in an excessively high threshold. In environments with good overall clarity, this high threshold can incorrectly classify healthy tissue edges or instrument outlines with variances greater than the threshold as degradation regions, causing false detections and forcing non-degraded areas to participate in residual diffusion evolution in subsequent procedures, thus failing to achieve the core objective of protecting clear tissue structures. In addition, the difference between the upper and lower limits of the adjustment factor needs to be greater than or equal to 0.1 and less than or equal to 0.3 to ensure that it can switch dynamically to adapt to the upper limit when there is a sudden outbreak of moderate to severe smoke. When a sudden dense fog is generated during the operation accompanied by the energy instrument point cutting or the irrigation fluid becomes turbid, the global contrast drops sharply, which will cause the overall maximum variance scalar value to drop significantly. If the adjustment factor does not have enough upper limit space, the degradation region judgment threshold will not be able to be adaptively raised, resulting in the spatial mask not being able to effectively cover these sudden dense fog regions with medium variance, thus causing insufficient robustness of the algorithm.

[0029] Step 1012: Based on the degradation region determination threshold, the local variance map is transformed into an initial binary mask. This process is represented as follows: ; Indicates the initial binary mask Medium pixel The value at; when When, it indicates that the corresponding pixel is a degraded area with blur or bubbles. When the value is 0, it indicates that the corresponding pixel is a region with a clear organizational structure.

[0030] Step 1013: Perform Gaussian smoothing on the initial binary mask to obtain the mask image, where Gaussian smoothing is represented as: ; in, Represents a mask image; Indicates Gaussian smoothing; The standard deviation represents the Gaussian kernel. When the standard deviation is less than the first preset value, it will not be able to effectively eliminate the visual artifacts at the boundary caused by the step fault. When the standard deviation is greater than the second preset value, it will cause the mask edge to diffuse excessively into the clear tissue area, destroying the original intention of local precision protection. When the standard deviation is 1, it achieves a balance between visual continuity at the boundary and spatial protection accuracy.

[0031] After the above processing, the mask image can be normalized to Furthermore, the closer the scalar value of the mask image is to 1, the more severe the degradation of the corresponding region. For such regions, this application will inject noise perturbation and reconstruct the image. When the scalar value of the mask image is closer to 0, the corresponding region is a clear tissue region. For such regions, this application will suppress the diffusion process and directly preserve the original pixels. Specifically, regions with a mask image scalar value of 0.5 represent the penumbra edge of bubbles, the periphery of gradually thinning smoke, or the transition zone between blurred and clear tissue in the surgical field. This intermediate value acts as a soft control gate, directly affecting the subsequent diffusion process. This means that at the current pixel, the method provided by this application both preserves the residual features of the original image to maintain the existing weak texture and introduces additional energy diffusion perturbation for repair.

[0032] By using a masked image, this application can characterize image degradation caused by non-uniform illumination, light scattering in liquid media, and selective absorption effects, resulting in non-uniformity and transient abrupt changes, thus avoiding its impact on the accuracy of subsequent image reconstruction.

[0033] Given the original image to be processed, the initial residual, and the mask image (the initial residual is an all-zero matrix), iterate through steps 102 to 104 from the maximum time step to the minimum time step: Step 102: Input the original image to be processed, the mask image and the residual into the residual diffusion bridge model, and output the prediction noise at the current time step. The residual is used to characterize the image degradation caused by bubble fogging, non-uniform illumination and light scattering in liquid medium and selective absorption effects.

[0034] In this step, if the current time step is the maximum time step, that is... The residual is the initial residual; when the current time step is the time step And t When, the residual is the time step. The intermediate residuals obtained.

[0035] Step 103: Calculate the denoised residual based on the predicted noise and the residual, and calculate the intermediate residual at the current time step based on the denoised residual and the mask image.

[0036] In some embodiments, the method for calculating the denoising residual is as follows: ; in, Indicates the first Denoising residuals at time steps; This represents the amplitude adjustment parameter, used to control the energy ratio of the residual and noise; Indicates the first The residual of the time step; Indicates prediction noise. This represents the original image to be processed. This represents the mask image corresponding to the original image to be processed. This represents the residual diffusion bridge model.

[0037] This step is a reverse reconstruction, belonging to the Markov chain state flow, that is, reconstructing the time step from the second residual of the final time step T. The residual.

[0038] In some embodiments, the method for calculating the intermediate residual is as follows: The denoised residual and the initial residual are weighted and fused pixel-by-pixel using the mask image. After fusion, the value of each pixel in the intermediate residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the denoised residual, plus... The sum of the products of the pixel values ​​corresponding to the initial residual is used in this embodiment to ensure the local controllability of the trajectory during the reverse diffusion process. Therefore, in each calculation, a mask image corresponding to the original image to be processed is introduced for physical constraint.

[0039] This process can be represented as: ; in, This represents the mask image corresponding to the original image to be processed. The initial residual is represented by an all-zero matrix. This is because, during model training, since the target image is unknown, the residual between the original image to be processed and the target image is also unknown. To facilitate subsequent inference, this application sets the initial value to an all-zero matrix. By setting the initial value to an all-zero matrix, perfect mathematical symmetry can be achieved on both the forward and reverse paths, and the following beneficial effects are achieved: 1) The reverse denoising shown in step 103 is performed under the spatiotemporal guidance of the spatial mask in each round of reverse diffusion. When processing the degraded region, since the first half of the formula is truncated by the mask and the second residual in the second half is pre-injected into the all-zero matrix, the reconstruction residual of the next time step of the region is forced to zero in each calculation, which fundamentally prevents the repair texture generated by the diffusion model in the iteration from overflowing or spreading to the clear healthy tissue region.

[0040] 2) At each discrete node of the time-axis evolution, the residual state of the clear structure region is always rigidly locked by an all-zero matrix, and does not participate in any noise removal or evolutionary energy transfer; when the reverse iteration reaches the last step, that is... In this way, the generated final prediction residual naturally retains a pure zero value in the non-degenerate region, thereby ensuring that the final clear image of the target achieves zero modification and zero distortion of the anatomical structure in the clear tissue structure region.

[0041] Furthermore, in some embodiments, this application employs a denoising diffusion implicit model at this stage. Compared to the random sampling process of the traditional denoising diffusion probabilistic model, the denoising diffusion implicit model introduces a non-Markov chain forward jump process, allowing deterministic sampling across discrete time nodes during inference. This significantly shortens the backsampling time while maintaining the determinism and temporal continuity of the enhancement results, effectively avoiding inter-frame flickering in video stream processing.

[0042] In some embodiments, this application leverages the efficient convergence of residual modeling to compress the number of iterations in the process from the traditional 1000 steps to [a smaller number]. Step, and preferably set The optimal number of steps is 50. The reason is that while fewer than 50 steps would increase speed, the discretization error of the denoising diffusion implicit model when approximating the continuous probability flow would increase due to the small number of sampling steps, potentially leading to minor graininess or stripe artifacts at the edges of the repaired bubbles. Conversely, more than 100 steps, while generating more detailed image textures, would increase computation time, failing to meet real-time requirements. Since the residual space eliminates complex global anatomical backgrounds and models only the noise distribution of local degradation features, its distribution structure is simpler than the full pixel space. Therefore, the model can achieve high-fidelity texture reconstruction with a lightweight, low sampling number.

[0043] Step 104: Determine if the current time step is the minimum iteration step, i.e. Check if the value is zero; if yes, proceed to step 105; otherwise, use the intermediate residual obtained at that time step as the residual and re-execute step 102.

[0044] Step 105: Obtain a clear image of the target based on the original image to be processed and the reconstruction residual.

[0045] In some embodiments, the method for calculating the sharp target image is as follows: the reconstruction residuals corresponding to the original image to be processed and the final time step are weighted and fused using the mask image corresponding to the original image to be processed to obtain the sharp target image, represented as: ; in, Represents a clear image of the target; This represents the original image to be processed; This represents the mask image corresponding to the original image to be processed. Indicates the final time step Reconstruction residuals at that time.

[0046] In some embodiments, to further eliminate subtle unnatural transitions, artificial cut marks, or residual computational noise that may be caused by discrete calculations or mask gradients at the boundary between clear tissue structure regions and degraded regions containing blur or bubbles, this application targets clear images... Bilateral filtering is performed, which maintains the high-frequency sharpness of the edge anatomy while specifically smoothing out minor transition artifacts at the mask boundaries, ensuring a high degree of continuity and consistency in the overall anatomical structure and visual presentation of the output image.

[0047] In some embodiments, when processing multiple consecutive frames of images, i.e., video streams, this application employs a sliding window processing mechanism for real-time throughput. The mask image is smoothly updated based on the local variance maps of adjacent frames to prevent visual flickering caused by bubbles or fogging areas during high-speed movement. It should also be noted that the method provided in this application supports parallel processing of image acquisition, mask generation, and residual reconstruction tasks in a multi-threaded manner on high-performance computing units, ensuring the stability of visual feedback throughout the surgical process.

[0048] Through this embodiment, the present application can achieve real-time enhancement and repair of visual images, which can reduce visual blurring caused by bubble atomization, thereby reducing the interruption of operation during surgery due to waiting for the visual field to recover. Compared with the traditional method that relies on natural dissipation, it significantly improves the continuity of the surgical process, reduces the fluctuation of surgical time, and helps to reduce operational risks and the burden on doctors.

[0049] In some embodiments, the training process of the residual diffusion bridge model is as follows: Figure 2 As shown, steps 201 to 206 are included: Step 201: Randomly select the original image and its corresponding clear image, calculate the true residual between the original image and the clear image, and construct the mask image of the original image.

[0050] In this step, the original image and the clear image are required to be exactly the same as in step 101. The original image is an image including blurred areas acquired under the scene of bubble atomization, non-uniform lighting and liquid medium. The clear image is an ideal reference image that is completely consistent with the content of the original image and the blurred areas have been repaired.

[0051] It should be noted that the true residual is the pixel difference between the original image and the sharpened image, expressed as... , Indicates a clear image. The original image is represented here; the process of constructing the mask image is similar to steps 1011 to 1013, and will not be repeated here.

[0052] Step 202: Randomly sample a time step and random noise. At this time step: Diffusion is performed based on the real residual and random noise to obtain a noisy term. The noisy residual is calculated based on the noisy term, the mask image and the real residual.

[0053] In urological surgery scenarios, most areas of the image, such as clear tissue surfaces not obscured by bubbles or fog, should be strictly preserved. Therefore, this application does not blindly predict the complete target image from end to end by the model. Instead, it decouples the image structure and reconstructs only the residual between the image and the clear image to achieve targeted and accurate repair of local degradation features.

[0054] In some embodiments, the diffusion process is as follows: ; in, Indicates the first The noise term obtained by time-step diffusion; This represents the amplitude adjustment parameter, used to control the energy ratio of the real residual signal to the noise signal at the current time step; Represents the true residual; Indicates the first Random noise at the time step.

[0055] This diffusion process describes the probabilistic path of the evolution of the true residual into standard Gaussian noise, at any evolution time step in this forward diffusion process. Within, based on the analytical properties of the edge distribution of the diffusion model, at any time step The intermediate residual states can be directly calculated from the actual original residuals using the formulas described above.

[0056] In some embodiments, the method for calculating the noisy residual is as follows: The noisy term and the true residual are weighted and fused pixel-by-pixel using a mask image. After fusion, the value of each pixel in the noisy residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the noisy term, plus... The sum of the products of the pixel values ​​corresponding to the true residuals is calculated using the following expression: ; Indicates at time step The calculated noisy residual; This indicates element-wise multiplication; This represents the true residual.

[0057] By introducing a mask image at each forward diffusion time step for forced constraint, it is possible to effectively prevent noise introduced during the diffusion process from contaminating clear tissue structure regions.

[0058] Furthermore, the noise-added residual calculation method provided in the above embodiments can ensure that when the scalar value of the mask image approaches 1, i.e. there is a degenerate region with blur or bubbles, the noise-added terms disturbed by noise are retained, so that the model can learn the texture reconstruction of degenerate features such as bubbles and fogging in this region through inverse evolution; while when the scalar value of the mask image approaches 0, i.e. a clear structural region, the noise disturbance terms can be suppressed, and the true original residuals that are not contaminated by noise are forcibly retained. At this time, the diffused noise is completely prohibited from entering the clear tissue region, thus eradicating the global image blurring problem caused by diffuse contamination.

[0059] Step 203: Input the original image, the noisy residual, the time step, and the mask image into the diffusion model to obtain the predicted noise for that time step.

[0060] Through steps 202 and 203, this application utilizes residuals as a modulation technique for forward noise injection and reverse denoising reconstruction to achieve targeted and precise repair of local degradation features.

[0061] Step 204: Calculate the loss function value based on the predicted noise and random noise.

[0062] In some embodiments, the diffusion model parameters are iteratively optimized by minimizing the mean squared error loss function between the predicted noise and the actual noise, as expressed as: ; in, This represents the mean squared error loss function value; This represents the introduced random noise; This represents the first prediction noise; Represents norm calculation.

[0063] Step 205: Determine whether the loss function value meets the iteration termination condition. If the iteration termination condition is not met, update the parameters of the diffusion model, select another original image and its corresponding clear image, and return to step 201.

[0064] Step 206: If the loss function value satisfies the iteration termination condition, then the current diffusion model is used as the residual diffusion bridge model.

[0065] In summary, the method provided in this application has the following beneficial effects: 1) Addressing the issues of localized blurring or occlusion caused by bubble atomization during urological surgery, and the non-uniform and transient degradation resulting from non-uniform illumination, light scattering in liquid media, and selective absorption effects, this application not only effectively repairs degraded areas, enhancing overall image contrast and clarity, but also ensures that key structural information such as tissue boundaries, textures, and blood vessels is well preserved while restoring image quality, thereby improving the recognizability and visual reliability of the surgical area. Furthermore, because it only specifically repairs degraded areas without processing the original clear tissue structures, it eliminates the need for global image brightness and color adjustments, effectively avoiding overexposure and color distortion. By compensating only for missing effective detail information without generating additional pixel structural features, it effectively avoids anatomical artifacts, meeting the high-precision enhancement requirements of medical-grade surgical applications.

[0066] 2) It can enhance and repair the acquired video frames in real time, reduce the blurring of vision caused by bubble atomization, thereby reducing the interruption of operation during the operation while waiting for the vision to recover. Compared with the traditional method that relies on natural dissipation, it significantly improves the continuity of the operation process, reduces the fluctuation of operation time, and reduces the operation risk.

[0067] Embodiments of this application also propose an image processing system that complements the method embodiments, including: The training image library stores the original images and their corresponding sharpened images.

[0068] The image acquisition module is used to acquire raw images to be processed in real time; the raw images to be processed are images acquired under scenarios of bubble atomization, non-uniform lighting, and liquid media.

[0069] The mask image construction module is used to calculate the local variance map of the original image to be processed and construct a mask image based on the local variance map. The original image to be processed is an image acquired under scenes of bubble atomization, non-uniform lighting, and liquid medium. The noise prediction module is used to iteratively execute the following steps from the maximum time step to the minimum time step, taking the original image to be processed, the initial residual, and the mask image (the initial residual is an all-zero matrix): inputting the original image to be processed, the mask image, and the residual into the residual diffusion bridge model, and outputting the predicted noise at the current time step; when the current time step is the maximum time step, the residual is the initial residual; when the current time step is the minimum time step... When, the residual is the time step. The intermediate residuals are obtained; the residuals are used to characterize the image degradation caused by bubble atomization, non-uniform illumination and light scattering in liquid media and selective absorption effects; the residual diffusion bridge model is obtained by training the diffusion model; the denoised residuals are calculated based on the predicted noise and the residuals, the intermediate residuals of the current time step are calculated based on the denoised residuals and the mask image, and the iteration is terminated when the current time step is the minimum time step, and the intermediate residuals of the current time step are used as the reconstruction residuals.

[0070] The sharp image output module is used to obtain a sharp image of the target based on the original image to be processed and the reconstruction residual.

[0071] Embodiments of this application also propose an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the medical-grade image processing method described in any of the above embodiments.

[0072] Embodiments of this application also propose a computer program product, including a computer program that, when executed by a processor, implements the medical-grade image processing method of any of the above embodiments.

[0073] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0074] Embodiments of this application also provide an electronic device 600, such as... Figure 3 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs executable by the processor 602; the processor 602 is used to execute the computer programs in the memory 601 to implement the medical-grade image processing method provided in any of the above embodiments.

[0075] The electronic device 600 also includes a communication interface 603. The processor 602, memory 601, and communication interface 603 are connected via a communication bus and communicate with each other.

[0076] Processor 602 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0077] Communication interface 603 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0078] The memory 601 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0079] In this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined. The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. A medical-grade image processing method, characterized in that, include: Calculate the local variance map of the original image to be processed, and construct a mask image based on the local variance map; The original image to be processed is an image acquired under scenarios of bubble atomization, non-uniform lighting, and liquid medium. Based on the original image to be processed, the initial residual, and the mask image, wherein the initial residual is an all-zero matrix, the following steps are iteratively executed from the maximum time step to the minimum time step: The original image to be processed, the masked image, and the residual are input into the residual diffusion bridge model, which outputs the prediction noise at the current time step. When the current time step is the maximum time step, the residual is the initial residual; when the current time step is the maximum time step, the residual is the maximum time step. When, the residual is the time step. The obtained intermediate residuals are used to characterize image degradation caused by bubble atomization, non-uniform illumination, light scattering in liquid media, and selective absorption effects. The residual diffusion bridge model is obtained by training a diffusion model. The denoising residual is calculated based on the predicted noise and residual. The intermediate residual at the current time step is calculated based on the denoising residual and the mask image. If the current time step is the minimum time step, the iteration is terminated, and the intermediate residual at the current time step is used as the reconstruction residual. A clear image of the target is obtained based on the original image to be processed and the reconstruction residual.

2. The medical-grade image processing method as described in claim 1, characterized in that, The construction of the mask image based on the local variance map includes: The degradation region determination threshold is calculated based on the local variance map; the degradation region determination threshold is the product of the maximum variance scalar value of all pixel spatial locations in the local variance map and the adjustment factor; the local variance map is used to characterize the sharpness distribution of the original image to be processed; Based on the aforementioned degradation region determination threshold, the local variance map is converted into an initial binary mask; The initial binary mask is Gaussian smoothed to obtain the mask image.

3. The medical-grade image processing method as described in claim 1, characterized in that, The step of calculating the intermediate residual at the current time step based on the denoised residual and the mask image includes: performing pixel-by-pixel weighted fusion of the denoised residual and the initial residual using the mask image; the value of each pixel in the fused intermediate residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the denoised residual, plus... The sum of the products of the pixel values ​​corresponding to the initial residual; where, This represents the mask image corresponding to the original image to be processed.

4. The medical-grade image processing method as described in claim 1, characterized in that, The step of obtaining a clear target image based on the original image to be processed and the reconstruction residual includes: weighted fusing the original image to be processed and the reconstruction residual using a mask image corresponding to the original image to be processed to obtain a clear target image.

5. The medical-grade image processing method as described in claim 1, characterized in that, When processing multiple consecutive frames of images, the mask image is updated smoothly based on the local variance maps of adjacent frames.

6. The medical-grade image processing method as described in claim 1, characterized in that, The calculation of the denoised residual based on the predicted noise and residual includes: ； in, Indicates the first Denoising residuals at time steps; This represents the amplitude adjustment parameter, used to control the energy ratio of the residual and noise; Indicates the first The residual of the time step; Indicates prediction noise, This represents the original image to be processed. This represents the mask image corresponding to the original image to be processed. This represents the residual diffusion bridge model.

7. The medical-grade image processing method as described in claim 1, characterized in that, The training process of the residual diffusion bridge model includes: Randomly select an original image and its corresponding clear image, calculate the true residual between the original image and the clear image, and construct a mask image of the original image; the original image is an image including blurred areas obtained under bubble atomization, non-uniform lighting and liquid medium scenes, and the clear image is an ideal reference image that is completely consistent with the content of the original image and the blurred areas have been repaired; A time step and random noise are randomly sampled. At that time step: a noisy term is obtained by diffusion based on the real residual and random noise. The noisy residual is calculated based on the noisy term, the mask image and the real residual. The original image, the noise-added residual, the time step, and the mask image are input into the diffusion model to obtain the predicted noise for that time step. The loss function value is calculated based on the predicted noise and random noise. If the loss function value does not meet the iteration termination condition, the parameters of the diffusion model are updated, and another original image and its corresponding clear image are selected. Otherwise, the current diffusion model is used as the residual diffusion bridge model.

8. The medical-grade image processing method as described in claim 7, characterized in that, The diffusion process is as follows: ； in, Indicates the first The noise term obtained by time-step diffusion; This represents the amplitude adjustment parameter, used to control the energy ratio of the residual and noise; Represents the true residual; Indicates the first Random noise at time steps.

9. The medical-grade image processing method as described in claim 7, characterized in that, The method for calculating the noise-added residual is as follows: The noise-added term and the true residual are weighted and fused pixel-by-pixel using the mask image. After fusion, the value of each pixel in the noise-added residual is equal to the product of the pixel value of the mask image at that position and the corresponding pixel value of the noise-added term, plus... The sum of the products of the pixel values ​​corresponding to the true residual; where, This represents the mask image corresponding to the original image.

10. An image processing system, characterized in that, include: The image acquisition module is used to acquire raw images to be processed in real time; the raw images to be processed are images acquired under scenarios of bubble atomization, non-uniform lighting, and liquid media. The mask image construction module is used to calculate the local variance map of the original image to be processed, and construct a mask image based on the local variance map; the original image to be processed is an image acquired under bubble atomization, non-uniform lighting and liquid medium scenarios; The noise prediction module is used to iteratively execute the following steps from the maximum time step to the minimum time step, taking the original image to be processed, the initial residual, and the mask image, where the initial residual is an all-zero matrix: inputting the original image to be processed, the mask image, and the residual into the residual diffusion bridge model, and outputting the predicted noise at the current time step; when the current time step is the maximum time step, the residual is the initial residual; when the current time step is the minimum time step... When, the residual is the time step. The obtained intermediate residuals are used to characterize image degradation caused by bubble atomization, non-uniform illumination, light scattering in liquid media, and selective absorption effects. The residual diffusion bridge model is obtained by training a diffusion model; the denoised residual is calculated based on the predicted noise and residual; the intermediate residual at the current time step is calculated based on the denoised residual and the mask image; and the iteration is terminated when the current time step is the minimum time step, and the intermediate residual at the current time step is used as the reconstruction residual. A clear image output module is used to obtain a target clear image based on the original image to be processed and the reconstruction residual.

11. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program executable by the processor; and the processor executes the computer program in the memory to implement the medical-grade image processing method as described in any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the medical-grade image processing method as described in any one of claims 1 to 9.

Images