An image enhancement processing method for a visual bronchoscope

Abstract

This invention belongs to the field of image processing technology and discloses an image enhancement method for visual bronchoscopes. This method is applied to a visual bronchoscope with an eccentrically positioned integrated imaging module at the distal end and an internal suction channel that is non-coaxial with the integrated imaging module. First, the method acquires the current frame's raw image data from the integrated imaging module and the real-time attitude data of the visual bronchoscope, and calculates global spatial parameters using pre-calibrated parameters. Then, image correction is performed based on the global spatial parameters to obtain a standardized corrected image. Corresponding enhancement processing is performed on the corrected image by region, and after feature alignment and fusion, coordinate transformation, an operational view image is obtained, which is then optimized and output in real time. This method, combined with an eccentric bronchoscope, can improve the clarity and uniformity of bronchoscope images, ensure the match between the view and the operational path, and meet clinical imaging needs.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and specifically to an image enhancement processing method for visual bronchoscopes. Background Technology

[0002] In respiratory medicine clinical practice, video bronchoscopes are core equipment for airway examination, lesion diagnosis, and interventional procedures. Video bronchoscopes with an eccentric distal imaging module and a built-in non-coaxial suction channel have been widely used in clinical settings due to their ability to perform both real-time imaging and negative pressure suction. However, current image processing technologies for these eccentric video bronchoscopes often directly adopt the general processing solutions used for conventional central imaging endoscopes. These technologies cannot adapt to the unique imaging defects inherent in the eccentric imaging structure, easily leading to problems such as uneven image illumination, distorted anatomical proportions, insufficient tissue detail recognition, and mismatch between the imaging view and the instrument operation path, thus failing to meet the needs of precise clinical diagnosis and treatment. Summary of the Invention

[0003] To solve, or at least partially solve, the above-mentioned technical problems, this application provides an image enhancement processing method for visual bronchoscopes.

[0004] This invention provides an image enhancement processing method for a video bronchoscope. The method is applied to a video bronchoscope, which includes an imaging catheter. An integrated imaging module is offset at the distal end of the imaging catheter, and a suction channel is provided inside the imaging catheter that is not coaxially arranged with the integrated imaging module. The method includes the following steps:

[0005] S1. Obtain the current frame raw image data collected by the integrated imaging module, and the real-time attitude data of the visual bronchoscope.

[0006] S2. Based on the pre-calibrated eccentric installation parameters and lens optical parameters of the integrated imaging module, and combined with the real-time attitude data, the global spatial parameters corresponding to the current frame are calculated. The global spatial parameters include a field distance distribution map that characterizes the distance distribution between the imaging target and the integrated imaging module at different positions within the field of view, an asymmetric illumination attenuation distribution model that characterizes the non-uniform illumination attenuation distribution caused by the eccentric structure within the field of view, and a mapping matrix between the imaging coordinate system and the suction channel feed coordinate system. The mapping matrix is ​​used to characterize the corresponding transformation relationship between the imaging coordinates and the instrument feed coordinates within the suction channel.

[0007] S3. Based on the global spatial parameters, perform asymmetric illumination compensation, two-order geometric correction, and regional color restoration sequentially on the original image data of the current frame to obtain a standardized corrected image;

[0008] S4. Based on the field of view object distance distribution map, the standardized correction image is divided into near module region, intermediate transition region and far module region according to the distance difference between the imaging target and the integrated imaging module, and spectral enhancement, texture detail enhancement and resolution optimization processing with corresponding weights are performed respectively to obtain multiple sets of enhanced images;

[0009] S5. After mapping multiple sets of enhanced images and standardized correction images to the same anatomical coordinate system to complete feature alignment, perform fusion processing, and then convert them into operation view images according to the mapping matrix. The image coordinates of the operation view images correspond one-to-one with the feed coordinates of the operating instruments in the suction channel.

[0010] S6. After performing post-processing optimization on the operation view image, it is output to the display terminal in real time.

[0011] Optionally, the eccentric mounting parameters include the eccentricity of the integrated imaging module relative to the imaging guide axis and the mounting angle, and the lens optical parameters include the lens distortion coefficient and the relative position parameters of the light source and the integrated imaging module.

[0012] Optionally, the real-time attitude data includes the real-time curvature and insertion depth of the imaging catheter;

[0013] The calculated global spatial parameters also include the asymmetric perspective distortion parameters corresponding to the current frame.

[0014] Optionally, the specific method of the asymmetric illumination compensation is as follows:

[0015] Based on the asymmetric illumination attenuation distribution model, the brightness attenuation region and the brightness excess region of the original image data of the current frame are determined. The brightness attenuation region is subjected to a corresponding brightness enhancement process, and the brightness excess region is subjected to a corresponding brightness suppression process, resulting in an illumination homogenized image.

[0016] Optionally, the execution order of the two-order geometric correction is as follows:

[0017] Based on the asymmetric perspective distortion parameters, the illumination homogenized image after asymmetric illumination compensation is corrected for off-center perspective distortion.

[0018] Based on the lens optical parameters, the illumination homogenized image after the off-center perspective distortion correction is subjected to lens inherent distortion correction to obtain a corrected image with true anatomical proportions.

[0019] Optionally, the specific method for the regional color restoration is as follows:

[0020] Based on the field-of-view object distance distribution map, the corrected image after the completion of the two-order geometric correction is divided into multiple regions corresponding to different object distances. White balance adjustment and color space conversion with corresponding gain are performed on different regions to obtain the standardized corrected image with consistent color reproduction.

[0021] Optionally, the specific method of spectral enhancement is as follows:

[0022] The standardized corrected image, after completing the color restoration of the sub-regions, is converted to a color space that matches the hemoglobin absorption spectrum;

[0023] The near module region, the intermediate transition region, and the far module region are respectively subjected to blue and green channel signal enhancement and red channel signal suppression with corresponding weights.

[0024] The spectral enhancement image of the corresponding region is obtained through nonlinear mapping processing.

[0025] Optionally, the specific method of enhancing texture details is as follows:

[0026] The spectrally enhanced image after the spectral enhancement is completed is decomposed into an illumination base layer that reflects changes in illumination and a detail texture layer that contains information about tissue details.

[0027] Brightness adjustment of the corresponding magnitude is performed on the illumination base layer of the near module region, the intermediate transition region, and the far module region, and detail enhancement of the corresponding intensity is performed on the detail texture layer of the near module region, the intermediate transition region, and the far module region to expand the color difference between the lesion tissue and the normal tissue in the corresponding region.

[0028] Optionally, the resolution optimization is specifically performed as follows:

[0029] Noise reduction is performed only on the near module region, 2x super-resolution reconstruction and corresponding intensity noise reduction are performed on the intermediate transition region, and 4x super-resolution reconstruction and corresponding intensity noise reduction are performed on the far module region, resulting in multiple sets of enhanced images with uniform resolution across the entire field of view.

[0030] Optionally, the fusion process adopts a pyramid fusion method, in which the image coordinates of any anatomical position in the converted operation view image correspond one-to-one with the feed coordinates of the operating instrument in the suction channel.

[0031] The method provided by this invention has the following beneficial effects:

[0032] This method is designed for a visual bronchoscope with an eccentrically set integrated imaging module. By acquiring the original image data of the current frame collected by the integrated imaging module and the real-time attitude data of the visual bronchoscope, and combining the pre-calibrated eccentric installation parameters and lens optical parameters, the global spatial parameters corresponding to the current frame are calculated. This enables the subsequent image processing to match the actual imaging state of the current frame and adapt to the real-time changes in imaging conditions during duct movement.

[0033] By sequentially performing asymmetric illumination compensation, two-order geometric correction, and regional color restoration on the current frame's original image data based on global spatial parameters, the problems of uneven unilateral illumination, perspective distortion, and inconsistent color restoration in different regions caused by eccentric imaging structures can be specifically addressed. This yields a standardized corrected image with accurate anatomical proportions and tissue color restoration, providing a stable and reliable image foundation for subsequent enhancement processing. By dividing the standardized corrected image into near-module, intermediate transition, and far-module regions based on the field-of-view distance distribution map, and performing enhancement processing with corresponding weights on each of the three regions, the system can adapt to the differences in imaging conditions in different regions, avoiding the problems of local over-enhancement or under-enhancement caused by global uniform enhancement, effectively improving the recognition of bronchial tissue and vascular structures in the image.

[0034] By mapping multiple sets of enhanced images and standardized corrected images to the same anatomical coordinate system for feature alignment and then performing fusion processing, and finally converting them into an operational view image coaxial with the suction channel feed path according to the mapping matrix, the anatomical positions in the image can accurately correspond to the actual instrument operation path, avoiding the problem of misalignment between the imaging view and the operation position, and reducing the risk of clinical operation. The real-time output images after post-processing optimization can stably adapt to the clinical use needs of off-center structure visual bronchoscopes, improving the accuracy of image observation and the convenience of operation during clinical diagnosis and treatment. Attached Figure Description

[0035] Figure 1 A schematic flowchart of an image enhancement processing method for a visual bronchoscope provided in this application embodiment;

[0036] Figure 2 A structural side view of a visual bronchoscope provided in an embodiment of this application;

[0037] Figure 3 A side view of the structure of a visual bronchoscope provided in this application embodiment.

[0038] 11. Imaging catheter; 12. Integrated imaging module; 13. Suctioning channel. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0040] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0041] See Figures 1 to 3 This invention provides an image enhancement processing method for a video bronchoscope. The method is applied to a video bronchoscope, which includes an imaging catheter. An integrated imaging module is eccentrically disposed at the distal end of the imaging catheter. The imaging catheter has a suction channel arranged non-coaxially with the integrated imaging module inside. The method includes the following steps:

[0042] S1. Acquire the current frame raw image data collected by the integrated imaging module, as well as the real-time attitude data of the visual bronchoscope.

[0043] In some implementations, real-time attitude data includes the real-time curvature and insertion depth of the imaging catheter;

[0044] The calculated global spatial parameters also include the asymmetric perspective distortion parameters corresponding to the current frame.

[0045] Specifically, the image enhancement processing method for a visual bronchoscope disclosed in this embodiment is applied to a visual bronchoscope with an eccentric integrated imaging module 12. For example... Figures 2 to 3 As shown, Figure 2 This is a side view of a visual bronchoscope. The viewing angle is perpendicular to the plane formed by the X and Y directions. The Z direction is not shown, but it is perpendicular to the plane formed by the X and Y directions. Figure 3This is a side view of a video bronchoscope at point AA, perpendicular to the YZ plane. The core component of this type of video bronchoscope is the imaging catheter 11. An integrated imaging module 12 is offset at the distal end of the imaging catheter 11. Inside the imaging catheter 11 is a suction channel 13, non-coaxially arranged with the integrated imaging module 12, allowing for simultaneous airway clearance during imaging observation. During clinical examination of the human bronchus, the imaging catheter 11 needs to gradually penetrate along the branching bronchial lumen. During this process, it undergoes varying degrees of bending and adjustment according to the physiological direction of the lumen. The imaging angle and distance of the offset integrated imaging module 12 dynamically change with the catheter's position, affecting the basic quality of the acquired images and placing higher demands on data synchronization and matching during image acquisition.

[0046] Therefore, in the implementation of this method, two sets of basic data are first acquired simultaneously: one is the raw image data of the current frame acquired by the integrated imaging module, and the other is the real-time attitude data of the visual bronchoscope. The raw image data of the current frame is acquired by the image sensor built into the integrated imaging module, which is the initial image information without additional processing, and completely preserves the original imaging information of the bronchial tissue. The real-time attitude data is acquired synchronously by the attitude detection element built into the visual bronchoscope, and the acquisition sequence is aligned with the acquisition sequence of the raw image data of the current frame, so that the two sets of data correspond to the catheter state and imaging state at the same moment.

[0047] Real-time attitude data specifically includes the real-time curvature and insertion depth of the imaging catheter. These two types of data comprehensively reflect the real-time position and morphology of the imaging catheter within the bronchial lumen. Specifically, the real-time curvature of the imaging catheter reflects the current imaging viewpoint shift of the integrated imaging module, while the insertion depth reflects the distance range between the integrated imaging module and the target tissue within the bronchial lumen. These two types of synchronously acquired real-time attitude data are combined with pre-calibrated fixed parameters to provide a complete real-time state basis for the subsequent calculation of global spatial parameters corresponding to the current frame. The calculated global spatial parameters also include the asymmetric perspective distortion parameters corresponding to the current frame, providing a benchmark for subsequent image correction and enhancement processing.

[0048] S2. Based on the pre-calibrated eccentric installation parameters of the integrated imaging module and the lens optical parameters, combined with real-time attitude data, the global spatial parameters corresponding to the current frame are calculated. The global spatial parameters include the field distance distribution map, which characterizes the distance distribution between the imaging target and the integrated imaging module at different positions in the field of view; the asymmetric illumination attenuation distribution model, which characterizes the non-uniform illumination attenuation distribution caused by the eccentric structure in the field of view; and the mapping matrix between the imaging coordinate system and the suction channel feed coordinate system. The mapping matrix is ​​used to characterize the corresponding transformation relationship between the imaging coordinates and the instrument feed coordinates in the suction channel.

[0049] In some implementations, the eccentric mounting parameters include the eccentricity of the integrated imaging module relative to the imaging guide axis and the mounting angle, and the lens optical parameters include the lens distortion coefficient and the relative position parameters of the light source and the integrated imaging module.

[0050] The pre-calibrated fixed parameters are calibrated and solidified after the visual bronchoscope hardware assembly is completed. These parameters are divided into two categories: eccentric mounting parameters of the integrated imaging module and lens optical parameters, serving as fixed benchmarks for global spatial parameter calculations. Eccentric mounting parameters include the eccentricity and installation angle of the integrated imaging module relative to the imaging catheter axis. This records the fixed installation position and angle of the integrated imaging module at the distal end of the imaging catheter, clarifying the fixed relative positional relationship between the imaging module, the imaging catheter axis, and the suction channel, which will not change with the usage status of the imaging catheter. Lens optical parameters include the lens distortion coefficient and the relative position parameters between the light source and the integrated imaging module. This records the inherent imaging characteristics of the imaging module's supporting optical components, as well as the fixed relative position between the illumination source and the imaging acquisition end, providing a fixed optical benchmark for subsequent illumination and distortion correction.

[0051] After acquiring the real-time attitude data corresponding to the current frame, the global spatial parameters corresponding to the current frame are calculated by combining the pre-calibrated eccentric installation parameters and lens optical parameters. The calculation process uses fixed pre-calibration parameters as the basic benchmark and synchronously acquired real-time attitude data as the correction basis, so that the calculated global spatial parameters match the position, shape and imaging state of the imaging duct at the time of acquisition of the current frame, avoiding the inability of fixed parameters to adapt to the constantly changing imaging conditions during bronchial examination.

[0052] The calculated global spatial parameters include a field-of-view distance distribution map, an asymmetric illumination attenuation distribution model, and a mapping matrix between the imaging coordinate system and the suction channel feed coordinate system. The field-of-view distance distribution map shows the distance distribution between the imaging target and the integrated imaging module at different locations within the current imaging field of view, clearly distinguishing the imaging distance differences in different regions of the field of view and providing a spatial distribution basis for subsequent regional image processing. The asymmetric illumination attenuation distribution model fully presents the brightness distribution differences within the current imaging field of view caused by the eccentrically mounted light source and imaging module, clarifying the illumination attenuation in different regions of the field of view and providing a correction basis for subsequent illumination homogenization processing. The mapping matrix between the imaging coordinate system and the suction channel feed coordinate system clarifies the corresponding transformation relationship between the image coordinates acquired by the integrated imaging module and the feed coordinates of the operating instruments within the suction channel, providing a benchmark for subsequent view transformation and clinical operation matching.

[0053] S3. Based on the global spatial parameters, perform asymmetric illumination compensation, two-order geometric correction, and regional color restoration sequentially on the original image data of the current frame to obtain a standardized corrected image.

[0054] In some implementations, the specific method of asymmetric illumination compensation is as follows:

[0055] Based on the asymmetric illumination attenuation distribution model, the brightness attenuation region and the brightness excess region of the original image data of the current frame are determined. The brightness attenuation region is subjected to brightness enhancement processing of corresponding magnitude, and the brightness excess region is subjected to brightness suppression processing of corresponding magnitude, so as to obtain an illumination homogenized image.

[0056] In some implementations, the order of execution for the two-order geometric correction is as follows:

[0057] Based on the asymmetric perspective distortion parameters, the eccentric perspective distortion correction is performed on the illumination homogenized image after asymmetric illumination compensation.

[0058] Based on the lens optical parameters, the illumination homogenized image after completing the off-center perspective distortion correction is subjected to lens inherent distortion correction to obtain a corrected image with true anatomical proportions.

[0059] In some implementations, the specific method for color reproduction by region is as follows:

[0060] Based on the field-of-view object distance distribution map, the corrected image after completing the two-order geometric correction is divided into multiple regions corresponding to different object distances. White balance adjustment and color space conversion with corresponding gain are performed on different regions to obtain a standardized corrected image with consistent color reproduction.

[0061] After calculating the global spatial parameters for the current frame, the original image data of the current frame is sequentially processed using these parameters, including asymmetric illumination compensation, two-order geometric correction, and regional color restoration, to obtain a standardized corrected image. This three-step process improves the imaging defects of the eccentric integrated imaging module. First, it corrects the illumination distribution deviation; then, it corrects the spatial geometric deformation; and finally, it unifies the color restoration effect across the entire field of view. This avoids the accumulation of errors from different correction stages, providing a standardized and defect-free image base for subsequent image enhancement processing.

[0062] Asymmetric illumination compensation is used to improve the structural defects of the integrated imaging module and the illumination source being off-center and non-coaxial. In conventional coaxially arranged endoscopes, illumination attenuation is symmetrically distributed from the center of the field of view outwards, which can be corrected using a general symmetric model. However, off-center arrangements result in unilateral non-uniform illumination attenuation in the field of view, exhibiting a distribution characteristic of higher brightness in the near-module area and lower brightness in the far-module area, and may even be accompanied by local occlusion shadows caused by the duct structure. The general correction model cannot adapt to this type of asymmetric brightness distribution. In the processing, based on the solved asymmetric illumination attenuation distribution model, the brightness distribution of the entire field of view of the current frame's original image data is first identified to determine the brightness attenuation area and the brightness attenuation area of ​​the current frame's original image data. Then, the brightness attenuation area is subjected to a corresponding brightness enhancement process, and the brightness attenuation area is subjected to a corresponding brightness suppression process, ultimately obtaining an illuminated and uniform image. This processing ensures consistent brightness in bronchial tissue imaging across the entire field of view, preventing the loss of tissue details caused by excessively high or low local brightness. It also provides a uniformly bright image basis for subsequent geometric distortion correction, reducing errors in deformation recognition and correction.

[0063] Two-order geometric correction is used to improve the dynamic imaging distortion caused by the off-center integrated imaging module. Conventional central coaxial endoscopes only need to correct the inherent distortion of the optical lens itself. However, the off-center integrated imaging module will produce asymmetric perspective distortion that changes in real time with the state of the imaging duct as it bends and deepens in the bronchus. This type of distortion cannot be eliminated by a general correction model with fixed parameters. The correction process is performed in a fixed sequence. First, based on the calculated asymmetric perspective distortion parameters, the image with asymmetric illumination compensation is corrected for off-center perspective distortion to correct the perspective distortion caused by the bending of the imaging duct and the angle of view of the integrated imaging module. Then, based on the pre-calibrated lens optical parameters, the image with asymmetric perspective distortion correction is corrected for the inherent distortion of the lens to correct the imaging distortion caused by the optical lens itself. Finally, a corrected image with accurate proportions of the anatomical structures is obtained. Performing the correction operations in this order can avoid the superposition of correction errors from dynamic perspective distortion and fixed lens distortion, ensuring that the imaging proportions of anatomical structures such as bronchial ducts and mucosal folds match the actual physiological structures. This prevents observation and operational deviations caused by structural deformation and provides an image basis with accurate spatial positioning for subsequent regional color restoration.

[0064] After completing two-order geometric correction and obtaining a corrected image with accurate anatomical proportions, regional motion blur suppression processing can be performed. Due to the eccentric structure of the integrated imaging module's imaging center and the imaging catheter's motion axis not coinciding, there are natural differences in the amplitude of imaging motion in different regions of the field of view when the imaging catheter moves forward, backward, or rotates to adjust the observation position during clinical operations. The degree of motion blur in the far module region is much higher than that in the near module region. Conventional global motion blur suppression processing is prone to problems such as over-processing of the near module region, resulting in artifacts, and insufficient blur suppression in the far module region. During processing, the motion speed and rotation angle data of the imaging catheter acquired in real time, as well as the field of view object distance distribution map, can be combined to perform motion blur suppression processing of corresponding intensities on different regions. The processed image is then sent to the subsequent regional color restoration stage, which can further improve the clarity of the corrected image.

[0065] Regional color restoration is used to improve full-field color deviation in off-center fields of view. While conventional central coaxial endoscopes can achieve color restoration through global white balance adjustment, significant differences in imaging distance and lighting conditions exist between different regions within an off-center field of view, leading to inconsistencies in color temperature and color representation. A globally uniform color adjustment cannot guarantee accurate restoration across the entire field of view. In this process, based on the calculated field-of-view distance distribution map, the image after double-order geometric correction is divided into multiple regions corresponding to different object distances. Then, white balance adjustments and color space conversions with corresponding gains are performed on different regions to maintain a uniform color representation of bronchial tissues across different object distances, ultimately resulting in a standardized corrected image with consistent color reproduction. This processing can correct color deviations within the field of view caused by different imaging distances and lighting conditions, ensuring accurate and uniform color reproduction of bronchial mucosa, blood vessels, and other tissues, avoiding misdiagnosis of lesions due to color deviations, and providing a standardized image foundation for subsequent regional image enhancement processing.

[0066] In addition, after completing the regional color restoration and obtaining standardized corrected images, instrument region recognition and adaptation processing can be performed simultaneously. Due to the non-coaxial, eccentric structure of the integrated imaging module and the suction channel, during interventional procedures such as clinical suctioning and biopsy, instruments extending from the suction channel will enter the imaging field of view at a fixed offset position. The metallic reflection and non-tissue imaging characteristics of such instrument regions can interfere with the subsequent weight calculation of regional enhancement, and may even lead to excessive enhancement of instrument reflection. During processing, the relative position parameters of the pre-calibrated suction channel and the integrated imaging module can be used to predict the range of instrument appearance in the field of view. Then, combined with image features, instrument region recognition is completed. The recognized instrument regions are only subjected to reflection suppression processing and are not included in the subsequent weight calculation of regional enhancement, which can avoid interference from the instrument regions to the enhancement processing, while not affecting the normal enhancement processing of the bronchial tissue region.

[0067] S4. Based on the field-of-view object distance distribution map, the standardized correction image is divided into near module region, intermediate transition region and far module region according to the distance difference between the imaging target and the integrated imaging module, and spectral enhancement, texture detail enhancement and resolution optimization processing with corresponding weights are performed respectively to obtain multiple sets of enhanced images.

[0068] In some implementations, the spectral enhancement is specifically achieved in the following ways:

[0069] The standardized and corrected image, after completing the color restoration of each region, is converted to a color space that matches the hemoglobin absorption spectrum;

[0070] For the near module area, intermediate transition area, and far module area, respectively, the blue and green channel signals are enhanced and the red channel signal is suppressed with corresponding weights;

[0071] The spectral enhancement image of the corresponding region is obtained through nonlinear mapping processing.

[0072] In some implementations, the specific methods for enhancing texture details are as follows:

[0073] The spectrally enhanced image is decomposed into an illumination base layer that reflects changes in illumination and a detail texture layer that contains information about tissue details.

[0074] The illumination base layer of the near module area, intermediate transition area, and far module area is adjusted to the corresponding magnitude, and the detail texture layer of the near module area, intermediate transition area, and far module area is enhanced to the corresponding intensity, thereby increasing the color difference between the lesion tissue and normal tissue in the corresponding area.

[0075] In some implementations, the specific methods of resolution optimization are as follows:

[0076] Noise reduction is performed only on the near module region, 2x super-resolution reconstruction and corresponding intensity noise reduction are performed on the intermediate transition region, and 4x super-resolution reconstruction and corresponding intensity noise reduction are performed on the far module region, resulting in multiple sets of enhanced images with uniform resolution across the entire field of view.

[0077] After obtaining the standardized and corrected image, the field of view is divided based on the calculated object distance distribution map. Due to the structural limitations of the eccentrically mounted integrated imaging module, there are inherent differences in imaging distance, lighting conditions, and detail rendering capabilities at different locations within the field of view. A globally uniform enhancement process may result in local over-enhancement or under-enhancement. Therefore, based on the distance difference between the imaging target and the integrated imaging module, the standardized and corrected image is divided into three continuous regions: a near-module region, an intermediate transition region, and a far-module region. Then, spectral enhancement, texture detail enhancement, and resolution optimization are performed on each of the three regions with corresponding weights, ultimately resulting in multiple sets of enhanced images. The division of the three regions matches the actual imaging state of the current frame, which can adapt to the changes in the field of view when the imaging catheter penetrates to different depths during bronchial examinations, avoiding the problem that fixed region division cannot match dynamic imaging conditions.

[0078] In this process, after defining the field of view and determining the boundaries of the near module region, intermediate transition region, and far module region, a smooth transition processing of the inter-frame region boundaries can be performed before performing the corresponding weighted enhancement processing. Due to the eccentric structure where the field of view axis of the integrated imaging module does not coincide with the rotation axis of the imaging catheter, the field of view will shift eccentrically with the catheter rotation when the doctor rotates the imaging catheter to adjust the observation angle during clinical operation. This causes significant jumps in the region division boundaries between consecutive frames, resulting in frequent switching of enhancement weights for the corresponding regions, leading to screen flicker and discontinuous enhancement effects. During processing, the imaging catheter rotation angle data of the current frame and previous consecutive frames can be combined to perform inter-frame smooth transition processing on the three region division boundaries of the current frame. After matching the region division ranges of consecutive frames, the corresponding weighted enhancement processing can be performed, ensuring the continuity and stability of the enhancement effect and avoiding screen flicker.

[0079] Spectral enhancement processing is used to overcome the hardware limitations of the off-center-mounted integrated imaging module. Due to the spatial constraints of the suction channel, the hardware size of the integrated imaging module is strictly compressed, making it impossible to integrate the dedicated filters and matching light source components required for narrowband imaging. Therefore, the optical effects of narrowband imaging are simulated through digital domain image processing. In the process, the standardized and calibrated image with completed regional color restoration is first converted to a color space that matches the hemoglobin absorption spectrum. Then, based on the imaging characteristics of the three different regions, corresponding weights are applied to enhance the blue and green channel signals and suppress the red channel signal. Finally, nonlinear mapping processing is used to obtain the spectral enhancement image of the corresponding region. The near-module region, with its close imaging distance and clear microvascular structure imaging, employs a lower channel enhancement weight to avoid over-enhancing of vascular structures and causing edge artifacts. The intermediate transition region uses a moderate channel enhancement weight to balance the contrast of vascular structures and the naturalness of the overall image. The far-module region, with its long imaging distance, weak microvascular signals, and low detail recognition, uses a higher channel enhancement weight to deeply enhance the imaging signal of superficial capillaries while suppressing interference signals from deep tissues. This ensures that the microvascular structure is presented uniformly and consistently across the entire field of view, improving the recognition of superficial bronchial mucosal lesions.

[0080] Texture detail enhancement processing is used to improve imaging defects in bronchial clinical examinations. When the imaging catheter bends along the physiological branches of the bronchus, uneven local illumination can occur. Furthermore, the discernibility of mucosal fine structures varies significantly in different regions under an off-center field of view. Therefore, a layered processing approach is used to avoid overexposure in bright areas and loss of detail in dark areas caused by global enhancement. In the processing, the spectrally enhanced image is first decomposed into an illumination basal layer reflecting illumination changes and a detail texture layer containing tissue detail information. Then, brightness adjustment and detail enhancement are performed according to the imaging characteristics of three different regions. Brightness adjustment is performed on the illumination basal layer of the three regions to further balance the illumination effect in different areas. Detail enhancement is performed on the detail texture layer of the three regions with corresponding intensities. Lower enhancement intensity is used in the near module region to avoid over-sharpening of normal mucosal folds, while higher enhancement intensity is used in the far module region to fully restore the fine structural information of the bronchial mucosa. Simultaneously, the color difference between lesion tissue and normal tissue in the corresponding region is increased, making the boundaries of lesion areas clearer, ultimately resulting in an enhanced image with improved texture detail.

[0081] Resolution optimization processing addresses the imaging characteristics of an off-center field of view. Differences in imaging distance across different regions lead to uneven resolution distribution within the field of view. The far module region, due to its long imaging distance, suffers from severe loss of high-frequency details and significant noise interference. Furthermore, bronchial examinations have strict requirements for real-time image output. Therefore, a region-specific differentiated processing approach is adopted, rather than a globally uniform super-resolution reconstruction. During processing, image processing operations of corresponding intensities are performed for the resolution and noise characteristics of the three different regions. For the near module region, only noise reduction is performed to remove random noise generated during imaging, preserving the original sufficient imaging details. For the intermediate transition region, 2x super-resolution reconstruction and corresponding noise reduction are performed to supplement the lost high-frequency details while suppressing accompanying noise signals. For the far module region, 4x super-resolution reconstruction and corresponding noise reduction are performed to deeply restore the details lost due to the long imaging distance, while significantly reducing noise interference. Ultimately, multiple enhanced images with uniform resolution across the entire field of view are obtained, ensuring stable and consistent imaging quality across the entire field of view.

[0082] S5. After mapping multiple sets of enhanced images and standardized correction images to the same anatomical coordinate system to complete feature alignment, perform fusion processing, and then convert them into operation view images according to the mapping matrix. The image coordinates of the operation view images correspond one-to-one with the feed coordinates of the operating instruments in the suction channel.

[0083] In some implementations, the fusion process employs a pyramid fusion method, where the image coordinates of any anatomical position in the converted operational view image correspond one-to-one with the feed coordinates of the operating instruments within the suction channel.

[0084] After obtaining multiple sets of enhanced images, the enhanced images and standardized correction images are preprocessed before fusion, and then the view transformation process is completed to finally obtain images that meet the needs of clinical examination and operation.

[0085] During processing, multiple sets of enhanced images and standardized correction images are first mapped to the same anatomical coordinate system to complete feature alignment. This process eliminates image boundary misalignment caused by regional differential processing, ensuring that the same anatomical structures and tissue features in different enhanced images are in matching spatial positions, avoiding problems such as feature shift and artifacts that affect observation during fusion. After feature alignment, a pyramid fusion method is used to fuse multiple sets of images. This fusion method can extract effective features from different enhanced images, taking into account the presentation of microvascular structures, mucosal texture details, and true tissue colors, avoiding the problem that a single enhancement mode cannot take into account multi-dimensional diagnostic information, and obtaining a fused image with multi-dimensional enhancement effects.

[0086] After fusion processing, based on the mapping matrix between the calculated imaging coordinate system and the suction channel feed coordinate system, the fused image is converted into an operational view image coaxial with the suction channel feed path. In the converted operational view image, the image coordinates of any anatomical position correspond one-to-one with the feed coordinates of the operating instrument within the suction channel. This processing can solve the problem of misalignment between the imaging view and the instrument feed path caused by the eccentric integrated imaging module, ensuring that the lesion location observed by the doctor in the view matches the actual feed position of the operating instrument, avoiding positioning deviations in clinical operation, and improving the accuracy and safety of endobronchial interventional procedures.

[0087] S6. After performing post-processing optimization on the operation view image, it is output to the display end in real time.

[0088] After the conversion of the operation view image is completed, post-processing optimization is performed on the operation view image, and then the processed image is output to the display terminal in real time for doctors to observe and operate in real time during the clinical examination process.

[0089] Specifically, during the post-processing optimization process, dynamic range compression and tone mapping adjustment are first performed on the operational view image. To address the issues of local light source reflection and loss of details in the dark areas of the deep lumen that are prone to occur during bronchial examinations, brightness suppression is performed on the bright areas of the image, and brightness adjustment is performed on the dark areas to preserve details. This ensures that the tissue details in both the bright and dark areas of the image can be fully presented without any local overexposure or underexposure.

[0090] Based on this, edge-aware sharpening processing is performed on the operational view image. Appropriate sharpening enhancement is applied to key observation areas such as bronchial mucosal folds, blood vessel edges, and lesion boundaries to improve the subjective clarity of the image. At the same time, excessive sharpening of flat areas of the image is avoided to prevent the introduction of additional noise interference and ensure the naturalness of the image.

[0091] During the post-processing optimization process, different clinical examination needs can be matched simultaneously. Multiple sets of display parameters corresponding to different observation focuses can be preset, and different presentation modes can be switched according to the actual needs of clinical operation, adapting to different clinical scenarios such as routine screening, key lesion observation, and interventional operation guidance.

[0092] All post-processing optimization operations are synchronized with the front-end image acquisition and processing flow. The processed images are output to the display end in real time at a stable frame rate, ensuring the smoothness of the imaging screen without obvious delays or stutters. It fully adapts to the real-time operation requirements of bronchoscopy clinical examinations, providing doctors with clear, accurate imaging screens that match their operating habits.

[0093] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application.

Claims

1. An image enhancement processing method for visual bronchoscopes, characterized in that, The method is applied to a video bronchoscope, which includes an imaging catheter. An integrated imaging module is offset at the distal end of the imaging catheter, and a suction channel is provided inside the imaging catheter, which is non-coaxially arranged with the integrated imaging module. The method includes the following steps: S1. Obtain the current frame raw image data collected by the integrated imaging module, and the real-time attitude data of the visual bronchoscope. S2. Based on the pre-calibrated eccentric installation parameters and lens optical parameters of the integrated imaging module, and combined with the real-time attitude data, the global spatial parameters corresponding to the current frame are calculated. The global spatial parameters include a field distance distribution map that characterizes the distance distribution between the imaging target and the integrated imaging module at different positions within the field of view, an asymmetric illumination attenuation distribution model that characterizes the non-uniform illumination attenuation distribution caused by the eccentric structure within the field of view, and a mapping matrix between the imaging coordinate system and the suction channel feed coordinate system. The mapping matrix is ​​used to characterize the corresponding transformation relationship between the imaging coordinates and the instrument feed coordinates within the suction channel. S3. Based on the global spatial parameters, perform asymmetric illumination compensation, two-order geometric correction, and regional color restoration sequentially on the original image data of the current frame to obtain a standardized corrected image; S4. Based on the field of view object distance distribution map, the standardized correction image is divided into near module region, intermediate transition region and far module region according to the distance difference between the imaging target and the integrated imaging module, and spectral enhancement, texture detail enhancement and resolution optimization processing with corresponding weights are performed respectively to obtain multiple sets of enhanced images; S5. After mapping multiple sets of enhanced images and standardized correction images to the same anatomical coordinate system to complete feature alignment, perform fusion processing, and then convert them into operation view images according to the mapping matrix. The image coordinates of the operation view images correspond one-to-one with the feed coordinates of the operating instruments in the suction channel. S6. After performing post-processing optimization on the operation view image, it is output to the display terminal in real time.

2. The method according to claim 1, characterized in that, The eccentric mounting parameters include the eccentricity of the integrated imaging module relative to the imaging guide axis and the mounting angle. The lens optical parameters include the lens distortion coefficient and the relative position parameters of the light source and the integrated imaging module.

3. The method according to claim 2, characterized in that, The real-time attitude data includes the real-time curvature and insertion depth of the imaging catheter; The calculated global spatial parameters also include the asymmetric perspective distortion parameters corresponding to the current frame.

4. The method according to claim 3, characterized in that, The specific method of asymmetric illumination compensation is as follows: Based on the asymmetric illumination attenuation distribution model, the brightness attenuation region and the brightness excess region of the original image data of the current frame are determined. The brightness attenuation region is subjected to a corresponding brightness enhancement process, and the brightness excess region is subjected to a corresponding brightness suppression process, resulting in an illumination homogenized image.

5. The method according to claim 4, characterized in that, The execution order of the two-order geometric correction is as follows: Based on the asymmetric perspective distortion parameters, the illumination homogenized image after asymmetric illumination compensation is corrected for off-center perspective distortion. Based on the lens optical parameters, the illumination homogenized image after the off-center perspective distortion correction is subjected to lens inherent distortion correction to obtain a corrected image with true anatomical proportions.

6. The method according to claim 5, characterized in that, The specific method for regional color restoration is as follows: Based on the field-of-view object distance distribution map, the corrected image after the completion of the two-order geometric correction is divided into multiple regions corresponding to different object distances. White balance adjustment and color space conversion with corresponding gain are performed on different regions to obtain the standardized corrected image with consistent color reproduction.

7. The method according to claim 6, characterized in that, The specific method of spectral enhancement is as follows: The standardized corrected image, after completing the color restoration of the sub-regions, is converted to a color space that matches the hemoglobin absorption spectrum; The near module region, the intermediate transition region, and the far module region are respectively subjected to blue and green channel signal enhancement and red channel signal suppression with corresponding weights. The spectral enhancement image of the corresponding region is obtained through nonlinear mapping processing.

8. The method according to claim 7, characterized in that, The specific method for enhancing texture details is as follows: The spectrally enhanced image after the spectral enhancement is completed is decomposed into an illumination base layer that reflects changes in illumination and a detail texture layer that contains information about tissue details. Brightness adjustment of the corresponding magnitude is performed on the illumination base layer of the near module region, the intermediate transition region, and the far module region, and detail enhancement of the corresponding intensity is performed on the detail texture layer of the near module region, the intermediate transition region, and the far module region to expand the color difference between the lesion tissue and the normal tissue in the corresponding region.

9. The method according to claim 8, characterized in that, The specific method for resolution optimization is as follows: Noise reduction is performed only on the near module region, 2x super-resolution reconstruction and corresponding intensity noise reduction are performed on the intermediate transition region, and 4x super-resolution reconstruction and corresponding intensity noise reduction are performed on the far module region, resulting in multiple sets of enhanced images with uniform resolution across the entire field of view.

10. The method according to claim 1, characterized in that, The fusion process employs a pyramid fusion method, where the image coordinates of any anatomical position in the resulting operational view image correspond one-to-one with the feed coordinates of the operating instruments within the suction channel.

Images