A multispectral endoscopic blood oxygen distribution imaging method and system

Abstract

The application discloses a multispectral endoscopic blood oxygen distribution imaging method and system. Narrowband RGB endoscopic images corresponding to multiple narrowband illumination conditions under the same endoscopic field of view are acquired, and channel signals for blood oxygen estimation are extracted respectively; an effective imaging area mask is generated based on quality determination results of at least two of overexposure, underexposure, strong reflection and dark area; the channel signals are corrected within the effective imaging area, and blood oxygen estimation features and reliability parameters are constructed; a preset blood oxygen mapping model is used to obtain candidate blood oxygen saturation estimation values, and the candidate blood oxygen saturation estimation values are weighted corrected, weighted reduced or removed based on the reliability parameters, so that a blood oxygen saturation distribution map and a confidence or low reliability area label are generated. The application can reduce the interference of low-quality imaging areas on blood oxygen estimation, and improve the stability and interpretability of the distribution map.

Description

Technical Field

[0001] This invention relates to the fields of medical endoscopic imaging, multispectral image processing, and physiological parameter imaging technology, and particularly to a multispectral endoscopic blood oxygen distribution imaging method and system. Background Technology

[0002] White light endoscopic images primarily reflect changes in tissue surface morphology and color, making it difficult to directly reflect functional information such as tissue hemoglobin oxygenation status and local hypoxia. Multispectral endoscopic imaging can utilize differences in hemoglobin, oxyhemoglobin, tissue scattering, and tissue absorption under different narrow-band illumination conditions or in different channels to generate images or parametric maps related to tissue blood oxygen saturation, thereby providing functional imaging information for endoscopic observation.

[0003] Existing multispectral or multichannel pulse oximetry schemes typically select multiple channels related to blood oxygenation status and construct blood oxygenation estimation features based on signals from multiple channels. Then, they use a preset model or a calibrated model to estimate blood oxygen saturation. These schemes can generate blood oxygen saturation distribution maps to a certain extent. However, in real endoscopic environments, image signals are also affected by factors such as strong reflections, overexposure, underexposure, dark areas, and inconsistent responses between different channels.

[0004] If all pixels or regions are directly input into the blood oxygen estimation model, low-quality areas may be mistaken for true blood oxygen changes, leading to artifacts or abnormal estimations in the blood oxygen saturation distribution map. Therefore, a blood oxygen saturation distribution imaging scheme suitable for real endoscopic imaging environments is needed. This scheme should identify the effective imaging region before blood oxygen estimation and introduce reliability constraints during the estimation process to improve the stability and interpretability of the blood oxygen distribution map. Summary of the Invention

[0005] The purpose of this invention is to provide a multispectral endoscopic blood oxygen distribution imaging method and system to solve the problem that existing endoscopic blood oxygen imaging methods are easily affected by low-quality imaging areas, resulting in insufficient stability and reliability of blood oxygen saturation distribution maps.

[0006] To achieve the above objectives, this invention provides a multispectral endoscopic blood oxygen distribution imaging method, comprising: acquiring multispectral endoscopic image data under the same endoscopic field of view, and extracting gray values ​​from multiple narrowband RGB endoscopic images as multiple channel signals for blood oxygen estimation; performing image quality determination based on the multispectral endoscopic image data or multiple channel signals, fusing at least two quality determination results to obtain an effective imaging region mask; performing normalization processing or reflectivity correction processing on the channel signals within the effective imaging region; constructing blood oxygen estimation features based on the corrected channel signals, and generating reliability parameters; inputting the blood oxygen estimation features into a preset blood oxygen mapping model to obtain candidate blood oxygen saturation estimates, and performing at least one of weighted correction, weight reduction, or elimination processing on the candidate blood oxygen saturation estimates based on the reliability parameters to obtain blood oxygen saturation estimates and corresponding confidence levels or low reliability region markers; generating and outputting a blood oxygen saturation distribution map, as well as the effective imaging region mask, confidence level, and / or low reliability region markers.

[0007] The present invention also provides a multispectral endoscopic blood oxygen distribution imaging system, including an image acquisition module, an effective region mask generation module, a preprocessing module, a blood oxygen feature and reliability construction module, a blood oxygen estimation module, and a distribution map output module.

[0008] Compared with the prior art, the present invention has at least the following beneficial effects: First, by generating an effective imaging region mask through multi-factor quality judgment before blood oxygen estimation, the influence of overexposure, underexposure, strong reflection, and dark areas on the blood oxygen estimation results can be reduced; Second, by using reliability parameters for weighting, downweighting, elimination, confidence output, or low-reliability region marking of blood oxygen estimation results, the blood oxygen saturation distribution map not only provides the estimated value but also reflects the reliability of the results in each region; Third, low-reliability regions can be eliminated, downweighted, or distinguished, which helps to reduce the risk of artifacts in endoscopic blood oxygen distribution imaging. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart of the blood oxygen saturation distribution imaging method provided in an embodiment of the present invention.

[0010] Figure 2 This is a schematic diagram of the blood oxygen saturation distribution imaging system provided in an embodiment of the present invention.

[0011] Figure 3 This is a schematic diagram illustrating the generation of the effective imaging region mask and reliability parameters provided in an embodiment of the present invention. Detailed Implementation

[0012] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Where there is no conflict, the technical features in the following embodiments can be combined with each other.

[0013] Example 1: Overall Process like Figure 1 As shown, this embodiment provides a multispectral endoscopic blood oxygen distribution imaging method. This method takes multispectral endoscopic image data under the same endoscopic field of view as input. First, it extracts grayscale values ​​from multiple narrowband RGB endoscopic images as multiple channel signals for blood oxygen estimation. Then, it generates an effective imaging region mask through image quality judgment, and performs signal correction, feature construction, reliability parameter generation, candidate blood oxygen estimation, reliability control, and distribution map output within the effective imaging region.

[0014] The multispectral endoscopic image data in this embodiment includes multiple narrowband RGB endoscopic images acquired under multiple narrowband illumination conditions. The channel signals used for blood oxygen estimation are extracted from the multiple narrowband RGB endoscopic images, specifically, they can be grayscale values ​​after grayscale processing, or they can be grayscale values ​​of the target color channel.

[0015] In this embodiment, the original RGB white light endoscopic image and the narrowband endoscopic images acquired under two narrowband illumination modes can be used as conventional display images or reference images for comparison with the effects of this embodiment, and are not necessary inputs for blood oxygen estimation. Blood oxygen estimation mainly uses multispectral endoscopic images acquired under multiple narrowband illumination conditions, and extracts grayscale values ​​from each narrowband RGB endoscopic image; multiple grayscale value channels are combined to form multiple channel signals for blood oxygen estimation.

[0016] Example 2: Image Data Source and Channel Signal Extraction In one implementation, the multispectral endoscopic image data originates from multiple narrowband RGB endoscopic images acquired under multiple narrowband illumination conditions. Each narrowband RGB endoscopic image can correspond to a narrowband illumination condition. During processing, grayscale values ​​or target color channel grayscale values ​​are extracted from each narrowband RGB endoscopic image as channel signals under the corresponding narrowband illumination condition.

[0017] Multiple narrowband RGB endoscopic images can correspond to multiple different narrowband responses. By converting each narrowband RGB endoscopic image to grayscale, or by selecting a target color channel from its R, G, and B color channels and extracting the grayscale value of the target color channel, multiple channel signals for blood oxygen estimation can be obtained.

[0018] In another implementation, the original RGB white light endoscopic image and the narrow-band endoscopic images acquired under two narrow-band illumination conditions are used only for display and comparison in the conventional white light mode or narrow-band mode, and are not used as necessary inputs for blood oxygen estimation. This avoids confusion between the conventional white light display mode or the two narrow-band display modes and the multispectral acquisition mode used for blood oxygen estimation in this embodiment.

[0019] Optionally, the multiple channel signals used for blood oxygen estimation can form 4 or 5 grayscale channels, or other numbers of grayscale channels can be formed depending on the actual multispectral endoscopic acquisition scheme. Different grayscale channels can correspond to blood oxygen sensitivity response, reference response, brightness response, or scattering response, respectively.

[0020] In this embodiment, no specific narrowband center wavelength or band range is limited; as long as multiple grayscale value channels for blood oxygen estimation can be extracted from multiple narrowband RGB endoscopic images, they can be used as multispectral endoscopic image data in this embodiment.

[0021] Optionally, the multiple channel signals used for blood oxygen estimation include at least two of the following: blood oxygen sensitive grayscale signal, reference response grayscale signal, brightness-related grayscale signal, and scattering-related grayscale signal.

[0022] Example 3: Effective Imaging Area Mask In this embodiment, before performing blood oxygenation estimation, image quality is first assessed based on multi-channel endoscopic image data or multiple channel signals used for blood oxygenation estimation, generating an effective imaging region mask. The effective imaging region mask is used to identify pixel locations or image regions suitable for blood oxygenation estimation, thereby preventing low-quality regions from directly participating in blood oxygenation estimation.

[0023] Specifically, at least two of the following can be generated: overexposure determination result, underexposure determination result, strong reflection determination result, and dark area determination result. Among them, the overexposure determination result is used to identify pixels whose grayscale value exceeds the overexposure threshold; the underexposure determination result is used to identify pixels whose grayscale value is below the underexposure threshold; the strong reflection determination result is used to identify pixels with strong reflection caused by specular reflection or local highlights; and the dark area determination result is used to identify pixels with insufficient lighting, shadows, or low channel response.

[0024] The aforementioned quality assessment results can be fused using logical operations, weighted scoring, thresholding, table lookup, or model-based methods to obtain candidate effective regions. Then, at least one of the following processes can be applied to the candidate effective regions: morphological denoising, hole filling, edge smoothing, and connected component filtering, to obtain an effective imaging region mask.

[0025] In one specific implementation, the effective imaging region mask is actually generated based on at least two of the following factors: overexposure, underexposure, strong reflection, and dark areas. By first identifying candidate invalid regions and then inverting them to obtain candidate effective regions, it is possible to reduce the number of pixels or regions that are clearly unsuitable for blood oxygen estimation from entering the subsequent blood oxygen mapping model.

[0026] Example 4: Construction of Channel Signal Correction and Blood Oxygen Estimation Features After obtaining the effective imaging region mask, within the effective imaging region corresponding to the mask, multiple channel signals used for blood oxygen estimation are normalized or reflectance corrected to obtain multiple corrected channel signals. This processing may include at least one of dark field correction, whiteboard correction, exposure normalization, inter-channel registration, brightness normalization, reference channel normalization, and local background correction.

[0027] For areas outside the effective imaging area mask, blood oxygen saturation estimates may not be calculated, or these areas may be marked as low-reliability regions. During display output, these areas can be distinguished from the effective imaging area through methods such as transparent display, grayscale display, mask occlusion, boundary marking, or confidence parameter annotation.

[0028] Blood oxygen estimation features can be constructed based on multiple calibration channel signals. In one embodiment, the normalized values ​​of multiple calibration channel signals corresponding to the same pixel location or the same image region are obtained respectively, and combined according to a preset channel order to form blood oxygen estimation features. These blood oxygen estimation features simultaneously reflect the tissue optical response under different narrowband responses and serve as input to a preset blood oxygen mapping model.

[0029] The normalized value can be determined based on the effective dynamic range of each channel, the whiteboard response, the reference channel response, or the local background response. The normalized values ​​of different channels correspond to each other at the same spatial location, and the specific normalization reference and channel combination order are determined based on the channel configuration and calibration data of the multispectral endoscopic acquisition system.

[0030] Example 5: Reliability parameters and their role in blood oxygen estimation In this embodiment, reliability parameters are generated based on an effective imaging area mask and multiple correction channel signals. These reliability parameters represent the confidence level of blood oxygenation estimation at corresponding pixel locations or image regions. The reliability parameters can be generated based on at least two of the following: exposure reliability components, strong reflection reliability components, and dark area reliability components.

[0031] For example, the reliability parameters can be obtained by weighted summation of the reliability components; alternatively, they can be obtained by weighted multiplication, minimum value taking, piecewise mapping, rule fusion, or table lookup fusion of the reliability components. The reliability parameters can be continuous or discrete levels.

[0032] In practical applications, reliability parameters are not limited to a single fixed formula, but can be generated using a variety of optional fusion methods based on image quality judgment results such as overexposure, underexposure, strong reflection, and dark areas.

[0033] The pre-defined blood oxygen mapping model first obtains candidate blood oxygen saturation estimates based on blood oxygen estimation features. Reliability parameters are used to perform at least one of the following processing on the candidate blood oxygen saturation estimates: weighted correction, weight reduction, or elimination, and generate a confidence level or low-reliability region marker corresponding to the processed blood oxygen saturation estimates. In this way, the reliability parameters participate in the confidence control of the blood oxygen estimation results, rather than being used only in the display stage.

[0034] During in vitro testing system validation, blood gas analyzer results can be used as the true values. For example, reference data can be obtained using an i-STAT blood gas analyzer, and the relative error of the blood oxygen saturation estimate can be calculated. This relative error can be used to verify the model's accuracy and can also be used during testing to extract and distinguish regions with different calculation accuracy.

[0035] Example 6: Establishment or Update of a Preset Blood Oxygen Mapping Model The preset blood oxygen mapping model can be a lookup table model, a linear regression model, a multinomial regression model, a least squares fitting model, a machine learning model, or a model combining a physical model and a data-driven model. The model input includes blood oxygen estimation features, and the model output includes candidate blood oxygen saturation estimates. After the candidate blood oxygen saturation estimates are processed by reliability parameters, the blood oxygen saturation estimates and corresponding confidence levels or low reliability warning information are obtained.

[0036] When establishing or updating a preset blood oxygen mapping model, calibration data can be used. The calibration data originates from in vitro phantom data and blood gas analyzer reference data, where the blood gas analyzer can be an i-STAT blood gas analyzer. The blood gas analyzer reference data is used for data acquisition and alignment of the model training set to determine the mapping relationship between blood oxygen estimation features and reference blood oxygen saturation.

[0037] When using calibration data to build or update a preset blood oxygen mapping model, only sample pixels or sample regions corresponding to the effective imaging area mask are used. Therefore, low-quality areas such as overexposed, underexposed, strongly reflective, and dark areas will not be used as valid training samples for determining model parameters, thus improving the stability of the model building or updating process.

[0038] Example 7: Blood Oxygen Saturation Distribution Map and Confidence Output A blood oxygen saturation distribution map is generated based on the estimated blood oxygen saturation values. The blood oxygen saturation distribution map can be a pixel-level distribution map or a region-level distribution map. Optionally, the estimated blood oxygen saturation values ​​can be mapped to pseudo-color and overlaid on a white light endoscopic image or other endoscopic images.

[0039] When outputting the blood oxygen saturation distribution map, an effective imaging area mask, a confidence map, an error indication map, or a low-reliability area marker map can be output simultaneously. For low-reliability areas, at least one of the following methods can be used to distinguish the output: transparent display, grayscale display, mask occlusion, boundary marking, or confidence parameter annotation.

[0040] In one verification method, the visual comparison of the blood oxygen saturation distribution before and after improvement can be used to observe the reduction in interference from strong reflections, overexposure, underexposure, or dark areas on the blood oxygen map after adding an effective imaging area mask and reliability parameters.

[0041] Example 8: System Structure like Figure 2 As shown in the figure, this embodiment also provides a multispectral endoscopic blood oxygen distribution imaging system. The system includes an image acquisition module, an effective region mask generation module, a preprocessing module, a blood oxygen feature and reliability construction module, a blood oxygen estimation module, and a distribution map output module.

[0042] The image acquisition module acquires multispectral endoscopic image data under the same endoscopic field of view and extracts grayscale values ​​from multiple narrowband RGB endoscopic images as multiple channel signals for blood oxygen estimation. The effective region mask generation module performs image quality assessment and generates an effective imaging region mask. The preprocessing module normalizes or corrects reflectance of the channel signals within the effective imaging region. The blood oxygen feature and reliability construction module constructs blood oxygen estimation features and generates reliability parameters. The blood oxygen estimation module obtains candidate blood oxygen saturation estimates based on the blood oxygen estimation features and performs at least one of the following processing methods—weighting correction, weight reduction, or elimination—on the candidate blood oxygen saturation estimates based on the reliability parameters to obtain the blood oxygen saturation estimate and corresponding confidence level or low-reliability region marker. The distribution map output module generates and outputs a blood oxygen saturation distribution map, the effective imaging region mask, confidence level, and / or low-reliability region marker.

[0043] like Figure 3 As shown, the effective imaging area mask and reliability parameters can be obtained by fusing multiple image quality judgment results. Therefore, this embodiment can shift the impact of low-quality areas such as overexposure, underexposure, strong reflection, and dark areas on blood oxygen estimation to the forefront in a real endoscopic environment, and output the blood oxygen estimation results and reliability information simultaneously.

Claims

1. A multispectral endoscopic blood oxygen distribution imaging method, characterized in that, include: S1, acquire multispectral endoscopic image data under the same endoscopic field of view, and extract multiple channel signals for blood oxygen estimation from the multispectral endoscopic image data; wherein, the multispectral endoscopic image data includes multiple narrowband RGB endoscopic images acquired under multiple narrowband illumination conditions, and the multiple channel signals for blood oxygen estimation are gray values ​​extracted from the multiple narrowband RGB endoscopic images respectively. S2, based on the multispectral endoscopic image data or the multiple channel signals used for blood oxygen estimation, image quality is determined, at least two quality determination results are generated respectively, and the at least two quality determination results are fused to obtain an effective imaging area mask; wherein, the at least two quality determination results include at least two of the following: overexposure determination result, underexposure determination result, strong reflection determination result, and dark area determination result; S3, within the effective imaging area corresponding to the effective imaging area mask, the multiple channel signals used for blood oxygen estimation are normalized or corrected for reflectivity to obtain multiple corrected channel signals. S4, construct blood oxygen estimation features based on the multiple correction channel signals, and generate reliability parameters based on the effective imaging area mask and / or the at least two quality judgment results; S5, input the blood oxygen estimation features into a preset blood oxygen mapping model to obtain candidate blood oxygen saturation estimates within the effective imaging area; perform at least one of the following processing on the candidate blood oxygen saturation estimates based on the reliability parameters: weighted correction, weight reduction, or elimination, to obtain blood oxygen saturation estimates and corresponding confidence levels or low-reliability region markers; S6. Generate a blood oxygen saturation distribution map based on the blood oxygen saturation estimate, and output the blood oxygen saturation distribution map, the effective imaging area mask, the confidence level, and / or the low reliability area marker.

2. The method according to claim 1, characterized in that, The multiple narrowband RGB endoscopic images correspond to different narrowband illumination conditions; Extracting the grayscale value from each narrowband RGB endoscopic image includes: performing grayscale processing on the narrowband RGB endoscopic image, or extracting the grayscale value of the target color channel from the target color channel of the narrowband RGB endoscopic image.

3. The method according to claim 1 or 2, characterized in that, The plurality of channel signals used for blood oxygen estimation include at least two of the following: blood oxygen sensitive grayscale signal, reference response grayscale signal, brightness-related grayscale signal, and scattering-related grayscale signal.

4. The method according to claim 1, characterized in that, Generating the effective imaging region mask includes: determining candidate invalid regions based on at least two of overexposed pixels, underexposed pixels, strongly reflective pixels, and dark area pixels; inverting the candidate invalid regions to obtain candidate effective regions; and performing at least one of morphological denoising, hole filling, edge smoothing, and connected component filtering on the candidate effective regions to obtain the effective imaging region mask.

5. The method according to claim 1, characterized in that, The normalization or reflectivity correction process includes at least one of dark field correction, whiteboard correction, exposure normalization, inter-channel registration, brightness normalization, reference channel normalization, and local background correction; and for areas outside the effective imaging area mask, the blood oxygen saturation estimate is not calculated, or these areas are marked as low reliability areas.

6. The method according to claim 1, characterized in that, The blood oxygen estimation feature is formed by combining the normalized values ​​of at least two of the corrected channel signals at the same pixel location or the same image region in a preset channel order.

7. The method according to claim 1, characterized in that, The reliability parameters are generated based on at least two of the exposure reliability components, strong reflection reliability components, and dark area reliability components, through at least one of the following methods: weighted summation, weighted product, minimum value, segmented mapping, rule fusion, or table lookup fusion. The reliability parameters are used to determine the weighted correction coefficient, weight reduction magnitude, or elimination condition of the candidate blood oxygen saturation estimates, and are used to generate the confidence level or low reliability region marker.

8. The method according to claim 1, characterized in that, The preset blood oxygen mapping model is established or updated using calibration data; the calibration data comes from in vitro phantom data and blood gas analyzer reference data; when establishing or updating the preset blood oxygen mapping model, only the sample pixels or sample regions corresponding to the effective imaging area mask are used to determine the model parameters of the preset blood oxygen mapping model.

9. A multispectral endoscopic blood oxygenation distribution imaging system, characterized in that, include: An image acquisition module is used to acquire multispectral endoscopic image data under the same endoscopic field of view, and extract multiple channel signals for blood oxygen estimation from the multispectral endoscopic image data; wherein, the multispectral endoscopic image data includes multiple narrowband RGB endoscopic images acquired under multiple narrowband illumination conditions, and the multiple channel signals for blood oxygen estimation are gray values ​​extracted from the multiple narrowband RGB endoscopic images respectively; An effective region mask generation module is used to determine the image quality based on the multispectral endoscopic image data or the multiple channel signals used for blood oxygen estimation, generate at least two quality determination results respectively, and fuse the at least two quality determination results to obtain an effective imaging region mask; wherein, the at least two quality determination results include at least two of the following: overexposure determination result, underexposure determination result, strong reflection determination result, and dark area determination result; The preprocessing module is used to perform normalization processing or reflectivity correction processing on the multiple channel signals used for blood oxygen estimation within the effective imaging area corresponding to the effective imaging area mask, so as to obtain multiple corrected channel signals. The blood oxygenation feature and reliability construction module is used to construct blood oxygenation estimation features based on the multiple correction channel signals, and generate reliability parameters based on the effective imaging area mask and / or the at least two quality judgment results; The blood oxygen estimation module is used to input the blood oxygen estimation features into a preset blood oxygen mapping model to obtain candidate blood oxygen saturation estimates within the effective imaging area, and to perform at least one of the following processing on the candidate blood oxygen saturation estimates based on the reliability parameters: weighted correction, weight reduction, or elimination, to obtain blood oxygen saturation estimates and corresponding confidence levels or low reliability region markers. The distribution map output module is used to generate a blood oxygen saturation distribution map based on the blood oxygen saturation estimate, and output the blood oxygen saturation distribution map, the effective imaging area mask, the confidence level, and / or the low reliability area marker.

10. The system according to claim 9, characterized in that, The image acquisition module is used to extract grayscale values ​​or target color channel grayscale values ​​from each narrowband RGB endoscopic image; the distribution map output module is used to distinguish and display low reliability areas in the blood oxygen saturation distribution map by at least one of the following methods: transparent display, grayscale display, mask occlusion, boundary marking, or confidence parameter annotation.

Images