Non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system and imaging method based on artificial intelligence

Abstract

The non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system and imaging method based on artificial intelligence adopt the absorption imaging brightening system of oxygenated hemoglobin and the two-dimensional real-time imaging system to realize a non-invasive, real-time imaging, high-precision reflective imaging blood oxygen meter that can eliminate external environmental light interference. This method not only solves the problem of limited monitoring range of contact oximeters, but also makes up for the defects of low precision and susceptibility to interference from external light sources of non-contact reflective blood oxygen imagers. And narrow-band filters are only required during the training process. Once the artificial intelligence model training is completed, blood oxygen imaging can be achieved only with the camera and the artificial intelligence model. This technical system can be extended to scenes with more light source interference such as laparoscopic surgery, and the blood flow state of important organs can be monitored in real time. It can be used for non-invasive vascular imaging, and can also monitor microcirculation congestion shock and intraoperative organ ischemia and infarction status assessment. Moreover, this patented technology can also be carried on mobile phones to realize home health monitoring. It further improves the scope of application of the hemoglobin measurement system, and is of great significance to the real-time evaluation of human blood flow status, clinical surgery and home health monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of biological image processing, and in particular to a non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system and an imaging method based on artificial intelligence. Background Art

[0002] Blood oxygen content is a commonly used physiological parameter in clinical practice, and is often used to assess the degree of hypoxia and ischemia of limb organs. With the development of electronic technology, blood oxygen content detection is no longer limited to invasive blood drawing tests. Portable oximeters based on the "non-invasive" concept are gradually used for daily blood oxygen detection to assess personal physiological conditions. At present, transmission oximeters and reflective oximeters are relatively mature and have been widely used in clinical practice.

[0003] Traditional invasive blood oxygen detection methods require arterial puncture, which is painful and complicated, time-consuming to analyze, and cannot detect blood oxygen saturation in real time. Although such methods have high accuracy, invasive detection may cause complications and cannot be detected in real time and continuously, which limits its clinical application.

[0004] Although the transmissive oximeter successfully avoids trauma and realizes real-time detection, it is limited by the measurement site, the measurement range is small, and the error is large under hypoxia. It cannot measure the blood oxygen saturation of larger or thicker local tissues or organs, such as the forehead, abdomen, cerebral blood vessels, muscle blood vessels, etc. It can only be tested at the end of the limbs where the incident light is easy to transmit, such as fingers or earlobes. At the same time, the transmissive oximeter is easily restricted by the movement of human limbs, the system consumes a lot of energy, and cannot be used for real-time detection of a large range of people.

[0005] Although the reflective oximeter also achieves non-invasive and real-time detection. However, the contact reflective oximeter is limited to the flat parts of the human skin, such as the forehead, wrist, etc., and the error is large under hypoxia. Although the non-contact reflective oximeter has solved the physical limitations between the patient and the instrument to a certain extent and expanded the detection range. However, the non-contact reflective oximeter is still not fully mature. The reflective blood oxygen probe can only receive reflected light, and most of the reflected light carrying physiological signals is easily interfered by external light, motion artifacts, power frequency noise, etc. The signal amplitude is small and the fluctuation is large, and the actual received useful signal is very weak. Therefore, the non-contact reflective blood oxygen detection method has low accuracy and poor real-time performance. It cannot perform effective blood oxygen detection under the influence of external light and other interference factors. The technical method needs to be further improved.

[0006] In addition, most existing reflective blood oxygen detection systems need to shield ambient light interference when in use, and require the use of special light sources, such as red light or near-infrared light. This means that in laparoscopic surgery, the existing blood oxygen imaging system needs to turn off the surgical light and turn on red light or near-infrared light imaging. For the surgeon, the surgical process must be suspended. This type of blood oxygen imaging system not only increases the complexity of the operation, but also prolongs the operation time and increases unpredictable risks.

[0007] In addition, in laparoscopic surgery, indocyanine green contrast agent is currently a commonly used vascular contrast agent for near-infrared fluorescence imaging, but this type of contrast agent is relatively toxic and has large side effects, which can easily cause patients to experience allergies, renal failure, and even shock symptoms. The invasiveness and large side effects of vascular contrast agents limit their clinical practice. At the same time, the lighting conditions of laparoscopic surgery also greatly limit the application of reflective oximeters.

[0008] Therefore, it is of far-reaching significance to develop a non-invasive, real-time imaging, high-precision non-contact reflective oximeter that can eliminate interference from external ambient light. Summary of the invention

[0009] In order to solve the technical problem of how to perform non-invasive, real-time, and accurate blood oxygen imaging of the human body without interference from external light sources, the present invention provides a non-invasive, real-time, contactless, and interference-resistant blood oxygen real-time imaging system and imaging method based on artificial intelligence, which can accurately locate oxygenated hemoglobin, distinguish blood vessels from surrounding background tissues, or be used to determine the degree of ischemia and hypoxia of tissues.

[0010] The technical solution adopted by the present invention is: a non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system based on artificial intelligence, including the following modules:

[0011] Light source emission and image acquisition module: emits light of a specific intensity and a specific wavelength, and respectively acquires light of a specific wavelength related to oxygenated hemoglobin and a background light image that does not contain any absorption peak band of oxygenated hemoglobin through the filtering function of the image acquisition element;

[0012] Absorbance image processing and imaging module:

[0013] Connected to the light source emission and image acquisition module, the hemoglobin absorbance image and the background light image are processed based on artificial intelligence to obtain the imaging of the spatial distribution of oxygenated hemoglobin;

[0014] Image output module: The obtained absorbance spatial distribution image is brightened by computer processing and overlaid on the image output of the laparoscope in real time in the form of a heat map or pseudo-color.

[0015] The light source emission and image acquisition module includes a 660nm narrowband filter with a bandwidth of 10nm and a CCD or CMOS element without a Bayer array. The 660nm narrowband filter with a bandwidth of 10nm filters a specific wavelength, and obtains an image with a high hemoglobin absorption degree and a background light image at a specific wavelength through the CCD or CMOS element without a Bayer array.

[0016] The light source emission and image acquisition module includes a CCD or CMOS element including a Bayer array. According to the mapping relationship between the constructed full-color RGB pixel value and its absorbance image at 660nm, the oxygenated hemoglobin absorbance image and the background light image are obtained using the same CCD or CMOS element including the Bayer array.

[0017] A non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging method comprises the following steps:

[0018] (1) Mapping construction of specific light wavelengths and RGB pixel values ​​under normal lighting conditions: The RGB pixel values ​​presented by the CCD / CMOS are presented under the illumination of a light source containing an absorption peak to obtain a full-color image. At the same time, the RGB pixel values ​​presented by the CCD / CMOS under 660nm and 600nm wavelength light are measured to obtain an image with a high degree of oxygenated hemoglobin absorption. The image is input into a computer to construct an artificial intelligence model, and the mapping relationship from the full-color image to the 660nm and 600nm wavelength images is obtained respectively. In this process, the artificial intelligence model is essentially a software-implemented filter, which is a supervised image-to-image mapping model;

[0019] (2) Obtaining the oxygenated hemoglobin absorption peak and background light image according to the mapping relationship during use: Under the same light source used when constructing the model, according to the mapping relationship between the full-color RGB pixel value and its absorbance image at 660nm constructed in advance, the oxygenated hemoglobin absorbance image is obtained using the same CCD or CMOS element, and under the same light source used when constructing the model, according to the mapping relationship between the full-color RGB pixel value and its absorbance image at 600nm constructed in advance, the background light image is obtained using the same CCD or CMOS element to obtain the surrounding tissue background image;

[0020] (3) Compare the absorbance spatial distribution image by computer: The absorbance spatial distribution image of oxygenated hemoglobin and the pixel values ​​of the background light image are calculated according to the Lambert-Beer law. The Lambert-Beer law refers to calculating the difference in absorption of oxygenated hemoglobin (HbO2) and reduced hemoglobin (Hb) to light of different wavelengths, collecting images of multiple wavelengths, and calculating the blood oxygen value through image processing. The calculation of blood oxygen content is directly related to the optical density value (OD), and the optical density value OD is defined as: OD = log (I0 / I), OD represents the attenuation of the outgoing light intensity I relative to the incident light intensity I0.

[0021] In the optical density calculation, the outgoing light intensity I is the pixel value of the corresponding channel, and the incident light intensity I0 is the incident light intensity of the corresponding color wavelength range. The blood oxygen saturation and the optical density ratio (ODR) of blood vessels at different wavelengths are approximately negatively correlated linearly, as shown in the formula.

[0022]

[0023] In the above formula, SatO2 represents blood oxygen saturation, ODR represents the optical density ratio of blood vessels at different wavelengths, I0|660nm is the incident light intensity at 660nm spectral range, and I0|600nm is the incident light intensity at 600nm spectral range. The incident light intensity can be obtained by measuring the light source spectrum in advance. 660 nm is the outgoing light absorbed by blood vessels in the 660nm spectral range, I 600 nm is the output light in the 600nm spectral range, a and b are the parameters used for linear fitting of blood oxygen value and ODR, which can be measured through experimental calibration.

[0024] (4) Image output: The obtained blood oxygen saturation spatial distribution image is brightened by computer processing and overlaid on the laparoscope image output in real time in the form of a heat map or pseudo-color.

[0025] The beneficial effects of the present invention are as follows: the present invention provides a non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system and imaging method based on artificial intelligence, which adopts an oxygenated hemoglobin light absorption imaging brightening system and a two-dimensional real-time imaging system, and realizes a non-invasive, real-time imaging, high-precision non-contact reflective oximeter that can eliminate external environmental light interference. This method not only solves the problem of limited monitoring range of contact oximeters, but also makes up for the defects of low accuracy and susceptibility to interference from external light sources of non-contact reflective oximeters. The technical system can be extended to scenes with more light source interference, such as laparoscopic surgery, to monitor the blood flow status of important organs in real time, and prevent further deterioration of microcirculation congestion shock and organ ischemia and infarction. It further improves the scope of application of the hemoglobin measurement system, and is of great significance to the real-time evaluation of human blood flow status and clinical surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of Example 1 of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with the accompanying drawings and the following embodiments. It should be understood that the accompanying drawings and the following embodiments are only used to illustrate the present invention rather than to limit the present invention.

[0028] Embodiment 1:

[0029] Bayer array-based imaging of oxygenated hemoglobin absorbance without narrow-band filters

[0030] The Bayer array is one of the main technologies for realizing color image capture by CCD or CMOS sensors. It is a 4×4 array consisting of 8 green, 4 blue and 4 red pixels. When converting grayscale images into color images, it will perform 9 operations in a 2×2 matrix to finally generate a color image. A color image can be regarded as the sum of all visible light bands. Therefore, the Bayer array can be used to replace narrow-band filters with the characteristics of different transmittances of different wavelengths. The alternative scheme can be achieved by pre-establishing a mapping from full-color images to specific wavelength images, and also by acquiring absorbance images of specific bands under surgical lights. Compared with the original scheme, this scheme only requires narrow-band filters when the mapping is established, and has the characteristic of not requiring the replacement of special equipment when used. In theory, blood oxygen imaging can even be achieved with a mobile phone camera.

[0031] Step 1: The mapping between specific light wavelengths and RGB pixel values ​​under normal lighting conditions is constructed by presenting the RGB pixel values ​​presented by CCD / CMOS under the illumination of a light source containing an absorption peak to obtain a full-color image. At the same time, the RGB pixel values ​​presented by CCD / CMOS under 660nm and 600nm wavelengths are measured to obtain an image with a high degree of oxygenated hemoglobin absorption. The image is input into a computer to construct an artificial intelligence model, and the mapping relationship from full-color image to 660nm and 600nm wavelength images is obtained respectively. In this process, the artificial intelligence model is essentially a filter implemented by software, realizing a supervised image-to-image mapping model.

[0032] Step 2: When using, obtain the oxygenated hemoglobin absorption peak and background light image according to the mapping relationship.

[0033] 1. Under the same light source used when building the model, the oxygenated hemoglobin absorbance image is obtained using the same CCD or CMOS element based on the mapping relationship between the full-color RGB pixel value and its absorbance image at 660nm constructed in advance.

[0034] 2. Under the same light source used when constructing the model, the background light image is obtained using the same CCD or CMOS element based on the mapping relationship between the full-color RGB pixel value and its absorbance image at 600nm constructed in advance to obtain the surrounding tissue background image.

[0035] Step 3: Compare the absorbance spatial distribution image by computer

[0036] The absorbance spatial distribution image of oxygenated hemoglobin absorbance image and background light image pixel values ​​are obtained according to the Lambert-Beer law. Lambert-Beer law refers to calculating the difference in absorption of oxygenated hemoglobin (HbO2) and reduced hemoglobin (Hb) to light of different wavelengths, collecting images of multiple wavelengths, and calculating the blood oxygen value through image processing. The calculation of blood oxygen content is directly related to the optical density value (OD), and the optical density value OD is defined as: OD = log (I0 / I), OD represents the attenuation of the outgoing light intensity I relative to the incident light intensity I0.

[0037] In the optical density calculation, the outgoing light intensity I is the pixel value of the corresponding channel, and the incident light intensity I0 is the incident light intensity of the corresponding color wavelength range. The blood oxygen saturation and the optical density ratio (ODR) of blood vessels at different wavelengths are approximately negatively correlated linearly, as shown in the formula.

[0038]

[0039] In the above formula, SatO2 represents blood oxygen saturation, ODR represents the optical density ratio of blood vessels at different wavelengths, I0|660nm is the incident light intensity at 660nm spectral range, and I0|600nm is the incident light intensity at 600nm spectral range. The incident light intensity can be obtained by measuring the light source spectrum in advance. 660 nm is the outgoing light absorbed by blood vessels in the 660nm spectral range, I 600 nm is the output light in the 600nm spectral range, a and b are the parameters used for linear fitting of blood oxygen value and ODR, which can be measured through experimental calibration.

[0040] Step 4: Image Output

[0041] The obtained absorbance spatial distribution image is brightened by computer processing and overlaid on the image output of the laparoscope in real time in the form of a heat map or pseudo-color.

[0042] in conclusion

[0043] The innovation of this application is to realize a new blood oxygen imaging system that can avoid interference from external light sources and can display the distribution information of oxygenated hemoglobin in space for real-time imaging of blood flow.

[0044] The invention is that by comparing images with high oxygenated hemoglobin absorption and background images with computers, real-time imaging of the spatial distribution of blood flow can be achieved. Special hardware is only required during the training process. Once the artificial intelligence model training is completed, blood oxygen imaging can be achieved with an ordinary camera. The angiography achieved in a non-invasive and side-effect-free manner solves the problem that existing reflective oximeters and perspective oximeters are easily interfered by external light sources.

[0045] Technical personnel should note that although the present invention has been described according to the above specific implementation methods, the inventive concept of the present invention is not limited to this invention, and any modification using the inventive concept will be included in the scope of protection of this patent.

[0046] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An imaging method using a non-invasive, real-time, non-contact, anti-interference blood oxygen real-time imaging system based on artificial intelligence, characterized in that: The following steps are involved: (1) Mapping construction of specific light wavelengths and RGB pixel values ​​under normal lighting conditions: The RGB pixel values ​​presented by the CCD / CMOS are displayed under the illumination of a light source containing an absorption peak to obtain a full-color image. At the same time, the RGB pixel values ​​presented by the CCD / CMOS under 660nm and 600nm wavelengths are measured to obtain an image with a higher absorbance of oxyhemoglobin. The image is input into a computer to construct an artificial intelligence model to obtain the mapping relationship from the full-color image to the 660nm and 600nm wavelength images respectively. (2) Obtaining the oxygenated hemoglobin absorption peak and background light image according to the mapping relationship during use: Under the same light source used when constructing the model, according to the mapping relationship between the full-color RGB pixel value and its absorbance image at 660nm constructed in advance, the oxygenated hemoglobin absorbance image is obtained using the same CCD or CMOS element, and under the same light source used when constructing the model, according to the mapping relationship between the full-color RGB pixel value and its absorbance image at 600nm constructed in advance, the background light image is obtained using the same CCD or CMOS element to obtain the surrounding tissue background image; (3) Comparison by computer to obtain the absorbance spatial distribution image: The absorbance spatial distribution image of oxygenated hemoglobin and the pixel values ​​of the background light image are calculated according to the Lambert-Beer law. The Lambert-Beer law refers to the calculation of the oxygenated hemoglobin (HbO ２ ) and reduced hemoglobin (Hb) absorb different wavelengths of light, collect images of multiple wavelengths, and calculate the blood oxygen value through image processing. The calculation of blood oxygen content is directly related to the optical density (OD). The optical density OD is defined as: OD =log( I0 / I), OD represents the attenuation of the outgoing light intensity I relative to the incident light intensity I0; In the optical density calculation, the outgoing light intensity I is the pixel value of the corresponding channel, and the incident light intensity I0 is the incident light intensity of the corresponding color wavelength range. The blood oxygen saturation and the optical density ratio of blood vessels at different wavelengths are approximately negatively correlated linearly, as shown in the formula: ； In the above formula, SatO2 represents blood oxygen saturation, ODR represents the optical density ratio of blood vessels at different wavelengths, I0|660nm is the incident light intensity at 660nm, and I0|600nm is the incident light intensity at 600nm. The incident light intensity can be obtained by measuring the light source spectrum in advance. 660 nm is the outgoing light absorbed by blood vessels in the 660nm spectral range, I 600 nm is the output light in the 600nm spectral range, a and b are the parameters used for linear fitting of blood oxygen value and ODR, which can be measured by experimental calibration; (4) Image output: The obtained absorbance spatial distribution image is brightened by computer processing and overlaid on the image output of the laparoscope in real time in the form of a heat map or pseudo-color; The imaging system comprises the following modules: Light source emission and image acquisition module: emits light of a specific intensity and a specific wavelength, and respectively acquires light of a specific wavelength related to oxygenated hemoglobin and a background light image that does not contain any absorption peak band of oxygenated hemoglobin through the filtering function of the image acquisition element; Absorbance image processing and imaging module: connected to the light source emission and image acquisition module, it performs artificial intelligence-based image processing on the hemoglobin absorbance image and the background light image to obtain the imaging of the spatial distribution of oxygenated hemoglobin; Image output module: The obtained absorbance spatial distribution image is brightened by computer processing and overlaid on the image output of the laparoscope in real time in the form of a heat map or pseudo-color.

2. The imaging method according to claim 1, characterized in that: The light source emission and image acquisition module includes a 660nm narrowband filter with a bandwidth of 10nm and a CCD or CMOS element without a Bayer array. The 660nm narrowband filter with a bandwidth of 10nm filters a specific wavelength, and obtains an image with a high hemoglobin absorption degree and a background light image at a specific wavelength through the CCD or CMOS element without a Bayer array.

3. The imaging method according to claim 1, characterized in that: The light source emission and image acquisition module includes a CCD or CMOS element including a Bayer array. According to the mapping relationship between the constructed full-color RGB pixel value and its absorbance image at 660nm, the oxygenated hemoglobin absorbance image and the background light image are obtained using the same CCD or CMOS element including the Bayer array.

Images