A non-invasive hemoglobin detection method based on hyperspectral image reconstruction

Abstract

The present application relates to a non-invasive hemoglobin detection method based on hyperspectral image reconstruction, belonging to the field of physiological signal detection. The present application irradiates human skin tissue by using a visible light source, simultaneously uses a hyperspectral camera and an RGB camera to simultaneously collect images of the irradiated skin area as a training set, trains a deep learning model, and realizes hyperspectral reconstruction of ordinary RGB images. A hemoglobin extraction feature extraction model for the visible light range diffuse reflectance spectrum is established using an algorithm, and the real hemoglobin content of the measured human body is output through a hemoglobin correction model.

Description

Technical Field

[0001] The present invention belongs to the field of human health detection technology, and relates to optical technology detection fields such as image processing and deep learning, and in particular to a method for non-invasive detection of hemoglobin concentration. Background Art

[0002] Hemoglobin is a key component of human blood, and its level is a crucial indicator for the clinical diagnosis of anemia and other blood disorders. According to the World Health Organization, 25% of the global population currently suffers from varying degrees of anemia, with rates reaching 42% among pregnant women, 30% among non-pregnant women, 47% among preschool children, and 12.7% among adolescents. These data indicate that anemia has become a global problem, not only endangering people's lives and health but also placing a heavy economic burden on society and families. Therefore, regular monitoring of hemoglobin concentration is crucial for the prevention and treatment of anemia.

[0003] Currently, the commonly used clinical methods for measuring hemoglobin concentration are still invasive methods, such as the ferric cyanide method and the ferric azide method. These methods are reliable, but they cause pain to the subjects and also carry the risk of wound infection. Invasive hemoglobin measurement methods are not suitable for pregnant women who are prone to anemia, newborns, people in resource-poor areas, and patients who need to monitor hemoglobin levels in real time during surgery. In addition, invasive detection methods are not suitable for hemoglobin screening in families or communities, which hinders the early diagnosis and prevention of anemia. Given the limitations of traditional hemoglobin measurement, non-invasive measurement methods have obvious advantages and application value. Optical measurement methods mainly use information closely related to hemoglobin concentration, such as the intensity, phase, and polarization angle of the transmitted light, as well as the absorption and scattering coefficients of the target tissue, to indirectly measure hemoglobin concentration by analyzing the captured information. Optical measurement methods have become the main technical means for non-invasive hemoglobin concentration detection due to their rapidity, non-invasiveness, multidimensional information, and real-time detection. Currently, the main optical non-invasive hemoglobin detection methods include photoacoustic spectroscopy, photoplethysmography, and spectral analysis. Photoacoustic spectroscopy requires a large photoacoustic spectroscopy system to detect hemoglobin concentration. The system equipment is expensive and not suitable for real-time monitoring of hemoglobin concentration at home. Non-invasive hemoglobin measurement methods based on photoplethysmography are mainly used to predict the overall trend of hemoglobin concentration, and their accuracy needs to be further improved. In response to this problem, the present invention proposes a new method for non-invasive detection of hemoglobin concentration, providing a practical technical solution to achieve non-invasive hemoglobin concentration detection.

[0004] The working principle of this non-invasive hemoglobin concentration detection method based on hyperspectral image reconstruction is to utilize the absorption and scattering of light by hemoglobin molecules in the blood, image the skin area through a visible light imaging device, select the area of ​​interest from the acquired image, reconstruct the RGB image in the photographed area into a hyperspectral image and extract the characteristic values ​​to obtain the hemoglobin concentration value. Summary of the Invention

[0005] This invention proposes a method for rapidly and noninvasively measuring human hemoglobin concentration using visible light imaging equipment under LED illumination. The method involves first acquiring an RGB image of the subject's skin, reconstructing it into a hyperspectral image, performing feature extraction, and predicting the hemoglobin concentration using a calibration model. This invention enables noninvasive hemoglobin testing, making it suitable for everyday hemoglobin testing for the general public and possessing broad development and application prospects. The method is described below. The method is implemented as follows: a noninvasive hemoglobin detection method based on hyperspectral image reconstruction, for noninvasively measuring human hemoglobin concentration. The method is characterized in that data acquisition is performed using a conventional CCD industrial camera, a hyperspectral camera, and a lens; and the illumination source is a white LED. The method first acquires RGB and hyperspectral images of a finger (including but not limited to a finger) as datasets for training a deep learning network. Simultaneously, a hemoglobin feature extraction model is generated using human skin simulation. The hyperspectral output of the deep learning network is then used for feature extraction. The feature values ​​are then input into the hemoglobin calibration model to obtain the subject's hemoglobin concentration.

[0006] The imaging device is located 30-40 cm from the fingertip, achieving non-invasive measurement of hemoglobin concentration values ​​and avoiding the risk of pain and infection for patients caused by invasive measurement.

[0007] The light source is an LED white light source.

[0008] The hemoglobin features include but are not limited to color information of multiple channels of the hyperspectral image, which can realize the extraction of hyperspectral image features.

[0009] The general prediction model includes but is not limited to a multi-scale mixed domain attention mechanism network prediction model for skin images and a hemoglobin feature extraction model. The prediction model uses a large number of RGB images and hyperspectral images as training samples. The feature extraction model is generated by Monte Carlo simulation, and the prediction ability of the model is achieved through cross-validation.

[0010] The blood glucose detection method comprises the following steps.

[0011] Step 1: Collect data sets to train deep learning models;

[0012] 1) Start the white light emitting LED light source to emit visible light to the fingers or other parts of the skin.

[0013] 2) Start the industrial camera and hyperspectral camera to ensure that no other light enters and affects the data collection of the imaging equipment.

[0014] 3) The measured position is placed at the angle between the light source and the imaging device, remains still for 10 seconds, and the RGB image is collected.

[0015] 4) The RGB image is input into the network prediction model to reconstruct multiple sets of hyperspectral images.

[0016] Step 2: Create a feature value extraction model;

[0017] 1) Design a skin diffuse reflection simulation experiment.

[0018] 2) Study the hemoglobin feature extraction algorithm.

[0019] 3) Evaluate the performance of the hemoglobin feature extraction model.

[0020] Step 3: Predicting hemoglobin concentration based on the extracted characteristic values;

[0021] 1) Establish a hemoglobin concentration prediction and correction model.

[0022] 2) Hemoglobin characteristics are input into the calibration model to predict hemoglobin concentration.

[0023] Preferably, the hyperspectral image is reconstructed using a multi-scale mixed-domain attention mechanism network prediction model for skin images.

[0024] Preferably, the hemoglobin extraction model adopts Monte Carlo simulation.

[0025] Beneficial effects

[0026] 1. The present invention provides a non-invasive hemoglobin concentration detection method based on hyperspectral image reconstruction. It does not require the detection instrument to come into contact with the object being detected, thereby improving comfort and avoiding detection errors caused by physiological changes caused by contact stimulation.

[0027] 2. The non-invasive hemoglobin detection method based on hyperspectral image reconstruction of the present invention does not require individual calibration or selection of measurement time. It is a universal model that realizes true non-invasive blood glucose detection for all people.

[0028] 3. The non-invasive hemoglobin detection method based on hyperspectral image reconstruction of the present invention only requires a conventional imaging device, has low requirements on resolution, does not require additional equipment, and has low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the method described in the present invention.

[0030] Figure 2 This is a flow chart of the non-invasive hemoglobin detection method based on hyperspectral image reconstruction described in the present invention.

[0031] Among them, 1-imaging device, 2-tested object, 3-LED light source. DETAILED DESCRIPTION

[0032] In order to make the purpose, advantages and features of the present invention clearer, the following is a further detailed description of a non-invasive hemoglobin detection method based on hyperspectral image reconstruction proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that: the accompanying drawings are all in a very simplified form and use non-precise proportions. They are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. The structure shown in the accompanying drawings is part of the actual structure; the acquisition site of the image signal of the present invention is not limited to the human face, and is also applicable to other parts of the human body. The present invention is also not limited to the prediction of hemoglobin content in normal people. The hyperspectral image reconstruction algorithm used in the present invention is not limited to a certain algorithm.

[0033] The embodiments of the present invention use normal healthy people as subjects.

[0034] This embodiment discloses a non-invasive hemoglobin detection method based on hyperspectral image reconstruction, and the image acquisition diagram is shown in FIG. Figure 1 The execution process is shown in the attached Figure 2 The specific contents are as follows.

[0035] Step 1: Use a hyperspectral camera and an ordinary industrial camera to collect hyperspectral images and RGB images containing skin tissue in the same illumination area to train the model.

[0036] Step 1-1: The subject sits still on a chair and starts the white light LED light source to ensure that the visible light is evenly irradiated on the fingers or other skin parts.

[0037] In steps 1-2, start the industrial camera and the hyperspectral camera, focus both cameras until the image is clearest, and perform whiteboard calibration to ensure that no other light affects the image acquisition of fingers or other parts.

[0038] In steps 1-3, place your palm at the measured location, keep a distance of 30-40 cm from the hyperspectral camera and industrial camera, and remain still for 10 seconds to collect RGB images and hyperspectral images.

[0039] In steps 1-4, the collected RGB images and hyperspectral images are used as training sets and as inputs to the multi-scale mixed domain attention mechanism network prediction model. The mathematical description of the reconstruction process is:

[0040] R(u,v,λ)=F(I RGB (u,v)) (1)

[0041] Where I RGB (u,v) represents the skin RGB image, F(.) represents the reconstruction model, and R(u,v,λ) represents the hyperspectral reflectance image. The original hyperspectral data for constructing the dataset can be expressed as:

[0042] I H (u,v,λ i )=I0(u,v,λ i )R(u,v,λ i ) (2)

[0043] Where I H (u,ν,λ i ) represents the hyperspectral data image, u and ν represent different pixel positions in the image space, and λ i Represents the band corresponding to each channel of the hyperspectral image; I0(u,ν,λ i ) represents the detection light source used when the original hyperspectral image was taken, R(u,ν,λ i ) is the reflectance spectrum of the skin.

[0044] Step 2: Create a hemoglobin feature extraction model through human skin diffuse reflectance experiment;

[0045] In step 2-1, Monte Carlo simulation is used to simulate light transmission through skin tissue under different physiological parameters, obtaining the corresponding skin diffuse reflectance spectra. A three-layer skin tissue model consisting of the epidermis, dermis, and subcutaneous tissue layers is then established. This skin tissue model is used to simulate the skin diffuse reflectance spectrum from 400nm to 700nm under changes in peripheral blood.

[0046] In step 2-2, a hemoglobin feature extraction model is established by using a typical supervised dimensionality reduction algorithm. The essence of this task is to project the spectral data into a low-dimensional space.

[0047] In step 2-3, the support vector regression algorithm is used to estimate the hemoglobin concentration of the simulation data, and the accuracy of the regression model is used as an evaluation index to evaluate the performance and applicability of the hemoglobin feature extraction model.

[0048] Step 3: Select the ROI area and input it into the feature extraction model to generate feature values, and then predict the hemoglobin concentration of the human body after correction.

[0049] In step 3-1, when selecting the region of interest (ROI), to remove noise introduced by camera detection, the mean of a 10×10 sub-pixel region is taken as a spatial feature vector. Adjacent 20×20 spatial feature vectors are then selected, each consisting of 29 elements. These 29 elements correspond to the reflectivity within 29 bandwidths, ranging from 415nm to 695nm, with intervals of 10nm. These spatial features are then input into the feature extraction model established in step 2.

[0050] The correction model is established by using support vector regression and random forest regression. The calculation method of the three evaluation indicators can be expressed as follows:

[0051]

[0052]

[0053]

[0054] Where N represents the number of samples in the dataset; y i represents the true value, Represents the predicted value of the model. Root mean square error (RMSE) and mean absolute error (MAE) are both used to measure the difference between the predicted result and the actual result. The smaller the value, the better the performance. Compared with MAE, RMSE is more affected by outliers. 2 The value range is (0, 1). When R 2 The closer it is to 1, the better the model fit is.

[0055] In step 3-2, the hemoglobin characteristics are input into the calibration model to predict the hemoglobin concentration.

[0056] In summary, the above are only preferred embodiments provided by the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the invention shall be included in the scope of protection of the present invention.

Claims

1. A non-invasive hemoglobin detection method based on hyperspectral image reconstruction, used for non-invasive detection of hemoglobin concentration, characterized in that: The method consists of hyperspectral image reconstruction, a hemoglobin feature extraction model, and a hemoglobin concentration prediction model. The hyperspectral reconstruction method is a multi-scale hybrid domain attention mechanism network model. The hemoglobin feature extraction model uses simulated data generated by Monte Carlo simulation to extract hemoglobin features through a supervised dimensionality reduction algorithm. The hemoglobin concentration prediction model uses two methods, support vector regression and random forest regression, for prediction. The method first collects RGB images and hyperspectral images to ensure that no other light affects the experimental collection. Light emitted by an LED light source is irradiated onto the test parts, including the fingertips of the human body. The diffuse reflectance spectrum data generated by the Monte Carlo simulation method is used to establish a hemoglobin feature extraction model. The collected RGB images are input into a multi-scale mixed domain attention mechanism network model to reconstruct the hyperspectral image. The hemoglobin concentration value of the measured object is then obtained through the hemoglobin concentration prediction model. The light source includes a white LED light source, and the multi-scale mixed-domain attention mechanism network model is trained using a large number of RGB images and hyperspectral images. The hemoglobin characteristics in the hyperspectral image include spectral diffuse reflectance feature information; The hemoglobin feature extraction model is based on simulated data generated by the Monte Carlo simulation method, extracts features through a supervised dimensionality reduction algorithm, and realizes the predictive ability of the model through cross-validation; the hemoglobin detection method comprises the following steps: Step 1: Collect data sets to train a multi-scale mixed-domain attention mechanism network model to reconstruct hyperspectral images; 1) Start the white light emitting LED light source to emit visible light to the finger; 2) Start the industrial camera and hyperspectral camera, focus them to the optimal position, and ensure that no other light enters to affect the data collection of the imaging equipment; 3) Place the finger at the measured position, keep the working distance from the two cameras, and remain still for 10 seconds to collect RGB images and hyperspectral images to create a data set; 4) Input the RGB image into the multi-scale mixed domain attention mechanism network model to reconstruct multiple sets of hyperspectral images; Step 2: Design a diffuse reflection simulation experiment and create a feature value extraction model; 1) Use the Monte Carlo simulation method to simulate the transmission of light in skin tissue under different physiological parameters and generate diffuse reflectance spectrum simulation data; 2) Use supervised dimensionality reduction algorithm to extract hemoglobin features from simulation data and establish a hemoglobin feature extraction model; 3) Use support vector regression and random forest regression algorithms to estimate the hemoglobin concentration of simulation data and evaluate the performance of the hemoglobin feature extraction model; Step 3: Select the ROI area and generate the eigenvalues, and then predict the hemoglobin concentration after correction; 1) Remove the noise introduced by the camera and select the area of ​​interest; 2) Input the hemoglobin characteristics into the hemoglobin concentration prediction model to predict the hemoglobin concentration.

Images