Blood glucose continuous value estimation method and system based on visible light, multi-band infrared and thermal infrared video

Abstract

The application belongs to the technical field of non-contact blood glucose monitoring, and specifically discloses a blood glucose continuous value estimation method and system based on visible light, multi-band infrared and thermal infrared video, which comprises the following steps: collecting visible light, multi-band infrared and thermal infrared videos of a target region of a subject, and performing pretreatment to obtain a region of interest (ROI); calculating the sensitivity of each band of the multi-band infrared video to a reference blood glucose value and the stability, calculating a comprehensive score, extracting a candidate band and constructing mechanism constraint features, constructing an interference representation and correcting the candidate blood glucose related features, performing crowd prior modeling and establishing an individual baseline correction mechanism; and generating a blood glucose continuous value by using a state space recursion method. According to the technical scheme, stable output of the blood glucose continuous value is realized on the basis of multi-modal video synchronous collection, in combination with crowd prior modeling, individual baseline correction, multi-region consistency evaluation and a continuous recursion updating mechanism.

Description

Technical Field

[0001] This invention belongs to the field of non-contact blood glucose monitoring technology, and relates to a method and system for estimating continuous blood glucose values ​​based on visible light, multi-band infrared and thermal infrared video. Background Technology

[0002] Blood glucose level is an important physiological indicator reflecting the state of glucose metabolism in the human body, and it has important application value in scenarios such as diabetes screening, daily health management, chronic disease monitoring, exercise recovery assessment, and family health monitoring.

[0003] Current blood glucose testing methods mainly include finger-prick blood sampling and continuous glucose monitoring. The former usually achieves high accuracy, but it is an invasive method and cannot meet the needs of high-frequency testing and long-term monitoring. The latter can achieve continuous monitoring, but it usually relies on patch or implantable devices, which still have certain limitations in terms of usage cost, wearing comfort, long-term compliance and convenience of daily use.

[0004] Currently, video-based non-invasive detection methods offer advantages such as being non-contact, low-burden, easy to deploy, and more suitable for everyday use, thus possessing significant application potential. Especially with the continuous development of camera devices and the improvement of infrared imaging capabilities, utilizing video signals to continuously estimate the metabolic state of the human body has gradually become a noteworthy technological direction.

[0005] Existing research and technical data indicate that within a certain wavelength range, changes in blood glucose levels cause detectable changes in absorbance and reflectance. Certain bands within the multi-band infrared range are sensitive to blood glucose-related optical changes. In specific implementations, the multi-band infrared band preferably covers the 700 nm to 1600 nm wavelength range. However, in actual human testing, the amplitude of blood glucose-related reflectance changes is usually small and easily affected by various factors such as moisture absorption, tissue background, skin texture, skin color differences, local temperature changes, and individual physiological differences. This results in insufficient stability of blood glucose-related features extracted under a single band or single modality.

[0006] Continuous blood glucose monitoring is not suitable for directly outputting values ​​from a single frame of image. Instead, it is better to use continuous video time windows as the basic estimation unit, and then continuously update them to form a continuous blood glucose value sequence. Without a reasonable continuous estimation mechanism, problems such as large jumps in results, significant short-term fluctuations, and insufficient stability of continuous output can easily occur.

[0007] Furthermore, individuals differ in age, gender, body mass index, skin color, basal metabolic rate, and glucose metabolism risk stratification, which can affect the mapping relationship between video features and blood glucose levels. If a uniform model is used directly without considering population differences and individual reference states, it can easily lead to problems such as decreased cross-population generalization performance, significant individual bias, and insufficient stability in continuous monitoring.

[0008] Therefore, existing technologies still lack a non-invasive video detection method for continuous blood glucose estimation scenarios. Summary of the Invention

[0009] The purpose of this invention is to address the aforementioned problems in existing technologies by proposing a method and system for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video.

[0010] To achieve the above objectives, the basic solution of this invention is: a method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video, comprising the following steps:

[0011] Using a visible light camera, a multi-band infrared camera, and a thermal infrared camera, visible light video, multi-band infrared video, and thermal infrared video are simultaneously acquired of the target area of ​​the subject, and preprocessed to obtain the region of interest (ROI).

[0012] The video data within the Region of Interest (ROI) is normalized and processed at the region level to extract video features. The normalization and region-level processing are used to eliminate the influence of illumination changes and device response differences between different time windows, making the features within different time windows comparable, and providing a unified input for subsequent multi-band feature screening, interference correction, and state space modeling.

[0013] The sensitivity of each band in multi-band infrared video data to reference blood glucose values, as well as the stability under different time windows, different ROIs and different individuals, are calculated, and the sensitivity and stability are combined to calculate a comprehensive score.

[0014] Based on the comprehensive score, candidate bands are extracted, and mechanistic constraint features are constructed using video features. The mechanistic constraint features and candidate band features are used together as input features for subsequent interference correction and continuous blood glucose value estimation.

[0015] By utilizing video features of candidate bands and mechanistic constraints, an interference characterization is constructed, and candidate blood glucose-related features are corrected. The interference characterization is used to describe non-target factors that are not directly related to blood glucose changes but affect video features, including surface appearance differences, local temperature changes, and moisture absorption. By constructing the interference characterization, non-blood glucose-related components can be removed from candidate blood glucose-related features, thereby improving the stability and accuracy of blood glucose estimation.

[0016] Incorporate demographic information to perform prior population modeling and establish an individual baseline correction mechanism;

[0017] Local observation results are generated on multiple ROIs in the target region, and consistency assessment and credibility weighting are performed on the local observation results.

[0018] A state-space recursive method is used to generate continuous blood glucose values.

[0019] The working principle and beneficial effects of this basic solution are as follows: This technical solution is based on multimodal video synchronous acquisition, uses multi-band infrared information as the main reflectance information source of blood glucose, uses visible light information for surface appearance difference compensation, uses thermal infrared information for thermal state correction, and further combines population prior modeling, individual baseline correction, multi-region consistency assessment and continuous recursive update mechanism to achieve stable output of continuous blood glucose values.

[0020] Furthermore, the target area of ​​the subject is the facial area, or one or more of the following: forehead, cheeks, sides of the nose, jaw, in front of the ear, or back of the hand.

[0021] The target area is preferably the facial area, but other suitable skin areas for non-invasive video capture, such as the forehead, cheeks, sides of the nose, jaw, in front of the ears, or the back of the hand, can also be selected according to the actual usage scenario for ease of use.

[0022] Furthermore, the method for preprocessing the acquired visible light video, multi-band infrared video, and thermal infrared video to obtain the region of interest (ROI) is as follows:

[0023] Let the time window length be The sliding step size is Then the first The visible light input, multi-band infrared input, and thermal infrared input within each time window are denoted as follows: , and ;

[0024] For multi-band infrared data, reflectance correction is performed using white-field and dark-field references. The corrected reflectance... ,for:

[0025] ,

[0026] in, Indicates the first The band in the first The original intensity within a time window, Indicates the dark field reference value. Indicates the whiteboard reference value;

[0027] For thermal infrared data, temperature conversion is performed through calibration parameters, and temperature drift compensation is performed in conjunction with an ambient temperature reference. Specifically:

[0028] The ambient reference temperature is estimated based on the ambient temperature sensor or background area temperature. Offset correction is applied to the thermal infrared measurements within the current time window, and linear or exponential smoothing compensation is performed in conjunction with the time-series temperature change trend to obtain stable thermal state data. The compensated thermal state data is denoted as... ;

[0029] In facial scenes, face detection is performed, and the positions of the forehead, left cheek, and right cheek regions are identified using a key point localization model. The key point localization model is preferably a deep learning-based face key point detection model, including but not limited to the MediaPipe Face Mesh model, the Dlib key point detection model, or a key point regression model based on a convolutional neural network. These regions are continuously tracked, and areas such as the eyes, eyebrows, mouth, nostrils, hair-covered areas, and obvious shadow areas are excluded, retaining only stable areas with skin exposure as regions of interest (ROIs).

[0030] After the original video is acquired, the multimodal data is preprocessed to reduce the impact of acquisition differences between different devices and environmental changes.

[0031] Furthermore, the video data within the Region of Interest (ROI) is normalized and processed at the region level to extract video features, as follows:

[0032] For visible light data, local illumination compensation is performed within the ROI range, and pixel intensity is normalized.

[0033] ,

[0034] in, and They represent the first Mean and standard deviation of ROI within each time window; The pixel intensity after normalization; Visible light input;

[0035] After normalization, appearance-related features are extracted from the visible light video, specifically:

[0036] Within the ROI, the color, brightness, texture, and reflectivity of the visible light image are statistically analyzed to form a visible light feature vector;

[0037] Skin color distribution characteristics are obtained by statistically analyzing the pixel distribution in different color channels or color spaces within the ROI, including color mean, color variance, or color histogram characteristics in RGB, HSV, or YCbCr color spaces.

[0038] The average brightness value is obtained by calculating the average value of the pixel brightness components within the ROI;

[0039] Channel ratios are obtained by calculating the ratios between different color channels, including the red-green channel ratio, the red-blue channel ratio, or the green-blue channel ratio.

[0040] Local texture statistics, including texture contrast, texture uniformity, texture entropy, and local gradient variation features, can be obtained through gray-level co-occurrence matrix, local binary mode, gradient statistics, or other texture description methods.

[0041] The specular reflection characteristics are obtained by statistically analyzing the proportion of bright pixels within the ROI, the proportion of areas where the brightness component exceeds a preset threshold, or the proportion of bright but low-saturation areas.

[0042] For multi-band infrared data, the regional reflectance characteristics and temporal variation characteristics of each band are calculated within the ROI. Let the _i_th band be the first band. The band in the first The ROI region within each time window is Regional reflectivity characteristics of this band Represented as:

[0043] ,

[0044] in, Indicates the first The band in the first Within the first time window The reflectance value at each pixel location. Indicates the number of pixels within the ROI;

[0045] Calculate the relative change of this band between adjacent time windows. Or offset relative to the individual baseline This is used to describe the temporal changes in multi-band infrared reflectance characteristics:

[0046] ,

[0047] or:

[0048] ,

[0049] in, Indicates the first Reference reflectance characteristics formed by each band during the baseline establishment phase;

[0050] The regional reflectivity characteristics are normalized and time-series smoothed. Regional reflectivity normalization is achieved through intra-band normalization, normalization relative to a reference band, or normalization relative to the average reflectivity level of multiple bands, expressed as:

[0051] ,

[0052] in, and They represent the first The mean and standard deviation of regional reflectance characteristics across multiple bands within a time window. To prevent extremely small constants with a denominator of zero; This represents the region-level reflectance characteristics after normalization and temporal smoothing.

[0053] The time series of each band is smoothed using a moving average method. The smoothed result is... for:

[0054] ,

[0055] in, Indicates the length of the smooth window; Indicates the first The band in the first Regional reflectance characteristics within a time window; Indicates the first The band in the first Reflectance characteristics smoothed by moving average within a time window;

[0056] For thermal infrared data, thermal state-related features are extracted within the ROI range. Let the first... The ROI within the time window is the first The temperature value at each pixel location The average temperature of the region Represented as:

[0057] ,

[0058] The regional temperature difference is determined by the highest temperature within the ROI. With the lowest temperature difference express:

[0059] ,

[0060] Heat distribution dispersion Represented by the variance or standard deviation of temperature values ​​within the ROI:

[0061] ,

[0062] Thermal gradients are represented by the temperature changes between adjacent pixels or the average value of temperature gradient magnitudes, and are used to describe the spatial distribution of temperature within an ROI.

[0063] rate of change of thermal state Represented by the change in average temperature between adjacent time windows:

[0064] ,

[0065] in, and They represent the first The first time window and the first The average temperature of the ROI within a time window.

[0066] After normalization, features such as skin color distribution, average brightness, channel ratio, local texture statistics, and specular reflection degree are further extracted from the visible light video. These features are used to construct an appearance interference characterization, which can suppress short-term fluctuations and local random noise.

[0067] Furthermore, the sensitivity of each band in the multi-band infrared video data to the reference blood glucose value was calculated, as well as its stability under different time windows, different ROIs, and different individuals. The sensitivity and stability were then combined to calculate a comprehensive score, specifically:

[0068] Let the first The sensitivity score for each band is Stability score: ,in:

[0069] The sensitivity score Used to characterize the The correlation between each band feature and the reference blood glucose value can be obtained through statistical correlation analysis, preferably using Pearson correlation coefficient, Spearman correlation coefficient, or other statistical indicators that can reflect the correlation of variables, specifically expressed as follows:

[0070] ,

[0071] in, Indicates the first The characteristic sequence of each band within each time window This represents a sequence of reference blood glucose values ​​within the corresponding time window;

[0072] The stability score Used to characterize the The degree of fluctuation of a band characteristic under different conditions is obtained by calculating the statistical dispersion of the band characteristic under different time windows, different ROIs, and different individual conditions. Variance, standard deviation, or coefficient of variation is preferably used as a measure, and is specifically expressed as follows:

[0073] ,

[0074] in, and They represent the first Mean and standard deviation of each band feature under multiple time windows, multiple ROIs, and multiple sample conditions. To prevent extremely small constants with a denominator of zero;

[0075] Among them, different time windows, different ROIs and different individual conditions refer to multiple time windows formed during continuous data collection, multiple local ROIs divided within the target area, and individual differences among different subjects.

[0076] Finally, the first Overall score of each band Represented as:

[0077] ,

[0078] in, and is a weighting parameter used to balance the impact of sensitivity and stability in the candidate band selection process.

[0079] During the training phase, the sensitivity of each band to the reference blood glucose value and its stability under different time windows, different ROIs and different individuals were calculated separately, and the two were combined to form a comprehensive score.

[0080] Furthermore, based on the comprehensive score, the steps for extracting candidate bands and constructing mechanistic constraint features are as follows:

[0081] In multi-band infrared video data, the extracted regional reflectance features and their time-smoothed features are used as the basic features. The corresponding comprehensive score is calculated for each band and sorted according to the comprehensive score.

[0082] The top scorers are selected based on their overall evaluation. Several candidate bands, among which The preset constants are preferably 3 to 10, forming a candidate band set;

[0083] Based on the candidate band set, a mechanism constraint feature is constructed for any two candidate bands. and The ratio feature, difference feature, and normalized difference feature are constructed respectively as follows:

[0084] ,

[0085] in, and This indicates the regional reflectance characteristics of the candidate band within the current time window, or its characteristics after time smoothing.

[0086] The mechanism constraint features and candidate band features together constitute the input features for subsequent interference correction and continuous blood glucose value estimation, which are used to enhance the expressive power of blood glucose-related information and improve feature stability.

[0087] Transforming single-band reflection information into combined features that are more suitable for describing blood glucose-related changes enhances the discriminability and stability of subsequent estimations.

[0088] Furthermore, using video features of candidate bands and mechanistic constraint features, an interference characterization is constructed, and candidate blood glucose-related features are corrected. The specific steps are as follows:

[0089] Let the first Candidate blood glucose-related features extracted within each time window are: The candidate blood glucose-related features are based on candidate band features in multi-band infrared video as the main information source, including regional reflectance features of candidate bands, reflectance features after time smoothing, and ratio features, difference features and normalized difference features constructed from candidate bands.

[0090] Let the first The appearance-related feature vectors extracted from visible light video within each time window are: The appearance-related feature vectors include mean brightness, channel ratio, skin color distribution, texture statistics, and reflectivity features; these feature vectors are converted into appearance interference representations through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now:

[0091] ,

[0092] in, This represents the appearance interference mapping function, whereby the appearance interference is characterized. Used to describe the effects of changes in illumination, skin color differences, surface texture differences, and specular reflection on candidate glucose-related features;

[0093] Let the first The thermal state-related feature vectors extracted from thermal infrared video within each time window are: The thermal state-related feature vector includes regional average temperature, regional temperature difference, thermal gradient, temperature dispersion, and temperature change rate characteristics. Through linear mapping, nonlinear mapping, attention weighting, or neural network mapping, the thermal state-related feature vector is converted into a thermal state disturbance representation. ,Right now:

[0094] ,

[0095] in, This represents the thermal state disturbance mapping function, wherein the thermal state disturbance characterization Used to describe the effects of local temperature changes, regional heat distribution differences, and perfusion status fluctuations on candidate blood glucose-related features;

[0096] Let the first The moisture-related feature vectors extracted from multi-band infrared video within each time window are: The water-related feature vector includes reflectance features of bands sensitive to water absorption and their ratios, differences, or combinations of normalized differences; the water-related feature vector is converted into water-related interference terms through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now:

[0097] ,

[0098] in, This represents the moisture interference mapping function, where the moisture-related interference term... Used to describe the effects of changes in skin moisture content and water absorption on multi-band infrared reflectance characteristics;

[0099] Candidate blood glucose-related features Interference correction was performed to obtain the corrected blood glucose-related features. Its expression is:

[0100] ,

[0101] in, , and The correction coefficient is used to characterize the degree of influence of appearance interference, thermal state interference, and moisture-related interference on candidate blood glucose-related features; when , , or When in vector form, the correction coefficients can be scalars, weight vectors, or mapping matrices, so that each interference characterization has a consistent dimension with the candidate blood glucose-related features;

[0102] The corrected glucose-related features This refers to the candidate blood glucose-related features Based on this, the feature representation obtained after stripping away non-target factors such as appearance differences, thermal state changes, and water absorption is used to characterize the changes in multimodal video optical response caused by blood glucose changes, and serves as input for subsequent population prior modeling, individual baseline correction, and state space recursion.

[0103] The visible light branch mainly provides interference information related to surface appearance, including skin color distribution, texture roughness, brightness shift, and local reflection status; the thermal infrared branch mainly provides interference information related to thermal status, including average temperature, local temperature difference, thermal gradient, and thermal drift.

[0104] Furthermore, by incorporating demographic information, the steps for prior population modeling are as follows:

[0105] Let the demographic information vector be... This includes age, gender, body mass index, skin color classification, and metabolic risk stratification. Discrete variables in the demographic information vector are encoded, and continuous variables are normalized to obtain the corresponding population prior vector. ;

[0106] Let the first The interference-corrected blood glucose-related features within each time window are: The corrected blood glucose-related features Prior vectors of the population The shared input group prior module obtains the current individual's position in the [missing information]. Prior correction for the population within a time window , represented as:

[0107] ,

[0108] in, This represents the mapping function learned from the training data;

[0109] The population prior correction term It is used to characterize the differences in baseline response among individuals of different ages, sexes, body mass indexes, skin color types, and metabolic risk stratifications under the same or similar corrected blood glucose-related characteristics, and serves as the initial offset constraint in subsequent blood glucose observation mapping or state space recursion processes.

[0110] Considering the differences among individuals in terms of age, gender, body mass index, skin color type, basal metabolic rate, and glucose metabolism risk stratification, demographic information is introduced for population prior modeling.

[0111] Furthermore, the method for establishing an individual baseline correction mechanism is as follows:

[0112] After the subjects completed target area localization and began multimodal video data acquisition, the previous... The first time window is used as the initialization phase, and the corrected features of this phase are used to establish a reference baseline for the individual in the current environment and state. Let the first time window be the initialization phase. The corrected candidate features for each initial time window are: Then individual baseline characteristics ,for:

[0113] ,

[0114] For any time window Calculate the offset of its current feature relative to the baseline. :

[0115] ,

[0116] in, Indicates time window Corrected candidate features;

[0117] During continuous monitoring, individual baselines are updated slowly according to high-quality time windows, and the update method is as follows:

[0118] ,

[0119] in, Indicates the update coefficients; Indicates time window Individual baseline characteristics.

[0120] This invention does not directly use absolute reflectance values ​​to estimate blood glucose, but rather focuses on the changes in the current state relative to an individual's reference state. This significantly reduces the impact of individual differences in factors such as innate skin color, basal temperature, skin thickness, and surface condition.

[0121] Furthermore, local observation results are generated for multiple ROIs within the target region, and consistency assessment and credibility weighting are performed on these local observation results. The specific method is as follows:

[0122] Let the first The local blood glucose observation values ​​given for the three ROI regions within each time window are as follows: , and Then regional consistency Its variance is expressed as:

[0123] ,

[0124] Weights are assigned to different regions based on image quality, occlusion, reflectivity, and thermal stability. This forms the weighted observations for that time window. :

[0125] ,

[0126] in, Represents the number of ROIs, and satisfies ;r represents the ROI number.

[0127] By generating local observations on multiple ROIs such as the forehead, left cheek, and right cheek, the impact of anomalous regions on the overall estimation results can be reduced, and a confidence score for the current time window can be generated simultaneously.

[0128] Furthermore, a state-space recursive method is used to generate continuous blood glucose values, specifically as follows:

[0129] Let the first The true blood glucose status within each time window is It changes slowly over time, and is represented as:

[0130] ,

[0131] in, Represents a state transition term. Indicates process noise;

[0132] Let the first The interference-corrected blood glucose-related features within each time window are: Individual baseline characteristics are The offset of the current feature relative to the individual baseline. Represented as:

[0133] ,

[0134] in, Used to indicate the degree of change in blood glucose-related characteristics relative to the individual's reference state within the current time window;

[0135] Observation model under the current time window Represented as:

[0136] ,

[0137] in, This represents the observation mapping function composed of baseline offset features and population priors. Indicates observation noise. This represents the prior vector of the population. This represents the current time window blood glucose observation result obtained jointly by the corrected blood glucose-related feature offset and the population prior;

[0138] Based on the estimated state from the previous moment and the observation results from the current moment, the blood glucose estimate for the current moment is updated as follows:

[0139] ,

[0140] in, This is the updated blood glucose estimate for the current moment. This represents the update gain, the size of which can be adjusted based on the reliability, self-consistency, and state stability of the current time window; the higher the quality of the current time window, the better. The larger the value, the stronger the impact of the current observation on the result update; the worse the quality of the current time window, The smaller the value, the more the system depends on the stable state of the previous moment.

[0141] This recursive method can effectively suppress transient noise and jumps caused by local anomalies, making the output continuous blood glucose values ​​more consistent with the actual change patterns.

[0142] Furthermore, according to The current value is used to determine whether it is in the normal, high or low range, and the direction of change between adjacent time windows is used to determine whether it is in an upward, downward or stable trend.

[0143] Let the slope of the change between two adjacent time windows be:

[0144] ,

[0145] Then when When the value is greater than a preset threshold, it is judged as an upward trend; when When the value is less than the negative threshold, it is judged as a downward trend; otherwise, it is judged as a stable trend.

[0146] By combining the fluctuation range over a continuous period, the duration of deviation from the individual's baseline, and the reliability of the current time window, a risk warning for hyperglycemia, hypoglycemia, or abnormal fluctuations is generated, specifically as follows:

[0147] Determine the preset threshold based on one or more of the following: medical reference range, individual historical baseline, monitoring scenario, or user settings;

[0148] When the estimated blood glucose value is consistently higher than the hyperglycemia threshold and the duration exceeds a preset time, it is considered a risk of hyperglycemia.

[0149] When the estimated blood glucose value remains below the hypoglycemia threshold for an extended period of time, it is considered a risk of hypoglycemia.

[0150] When the magnitude or slope of the change in the estimated blood glucose value exceeds the preset fluctuation threshold within a short period of time, it is judged as an abnormal fluctuation risk.

[0151] After obtaining continuous blood glucose estimates Furthermore, this invention can also generate derived results such as interval judgment, trend analysis, and risk warning.

[0152] Furthermore, using real blood glucose reference values ​​as a monitoring signal, the candidate band screening parameters, interference decomposition correction parameters, population prior mapping parameters, individual baseline update parameters, and state space recursion parameters are jointly optimized or optimized in stages.

[0153] The candidate band selection parameters include weight parameters in the comprehensive score. and and the number of candidate bands ;

[0154] The interference decomposition and correction parameters include the appearance interference correction coefficient. Thermal state interference correction coefficient Moisture-related interference correction coefficient ;

[0155] The population prior mapping parameters include the population prior mapping function. Model parameters;

[0156] The individual baseline update parameters include the baseline update coefficient. ;

[0157] The state-space recursive parameters include state transition terms. Process noise parameters, observation mapping function Model parameters, observation noise parameters, and update gain ;

[0158] Main loss function Using mean square error form:

[0159] ,

[0160] Where N is the number of training samples or the number of time windows;

[0161] Adding trend consistency loss and smoothing constraint loss ,for:

[0162] ,

[0163] ,

[0164] Total loss function ,for:

[0165] ,

[0166] in, and This represents the weighting parameter.

[0167] The present invention also provides a continuous blood glucose monitoring system, which includes a video acquisition unit, a processing unit, and a display unit; the acquisition unit includes a visible light camera, a multi-band infrared camera, and a thermal infrared camera, used to simultaneously acquire visible light video, multi-band infrared video, and thermal infrared video of the target area of ​​the subject and transmit them to the processing unit; the processing unit executes the continuous blood glucose estimation method of the present invention to obtain continuous blood glucose values ​​and displays them through the display unit.

[0168] The continuous blood glucose monitoring system of the present invention can effectively handle individual differences and multi-source interference by comprehensively utilizing visible light, multi-band infrared and thermal infrared video information, and achieve stable output of continuous blood glucose values. It is non-invasive and non-implantable, making it more practical.

[0169] By performing joint optimization or phased optimization, the model can not only improve the accuracy of numerical regression, but also enhance its trend preservation ability and continuous output stability. Attached Figure Description

[0170] Figure 1 This is a flowchart illustrating the continuous blood glucose value estimation method based on visible light, multi-band infrared, and thermal infrared video of the present invention.

[0171] Figure 2 This is a schematic diagram of the data preprocessing and ROI partitioning process for the continuous blood glucose value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention.

[0172] Figure 3 This is a flowchart illustrating the candidate band selection and mechanism constraint feature construction of the blood glucose continuous value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention.

[0173] Figure 4 This is a flowchart illustrating the interference decomposition and multimodal correction of the continuous blood glucose value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention.

[0174] Figure 5 This is a flowchart illustrating the population prior modeling and individual baseline correction of the blood glucose continuous value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention.

[0175] Figure 6This is a flowchart illustrating the multi-regional consistency assessment and credibility weighting of the blood glucose continuous value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention.

[0176] Figure 7 This is a schematic diagram of the state-space recursion process of the blood glucose continuous value estimation method based on visible light, multi-band infrared and thermal infrared video of the present invention. Detailed Implementation

[0177] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0178] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0179] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0180] This invention discloses a method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video. It is based on synchronous acquisition of multimodal video, uses multi-band infrared information as the main reflectance information source of blood glucose, uses visible light information for surface appearance difference compensation, and uses thermal infrared information for thermal state correction. Furthermore, it combines population prior modeling, individual baseline correction, multi-region consistency assessment, and a continuous recursive update mechanism to achieve stable output of continuous blood glucose values.

[0181] like Figure 1 As shown, the method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video includes the following steps:

[0182] Using a visible light camera, a multi-band infrared camera, and a thermal infrared camera, visible light video, multi-band infrared video, and thermal infrared video are simultaneously acquired from the target area of ​​the subject, and preprocessed to obtain the region of interest (ROI). Preferably, the target area of ​​the subject is the facial area, or one or more of the following: forehead, cheeks, sides of the nose, jaw, in front of the ear, or back of the hand. Other skin areas suitable for non-invasive video acquisition can also be used.

[0183] During the acquisition process, visible light video is used to characterize the color, texture, brightness, and appearance of the skin surface; multi-band infrared video is used to characterize reflectance changes in various bands within the range of 700 nm to 1600 nm; and thermal infrared video is used to characterize changes in body surface temperature, heat distribution, and local perfusion status. The three types of video can be acquired synchronously using a unified trigger signal, aligned after acquisition using a unified timestamp, or sliced ​​synchronously using a unified time window.

[0184] Normalization and region-level processing are performed on video data within the Region of Interest (ROI) to extract video features. Normalization and region-level processing are used to eliminate the influence of illumination changes and device response differences between different time windows, making the features within different time windows comparable, and providing a unified input for subsequent multi-band feature screening, interference correction, and state space modeling.

[0185] The sensitivity of each band in multi-band infrared video data to reference blood glucose values, as well as the stability under different time windows, different ROIs and different individuals, are calculated, and the sensitivity and stability are combined to calculate a comprehensive score.

[0186] Based on the comprehensive score, candidate bands are extracted, and mechanistic constraint features are constructed using video features. The mechanistic constraint features and candidate band features are used together as input features for subsequent interference correction and continuous blood glucose value estimation.

[0187] By utilizing the video features of candidate bands and mechanistic constraints, an interference characterization is constructed, and candidate blood glucose-related features are corrected. The interference characterization is used to describe non-target factors that are not directly related to blood glucose changes but affect video features, including surface appearance differences, local temperature changes, and moisture absorption. By constructing the interference characterization, non-blood glucose-related components can be removed from candidate blood glucose-related features, thereby improving the stability and accuracy of blood glucose estimation.

[0188] Incorporate demographic information to perform prior population modeling and establish an individual baseline correction mechanism;

[0189] Local observation results are generated on multiple ROIs in the target region, and consistency assessment and credibility weighting are performed on the local observation results.

[0190] A state-space recursive method is used to generate continuous blood glucose values.

[0191] The present invention aims to address how to improve the discriminability of blood glucose-related features under non-invasive video conditions, reduce the impact of various interference factors on the estimation results, and establish a continuous blood glucose estimation method that takes into account population difference modeling, individual baseline correction, and continuous and stable output, thereby improving the stability, continuity, and applicability of blood glucose estimation in different populations, environments, and states.

[0192] In a preferred embodiment of the present invention, such as Figure 2 As shown, the method for preprocessing the acquired visible light video, multi-band infrared video, and thermal infrared video to obtain the region of interest (ROI) is as follows:

[0193] Let the time window length be The sliding step size is Then the first The visible light input, multi-band infrared input, and thermal infrared input within each time window are denoted as follows: , and This ensures that different modes have corresponding relationships within the same time period, laying the foundation for subsequent joint processing.

[0194] Preprocessing includes time synchronization, spatial registration, multi-band infrared reflectance correction, thermal infrared temperature calibration, environmental temperature drift compensation, and dead pixel repair to reduce the impact of acquisition differences between different devices and environmental changes. For multi-band infrared data, reflectance correction is performed using white board references and dark field references. The corrected reflectance... ,for:

[0195] ,

[0196] in, Indicates the first The band in the first The original intensity within a time window, Indicates the dark field reference value. Indicates the whiteboard reference value;

[0197] For thermal infrared data, temperature conversion is completed through calibration parameters.

[0198] Temperature drift compensation is performed in conjunction with an ambient temperature reference, specifically as follows:

[0199] The ambient reference temperature is estimated based on the ambient temperature sensor or background area temperature. Offset correction is applied to the thermal infrared measurements within the current time window, and linear or exponential smoothing compensation is performed in conjunction with the time-series temperature change trend to obtain stable thermal state data. The compensated thermal state data is denoted as... In facial scenes, face detection is performed, and the positions of the forehead, left cheek, and right cheek regions are identified using a keypoint localization model. The keypoint localization model is preferably a deep learning-based facial keypoint detection model, including but not limited to the MediaPipe Face Mesh model, the Dlib keypoint detection model, or a keypoint regression model based on a convolutional neural network. These regions are continuously tracked, and areas such as the eyes, eyebrows, mouth, nostrils, hair-covered areas, and areas with obvious shadows are excluded, retaining only stable regions with exposed skin as Regions of Interest (ROIs). In continuous video, the ROI is dynamically updated according to the keypoint positions to ensure that relatively consistent anatomical regions are extracted across different time windows.

[0200] In a preferred embodiment of the present invention, the video data within the Region of Interest (ROI) is normalized and processed at the region level to extract video features, as follows:

[0201] For visible light data, local illumination compensation is performed within the ROI range, and pixel intensity is normalized.

[0202] ,

[0203] in, and They represent the first Mean and standard deviation of ROI within each time window; For Return

[0204] After normalization, appearance-related features are extracted from the visible light video, specifically:

[0205] Within the ROI, the color, brightness, texture, and reflectivity of the visible light image are statistically analyzed to form a visible light feature vector;

[0206] Skin color distribution characteristics are obtained by statistically analyzing the pixel distribution in different color channels or color spaces within the ROI, including color mean, color variance, or color histogram characteristics in RGB, HSV, or YCbCr color spaces.

[0207] The average brightness value is obtained by calculating the average value of the pixel brightness components within the ROI;

[0208] Channel ratios are obtained by calculating the ratios between different color channels, including the red-green channel ratio, the red-blue channel ratio, or the green-blue channel ratio.

[0209] Local texture statistics, including texture contrast, texture uniformity, texture entropy, and local gradient variation features, can be obtained through gray-level co-occurrence matrix, local binary mode, gradient statistics, or other texture description methods.

[0210] The specular reflection characteristics are obtained by statistically analyzing the proportion of bright pixels within the ROI, the proportion of areas with brightness components exceeding a preset threshold, or the proportion of bright but low-saturation areas. Visible light features are used to describe the impact of illumination changes, skin color differences, skin surface texture differences, and local reflectivity on subsequent blood glucose-related features.

[0211] For multi-band infrared data, calculate the regional reflectance characteristics and temporal variation characteristics of each band within the ROI. Let the... The band in the first The ROI region within each time window is Regional reflectivity characteristics of this band It can be represented as:

[0212] ,

[0213] in, Indicates the first The band in the first Within the first time window The reflectance value at each pixel location. This indicates the number of pixels within the ROI.

[0214] Calculate the relative change of this band between adjacent time windows. Or offset relative to the individual baseline This is used to describe the temporal changes in multi-band infrared reflectance characteristics:

[0215] ,

[0216] or:

[0217] ,

[0218] in, Indicates the first The reference reflectance characteristics formed by each band during the baseline establishment phase.

[0219] To mitigate the impact of variations in overall illumination intensity, differences in device response, and local random noise, the regional reflectivity characteristics are normalized and temporally smoothed. This regional reflectivity normalization can be achieved through intra-band normalization, normalization relative to a reference band, or normalization relative to the average reflectivity level across multiple bands. For example, it can be expressed as:

[0220] ,

[0221] in, and They represent the first The mean and standard deviation of regional reflectance characteristics across multiple bands within a time window. To prevent extremely small constants with a denominator of zero. This represents the region-level reflectance characteristics after normalization and temporal smoothing.

[0222] The time series of each band is smoothed using a moving average method. The smoothed result is... Written as:

[0223] ,

[0224] in, Indicates the length of the smooth window; Indicates the first The band in the first Regional reflectance characteristics within a time window; Indicates the first The band in the first Reflectance characteristics smoothed by moving average within a time window. Through the above processing, the reflectance baseline, relative offset information, and temporal variation information related to blood glucose changes are preserved and used as the basic input for subsequent candidate band selection and mechanism constraint feature construction.

[0225] For thermal infrared data, extract thermal state-related features within the ROI. Let the... The ROI within the time window is the first The temperature value at each pixel location The average temperature of the region It can be represented as:

[0226] ,

[0227] The regional temperature difference can be determined by the highest temperature within the ROI. With the lowest temperature difference express:

[0228] ,

[0229] Heat distribution dispersion Represented by the variance or standard deviation of temperature values ​​within the ROI:

[0230] ,

[0231] Thermal gradients can be represented by the temperature changes between adjacent pixels or the average value of the temperature gradient, and are used to describe the spatial distribution of temperature within an ROI.

[0232] rate of change of thermal state It can be represented by the change in average temperature between adjacent time windows:

[0233] ,

[0234] in, and They represent the first The first time window and the first The average temperature of the ROI within each time window. The aforementioned thermal infrared features are used to reflect the current thermal state level, thermal distribution differences, and trends over time in the region, and serve as the basic input for constructing subsequent thermal state disturbance characterizations.

[0235] In a preferred embodiment of the present invention, the sensitivity of each band in multi-band infrared video data to a reference blood glucose value is calculated, as well as its stability under different time windows, different ROIs, and different individuals. The sensitivity and stability are then fused to calculate a comprehensive score. Specifically:

[0236] This invention does not directly input all bands into a unified model with equal weights. Instead, it first selects candidate bands with high blood glucose relevance and good stability from multiple bands. Specifically, during the training phase, the sensitivity of each band to the reference blood glucose value and its stability under different time windows, different ROIs, and different individuals are calculated separately, and the two are combined to form a comprehensive score.

[0237] Let the first The sensitivity score for each band is Stability score: ,in:

[0238] The sensitivity score Used to characterize the The correlation between each band feature and the reference blood glucose value can be obtained through statistical correlation analysis, preferably using Pearson correlation coefficient, Spearman correlation coefficient, or other statistical indicators that can reflect the correlation of variables, specifically expressed as follows:

[0239] ,

[0240] in, Indicates the first The characteristic sequence of each band within each time window This represents a sequence of reference blood glucose values ​​within the corresponding time window;

[0241] The stability score Used to characterize the The degree of fluctuation of a band characteristic under different conditions is obtained by calculating the statistical dispersion of the band characteristic under different time windows, different ROIs, and different individual conditions. Variance, standard deviation, or coefficient of variation is preferably used as a measure, and is specifically expressed as follows:

[0242] ,

[0243] in, and They represent the first Mean and standard deviation of each band feature under multiple time windows, multiple ROIs, and multiple sample conditions. To prevent extremely small constants with a denominator of zero;

[0244] Among them, different time windows, different ROIs and different individual conditions refer to multiple time windows formed during continuous data collection, multiple local ROIs divided within the target area, and individual differences among different subjects.

[0245] Finally, the first Overall score of each band Represented as:

[0246] ,

[0247] in, and is a weighting parameter used to balance the impact of sensitivity and stability in the candidate band selection process.

[0248] In a preferred embodiment of the present invention, such as Figure 3 As shown, the steps for extracting candidate bands and constructing mechanism-constrained features based on comprehensive scoring are as follows:

[0249] In multi-band infrared video data, the extracted regional reflectance features and their time-smoothed features are used as the basic features. The corresponding comprehensive score is calculated for each band and sorted according to the comprehensive score.

[0250] The top scorers are selected based on their overall evaluation. Several candidate bands, among which The preset constants are preferably 3 to 10, forming a candidate band set;

[0251] Based on the candidate band set, a mechanism constraint feature is constructed for any two candidate bands. and The ratio feature, difference feature, and normalized difference feature are constructed respectively as follows:

[0252] ,

[0253] in, and This indicates the regional reflectance characteristics of the candidate band within the current time window, or its characteristics after time smoothing.

[0254] The mechanism constraint features and candidate band features together constitute the input features for subsequent interference correction and continuous blood glucose value estimation, which are used to enhance the expressive power of blood glucose-related information and improve feature stability.

[0255] In this way, single-band reflection information can be transformed into combined features that are more suitable for describing blood glucose-related changes, thereby enhancing the discriminability and stability of subsequent estimations.

[0256] In a preferred embodiment of the present invention, such as Figure 4 As shown, interference characterization is constructed using video features of candidate bands and mechanistic constraint features, and candidate blood glucose-related features are corrected. The specific steps are as follows:

[0257] Since blood glucose-related signals show relatively small variations in video and are easily affected by factors such as skin color, texture, moisture absorption, and local temperature changes, further interference decomposition and correction are needed based on the candidate band features.

[0258] Therefore, this invention utilizes visible light branching and thermal infrared branching to construct interference characterization.

[0259] The visible light branch mainly provides interference information related to surface appearance, including skin color distribution, texture roughness, brightness shift, and local reflection status; the thermal infrared branch mainly provides interference information related to thermal status, including average temperature, local temperature difference, thermal gradient, and thermal drift.

[0260] Let the first Candidate blood glucose-related features extracted within each time window are: The candidate blood glucose-related features are based on candidate band features in multi-band infrared video as the main information source, including regional reflectance features of candidate bands, reflectance features after time smoothing, and ratio features, difference features and normalized difference features constructed from candidate bands.

[0261] Let the first The appearance-related feature vectors extracted from visible light video within each time window are: The appearance-related feature vectors include mean brightness, channel ratio, skin color distribution, texture statistics, and reflectivity features; these feature vectors are converted into appearance interference representations through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now:

[0262] ,

[0263] in, This represents the appearance interference mapping function, whereby the appearance interference is characterized. Used to describe the effects of changes in illumination, skin color differences, surface texture differences, and specular reflection on candidate glucose-related features;

[0264] Let the first The thermal state-related feature vectors extracted from thermal infrared video within each time window are: The thermal state-related feature vector includes regional average temperature, regional temperature difference, thermal gradient, temperature dispersion, and temperature change rate characteristics. Through linear mapping, nonlinear mapping, attention weighting, or neural network mapping, the thermal state-related feature vector is converted into a thermal state disturbance representation. ,Right now:

[0265] ,

[0266] in, This represents the thermal state disturbance mapping function, wherein the thermal state disturbance characterization Used to describe the effects of local temperature changes, regional heat distribution differences, and perfusion status fluctuations on candidate blood glucose-related features;

[0267] Let the first The moisture-related feature vectors extracted from multi-band infrared video within each time window are: The water-related feature vector includes reflectance features of bands sensitive to water absorption and their ratios, differences, or combinations of normalized differences; the water-related feature vector is converted into water-related interference terms through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now:

[0268] ,

[0269] in, This represents the moisture interference mapping function, where the moisture-related interference term... Used to describe the effects of changes in skin moisture content and water absorption on multi-band infrared reflectance characteristics;

[0270] Candidate blood glucose-related features Interference correction was performed to obtain the corrected blood glucose-related features. Its expression is:

[0271] ,

[0272] in, , and The correction coefficient is used to characterize the degree of influence of appearance interference, thermal state interference, and moisture-related interference on candidate blood glucose-related features; when , , or When in vector form, the correction coefficients can be scalars, weight vectors, or mapping matrices, so that each interference characterization has a consistent dimension with the candidate blood glucose-related features;

[0273] Corrected blood glucose-related features This refers to the candidate blood glucose-related features Based on this, the feature representation obtained after stripping away non-target factors such as appearance differences, thermal state changes, and water absorption is used to characterize the changes in multimodal video optical response caused by blood glucose changes, and serves as input for subsequent population prior modeling, individual baseline correction, and state space recursion.

[0274] After the above correction, more residual information related to blood glucose changes but not directly related to surface appearance differences and thermal state fluctuations can be extracted from the original principal features, making subsequent estimations more stable.

[0275] In a preferred embodiment of the present invention, such as Figure 5 As shown, the steps for incorporating demographic information and performing prior population modeling are as follows:

[0276] Considering the differences among individuals in terms of age, gender, body mass index, skin color type, basal metabolic rate, and glucose metabolism risk stratification, demographic information is introduced for population prior modeling.

[0277] Let the demographic information vector be... This includes age, gender, body mass index, skin color classification, and metabolic risk stratification. Discrete variables in the demographic information vector are encoded, and continuous variables are normalized to obtain the corresponding population prior vector. ;

[0278] Let the first The interference-corrected blood glucose-related features within each time window are: The corrected blood glucose-related features Prior vectors of the population The shared input group prior module obtains the current individual's position in the [missing information]. Prior correction for the population within a time window , represented as:

[0279] ,

[0280] in, This represents the mapping function learned from the training data;

[0281] The population prior correction term This method is used to characterize the differences in baseline responses among individuals of different ages, sexes, body mass indexes, skin color patterns, and metabolic risk stratifications under the same or similar corrected blood glucose-related features. It serves as the initial offset constraint in subsequent blood glucose observation mapping or state-space recursion processes. Through this processing, the same video features can correspond to different initial estimation constraints under different age, sex, BMI, or skin color conditions, thereby reducing systematic bias between different populations.

[0282] This invention improves the adaptability of the model when applied across different population groups. By incorporating demographic information such as age, gender, body mass index, and skin color pattern to construct a population prior, and then combining this with individual multimodal reference states to establish an individual baseline, the systematic impact of inherent differences between different population groups on blood glucose estimation results can be reduced, thereby improving the applicability and stability of the model in different populations.

[0283] In a preferred embodiment of the present invention, the method for establishing an individual baseline correction mechanism is as follows:

[0284] After the subject has entered and completed the target area localization and started multimodal video data acquisition (in the initial monitoring phase), the following selections are made. The first time window is used as the initialization phase, and the corrected features of this phase are used to establish a reference baseline for the individual in the current environment and state. Let the first time window be the initialization phase. The corrected candidate features for each initial time window are: Then individual baseline characteristics ,for:

[0285] ,

[0286] For any time window Calculate the offset of its current feature relative to the baseline. :

[0287] ,

[0288] in, Indicates time window Corrected candidate features;

[0289] This invention does not directly use absolute reflectance values ​​to estimate blood glucose, but rather focuses on the changes in the current state relative to an individual's reference state. This significantly reduces the impact of individual differences in factors such as innate skin color, basal temperature, skin thickness, and surface condition.

[0290] During continuous monitoring, individual baselines are updated slowly according to high-quality time windows, and the update method is as follows:

[0291] ,

[0292] in, Indicates the update coefficients; Indicates time window The system establishes individual baseline characteristics. Through this dynamic update mechanism, the system can adapt to changes in the environment and individual states.

[0293] In a preferred embodiment of the present invention, such as Figure 6 As shown, local observation results are generated on multiple ROIs within the target region, and consistency assessment and credibility weighting are performed on these local observation results. The specific method is as follows:

[0294] To reduce the impact of local occlusion, reflection, thermal drift, or abnormal conditions in a single area on the final results, this invention generates local observation results on multiple ROIs, such as the forehead, left cheek, and right cheek, and performs consistency analysis on these local results.

[0295] Let the first The local blood glucose observation values ​​given for the three ROI regions within each time window are as follows: , and The optimal number of ROIs is three, but it is not limited to three. It can also be set to two or more regions depending on the actual situation, thus ensuring regional consistency. Its variance is expressed as:

[0296] ,

[0297] when When the value is small, it indicates that observations across different regions are relatively consistent, and the reliability of the current time window is high; when... A larger value indicates that certain areas may be subject to localized disturbances.

[0298] Weights are assigned to different regions based on image quality, occlusion, reflectivity, and thermal stability. This forms the weighted observations for that time window. :

[0299] ,

[0300] in, Represents the number of ROIs, and satisfies ;r represents the ROI index. This method reduces the impact of outlier regions on the overall estimation results and simultaneously generates a confidence score for the current time window.

[0301] This invention can improve robustness under local anomaly conditions. Through multi-region consistency assessment and confidence weighting mechanism, when a local area is affected by occlusion, reflection, or local thermal drift, its impact on the final result can be automatically reduced, thereby reducing the damage of local anomalies to the overall estimation result.

[0302] In a preferred embodiment of the present invention, such as Figure 7 As shown, a state-space recursive method is used to generate continuous blood glucose values, specifically:

[0303] After obtaining the weighted observations for the current time window, instead of directly outputting blood glucose values ​​using window-by-window independent regression, continuous blood glucose values ​​are generated using the state-space recursion approach.

[0304] Let the first The true blood glucose status within each time window is It changes slowly over time, and is represented as:

[0305] ,

[0306] in, Represents a state transition term. Indicates process noise;

[0307] Let the first The interference-corrected blood glucose-related features within each time window are: Individual baseline characteristics are The offset of the current feature relative to the individual baseline. Represented as:

[0308] ,

[0309] in, Used to indicate the degree of change in blood glucose-related characteristics relative to the individual's reference state within the current time window;

[0310] Observation model under the current time window Represented as:

[0311] ,

[0312] in, This represents the observation mapping function composed of baseline offset features and population priors. Indicates observation noise. This represents the prior vector of the population. This represents the current time window blood glucose observation result obtained jointly by the corrected blood glucose-related feature offset and the population prior;

[0313] Based on the estimated state from the previous moment and the observation results from the current moment, the blood glucose estimate for the current moment is updated as follows:

[0314] ,

[0315] in, This is the updated blood glucose estimate for the current moment. This represents the update gain, the size of which can be adjusted based on the reliability, self-consistency, and state stability of the current time window; the higher the quality of the current time window, the better. The larger the value, the stronger the impact of the current observation on the result update; the worse the quality of the current time window, The smaller the value, the more the system relies on the steady state of the previous moment. This recursive method can effectively suppress jumps caused by instantaneous noise and local anomalies, making the output continuous blood glucose value more consistent with the actual change pattern.

[0316] This invention improves output stability in continuous monitoring scenarios. Because the system does not output independently using a single time window, but updates blood glucose estimation results through a continuous recursive approach, it reduces output jumps caused by instantaneous noise, local modal quality degradation, and short-term abnormal disturbances, making the continuous value sequence smoother and more reliable.

[0317] In a preferred embodiment of the present invention, according to The current value is used to determine whether it is in the normal, high or low range, and the direction of change between adjacent time windows is used to determine whether it is in an upward, downward or stable trend.

[0318] Let the slope of the change between two adjacent time windows be:

[0319] ,

[0320] Then when When the value is greater than a preset threshold, it is judged as an upward trend; when When the value is less than the negative threshold, it is judged as a downward trend; otherwise, it is judged as a stable trend.

[0321] By combining the fluctuation range over a continuous period, the duration of deviation from the individual's baseline, and the reliability of the current time window, a risk warning for hyperglycemia, hypoglycemia, or abnormal fluctuations is generated, specifically as follows:

[0322] Determine the preset threshold based on one or more of the following: medical reference range, individual historical baseline, monitoring scenario, or user settings;

[0323] When the estimated blood glucose value is consistently higher than the hyperglycemia threshold and the duration exceeds a preset time, it is considered a risk of hyperglycemia.

[0324] When the estimated blood glucose value remains below the hypoglycemia threshold for an extended period of time, it is considered a risk of hypoglycemia.

[0325] When the magnitude or slope of the change in the estimated blood glucose value exceeds the preset fluctuation threshold within a short period of time, it is judged as an abnormal fluctuation risk.

[0326] The entire process revolves around the estimation of continuous blood glucose values, with other results being further analyses based on these continuous values.

[0327] In a preferred embodiment of the present invention, the real blood glucose reference value is used as a monitoring signal to jointly optimize or optimize the candidate band screening parameters, interference decomposition correction parameters, population prior mapping parameters, individual baseline update parameters and state space recursion parameters in stages.

[0328] The candidate band selection parameters include weight parameters in the comprehensive score. and and the number of candidate bands ;

[0329] The interference decomposition and correction parameters include the appearance interference correction coefficient. Thermal state interference correction coefficient Moisture-related interference correction coefficient ;

[0330] The population prior mapping parameters include the population prior mapping function. Model parameters;

[0331] The individual baseline update parameters include the baseline update coefficient. ;

[0332] The state-space recursive parameters include state transition terms. Process noise parameters, observation mapping function Model parameters, observation noise parameters, and update gain ;

[0333] Main loss function Using mean square error form:

[0334] ,

[0335] Where N is the number of training samples or the number of time windows;

[0336] To ensure that the continuous output results are as close as possible to the actual blood glucose values, while also exhibiting good trend consistency and time smoothness, a trend consistency loss is added. and smoothing constraint loss ,for:

[0337] ,

[0338] ,

[0339] Total loss function ,for:

[0340] ,

[0341] in, and This represents the weight parameters. Through this training method, the model can not only improve the accuracy of numerical regression, but also enhance its trend-preserving ability and continuous output stability.

[0342] This invention forms a complete technical route around continuous blood glucose value estimation, namely, firstly, multimodal synchronous acquisition and preprocessing are completed, then ROI selection and regional feature extraction are performed, on this basis, candidate band screening and mechanism-constrained feature construction are carried out, then the main features are decomposed and corrected for interference through visible light and thermal infrared information, and a more stable blood glucose-related characterization is obtained by combining population prior modeling and individual baseline correction, and finally, the stable output of continuous blood glucose values ​​is achieved through multi-region consistency assessment and state space recursion, and further results such as interval, trend and risk indication are generated.

[0343] This invention improves the distinguishability of blood glucose-related features. By using multi-band infrared as the primary source of sensitive information for blood glucose and organizing band information through candidate band screening, ratio difference construction, and mechanism-constrained feature construction, it can not only utilize the effective information of each band itself, but also further utilize the difference relationship, ratio relationship, and normalized offset relationship between bands to form a more targeted joint expression, thereby enhancing the characterization ability of blood glucose-related optical changes.

[0344] This invention can reduce the interference of non-target factors on blood glucose estimation results. By using visible light to characterize surface appearance interference and thermal infrared light to characterize temperature and thermal state interference, and then correcting the main blood glucose sensitivity information in multi-band infrared light, it is beneficial to reduce the influence of skin color, texture, surface condition, moisture absorption, temperature changes and perfusion state changes on the estimation results, thereby improving the system's anti-interference ability.

[0345] This invention enables the creation of output formats more suitable for practical applications. Since the system uses continuous blood glucose values ​​as its core output and can further derive interval judgments, trend analyses, and risk alerts, it is more suitable for use in scenarios such as daily health management, glucose metabolism risk screening, exercise recovery monitoring, and family health monitoring.

[0346] This invention improves the discriminability, stability, continuity, and practical applicability of non-invasive video blood glucose estimation by using multi-band infrared as the primary source of sensitive blood glucose information, with visible light and thermal infrared serving as auxiliary correction, and combining population prior modeling, individual baseline correction, and a continuous recursive output mechanism.

[0347] This invention also provides a continuous blood glucose monitoring system, comprising a video acquisition unit, a processing unit, and a display unit. The acquisition unit includes a visible light camera, a multi-band infrared camera, and a thermal infrared camera, used to simultaneously acquire visible light video, multi-band infrared video, and thermal infrared video of the target area of ​​the subject and transmit them to the processing unit. The processing unit executes the continuous blood glucose estimation method of this invention to obtain continuous blood glucose values, which are then displayed by the display unit. This continuous blood glucose monitoring system effectively handles individual differences and multi-source interference by comprehensively utilizing visible light, multi-band infrared, and thermal infrared video information, and achieves stable output of continuous blood glucose values. It is non-invasive and non-implantable, making it more practical.

[0348] The specific embodiments described herein are merely illustrative examples of the present invention. Those skilled in the art can make various modifications or additions to the described embodiments or use similar methods to substitute them, without departing from the technology of the present invention or exceeding the scope defined by the appended claims.

[0349] In the embodiments of this application, terms such as "fixed," "fixed connection," and "fixed connection" refer to common fixing methods in the prior art, such as welding, riveting, and screws. "Rotary connection" refers to common rotary connection methods in the prior art, such as hinges and bearing rotation. If electrical components are provided, the functions, control, and power supply methods of all electrical components are common technical means in the prior art. This application has not improved them and they are not within the protection scope of this application. Therefore, this application will not elaborate on them.

[0350] Furthermore, the selection of materials and strength limitations for all components in this application can be made and arranged by those skilled in the art based on the site environment and the requirements of relevant national or industry standards, and are not within the scope of protection of this application. Therefore, this application will not elaborate on these points.

Claims

1. A method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video, characterized in that, Includes the following steps: Using a visible light camera, a multi-band infrared camera, and a thermal infrared camera, visible light video, multi-band infrared video, and thermal infrared video are simultaneously acquired of the target area of ​​the subject, and preprocessed to obtain the region of interest (ROI). Normalize and perform region-level processing on the video data within the Region of Interest (ROI) to extract video features; The sensitivity of each band in multi-band infrared video data to reference blood glucose values, as well as the stability under different time windows, different ROIs and different individuals, are calculated, and the sensitivity and stability are combined to calculate a comprehensive score. Based on the comprehensive score, candidate bands are extracted, and mechanism-constrained features are constructed using video features; By utilizing video features of candidate bands and mechanistic constraint features, interference characterization is constructed, and candidate blood glucose-related features are corrected; demographic information is introduced to perform population prior modeling, and an individual baseline correction mechanism is established. Local observation results are generated on multiple ROIs in the target region, and consistency assessment and credibility weighting are performed on the local observation results. A state-space recursive method is used to generate continuous blood glucose values.

2. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, The target area for the subject is the facial area, or one or more of the following: forehead, cheeks, sides of the nose, jaw, in front of the ear, or back of the hand.

3. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, The method for preprocessing the acquired visible light video, multi-band infrared video, and thermal infrared video to obtain the region of interest (ROI) is as follows: Let the time window length be The sliding step size is Then the first The visible light input, multi-band infrared input, and thermal infrared input within each time window are denoted as follows: , and ; For multi-band infrared data, reflectance correction is performed using white-field and dark-field references. The corrected reflectance... ,for: ， in, Indicates the first The band in the first The original intensity within a time window, Indicates the dark field reference value. Indicates the whiteboard reference value; For thermal infrared data, temperature conversion is performed through calibration parameters, and temperature drift compensation is performed in conjunction with an ambient temperature reference. Specifically: The ambient reference temperature is estimated based on the ambient temperature sensor or background area temperature. Offset correction is applied to the thermal infrared measurements within the current time window, and linear or exponential smoothing compensation is performed in conjunction with the time-series temperature change trend to obtain stable thermal state data. The compensated thermal state data is denoted as... ; In facial scenes, face detection is performed, and the location of the forehead, left cheek, and right cheek areas is identified using a key point localization model. These areas are continuously tracked, and areas such as eyes, eyebrows, mouth, nostril edges, hair-covered areas, and obvious shadow areas are excluded. Only stable areas with exposed skin are retained as regions of interest (ROIs).

4. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, The following method is used to normalize and perform region-level processing on the video data within the Region of Interest (ROI) to extract video features: For visible light data, local illumination compensation is performed within the ROI range, and pixel intensity is normalized. ， in, and They represent the first Mean and standard deviation of ROI within each time window; The pixel intensity after normalization; Visible light input; After normalization, appearance-related features are extracted from the visible light video, specifically: Within the ROI, the color, brightness, texture, and reflectivity of the visible light image are statistically analyzed to form a visible light feature vector; Skin color distribution characteristics are obtained by statistically analyzing the pixel distribution in different color channels or color spaces within the ROI, including color mean, color variance, or color histogram characteristics in RGB, HSV, or YCbCr color spaces. The average brightness value is obtained by calculating the average value of the pixel brightness components within the ROI; Channel ratios are obtained by calculating the ratios between different color channels, including the red-green channel ratio, the red-blue channel ratio, or the green-blue channel ratio. Local texture statistics, including texture contrast, texture uniformity, texture entropy, and local gradient variation features, can be obtained through gray-level co-occurrence matrix, local binary mode, gradient statistics, or other texture description methods. The specular reflection characteristics are obtained by statistically analyzing the proportion of bright pixels within the ROI, the proportion of areas where the brightness component exceeds a preset threshold, or the proportion of bright but low-saturation areas. For multi-band infrared data, the regional reflectance characteristics and temporal variation characteristics of each band are calculated within the ROI. Let the _i_th band be the first band. The band in the first The ROI region within each time window is Regional reflectivity characteristics of this band Represented as: ， in, Indicates the first The band in the first Within the first time window The reflectance value at each pixel location. Indicates the number of pixels within the ROI; Calculate the relative change of this band between adjacent time windows. Or offset relative to the individual baseline Used to describe the temporal changes of multi-band infrared reflectance characteristics: ， or: ， in, Indicates the first Reference reflectance characteristics formed by each band during the baseline establishment phase; The regional reflectivity characteristics are normalized and temporally smoothed. Regional reflectivity normalization is achieved through intra-band normalization, normalization relative to a reference band, or normalization relative to the average reflectivity level of multiple bands, expressed as: ， in, and They represent the first The mean and standard deviation of regional reflectance characteristics across multiple bands within a time window. To prevent extremely small constants with a denominator of zero; This represents the region-level reflectance characteristics after normalization and temporal smoothing. The time series of each band is smoothed using a moving average method. The smoothed result is... for: ， in, Indicates the length of the smooth window; Indicates the first The band in the first Regional reflectance characteristics within a time window; Indicates the first The band in the first Reflectance characteristics smoothed by moving average within a time window; For thermal infrared data, thermal state-related features are extracted within the ROI range. Let the first... The ROI within the time window The temperature value at each pixel location The average temperature of the region Represented as: ， The regional temperature difference is determined by the highest temperature within the ROI. With the lowest temperature difference express: ， Heat distribution dispersion Represented by the variance or standard deviation of temperature values ​​within the ROI: ， Thermal gradients are represented by the temperature changes between adjacent pixels or the average value of temperature gradient magnitudes, and are used to describe the spatial distribution of temperature within an ROI. rate of change of thermal state Represented by the change in average temperature between adjacent time windows: ， in, and They represent the first The first time window and the first The average temperature of the ROI within a time window.

5. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, The sensitivity of each band in multi-band infrared video data to reference blood glucose values ​​was calculated, as well as its stability under different time windows, different ROIs, and different individuals. The sensitivity and stability were then combined to calculate a comprehensive score. Specifically: Let the first The sensitivity score for each band is Stability score: ,in: The sensitivity score Used to characterize the The correlation between each band characteristic and the reference blood glucose value can be obtained through statistical correlation analysis, and is specifically expressed as follows: ， in, Indicates the first The characteristic sequence of each band within each time window This represents a sequence of reference blood glucose values ​​within the corresponding time window; The stability score Used to characterize the The degree of fluctuation of a band feature under different conditions is obtained by calculating the statistical dispersion of the band feature under different time windows, different ROIs, and different individual conditions, specifically expressed as follows: ， in, and They represent the first Mean and standard deviation of each band feature under multiple time windows, multiple ROIs, and multiple sample conditions. To prevent extremely small constants with a denominator of zero; Finally, the first Overall score of each band Represented as: , in, and This is a weighting parameter used to balance the impact of sensitivity and stability in the candidate band selection process.

6. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 5, characterized in that, The steps for extracting candidate bands and constructing mechanism-constrained features based on comprehensive scoring are as follows: In multi-band infrared video data, the extracted regional reflectance features and their time-smoothed features are used as the basic features. The corresponding comprehensive score is calculated for each band and sorted according to the comprehensive score. The top scorers are selected based on their overall evaluation. Several candidate bands, among which A set of candidate bands is formed by setting a preset constant. Based on the candidate band set, a mechanism constraint feature is constructed for any two candidate bands. and The ratio feature, difference feature, and normalized difference feature are constructed respectively as follows: ， in, and This indicates the regional reflectance characteristics of the candidate band within the current time window, or its characteristics after time smoothing.

7. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, By utilizing video features of candidate bands and mechanistic constraints, an interference characterization is constructed, and candidate blood glucose-related features are corrected. The specific steps are as follows: Let the first Candidate blood glucose-related features extracted within each time window are: This includes regional reflectance characteristics of candidate bands, reflectance characteristics after time smoothing, and ratio characteristics, difference characteristics, and normalized difference characteristics constructed from candidate bands. Let the first The appearance-related feature vectors extracted from visible light video within each time window are: The appearance-related feature vectors include mean brightness, channel ratio, skin color distribution, texture statistics, and reflectivity features; these feature vectors are converted into appearance interference representations through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now: ， in, Represents the appearance interference mapping function, and the appearance interference characterization. Used to describe the effects of changes in illumination, skin color differences, surface texture differences, and specular reflection on candidate glucose-related features; Let the first The thermal state-related feature vectors extracted from thermal infrared video within each time window are: The thermal state-related feature vector includes regional average temperature, regional temperature difference, thermal gradient, temperature dispersion, and temperature change rate characteristics. Through linear mapping, nonlinear mapping, attention weighting, or neural network mapping, the thermal state-related feature vector is converted into a thermal state disturbance representation. ,Right now: ， in, This represents the thermal state disturbance mapping function, wherein the thermal state disturbance characterization Used to describe the effects of local temperature changes, regional heat distribution differences, and perfusion status fluctuations on candidate blood glucose-related features; Let the first The moisture-related feature vectors extracted from multi-band infrared video within each time window are: The water-related feature vector includes reflectance features of bands sensitive to water absorption and their ratios, differences, or combinations of normalized differences; the water-related feature vector is converted into water-related interference terms through linear mapping, nonlinear mapping, attention weighting, or neural network mapping. ,Right now: ， in, This represents the moisture interference mapping function, where the moisture-related interference term... Used to describe the effects of changes in skin moisture content and water absorption on multi-band infrared reflectance characteristics; Candidate blood glucose-related features Interference correction was performed to obtain the corrected blood glucose-related features. Its expression is: ， in, , and The correction coefficient is used to characterize the degree of influence of appearance interference, thermal state interference, and moisture-related interference on candidate blood glucose-related features; when , , or When in vector form, the correction coefficients can be scalars, weight vectors, or mapping matrices, so that each interference characterization has the same dimension as the candidate blood glucose-related features; The corrected glucose-related features Refers to candidate blood glucose-related features Based on this, the feature expression obtained after removing non-target factors such as appearance differences, thermal state changes and moisture absorption is used to characterize the changes in multimodal video optical response caused by blood glucose changes.

8. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, The steps for incorporating demographic information to perform prior population modeling are as follows: Let the demographic information vector be... This includes age, gender, body mass index, skin color classification, and metabolic risk stratification. Discrete variables in the demographic information vector are encoded, and continuous variables are normalized to obtain the corresponding population prior vector. ; Let the first The interference-corrected blood glucose-related features within each time window are: Corrected blood glucose-related features Prior vectors of the population The shared input group prior module obtains the current individual's position in the [missing information]. Prior correction for the population within a time window , is represented as: ， in, This represents the mapping function learned from the training data; The population prior correction term It is used to characterize the differences in baseline response among individuals of different ages, sexes, body mass indexes, skin color types, and metabolic risk stratifications under the same or similar corrected blood glucose-related characteristics, and serves as the initial offset constraint in subsequent blood glucose observation mapping or state space recursion processes.

9. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 8, characterized in that, The method for establishing an individual baseline correction mechanism is as follows: After the subjects completed target area localization and began multimodal video data acquisition, the previous... The first time window is used as the initialization phase, and the corrected features of this phase are used to establish a reference baseline for the individual in the current environment and state. Let the first time window be the initialization phase. The corrected candidate features for each initialization time window are: Then individual baseline characteristics ,for: ， For any time window Calculate the offset of its current feature relative to the baseline. : ， in, Indicates time window Corrected candidate features; During continuous monitoring, individual baselines are updated slowly according to high-quality time windows, and the update method is as follows: ， in, Indicates the update coefficients; Indicates time window Individual baseline characteristics.

10. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, Local observation results are generated on multiple ROIs within the target region, and consistency assessment and credibility weighting are performed on these local observation results. The specific method is as follows: Let the first The local blood glucose observation values ​​given for the three ROI regions within each time window are as follows: , and Then regional consistency Its variance is expressed as: ， Weights are assigned to different regions based on image quality, occlusion, reflectivity, and thermal stability. This forms the weighted observations for that time window. : ， in, Represents the number of ROIs, and satisfies ;r represents the ROI number.

11. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 1, characterized in that, A state-space recursive method is used to generate continuous blood glucose values, specifically as follows: Let the first The true blood glucose status within each time window is It changes slowly over time, and is represented as: ， in, Represents a state transition term. Indicates process noise; Let the first The interference-corrected blood glucose-related features within each time window are: Individual baseline characteristics are The offset of the current feature relative to the individual baseline. Represented as: ， in, Used to indicate the degree of change in blood glucose-related characteristics relative to the individual's reference state within the current time window; Observation model under the current time window Represented as: ， in, This represents the observation mapping function composed of baseline offset features and population priors. Indicates observation noise. This represents the prior vector of the population. This represents the current time window blood glucose observation result obtained jointly by the corrected blood glucose-related feature offset and the population prior; Based on the estimated state from the previous moment and the observation results from the current moment, the blood glucose estimate for the current moment is updated as follows: ， in, This is the updated blood glucose estimate for the current moment. This represents the update gain, the size of which can be adjusted based on the reliability, self-consistency, and state stability of the current time window; the higher the quality of the current time window, the better. The larger the value, the stronger the impact of the current observation on the result update; the worse the quality of the current time window, The smaller the value, the more the system depends on the stable state of the previous moment.

12. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 11, characterized in that, according to The current value is used to determine whether it is in the normal, high or low range, and the direction of change between adjacent time windows is used to determine whether it is in an upward, downward or stable trend. Let the slope of the change between two adjacent time windows be: ， Then when When the value is greater than a preset threshold, it is judged as an upward trend; when When the value is less than the negative threshold, it is judged as a downward trend; otherwise, it is judged as a stable trend. By combining the fluctuation range over a continuous period, the duration of deviation from the individual's baseline, and the reliability of the current time window, a risk warning for hyperglycemia, hypoglycemia, or abnormal fluctuations is generated, specifically as follows: Determine the preset threshold based on one or more of the following: medical reference range, individual historical baseline, monitoring scenario, or user settings; When the estimated blood glucose value is consistently higher than the hyperglycemia threshold and the duration exceeds a preset time, it is considered a risk of hyperglycemia. When the estimated blood glucose value remains below the hypoglycemia threshold for an extended period of time, it is considered a risk of hypoglycemia. When the magnitude or slope of the change in the estimated blood glucose value exceeds the preset fluctuation threshold within a short period of time, it is judged as an abnormal fluctuation risk.

13. The method for estimating continuous blood glucose values ​​based on visible light, multi-band infrared, and thermal infrared video according to claim 11, characterized in that, Using real blood glucose reference values ​​as a monitoring signal, the candidate band screening parameters, interference decomposition correction parameters, population prior mapping parameters, individual baseline update parameters, and state space recursion parameters are jointly or in stages optimized. The candidate band selection parameters include weight parameters in the comprehensive score. and and the number of candidate bands ; The interference decomposition and correction parameters include the appearance interference correction coefficient. Thermal state interference correction coefficient Moisture-related interference correction factor ; The population prior mapping parameters include the population prior mapping function. Model parameters; The individual baseline update parameters include the baseline update coefficient. ; The state-space recursive parameters include state transition terms. Process noise parameters, observation mapping function Model parameters, observation noise parameters, and update gain ; Main loss function Using mean square error form: ， Where N is the number of training samples or the number of time windows; Adding trend consistency loss and smoothing constraint loss ,for: ， ， Total loss function ,for: ， in, and This represents the weighting parameter.

14. A continuous glucose monitoring system, characterized in that, It includes a video acquisition unit, a processing unit, and a display unit; The acquisition unit includes a visible light camera, a multi-band infrared camera, and a thermal infrared camera, which are used to simultaneously acquire visible light video, multi-band infrared video, and thermal infrared video of the target area of ​​the subject and transmit them to the processing unit. The processing unit performs the method described in any one of claims 1-13 to obtain continuous blood glucose values ​​and displays them through the display unit.

Images