Hyperspectral-thermal imaging diabetic foot multi-point risk assessment method and system based on condition number adaptive unmixing

Abstract

The invention relates to a hyperspectral-thermal imaging diabetic foot multi-point risk assessment method and system based on condition number adaptive unmixing, and belongs to the technical field of medical images and artificial intelligence. According to the system, a plantar hyperspectral data cube and a thermal image are synchronously collected, a linear model of hemoglobin chromophores and a scattering substrate is established through the expansion correction Beer-Lambert law, non-negative constraint least square with robust loss is adopted for solving, and a physiological parameter graph and a pixel-level confidence graph are output; performing temperature calibration, foot segmentation and double-foot registration on the thermal image, combining Getis-Ord Gi * space statistics and DBSCAN anomaly cluster detection, and calculating a thermal anomaly index; physiological and thermal features are extracted from a plurality of ROIs such as heels and metatarsal bones based on a foot anatomy template, and an interpretable machine learning model is input to output risk grades, early warning positions and confidence coefficients. The method is oriented to noninvasive and automatic diabetic foot early risk early warning, and has higher robustness and repeatability in a complex nerve-ischemic lesion scene.

Description

A Hyperspectral-Thermal Imaging Method and System for Multi-Point Risk Assessment of Diabetic Foot Based on Condition Number Adaptive Unmixing Technical Field

[0001] This invention belongs to the field of medical imaging and artificial intelligence technology, and relates to a hyperspectral-thermal imaging method and system for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing. Background Technology

[0002] Diabetic foot ulcer (DFU) is one of the most serious and common complications of diabetes, imposing a heavy economic burden on global healthcare systems and severely impacting patients' quality of life. Epidemiological studies show that the lifetime risk of developing foot ulcers in diabetic patients is approximately 19% to 34%, and about 50% of DFUs will become infected, with 15% to 20% of moderate to severe infections ultimately requiring lower limb amputation. The core pathophysiological mechanism of DFU is a complex "triad": the interaction of neuropathy, ischemia / vascular disease, and infection is key to its development. Microcirculatory and endothelial dysfunction appears early and affects tissue oxygen metabolism and repair. Microcirculatory dysfunction plays a crucial initiating role in the development of DFU, with pathological changes including vascular endothelial damage, autonomic nervous system dysregulation, impaired vasodilation, and decreased blood flow reserve. These changes ultimately lead to tissue ischemia and hypoxia, energy metabolism disorders, thereby triggering nerve damage and ulcer formation. Therefore, accurate and sensitive detection of early microcirculatory dysfunction before irreversible tissue damage and ulceration occur is crucial for preventing DFU.

[0003] However, existing routine clinical screening methods have significant limitations in detecting early, functional lesions. For example, while methods such as using the 10g Semimes-Weinstein monofilament test (10g-MF) to assess sensory neuropathy or measuring the ankle-brachial index (ABI) to assess large vessel disease are standard procedures recommended by clinical guidelines, they primarily reflect existing structural damage or severe functional loss, and are insufficiently sensitive to early microcirculatory functional disorders before ulceration. Furthermore, although transcutaneous oxygen partial pressure (TcPO2) measurement can reflect local oxygen supply, it is cumbersome, time-consuming, and a contact-based measurement, unsuitable for large-scale, rapid screening scenarios. Existing research indicates that only 32.1% of general practitioners explicitly state that they always or frequently perform foot risk screening for diabetic patients, and the annual screening coverage rate in some countries ranges from 15.7% to 64.8%, showing a generally low screening rate.

[0004] In recent years, optical imaging technology has shown great potential in the field of non-invasive medical diagnostics. Among them, NIRS typically operates at 700-1000 nm and can quantitatively quantify... And estimate Related to hemodynamic parameters. However, NIRS alone cannot obtain surface temperature information closely related to autonomic nerve function and inflammatory status. On the other hand, infrared thermography (IRT) technology can generate temperature distribution maps by capturing infrared radiation on the surface of the foot, effectively identifying hot spots caused by inflammation or cold spots caused by local ischemia. Clinical studies have confirmed that a temperature difference of more than 2.2°C between corresponding areas of the left and right feet is a strong indicator of impending ulceration. Nevertheless, IRT cannot penetrate the skin and cannot provide crucial information about oxygen metabolism in deep tissues.

[0005] Furthermore, one of the most complex clinical challenges is the ambiguity of diagnosis, particularly in neuro-ischemic lesions. These patients present with both nerve damage and insufficient vascular supply, and single-modality imaging results can be contradictory: NIRS may show low tissue oxygen saturation (ischemic signal), while IRT may anomalously show high temperature (inflammatory signal) due to inflammatory response or arteriovenous shunting. This combination of "hypoxia + hyperthermia" represents an extremely high risk of ulceration, but any single-modality analysis may lead to an incomplete or even erroneous diagnosis.

[0006] Therefore, there is an urgent need in this field for a novel technical solution that can overcome all the above-mentioned deficiencies: such a solution should be able to (1) integrate deep physiological information (NIRS) and surface functional information (IRT); and (2) intelligently fuse multimodal data through advanced algorithms to resolve diagnostic ambiguities and learn complex pathological patterns from the data. Such a system will be able to achieve truly predictive screening, rather than just diagnosis, thereby meeting a significant unmet clinical need in the field of DFU prevention. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a hyperspectral-thermal imaging method and system for multi-point risk assessment of diabetic foot based on conditional number adaptive unmixing. This method and system address the problems of existing diabetic foot screening technologies, such as insensitivity to early microcirculatory dysfunction, limitations of single-modal information, and inability to resolve ambiguities in complex pathological diagnoses. It provides an automated, non-invasive assessment system and method that integrates hyperspectral imaging, thermal imaging, and machine learning. This system extracts and intelligently fuses multi-dimensional physiological and thermal information from key anatomical regions of the foot to achieve early and accurate assessment of the risk of diabetic foot ulcers, thereby providing a highly efficient, reliable, and widely deployable preventive assessment tool for clinical use.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing, the method specifically includes the following steps: S1, Subject preparation and environmental steady state: The subject's feet are placed in a temperature and humidity controlled detection environment and kept still or in a static position for no less than a preset time to achieve thermal equilibrium; S2, Equipment calibration, spatial mapping and synchronous triggering: Dark field / whiteboard correction is performed on the near-infrared hyperspectral imaging unit and emissivity and environmental compensation calibration is performed on the infrared thermal imaging unit; the spatial mapping relationship between the two modes is calculated based on the factory calibration parameters or calibration board; synchronous triggering of the two modal devices is achieved through the control unit and millisecond-level timestamp annotation is performed; S3, Baseline multimodal data acquisition: Near-infrared hyperspectral data cube and infrared thermal image are acquired synchronously in a resting state, and recorded... Record metadata such as environmental parameters, imaging angle, and subject ID during data acquisition; S4, Spectral preprocessing, physiological parameter inversion, and confidence assessment: Perform dark / white correction, smoothing, and detrending processing on hyperspectral data; construct an extended modified Beer-Lambert law model including hemoglobin chromophores and scattering substrate; use non-negative constrained least squares solution with robust loss to obtain oxyhemoglobin, deoxyhemoglobin, total hemoglobin, and tissue oxygen saturation parameter maps; monitor the condition number of the weighted normal equation during the solution process and adaptively adjust band weights or remove abnormal bands based on residuals; output pixel-level confidence maps based on residual norm and leverage value; S5, Thermal image preprocessing and thermal anomaly detection: perform noise suppression and temperature calibration on infrared thermal images; complete foot segmentation and bipedal registration; calculate temperature difference indices for corresponding areas of the bipedalities; utilize Getis-Ord... Gi* spatial statistics and density-based clustering algorithms identify hot / cold spots and calculate thermal anomaly indices to quantify and rank candidate abnormal regions; S6, ROI definition and multi-point mapping: Based on a preset foot anatomy template, several regions of interest (ROIs) are automatically or semi-automatically divided, and the spatial mapping relationship is used to map the ROIs between hyperspectral parameter maps and thermal parameter maps; S7, feature extraction and oxygen-thermal discrepancy index construction: Statistical / kinetic features derived from NIRS and statistical / textural features derived from IRT are extracted within each ROI, and based on tissue oxygen saturation and thermal anomaly indices, a multi-point mapping index is constructed. An oxygen-thermal contradiction index is established to quantify cross-modal contradiction patterns such as "hypoxia + hyperthermia"; features of each ROI are concatenated with the oxygen-thermal contradiction index to form a fused feature vector; S8, risk assessment and interpretable output based on machine learning: the fused feature vector is input into a pre-trained classification or regression model, and the risk score or risk level, the location and confidence of suspected abnormal ROIs are output, along with interpretable information based on feature importance; S9, quality control, report generation and storage: when the pixel-level confidence is lower than a preset threshold, bipedal registration fails, or spatial mapping error exceeds the limit, a resampling prompt or short-term repeated sampling is triggered;The system automatically generates visual reports and stores raw data, analysis results, timestamps, quality control information, and subject IDs in a local or cloud database for follow-up and historical comparative analysis.

[0009] Furthermore, in step S4, during the solution process, robust regression with NNLS combined with Huber loss is adopted, and an adaptive band weight strategy and condition number threshold monitoring are introduced; when the condition number exceeds the threshold, the system reduces the weight of high residual bands or removes noisy bands to ensure the stability of the solution.

[0010] Furthermore, in step S5, thermal anomaly detection simultaneously employs Getis-Ord Gi* statistics and DBSCAN anomaly cluster detection, and quantifies and sorts candidate anomaly regions based on the Thermal Abnormality Index (TAI); when the temperature difference between the ROIs corresponding to both feet reaches a preset threshold or the TAI exceeds a preset threshold, the system outputs a high-risk warning.

[0011] Furthermore, in step S7, the oxygen-thermal contradiction index is composed of at least the tissue oxygen saturation statistics within the ROI and the thermal abnormality index within the ROI, and is used to characterize the cross-modal combination pattern of "hypoxia + high temperature" or "hypoxia + low temperature" under complex pathological conditions such as neuro-ischemic conditions; the machine learning model uses the oxygen-thermal contradiction index as an explicit input feature for risk classification and grading.

[0012] Furthermore, in step S9, quality control is based on a comprehensive judgment of pixel-level confidence map, image saturation, motion artifact index, and registration error; when quality control fails, the system records the reason for failure in the report and provides resampling suggestions related to the acquisition posture.

[0013] The present invention also provides a hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing. The system includes a near-infrared hyperspectral imaging unit, an infrared thermal imaging unit, an integrated illumination and foot fixation bracket, a data acquisition and control unit, a data processing and analysis unit, and a human-computer interaction display and report output module; the data processing and analysis unit is configured to perform the method described above.

[0014] Furthermore, the near-infrared hyperspectral imaging unit covers the 700-930 nm range and has ≥200 bands; the infrared thermal imaging unit is a long-wave infrared thermal imager with a thermal sensitivity NETD≤50 mK and supports at least 640×480 pixel resolution; the data processing and analysis unit includes a multi-core CPU and a dedicated GPU or NPU to accelerate preprocessing and model inference.

[0015] Furthermore, the data acquisition and control unit includes a synchronization triggering and timestamp module, a cross-modal spatial mapping and registration module, and an ROI template management module, which are used to realize the corresponding analysis of hyperspectral parameter maps and thermal parameter maps on the same anatomical region.

[0016] Furthermore, the system also includes a plantar imaging ablation device and an environmental parameter detection unit, used to record ambient temperature and humidity and compensate for thermal image temperature to improve the repeatability of cross-batch acquisition.

[0017] Furthermore, the system also includes a quality control module, which is used to automatically detect image frame loss, oversaturation, registration failure, spatial mapping error or motion artifacts, and provide resampling suggestions or specific operation prompts through touch screen and voice prompts when quality control abnormalities are detected.

[0018] The beneficial effects of this invention are as follows: 1) Improved diagnostic accuracy and information dimension: This invention constructs a more comprehensive physiological and pathological profile by integrating deep tissue oxygen metabolism information provided by NIRS and surface temperature distribution information provided by IRT.

[0019] 2) It resolves the diagnostic ambiguity of complex pathologies and enhances diagnostic robustness: This invention extracts features from multiple key anatomical points and uses a machine learning model for intelligent fusion. This model can be trained to identify complex pathological patterns that may produce contradictory signals under a single modality, such as the unique high-risk feature of "hypoxia" and "hyperthermia" coexisting in neuro-ischemic foot lesions. This capability is significantly superior to separate single-modal analysis or simple feature superposition, thereby greatly improving the overall diagnostic accuracy and robustness in complex clinical scenarios.

[0020] 3) Achieves high interpretability and clinical applicability: Unlike deep learning "black box" models, this invention adopts a path based on feature engineering and classical machine learning, making its diagnostic decision-making process more interpretable. Clinicians can trace which specific physiological indicators (such as "high temperature in the first metatarsal head") in which specific regions contribute most to the final risk assessment, which helps build clinical trust and facilitates integration with existing medical knowledge.

[0021] 4) Achieves a non-contact, efficient, and automated screening process: The entire system is designed for non-contact operation, avoiding the risk of cross-infection and improving patient acceptance. Combining standardized anatomical region definitions and automated feature extraction and analysis processes, this invention enables rapid and repeatable screening, making it ideal for large-scale, routine diabetic foot risk screening in primary care clinics or large hospitals, overcoming the limitations of existing technologies that are cumbersome and reliant on expert experience.

[0022] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the preferred embodiments of this invention will be described in detail below with reference to the accompanying drawings, in which: Figure 1 is a schematic diagram of the overall structure of the multi-point screening system for diabetic foot according to this invention; Figure 2 is a flowchart of the data acquisition process of the diabetic foot screening system according to this invention; Figure 3 is a flowchart of the multi-band near-infrared hyperspectral data processing and physiological parameter calculation process according to this invention; Figure 4 is a flowchart of the infrared thermal imaging foot thermal abnormality detection method according to this invention; Figure 5 is a flowchart of the multimodal feature fusion and machine learning method for diabetic foot diagnosis according to this invention; Figure 6 is a schematic diagram of the output module and clinical application according to this invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] Figure 1 is a schematic diagram of the overall structure of the multi-point screening system for diabetic foot according to the present invention. Referring to Figure 1, the present invention provides a multi-point screening system for diabetic foot based on hyperspectral imaging and infrared thermal imaging. Its overall architecture adopts a modular design, integrating a hyperspectral imaging unit (101), an infrared thermal imaging unit (102), an illumination and fixation bracket (103), a data acquisition and control unit (104), a human-computer interaction and safety module (105), a plantar imaging ablation device (106), and data and video transmission cables (107). The functional modules are interconnected through standardized hardware interfaces and a unified software platform to form a complete intelligent screening platform.

[0026] Specifically, the near-infrared hyperspectral imaging unit (101) includes an industrial-grade hyperspectral camera with a spectral coverage range of 700~930 nm, capable of acquiring images in more than 200 narrowband bands. The camera has a resolution of at least 1280×1024 pixels and a frame rate of up to 30 FPS. The hyperspectral camera is connected to the data acquisition unit via a USB 3.0 or PCIe interface to achieve high-bandwidth image data transmission.

[0027] Specifically, the infrared thermal imaging unit (102) is a long-wave infrared thermal imager with a spectral range covering 8~14μm. It has a thermal sensitivity of less than 50mK (NETD) and a temperature measurement accuracy of ±0.5℃, capable of distinguishing clinically significant temperature differences of ≥2.0℃ in diabetic foot ulcers. This unit uses an uncooled focal plane array (FPA) detector, supports a resolution of at least 640×480 pixels, and has a frame rate of ≤50Hz. The infrared thermal imager and the hyperspectral camera are physically arranged in parallel or coaxially integrated to ensure that their imaging fields of view are basically overlapped, so as to achieve spatial registration of cross-modal data.

[0028] Specifically, the integrated lighting and mounting bracket (103) includes a ring-shaped near-infrared LED lighting source and a lamp source bracket. The lighting source uses a high-power narrowband LED with a wavelength matching the sensitive range of a hyperspectral camera (700–930 nm), and the illumination uniformity is ≥90%.

[0029] Specifically, the data acquisition and control unit (104) is an embedded industrial computer, configured with a multi-core CPU (≥8 cores) and an independent GPU module, and equipped with 4×USB3.0, 2×Gigabit Ethernet, and 5G / Wi-Fi wireless communication interfaces. This unit is responsible for unified timing management, data acquisition, cache storage, and preliminary processing of multimodal hardware. Internally, it runs a Linux-based real-time operating system, supporting hardware interrupt triggering and high-precision timestamp marking. The GPU is used to accelerate hyperspectral data preprocessing and machine learning inference computation.

[0030] Specifically, the human-computer interaction and safety module (105) includes a high-brightness touchscreen of ≥10.1 inches, which displays hyperspectral pseudo-color images and thermal distribution maps in real time. The interface supports multi-language switching and touch operation. The system also integrates a voice broadcast module and an audible and visual alarm, which can prompt the operator to make adjustments in real time when there is a posture deviation, signal saturation, or temperature exceeding the threshold. This module complies with the IEC 60601 electrical safety standard for medical devices. The shell is made of medical-grade ABS + rubber composite material, and the protection level reaches IP54, ensuring the safety and durability of the equipment in hospital outpatient clinics, wards, or screening sites.

[0031] Specifically, the foot imaging ablation device (106) is an integrated black background panel made of a black composite material with a high extinction coefficient. This provides a uniform, low-reflection background during foot photography, effectively eliminating stray light and environmental interference, thereby improving image contrast and the accuracy of subsequent feature extraction. The device is designed with a hollow support structure, allowing the subject to insert their foot and maintain a natural posture, ensuring a stable and consistent background throughout the shooting process.

[0032] Specifically, the data and video transmission cable (107) includes a high-speed data cable connecting the hyperspectral imaging unit (101) and the infrared thermal imaging unit (102) to the data acquisition and control unit (104), and a video signal cable connecting the data acquisition and control unit (104) to the human-computer interaction and security module (105). The high-speed data cable preferably uses an industrial-grade USB 3.0 or gigabit Ethernet cable with a locking mechanism to ensure the stability and integrity of image data during transmission; the video signal cable uses a high-quality HDMI or DisplayPort cable to ensure that visualization results such as pseudo-color images and heat maps can be presented on the display screen in high definition and with low latency. All cables are shielded to reduce electromagnetic interference and ensure the overall stable and reliable operation of the system.

[0033] Referring to Figure 2, this invention further provides a data acquisition method based on the above system. This method, through a standardized experimental procedure, ensures the accuracy and repeatability of the collected data, thereby meeting the needs of subsequent clinical analysis and intelligent judgment. The overall process includes four stages: subject preparation, equipment calibration, data acquisition, data storage, and quality control. Specifically, the subject preparation steps include: the subject sits quietly in a temperature-controlled testing environment for at least 15 minutes to allow the body to reach thermal equilibrium. The laboratory temperature range can be controlled between 22 and 25°C, and the relative humidity maintained between 40% and 60%. During this period, the subject must expose both feet to avoid the influence of tight shoes and socks on blood flow.

[0034] Specifically, the device calibration steps include: after starting the system, firstly, performing dark current and whiteboard calibration on the hyperspectral imaging unit to calculate the true spectral reflectance and eliminate interference from sensor noise and uneven illumination; then, calibrating the emissivity and ambient temperature of the infrared thermal imaging unit, wherein the emissivity of human skin is set to 0.98 to ensure temperature measurement accuracy.

[0035] Specifically, the data acquisition steps include: placing the subject's feet on an illumination and fixation support, and cooperating with a foot imaging ablation device to ensure a clean and uniform background. Under the control unit's scheduling, the system synchronously triggers the hyperspectral imaging unit and the infrared thermal imaging unit, acquiring a multi-band spectral cube and thermal image matrix of the foot in a resting state.

[0036] Specifically, the data storage steps include: after the acquisition is completed, the data acquisition and control unit stores the hyperspectral and infrared thermal image files in a standardized format, and adds the subject's basic information, acquisition time and environmental conditions to form a complete data record.

[0037] Specifically, the quality control steps include: after the acquisition is completed, the system automatically runs a data integrity detection program. If any abnormalities are found, such as missing frames in the image, signal oversaturation, or registration failure, the system will prompt the user on the interactive interface and require re-acquisition, thereby ensuring that the data entering subsequent analysis all meet the quality requirements.

[0038] Please refer to Figure 3. The present invention further provides a data processing and physiological parameter calculation method suitable for multi-band near-infrared hyperspectral imaging. The method includes, from bottom to top: spectral calibration and preprocessing, effective optical path modeling, extended MBLL spectral unmixing, selection of quantitative bands and condition number control for blood oxygen and total hemoglobin, and quality control.

[0039] Specifically, the spectral calibration and preprocessing steps include: for each pixel at the Wavelength ( The original grayscale under) Perform dark field / white board normalization to obtain reflectivity. : in For pixels At wavelength The original grayscale; For dark field reference; Use a whiteboard as a reference. Then calculate the absorbance: Savitzky-Golay smoothing and mild detrending were applied to the spectral sequences to suppress high-frequency noise and slow baseline drift.

[0040] Specifically, the effective optical path modeling step, based on the definition of detection geometry and tissue scattering, is... Effective optical path length in the band: in, The equivalent average propagation distance related to the system's illumination-imaging geometry can be determined in the following ways: First, by integrating a ranging / displacement sensor into the system to measure the distance between the light source / camera and the tissue surface in real time and converting it into the equivalent average propagation distance. Secondly, a fixed bracket / limiting structure is used to maintain a constant shooting distance during data acquisition, so that... Take it as a preset constant. The specific value can be determined by combining calibration experiments, Monte Carlo simulations, or empirical models. This is a wavelength-dependent diffusion path factor. To balance accuracy and feasibility, A quadratic polynomial approximation can be used: in For reference wavelength, Obtained through preliminary experiments or prior fitting from literature; it can also be updated at the individual level through short-term calibration. .

[0041] Specifically, the extended MBLL spectral unmixing step includes: setting the chromatin basis vector as... ;exist Constructing an extinction coefficient matrix across the bands: its first Various groups of people The molar extinction coefficient at that point, Let be the number of chromatic groups. Further define the optical path diagonal matrix: To achieve absorption-scattering separation, a low-dimensional scattering substrate is introduced. (like or (Form) to characterize the slowly varying scattering background. Therefore, the linear model for the pixel-level spectrum is written as: in This refers to relative absorbance. This is the healthy control area. The relative concentration of chromophores. The scattering coefficient is... The residuals are obtained. Noise-weighted and regularized non-negative constrained least squares are used to solve for all bands: in This is the band weight matrix. These are the Tikhonov regularity coefficients. This is used to suppress excessively large parameters and improve stability. In the preferred implementation, robust regression using NNLS combined with Huber loss is employed to resist anomalous bands and motion artifacts.

[0042] Specifically, the blood oxygen and total hemoglobin quantification steps obtain absolute or relative concentrations from the unmixing results. , Calculate total hemoglobin: And calculate tissue oxygen saturation: After completing the calculations at the pixel level, the median value of the ROI and the IQR are generated based on the foot anatomical partitions to improve statistical robustness.

[0043] Specifically, the band selection and condition number control steps, in order to avoid Ill-conditioned, the system monitors the weighted normal equation in real time. The condition number ( ).when At that time, an adaptive band weighting strategy is activated, which maximizes the coverage of the hemoglobin-sensitive area through a greedy algorithm. This involves adjusting the weight of noise bands or removing abnormal bands to ensure robustness and reversibility, while preserving... The main contribution of band information.

[0044] Specifically, the quality control and confidence assessment step outputs the residual norm for each pixel. The system evaluates the leverage value to generate a pixel-level confidence map. When the confidence level falls below a threshold or the motion artifact exceeds a threshold, the system prompts for resampling or automatically triggers short-term resampling. A bootstrap method is used to assess the 95% confidence interval for time-series indicators as part of the clinical report.

[0045] Referring to Figure 4, this invention further provides a data processing and thermal anomaly detection method based on infrared thermal imaging. This method is used to analyze the surface temperature distribution and abnormal lesion characteristics of the foot in diabetic foot patients. The method includes five steps: image preprocessing, foot region segmentation, temperature field calibration, thermal feature extraction, and anomaly detection.

[0046] Specifically, the image preprocessing step involves noise suppression and pseudo-color restoration of the original infrared thermal image. Gaussian filtering or nonlocal mean filtering is used to reduce thermal noise interference, and the grayscale thermal image is mapped to a temperature matrix. ,in Represents pixel coordinates.

[0047] Specifically, the foot region segmentation step uses a boundary condition algorithm to remove non-foot regions. The resulting foot region mask is denoted as... Multiplying it by the temperature matrix yields the effective temperature map of the foot.

[0048] Specifically, the temperature field calibration step maps the grayscale values ​​output by the infrared sensor to absolute temperature based on the calibration curve of the imaging device. An environmental compensation factor is also introduced. Used to correct for the effects of room temperature, humidity, and background radiation: in, To compensate for the temperature bias in the environment, The temperature value is obtained from the initial conversion.

[0049] Specifically, the thermal feature extraction calculates the statistical temperature parameters of the entire foot and local areas, including: (1) average temperature (2) Maximum temperature (3) Minimum temperature (4) Temperature difference index (5) Symmetrical differences between the two feet .in The statistical range is the pre-selected ROI region. The ROI region number is used. A value greater than 2.2℃ is generally considered to indicate abnormal blood supply or potential infection foci.

[0050] Specifically, in addition to comparing both feet, the hot / cold spot detection system further performs statistical anomaly analysis on local areas within a single foot. Localized high temperatures are usually associated with inflammation and infection, while localized low temperatures may indicate ischemia.

[0051] (1) Statistical Hotspot Analysis (Getis-Ord Gi*): Identifies significant high- or low-value clusters using spatial statistical methods. The Gi* statistic is defined as: in, It is the target pixel index. It is a neighborhood pixel index. For the temperature of neighboring pixels, This is the spatial weight matrix. The global average temperature. Standard deviation The number of effective pixels within the study area, For pixels and Euclidean distance, For the cutoff radius, The attenuation scale. If... At the significance level If the value is positive, the pixel and its neighborhood are identified as a "hot spot area"; if the value is negative, it is identified as a "cold spot area".

[0052] (2) Density-based anomaly detection (DBSCAN): Treating hot pixels as two-dimensional coordinates The combined data with temperature values ​​is used to divide the data into normal and abnormal clusters using the DBSCAN clustering algorithm. Most normal areas are clustered into the main cluster, while pixels with extremely high or low temperatures are identified as small clusters or noise points because they deviate from the normal clusters, thus enabling the detection of asymmetric or early minor anomalies.

[0053] Specifically, the anomaly detection and discrimination involves calculating the Thermal Abnormality Index (TAI) to quantitatively evaluate candidate regions. in, The average temperature of the suspected lesion area. The temperature of the ROI corresponding to the contralateral foot. This represents the standard deviation of the control area.

[0054] Referring to Figure 5, this invention further provides a multimodal feature fusion and machine learning diagnostic method. This method, based on dual-modal information from near-infrared hyperspectral data and infrared thermal imaging data, enables early diagnosis and risk stratification of diabetic foot. The method includes four steps: feature extraction, feature fusion, classification and regression modeling, and diagnostic report generation.

[0055] Specifically, the NIRS feature extraction involves unmixing near-infrared hyperspectral data and modeling it with physiological parameters to obtain dynamic features such as blood oxygen saturation (StO2), total hemoglobin (HbT), oxyhemoglobin (HbO2), and deoxyhemoglobin (HHb). Simultaneously, band-level statistical features (mean, standard deviation, principal component score) are extracted to form a structured spectral feature vector.

[0056] Specifically, the IRT feature extraction involves preprocessing and segmenting the foot infrared thermal image to extract global and local temperature features, including average temperature, maximum temperature, minimum temperature, bipedal symmetry difference, hot / cold spot distribution index (Gi*), and thermal anomaly index (TAI). Furthermore, gray-level co-occurrence matrix texture features (contrast, homogeneity, entropy) are extracted to enhance the discriminative power of local thermal patterns.

[0057] Specifically, the construction of the oxygen-thermal contradictory index involves explicitly quantifying the contradictory combination of "hypoxia + hyperthermia" under complex conditions such as neuro-ischemic states. The system constructs an Oxygen-Thermal Contradictory Index (OTCI) within each Region of Interest (ROI). Preferably, the OTCI can be composed of the standardized missing value of tissue oxygen saturation and the standardized deviation of the thermal anomaly index, for example: in For the Sigmoid function, This is the thermal anomaly threshold. Blood oxygen threshold , The OTCI is a scale parameter; when the OTCI exceeds the preset threshold, the system marks the ROI as a high-risk conflict mode and outputs the corresponding OTCI value and triggering reason in the report.

[0058] Specifically, the multimodal feature fusion standardizes the NIRS and IRT feature vectors, and then performs dimensionality reduction through feature concatenation and principal component analysis (PCA) / linear discriminant analysis (LDA). The fused feature vectors retain both spectral dynamic information and thermal and spatial distribution characteristics, thereby achieving cross-modal complementarity.

[0059] Specifically, during feature fusion, the NIRS and IRT features of each ROI are concatenated with the OTCI as explicit input features; to maintain interpretability, the system simultaneously provides the contribution of the OTCI to risk classification in the model inference output.

[0060] Specifically, the classification and regression modeling incorporates fused features into various classic machine learning models, including Support Vector Machine (SVM), Random Forest (RF), and Gradient Boosting Decision Tree (GBDT). Model outputs include: (1) Risk grading: low / medium / high risk; (2) Lesion type discrimination: inflammatory, ischemic, or mixed; (3) Physiological parameter regression prediction: quantitative indicators such as StO2, HbT, HbO2, and HHb. To enhance robustness, ensemble learning strategies based on Bagging or Stacking are preferred, fusing results from multiple classifiers.

[0061] Specifically, the diagnostic report is generated automatically by the system based on the output of the machine learning model. The report includes: (1) numerical physiological parameters and statistical intervals; (2) a map of abnormal temperature areas in the foot; (3) risk grading conclusions and corresponding confidence levels; and (4) suggestions for possible lesion types and early intervention.

[0062] Specifically, the report should ideally include: StO2 / HbT / temperature difference / TAI statistics for each ROI, pixel-level confidence level summary, and OTCI list with corresponding hot / cold spot location markings, to support doctors in quickly interpreting risk patterns of "ischemia-predominant / inflammation-predominant / contradictory mixed".

[0063] Specifically, this method trains and validates on a multimodal database containing sufficient labeled samples, uses cross-validation to evaluate generalization ability, and performs interpretability analysis through feature importance ranking (such as the Gini index based on RF or the weight vector of SVM) to ensure that the final output has clinical understandability and credibility.

[0064] Referring to Figure 6, this invention further provides an embodiment of system output and clinical application. After completing multimodal data acquisition, feature extraction, and fusion analysis, the system can autonomously generate clinically relevant diagnostic outputs and decision support content, thereby directly serving the clinical needs of doctors and patients.

[0065] Specifically, the system output module includes the following parts: (1) Quantitative parameter output: The system can output key physiological and thermal parameters in numerical and tabular form, such as near-infrared spectral indicators and infrared thermal imaging indicators. Infrared spectral indicators include blood oxygen saturation, total hemoglobin concentration, oxyhemoglobin, deoxyhemoglobin, etc.; infrared thermal imaging indicators include bipedal temperature difference, temperature asymmetry index, hot spot / cold spot distribution characteristics, etc., and thermal anomaly index TAI, etc. Furthermore, the system can also output the oxygen-thermal contradiction index OTCI and its triggering cause in each region of interest (ROI) to display and characterize contradictory patterns such as "low blood oxygen + high temperature".

[0066] (2) Risk grading output: The system outputs the risk level of diabetic foot based on a pre-trained machine learning classification / regression model, and provides a confidence score for each level. The confidence level ranges from 0 to 1, and doctors can adjust the diagnostic reference weights according to the confidence level.

[0067] (3) Lesion type indication: The system can automatically distinguish the type of inflammation, ischemic or mixed lesions based on multimodal features and indicate them in the output as text labels. For example, "Left and right plantar regions: ischemic abnormality, high risk, confidence level 0.87".

[0068] (4) Visualized reports: The system can generate standardized reports, including visualization of hot / cold spot distribution in IRT thermal images; comparison of NIRS feature curves with key spectral indicators; and ranking of feature importance output by machine learning models to help doctors understand the basis of model judgment.

[0069] (5) Long-term follow-up and database integration: All results output by the system can be automatically stored in the electronic medical record database and support integration with hospital information systems, image archiving and communication systems, facilitating cross-departmental and cross-time point case management. This function supports dynamic monitoring of patients, generating longitudinal follow-up curves for efficacy evaluation and prognosis prediction.

[0070] (6) Quality control and re-sampling prompts: Output quality control pass / fail, reasons for failure and re-sampling suggestions.

[0071] Specifically, the clinical applications include, but are not limited to, the following scenarios: (1) Outpatient screening: rapid examination in high-risk groups of diabetic foot to provide immediate risk assessment and help doctors decide whether further imaging or vascular examination is needed; (2) Inpatient monitoring: regular monitoring of NIRS and IRT data during hospitalization of diabetic foot patients to dynamically track foot perfusion and thermal status and evaluate the effectiveness of treatment plans; (3) Community and home scenarios: deployment of this system through portable devices to enable follow-up by community doctors or home self-testing, combined with cloud analysis results to provide timely warnings of potential complications; (4) Clinical research: the standardized multimodal indicators output by the system can be used for the construction of large-scale clinical databases to provide high-quality data support for disease classification research and prognostic model construction.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing, characterized in that: The method specifically includes the following steps: S1, Subject preparation and environmental stabilization: The subject's feet are placed in a temperature and humidity controlled testing environment and the subject sits or remains still for at least a preset time to achieve thermal equilibrium; S2, Equipment calibration, spatial mapping, and synchronous triggering: Dark field / whiteboard calibration is performed on the near-infrared hyperspectral imaging unit, and emissivity and environmental compensation calibration is performed on the infrared thermal imaging unit; the spatial mapping relationship between the near-infrared hyperspectral imaging mode and the infrared thermal imaging mode is calculated based on the factory calibration parameters or calibration board; synchronous triggering of the two modes of equipment is achieved through the control unit, and millisecond-level timestamp annotation is performed; S3, Baseline multimodal data acquisition: Near-infrared hyperspectral data cubes and infrared thermal images are acquired synchronously in a resting state, and environmental parameters, imaging angle, and subject information are recorded during acquisition. ID metadata; S4, Spectral preprocessing, physiological parameter inversion and confidence assessment: Dark / white correction, smoothing and detrending processing are performed on hyperspectral data. An extended modified Beer-Lambert law model including hemoglobin chromophores and scattering substrate is constructed. Non-negative constrained least squares solution with robust loss is used to obtain the parameters of oxyhemoglobin, deoxyhemoglobin, total hemoglobin and tissue oxygen saturation. During the solution process, the condition number of the weighted normal equation is monitored and the band weights are adaptively adjusted or abnormal bands are removed based on the residuals. Pixel-level confidence maps are output based on the residual norm and leverage value; S5, Thermal image preprocessing and thermal anomaly detection: Noise suppression and temperature calibration are performed on infrared thermal images. Foot segmentation and bipedal registration are completed, and the temperature difference index of the corresponding areas of the bipedalities is calculated. Getis-Ord is used... Gi* spatial statistics and density-based clustering algorithms identify hot / cold spots and calculate thermal anomaly indices to quantify and rank candidate anomaly regions; S6, ROI definition and multi-point mapping: Based on a preset foot anatomy template, several regions of interest (ROIs) are automatically or semi-automatically divided, and the spatial mapping relationship is used to map the ROIs between hyperspectral parameter maps and thermal parameter maps; S7, feature extraction and oxygen-thermal contradiction index construction: Statistical / kinetic features derived from NIRS and statistical / textural features derived from IRT are extracted within each ROI, and an oxygen-thermal contradiction index is constructed based on tissue blood oxygen saturation and thermal anomaly indicators to quantify the "low blood oxygen + high temperature" cross-modal contradiction pattern; Each ROI feature is concatenated with the oxygen-thermal contradiction index to form a fused feature vector; S8, Risk assessment and interpretable output based on machine learning: The fused feature vector is input into a pre-trained classification or regression model, and the risk score or risk level, the location and confidence of suspected abnormal ROIs are output, along with interpretable information based on feature importance; S9, Quality control, report generation and storage: When the pixel-level confidence is lower than a preset threshold, bipedal registration fails, or spatial mapping error exceeds the limit, a resampling prompt or short-term repeated sampling is triggered; The system automatically generates a visual report and stores the original data, analysis results, timestamps, quality control information, and subject IDs in a local or cloud database for follow-up and historical comparative analysis.

2. The hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing as described in claim 1, characterized in that: In step S4, during the solution process, robust regression using non-negative least squares (NNLS) combined with Huber loss is adopted, and an adaptive band weighting strategy and condition number threshold monitoring are introduced. When the condition number exceeds the threshold, the system reduces the weight of high residual bands or removes noisy bands to ensure the stability of the solution.

3. The hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing as described in claim 2, characterized in that: In step S5, the thermal anomaly detection simultaneously employs Getis-Ord Gi* hotspot statistics and density-based spatial clustering of applications with noise (DBSCAN) to detect anomaly clusters, and quantifies and sorts candidate anomaly regions based on the Thermal Abnormality Index (TAI). When the temperature difference between the ROIs corresponding to both feet reaches a preset threshold or the TAI exceeds a preset threshold, the system outputs a high-risk warning.

4. The hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing as described in claim 3, characterized in that: In step S7, the oxygen-thermal contradiction index is composed of at least the tissue oxygen saturation statistics within the ROI and the thermal abnormality index within the ROI, and is used to characterize the cross-modal combination pattern of "hypoxia + high temperature" or "hypoxia + low temperature" under complex pathological conditions such as neuro-ischemic conditions; the machine learning model uses the oxygen-thermal contradiction index as an explicit input feature for risk classification and grading.

5. The hyperspectral-thermal imaging method for multi-point risk assessment of diabetic foot based on condition number adaptive unmixing according to claim 4, characterized in that: In step S9, quality control is based on a comprehensive judgment of pixel-level confidence map, image saturation, motion artifact index, and registration error. When quality control fails, the system records the reason for failure in the report and provides resampling suggestions related to the acquisition posture.

6. A hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing, characterized in that: The system includes a near-infrared hyperspectral imaging unit, an infrared thermal imaging unit, an integrated lighting and foot fixation bracket, a data acquisition and control unit, a data processing and analysis unit, and a human-computer interaction display and report output module; the data processing and analysis unit is configured to perform the method of any one of claims 1 to 5.

7. A hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing as described in claim 6, characterized in that: The near-infrared hyperspectral imaging unit covers the 700-930 nm range and has ≥100 bands; the infrared thermal imaging unit is a long-wave infrared thermal imager with a thermal sensitivity NETD≤50 mK and supports at least 640×480 pixel resolution; the data processing and analysis unit includes a multi-core CPU and a dedicated GPU or NPU to accelerate preprocessing and model inference.

8. A hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing according to claim 6, characterized in that: The data acquisition and control unit includes a synchronization triggering and timestamp module, a cross-modal spatial mapping and registration module, and an ROI template management module, which are used to realize the corresponding analysis of hyperspectral parameter maps and thermal parameter maps on the same anatomical region.

9. A hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing according to claim 6, characterized in that: The system also includes a plantar imaging ablation device and an environmental parameter detection unit, used to record ambient temperature and humidity and compensate for thermal image temperature to improve the repeatability of cross-batch acquisition.

10. A hyperspectral-thermal imaging multi-point risk assessment system for diabetic foot based on condition number adaptive unmixing according to claim 6, characterized in that: The system also includes a quality control module, which is used to automatically detect image frame loss, oversaturation, registration failure, spatial mapping error or motion artifacts, and provide re-sampling suggestions or specific operation prompts through touch screen and voice prompts when quality control abnormalities are detected.

Images