Mobile terminal diabetic foot risk assessment system and method based on multi-parameter fusion

Abstract

The invention provides a mobile terminal diabetic foot risk assessment system and method based on multi-parameter fusion. The system comprises a data acquisition module, a feature extraction module, a fusion module, a model deployment and reasoning module and a risk assessment module. The data acquisition module is used for acquiring a foot infrared image, a temperature matrix and structured physiological data; the feature extraction module extracts temperature, texture and physiological data features of the image; the feature fusion module carries out fusion optimization on multi-source features through standardization, PCA dimension reduction and an attention mechanism; the model deployment and reasoning module performs risk prediction by adopting machine learning algorithms such as a support vector machine model and outputs a diabetic foot risk probability; and the risk assessment module displays the risk level and the temperature heat map on the mobile terminal in an image-text mode. The intelligent diabetic foot screening method can realize intelligent diabetic foot screening in home and grassroots scenes, and has high accuracy, convenience and good popularization value.

Description

Technical Field

[0001] The present invention relates to the technical field of diabetic foot risk assessment, and in particular to a diabetic foot risk assessment system and method for completing image acquisition, information fusion and intelligent reasoning on a mobile device. Background Art

[0002] Diabetic foot (DF) is a foot ulcer or deep tissue damage caused by distal lower limb neuropathy or vascular disease in patients with diabetes mellitus (DM). The "Guidelines for the Diagnosis and Treatment of Diabetic Foot in China (2024 Edition)" states that DF is a serious chronic complication that can lead to disability and death in diabetic patients. It has a high incidence rate, is difficult to treat, and is extremely expensive. The guidelines also state that "prevention of DF is better than cure. Strengthening the management of high-risk diabetic foot and conducting early screening for high-risk individuals can prevent or delay the onset of DF." Early screening and timely intervention for DF are crucial for preventing complications.

[0003] In recent years, many diabetic foot screening devices and methods have emerged. For example, the Chinese patent "CN202510144430.9-A diabetic foot screening device" proposes a new screening device that achieves rapid detection of the patient's feet by designing key components such as a base, a detection plate, acupuncture components and a drive mechanism. However, the device uses a needle puncture detection method, which may cause pain and discomfort to the patient, and there are problems with a single detection method and insufficient data dimensions, making it difficult to meet the needs of more accurate diabetic foot screening. The Chinese patent "202411595210.X-Diabetic foot risk assessment method and system based on big data" proposes to collect multi-dimensional physiological data, build a personalized health baseline model to predict the safety baseline, and compare physiological characteristics with the baseline to assess the risk of diabetic foot, which has advantages in early warning and personalized health management. However, this method has significant shortcomings: First, it does not test the key physiological indicator of diabetic patients, the foot temperature, which is closely related to the occurrence and development of diabetic foot. The lack of data in this dimension will affect the comprehensiveness of the assessment; second, because the system data requires the joint collection of multiple devices, it is not suitable for diabetic patients to self-test at home. It lacks convenience and practicality, and it is difficult to meet the daily self-monitoring needs of patients. There are limitations in actual application scenarios, and improvements are needed to enhance the detection dimension and applicability. Existing screening methods rely on professional equipment and medical staff, making them difficult to popularize at the grassroots level and in home scenarios. There are also technical bottlenecks such as a single dimension and uncorrected temperature interference. Therefore, there is an urgent need for a risk assessment system that supports mobile deployment and integrates multi-source information. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a diabetic foot risk assessment system and method based on multi-parameter fusion, adopts a modular structural design, constructs a complete intelligent screening process, realizes integrated collection, processing and display on the mobile terminal, and overcomes the problems of insufficient dimensions and poor scene adaptability of existing technologies.

[0005] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is:

[0006] A diabetic foot risk assessment system based on multi-parameter fusion includes five modules: data acquisition module, feature extraction module, feature fusion module, model deployment and reasoning module, and risk assessment module.

[0007] The data acquisition module is responsible for acquiring infrared images of the soles of the feet, visible light images, temperature matrices, and the user's structured physiological data, such as age, gender, course of disease, and complications. The images are collected through infrared thermal imaging equipment with wireless communication capabilities, and the structured data are filled in through electronic forms.

[0008] The feature extraction module is used to process image and temperature data, extract texture features (such as contrast, entropy, energy, correlation) and temperature gradients in the region of interest; standardize structured data and unify feature formats.

[0009] The feature fusion module combines image features, temperature features, and structural features into a unified feature vector, performs dimensionality reduction through PCA, and further introduces an attention mechanism to automatically assign importance weights to different features, thereby improving overall expressiveness.

[0010] The model inference module loads a trained support vector machine (SVM) model, predicts the fused features, and outputs a probability value for diabetic foot risk. The model uses a Gaussian kernel function to improve classification capabilities and Platt Scaling to convert the results into a probability score, which is then deployed and run locally on the mobile device.

[0011] The risk assessment module divides risks into low, medium and high levels based on the prediction results, and combines heat maps and key indicators to generate an assessment report with pictures and text on the mobile terminal for users to view and save.

[0012] A mobile terminal diabetic foot risk assessment method based on multi-parameter fusion includes the following steps:

[0013] Step 1: Collect infrared images, visible light images, and temperature matrices of the plantar surface, and input structured physiological data, including age, gender, disease course, and complications;

[0014] Step 2: Feature extraction and standardization: Extract the infrared image texture features including contrast, entropy, energy, and correlation, extract the temperature gradient within the selected region of interest (ROI), extract the temperature and structured data feature pairs and perform standardization, feature fusion, and dimensionality reduction.

[0015] Step 3: Concatenate the image and the structured feature vector, perform dimensionality reduction, introduce the attention mechanism to weight the feature importance, and output the fused feature vector;

[0016] Step 4: Model inference: load the pre-trained machine learning model, input the fused feature vector for inference, and map the output risk probability;

[0017] Step 5: Results display and reporting: Finally, the risk level, risk score and heat map are displayed on the human-computer interaction interface, key indicator summaries are summarized, and an assessment report is generated and saved.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention uses infrared thermal imaging, which is fast, non-contact, and non-invasive. It is simple and easy to operate, and can complete diabetic foot risk screening without the need for professional operation, thereby improving grassroots universality.

[0020] The present invention collects multiple parameters including infrared images, temperature information, and structured physiological data, and through feature screening and weight fusion, uses a lightweight machine learning algorithm to improve the accuracy of diabetic foot risk prediction.

[0021] The present invention supports visual output of results, interpretation of results, and tracking of risk trends. It is suitable for various application scenarios such as home self-testing and primary healthcare, and has good social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for use in the embodiments:

[0023] Figure 1 It is a schematic diagram of the overall structure of the present invention;

[0024] Figure 2 is a collection environment diagram in one embodiment;

[0025] Figure 3 A diagram showing a data collection and preprocessing structure in one embodiment;

[0026] Figure 4 This is a diagram of the multimodal feature fusion structure of the present invention;

[0027] Figure 5 This is a diagram of the model training and reasoning execution structure of the present invention;

[0028] Figure 6 The present invention is a flowchart of a mobile terminal diabetic foot risk assessment method based on multi-parameter fusion in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. The following embodiments only reflect several implementation forms of the present invention, and their descriptions are relatively specific and detailed, but this does not mean that they can be regarded as limiting the scope of protection of the patent of the present invention. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0030] The various technical features in the following embodiments may be combined arbitrarily. For the sake of brevity, this specification does not describe all possible combinations. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be covered by this specification.

[0031] Unless otherwise defined, technical or scientific terms used in the present disclosure should have the same meaning as commonly understood by a person having ordinary skills in the field to which the present disclosure belongs.

[0032] In order to solve the problems that the current screening method relies on professional equipment or personnel, is difficult to popularize in grassroots and home scenarios, and has technical bottlenecks such as single dimension and uncorrected temperature interference, the present invention provides a mobile terminal diabetic foot risk assessment method and system based on multi-parameter fusion. Figure 1 As shown in Figure 1, it specifically includes data acquisition module, feature extraction module, feature fusion module, model deployment and reasoning module and risk assessment module. It uses the trained machine learning method to perform reasoning and generate the final screening results. Figure 2 As shown, it is a collection environment diagram in one embodiment of the present invention, which includes a test subject 1 in a supine position, an infrared thermal imaging camera 2 connected to a mobile terminal device via Bluetooth or Wi-Fi, and a mobile terminal device 3 for information storage.

[0033] The data acquisition module is used to collect images and structured data of the user's foot soles. Specifically, it includes a Bluetooth / Wi-Fi connection module, a personal information registration module, an infrared image acquisition module, and a structured physiological data collection module. The Bluetooth / Wi-Fi module is used to connect a mobile device and an infrared camera; the personal information registration module ensures the independence and security of user information; the infrared image module is used to collect images of the user's foot soles; and structured physiological data is collected using an electronic spreadsheet. Infrared thermal imaging equipment with wireless communication capabilities (Wi-Fi, Bluetooth) is used to collect infrared images and visible light images of the foot soles, along with corresponding temperature matrix data. Structured physiological data is collected using an electronic spreadsheet, including but not limited to basic user information (name, gender, age, height, weight), diabetes duration, complications (hypertension, cardiovascular disease, kidney disease, eye disease, etc.), and treatment information. All data is packaged and encapsulated using unique identification numbers and temporarily stored in a local cache area to serve as input for the subsequent feature extraction module.

[0034] Furthermore, the feature extraction module is mainly responsible for data processing, feature construction and standardization of images and structured information, including image preprocessing, temperature feature extraction, structured data processing and multi-index correlation analysis, and outputs images and structured feature vectors with unified structure, standardized dimensions and sufficient information for subsequent module integration.

[0035] In the feature extraction module, Figure 3 As shown, for image data, the visible light image (RGB) is first converted to a grayscale image , using the fixed weight method:

[0036] , where G is the green channel value, R is the red channel value, and B is the blue channel value.

[0037] The global threshold method is then used for binary segmentation, and the plantar boundary is extracted and matched to the infrared image through erosion, dilation, and contour detection operations. The infrared image is enhanced using adaptive histogram equalization (CLAHE) to enhance image contrast and improve texture separability; a sharpening convolution kernel is then used to enhance image details, and median filtering is used to complete noise reduction. The gray-level co-occurrence matrix (GLCM) method is used to extract image texture features and define the gray-level co-occurrence matrix. , where i and j are grayscale values, d is the distance between pixels, and θ is the direction, extracting contrast (Contrast) Entropy Energy Correlation and other indicators. is the joint probability of gray values ​​i and j appearing together in a given direction θ and distance d in the normalized gray-level co-occurrence matrix GLCM, 、 They represent the probability-weighted average of the mean of the row of grayscale value i and the mean of the column of grayscale value j in GLCM, 、 Represents the probability-weighted standard deviation of the grayscale value i in the row and grayscale value j in the column in the GLCM; these indicators reflect the clarity, regularity, and directionality of the image texture. For the temperature matrix, the system calculates its mean, maximum, minimum, and standard deviation, and draws a temperature histogram to analyze the thermal distribution of the sole of the foot. Select a region of interest (ROI) and calculate the temperature gradient within the ROI. , reflecting the rate of change of temperature in space. For a two-dimensional temperature matrix, the approximate gradient in the x and y directions is calculated using the difference , :

[0038] ;

[0039] ;

[0040] in, Temperature right Coordinates and Partial derivatives of coordinates, It is the temperature at a certain position in the two-dimensional temperature matrix, and the temperature gradient is obtained by synthesis :

[0041] ;

[0042] Structural features are formed by normalizing (Min-Max) or standardizing (Z-score) variables such as age, weight, and diabetes duration to eliminate dimensionality. Outliers in the temperature matrix are corrected using the 3σ principle. For high-dimensional image features (such as multiple texture indices derived from the gray-level co-occurrence matrix), principal component analysis (PCA) is used to retain principal components with a cumulative contribution greater than 90% to reduce redundant features and improve modeling efficiency.

[0043] Furthermore, the purpose of the feature fusion module is to jointly process the multi-source heterogeneous image texture features, temperature statistical features and structured data features, build a unified and efficient feature vector input space, and output a uniform length, normalized, representative fusion feature vector. , used as direct input to the model deployment and inference modules.

[0044] The feature fusion module, such as Figure 4 As shown, the extracted 、 、 、 、 , these image features are concatenated into feature vectors , and then combined with structured features , building a unified and efficient feature vector input space :

[0045] ,in 、 They are image feature space and structured feature space respectively. To improve the fusion effect, the system introduces the attention mechanism to automatically assign weights of different modal features and calculate the weight vector :

[0046] ,in, is the activation function, is the weight matrix, is the bias vector; get the feature vector after weighted fusion , expressed as:

[0047] ,in , are the weights of image features and structural features, respectively, and are both learnable parameters.

[0048] Furthermore, the model deployment and inference module is based on the trained support vector machine model, adopts kernel function mapping to enhance nonlinear classification capabilities, and adjusts the model format for deployment on mobile terminals to achieve risk level prediction. The risk probability is obtained by combining the posterior probability mapping, realizing the modeling, training, model compression and mobile deployment of the fused feature vector, supporting fast inference and outputting continuous risk probability values, providing accurate quantitative basis for the final risk assessment module.

[0049] The model deployment and reasoning modules are as follows Figure 5 As shown in Figure 2, the fused feature set is fed into the model for inference. The C-Support Vector Classification (C-SVC) SVM model is used, and its training objective is to construct the optimal hyperplane:

[0050] ;

[0051] The optimization objective function is: ;

[0052] Constraints ,in: is the normal vector of the hyperplane; is the bias term; is the feature mapping function; is a soft interval variable; is the penalty factor.

[0053] The Gaussian kernel function (RBF) is used to improve the modeling capability of nonlinear separable samples. The kernel function expression is: In the training phase, parameters are optimized jointly by grid search and cross validation. and , improve the generalization ability of the model and reduce the risk of overfitting. The support vector, bias term and kernel function parameters are serialized and stored, which can be exported to ONNX or TensorFlow Lite format, and adapted to mobile platforms such as Android. Finally, the local model is used to perform forward inference on the input vector calculate :

[0054] ,in is the trained Lagrange multiplier, is the support vector, is the bias term;

[0055] And judge the label based on its positive or negative. Furthermore, in order to improve user comprehensibility, the system uses the PlattScaling method to Convert to probability score: , where A and B are the parameters fitted during the training phase.

[0056] Furthermore, the risk assessment module is used to visualize model inference results in various forms, analyze data, interpret risks, and generate assessment reports. Serving as the system's human-computer interface, it enhances the interpretability and clinical practicality of model predictions, enabling full-process output capabilities including risk visualization and pushable recommendations. A graphical interface displays plantar heatmaps, scores, and high-risk alerts, and generates standardized screening reports, allowing data to be viewed and reviewed via mobile devices.

[0057] The risk assessment module maps the risk probability values ​​output by the model into three risk levels: P < 0.3 indicates low risk, 0.3 ≤ P < 0.8 indicates medium risk, and P ≥ 0.8 indicates high risk. These are color-coded and displayed through a graphical interface. A structured diabetic foot assessment report is generated on the human-computer interface, including basic user information, assessment time, image analysis results, and model output results.

[0058] like Figure 6 As shown, a mobile terminal diabetic foot risk assessment method based on multi-parameter fusion is provided, which includes the following steps:

[0059] Step 1: Collect infrared images, visible light images, and temperature matrices of the soles of the feet, and enter structured data, including age, gender, course of disease, complications, etc.

[0060] Step 2: Feature extraction and standardization: Extract infrared image texture features (contrast, entropy, energy, correlation), extract temperature gradients within the selected region of interest (ROI), extract temperature and structured data feature pairs and perform standardization, feature fusion and dimensionality reduction;

[0061] Step 3: Concatenate the image and the structured feature vector, use PCA to reduce the dimension, retain the main information, introduce the attention mechanism to weight the feature importance, and output the fused feature vector;

[0062] Step 4: Model inference: load the pre-trained machine learning model, input the feature vector for inference, and use the PlattScaling method to map the output risk probability;

[0063] Step 5: Results display and reporting: Finally, the risk level, risk score and heat map are displayed on the human-computer interaction interface, key indicator summaries are summarized, and an assessment report is generated and saved.

[0064] It should be understood that although the steps in the flowcharts of the present invention are presented in sequence as indicated by arrows, this does not mean that they must be executed strictly in this order. Unless otherwise expressly stated, there is no hard limit to the order in which these steps are executed, and other orders may be used. In addition, at least a portion of the steps in each embodiment may include multiple sub-steps or stages, and these sub-steps or stages do not have to be completed at the same time, but can be executed at different time points. Moreover, the execution order of these sub-steps or stages is not necessarily carried out in sequence, and they can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

Claims

1. A mobile terminal diabetic foot risk assessment system based on multi-parameter fusion, characterized by: It includes data acquisition module, feature extraction module, feature fusion module, model deployment and reasoning module, and diabetic foot risk assessment module; The data acquisition module is used to collect infrared images, visible light images, temperature matrices, and structured physiological data of the subject's soles, and perform preliminary preprocessing on the collected data; The feature extraction module is used for image enhancement, segmentation and texture feature extraction; The feature fusion module fuses multi-source features of images, temperature matrices, and structured physiological data into a unified feature vector and optimizes its representation capability; The model deployment and inference module is used to feed the fused feature vector into a trained machine learning model deployed locally on the mobile device to determine the risk level and output the corresponding probability; The diabetic foot risk assessment module obtains the model output results, integrates them and displays the risk level and image analysis results on the interactive interface of the mobile terminal.

2. The system according to claim 1, wherein: The data acquisition module includes an image acquisition device, which includes an infrared thermal imager with Bluetooth and WI-FI functions and a smart mobile device. The subject can lie on a supine examination bed with the imaging end of the infrared thermal imager aimed at the subject's feet. The infrared thermal imager can communicate with the feature extraction module and the feature fusion module.

3. The system according to claim 1, wherein: Structured physiological data includes but is not limited to: user basic information, diabetes course, complications and treatment information; user basic information includes name, gender, age, height, and weight, and complications include hypertension, cardiovascular disease, kidney disease, and eye disease.

4. The system according to claim 1, wherein: in, The feature extraction module uses the texture analysis method gray level co-occurrence matrix GLCM on the infrared image to define the gray level co-occurrence matrix , the extracted texture features include contrast Con, entropy Ent, energy E and correlation Corr; ； ； ； ； Where i and j are grayscale values, d is the distance between pixels, and θ is the direction; is the joint probability of gray values ​​i and j appearing together in a given direction θ and distance d in the normalized gray-level co-occurrence matrix GLCM, 、 They represent the probability-weighted average of the mean of the row of grayscale value i and the mean of the column of grayscale value j in GLCM, 、 They represent the probability-weighted standard deviation of the gray value i row and gray value j column in the GLCM respectively; Select the region of interest (ROI) and calculate the temperature gradient within the ROI. The calculation method is as follows: ； ； The temperature gradient is obtained by comprehensive ,in, is the temperature at a certain position in the two-dimensional temperature matrix, Representatives in The rate of change in direction, Representatives in The rate of change in direction.

5. The system according to claim 1, wherein: in, The feature fusion module performs feature dimensionality reduction and fusion on multi-source features, and introduces an attention mechanism to automatically learn the importance weights of different features to enhance the key feature expression capabilities: ， ； in, is the weight matrix, is the bias vector, and are the attention weights of image features and structural features respectively, and are image and structured feature vectors respectively, is the weight vector, is the fused feature vector.

6. The system according to claim 5, characterized in that The fused feature set is fed into the model deployment and inference module for inference. The SVM model is used, and its training goal is to construct the optimal hyperplane: ； The optimization objective function is: , constrained to ,in: is the normal vector of the hyperplane; is the bias term; is the feature mapping function; is a soft interval variable; is the penalty factor; The Gaussian kernel function is used to improve the modeling ability of nonlinear separable samples. The kernel function expression is: In the training phase, parameters are optimized jointly by grid search and cross validation. and Finally, the local model is used to perform forward reasoning on the input vector calculate ,in is the trained Lagrange multiplier, is the support vector, is the bias term, and the label is determined based on its positive or negative value.

7. The system according to claim 1, characterized in that in, The model deployment and inference module uses nonlinear kernel function to perform support vector machine modeling and combines Platt Scaling method for probability mapping , where parameters A and B are fitted by minimizing the log-likelihood during the training phase to output the diabetic foot risk grade.

8. The system according to claim 7, characterized in that in, The risk assessment module divides the risk probability output by reasoning into multiple risk levels, and generates a visual risk assessment report based on the plantar infrared thermal image, temperature characteristics and key indicators.

9. The system according to claim 8, characterized in that According to the risk probability value output by the model, it is mapped into three risk levels: P < 0.3 indicates low risk, 0.3 ≤ P < 0.8 indicates medium risk, and P ≥ 0.8 indicates high risk.

10. A mobile terminal diabetic foot risk assessment method based on multi-parameter fusion, characterized in that: The following steps are involved: Step 1: Collect infrared images, visible light images, and temperature matrices of the plantar surface, and input structured physiological data, including age, gender, disease course, and complications; Step 2: Feature extraction and standardization: Extract the infrared image texture features including contrast, entropy, energy, and correlation, extract the temperature gradient within the selected region of interest (ROI), extract the temperature and structured data feature pairs and perform standardization, feature fusion, and dimensionality reduction. Step 3: Concatenate the image and the structured feature vector, perform dimensionality reduction, introduce the attention mechanism to weight the feature importance, and output the fused feature vector; Step 4: Model inference: load the pre-trained machine learning model, input the fused feature vector for inference, and map the output risk probability; Step 5: Results display and reporting: Finally, the risk level, risk score and heat map are displayed on the human-computer interaction interface, key indicator summaries are summarized, and an assessment report is generated and saved.

Images