Assessment method of diabetic foot based on wound image recognition

Abstract

The invention discloses a diabetic foot assessment method based on wound image recognition. The method comprises the following steps: processing a foot infrared thermal imaging image to generate a temperature anomaly graph; fusing the foot wound image and the temperature anomaly graph to generate a fused image; constructing a foot blood flow analysis model, extracting a blood flow influence factor, and correcting the fused image through the blood flow influence factor; and constructing a joint assessment model, taking the corrected fusion image and historical data of the diabetic foot patient as input of the joint assessment model, predicting a risk assessment level of the diabetic foot, and outputting a disease assessment result of the diabetic foot patient according to the risk assessment level. According to the method, multi-modal medical image fusion analysis is realized, the accuracy of wound area positioning and pathological state expression is improved, the accuracy of lesion area recognition is improved, the illness worsening risk can be predicted in advance, clinical decision can be assisted, the risk of serious complications such as amputation of a patient is reduced, and the prognosis and life quality of the patient are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an evaluation method for diabetic foot based on wound image recognition.

Background Art

[0002] Diabetic foot (DF) is one of the severe complications in the late stage of diabetes. Due to its high disability and fatality rates, as well as great treatment difficulty and long treatment cycle, it causes a heavy burden on patients.

[0003] Traditional evaluations of diabetic foot mostly rely on manual visual judgment and basic medical indicators, suffering from problems such as strong evaluation subjectivity, insufficient accuracy, and delayed diagnosis, making it difficult to achieve early accurate diagnosis and dynamic prediction of the disease condition.

[0004] Therefore, there is an urgent need for a more accurate, multi-modal data integrated, and personalized risk trend prediction capable diabetic foot evaluation method to improve the timeliness and accuracy of diagnosis and treatment for diabetic foot patients.

Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides an evaluation method for diabetic foot based on wound image recognition.

[0006] In a first aspect, an embodiment of the present invention provides an evaluation method for diabetic foot based on wound image recognition, the method comprising:

[0007] S1. Obtain the infrared thermal imaging image and color image of the foot of a target diabetic foot patient, and perform preprocessing on each of them respectively, and process the infrared thermal imaging image of the foot to generate a temperature anomaly map;

[0008] S2. Remove interference factors from the color image of the foot, mark the real wound of the diabetic foot to generate a foot wound image, and fuse the foot wound image with the temperature anomaly map to generate a fused image;

[0009] S3. Construct a foot blood flow analysis model, perform feature analysis through the foot blood flow analysis model, extract blood flow influencing factors, and correct the fused image through the blood flow influencing factors;

[0010] S4. Construct a joint evaluation model, use the corrected fused image and the historical data of the diabetic foot patient as the input of the joint evaluation model, predict the risk evaluation level of the diabetic foot, and output the disease evaluation result of the diabetic foot patient according to the risk evaluation level.

[0011] As described above in the aspect and any possible implementation manner, a further implementation manner is provided, wherein the processing the infrared thermal imaging image of the foot to generate a temperature anomaly map specifically includes:

[0012] S11. Convert the original gray value of the foot infrared thermal imaging image into the actual temperature value to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x, y) = a·I(x, y) + b, where T(x, y) represents the actual temperature value at pixel (x, y), I(x, y) represents the original gray value at pixel (x, y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager;

[0013] S12. Perform Gaussian filtering on the converted temperature matrix to remove noise;

[0014] S13. Use the foot color image to extract the foot area in the foot infrared thermal imaging image;

[0015] S14. Mirror-align the foot areas of the left and right feet through the image coordinate system. The alignment formula is defined as: x′ R = 2·X c - x L , y′ R = y L , where x L and y L represent the pixel coordinates within the left foot area, x′ R and y′ R represent the mirror pixel coordinates of the right foot corresponding to the left foot, and X c represents the abscissa of the symmetry axis of the left and right feet;

[0016] S15. Calculate the temperature difference at the symmetric positions of the two feet to obtain a temperature difference matrix, and perform binary processing on the temperature difference matrix according to a preset temperature difference threshold to obtain an abnormal area mask. Among them, the temperature difference matrix is defined as: ΔT(x L , y L ) = T L (x L , y L ) - T R (x′ R , y′ R ), where ΔT(x L , y L ) represents the temperature difference at the corresponding positions of the left and right feet, T L (x L , y L ) represents the temperature value at the left foot position, and T R (x′ R , y′ R ) represents the temperature value at the corresponding mirror position of the right foot; The calculation formula for the abnormal area mask is: Among them, A(x, y) represents the binary mask of the abnormal area, 1 represents abnormal, 0 represents normal, ΔT(x, y) represents the temperature difference of the pixels in the corresponding areas of the left and right feet, and ΔT thrIndicates the temperature difference threshold;

[0017] S16, mapping the abnormal area mask back to the foot infrared thermal imaging image to generate a temperature anomaly map I temp (x,y).

[0018] According to the above aspects and any possible implementation, an implementation is further provided, wherein S13 specifically includes:

[0019] S131. Perform color balancing on the foot color image and convert it to the HSV color space. Then, use a pre-set threshold to segment the hue and saturation to obtain a preliminary foot region: Among them, H(x,y) and S(x,y) represent the hue and saturation in the HSV color space respectively, and M C (x,y) represents the initial generated color image foot mask, H min and H max Indicates the hue range of the foot skin color in the HSV space, S min Indicates the minimum saturation threshold, indicating;

[0020] S132, to M C (x,y) uses morphological processing to repair edge discontinuities and obtain a smooth and coherent foot contour mask, namely M foot (x,y);

[0021] S133, foot color image I C (x,y) using Canny edge detection E C (x,y)=Canny(I C (x,y)) is processed to obtain the edge graph E C (x,y), the edge graph E C (x,y) and M foot (x,y) performs C foot (x,y)=E C (x,y)·M foot (x,y) operation to obtain the final foot contour map C foot (x,y);

[0022] S134, final foot contour diagram C foot (x, y) performs morphological closing operation to generate a continuous closed contour, where the expansion operation is performed first: Then perform the corrosion operation: Among them, S represents the structural element, M final (x,y) represents the final generated foot contour binary mask;

[0023] S135. Align the foot contour mask M obtained from the foot infrared thermal imaging image and the foot color image, and apply an affine transformation for correction, M final (x, y) and apply an affine transformation for correction, M aligned (u, v) = T affine (M final (x, y)), where T affine represents the affine transformation for realizing the geometric registration of the color and infrared images, and M aligned (u, v) represents the binary mask of the foot after the affine transformation;

[0024] S136. Use the corrected foot mask to extract the region of the foot infrared thermal imaging image to obtain the foot temperature map T foot (u, v), T foot (u, v) = I T (u, v) × M aligned (u, v), where I T (u, v) represents the infrared image after the affine transformation, and T foot (u, v) represents the infrared temperature map of the extracted foot region.

[0025] For the aspects and any possible implementation manners as described above, a further implementation manner is provided. In S2, the foot wound image and the temperature anomaly map are fused to generate a fused image, which specifically includes:

[0026] S21. Use the color segmentation method in the HSV color space to extract the suspected wound region of the foot color mouth image, and define a threshold function: where H(x, y), S(x, y), and V(x, y) respectively represent the hue, saturation, and brightness of the pixel in the HSV color space, and H L , H H , S min and V max respectively represent preset thresholds for preliminarily distinguishing the normal skin region and the suspected wound region;

[0027] S22. Perform texture feature analysis on the suspected wound region M initial (x, y) to distinguish diabetic foot wounds from interference factors, and specifically calculate the texture features using the gray-level co-occurrence matrix, where I gray (x, y) represents the gray-scale conversion value of the foot color mouth image, d represents the pixel distance, θ represents the analysis direction, and W and H respectively represent the width and height of the image; calculate the contrast and homogeneity of each pixel neighborhood;

[0028] S23. Set a texture feature threshold function to remove the interference of the texture obvious region: Among them, Co(x,y) represents contrast, Ho(x,y) represents homogeneity, and T c and T h represent the texture feature thresholds respectively, which are used to distinguish real wounds from natural skin textures and scab tissues;

[0029] S24. Define the edge irregularity coefficient I of the wound area irregular , set the irregularity threshold T irregular , remove the common wound areas with high regularity, and obtain the final diabetic foot wound mask M wound (x,y) as follows: Among them, P represents the perimeter of the suspected wound area, and A represents the area of the area.

[0030] In the aspects and any possible implementation manners as described above, a further implementation manner is provided. In S2, the interference factors in the foot color image are removed, specifically including:

[0031] S25. Perform image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x,y), I wound (x,y) = I C (x,y) × M wound (x,y) × γ, where I C (x,y) represents the original foot color image, and γ represents the enhancement coefficient;

[0032] S26. Fuse the foot wound image with the temperature anomaly map to generate a fused image. Among them, define the fusion coefficient: Define the fused image: I fused (x,y) = w(x,y) × I temp (x,y) + [1 - w(x,y)] × I wound (x,y), where w(x,y) represents the fusion coefficient, I fused (x,y) represents the fused image, I temp (x,y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0 < λ < 1, and ε represents a very small positive number.

[0033] In the aspects and any possible implementation manners as described above, a further implementation manner is provided. S3 specifically includes:

[0034] S31. Obtain the blood perfusion images of diabetic foot patients and non-diabetic foot patients collected in advance, use the median absolute deviation MAD to remove local abnormal blood flow values, and obtain the denoised images. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0035] S32. Calculate the first average blood perfusion intensity of the wound area of diabetic foot patients Calculate the second average blood perfusion intensity of the standard measurement area of non-diabetic foot patients And according to the first average blood perfusion intensity And the second average blood perfusion intensity Calculate the blood flow impact factor K F , where F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the set of pixels in the wound area of the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of the diabetic foot patient group, ΩN,standard represents the set of pixels in the standard measurement area of the non-diabetic foot healthy control group, N N,standard represents the total number of pixels in the standard measurement area of the non-diabetic foot healthy control group, and ε represents a very small positive number;

[0036] S33. Use the blood flow impact factor to correct the fused image, and define the formula for correcting the fused image:

[0037] I corr (x,y) = I fused (x,y) × (1 + β × (1 - K F )), I corr (x,y) represents the fused image corrected by blood flow, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0038] In the above aspects and any possible implementation manners, a further implementation manner is provided. The S3 specifically includes:

[0039] S31. Obtain the blood perfusion images of diabetic foot patients and non - diabetic foot patients collected in advance, obtain the blood perfusion image of the target diabetic foot patient, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain the denoised image. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0040] S32. Calculate the first average blood perfusion intensity of the wound area of diabetic foot patients Calculate the second average blood perfusion intensity of the standard measurement area of non - diabetic foot patients And according to the first average blood perfusion intensity and the second average blood perfusion intensity calculate the blood flow influence factor K F , calculate the individual offset factor K of the target diabetic foot patient T , according to the blood flow influence factor K F and the individual offset factor K T calculate the final correction factor K final ; Among them, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non - diabetic foot patients, ΩD,wound represents the set of pixels in the wound area of the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of the diabetic foot patient group, ΩN,standard represents the set of pixels in the standard measurement area of the non - diabetic foot healthy control group, N N,standard represents the total number of pixels in the standard measurement area of the non - diabetic foot healthy control group, ε represents a very small positive number, represents the average blood flow intensity of the wound area of the blood perfusion image of the target diabetic foot patient, represents the average blood flow intensity of the healthy foot area of the blood perfusion image of the target diabetic foot patient, represents an adjustable parameter,

[0041] S33. Through the final correction factor K finalCorrect the fused image and define the formula for the corrected fused image:

[0042] I corr (x,y) = I fused (x,y) × (1 + β × (1 - K final ))), I corr (x,y) represents the fused image corrected by blood flow, and I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0043] For the aspects and any possible implementation manners as described above, a further implementation manner is provided. Specifically, S4 includes:

[0044] Construct and train a joint evaluation model combining CNN and LSTM;

[0045] Use the corrected fused image I corr (x,y) and the historical data of diabetic foot patients with a preset time length as the input of the joint evaluation model, and define the input data matrix: X image = I corr (x,y), X image represents the image input data, and the size is uniformly 128×128×3. X t = [HbA1c t , blood glucose t , wound area t , number of infections t , BMI t , X history represents the historical data, and T represents the time length of the historical data;

[0046] According to the model, output the risk assessment level and evolution risk level of diabetic foot, and output the disease assessment result of diabetic foot patients according to the risk assessment level and evolution risk level.

[0047] For the aspects and any possible implementation manners as described above, a further implementation manner is provided. Specifically, the construction and training of the joint evaluation model combining CNN and LSTM includes:

[0048] Collect the medical data of diabetic foot patients and non-diabetic foot patients, and preprocess the processed fused image and the historical index data of the patients to form a data set;

[0049] Randomly divide the data set into a training set and a validation set. 80% of the patient data is the training set, and 20% of the patient data is the validation set;

[0050] Adopt the CNN module structure for spatial feature extraction:

[0051] First Convolutional Layer Conv1: Convolution kernel 3×3×32, ReLU activation function; First Pooling Layer MaxPool: Pooling size 2×2, stride 2; Second Convolutional Layer Conv2: Convolution kernel 3×3×64, ReLU activation function; Second Pooling Layer MaxPool2: Pooling size 2×2, stride 2; Feature flattening: Flatten the two-dimensional feature map into a one-dimensional feature vector; Fully connected layer: Output the image feature vector F cnn ∈R 128 ;

[0052] Among them, the formula of the CNN module is expressed as: F cnn = f cnn (X image ; θ cnn ), θ cnn represents the set of trainable parameters of the CNN;

[0053] The temporal feature extraction adopts the LSTM module structure:

[0054] Build an LSTM network: Input dimension: X history ∈R T×d , d represents each temporal feature dimension, Output dimension: The feature vector of the last hidden layer F lstm ∈R 64 ;

[0055] The formula of the LSTM cell is as follows: Input gate: i t = σ(W xi X t + W hi h t-1 + b i ), Forgetting gate: f t = σ(W xf X t + W hf h t-1 + b f ), Output gate: o t = σ(W xo X t + W ho h t-1 + b o ), Cell state: c t = f t ⊙ c t-1 + i t ⊙ tanh(W xc X t + W hc h t-1 + b c ), Hidden state: h t = o t ⊙ tanh(c t), Final temporal feature output: F lstm = h T , where σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, and i t represents the input gate, which controls the degree of update of the input features to the cell state, and f t represents the forget gate, which controls the retention or discard of past information, and o t represents the output gate, which controls the hidden state of the final output, and c t represents the cell state, which is used for long-term memory of information, and h t represents the hidden state, the expression of the final time series features, W ** and b * represent the trainable parameters of the LSTM. Among them, W ** represents the weight, and b * represents the bias;

[0056] CNN feature and LSTM feature fusion: F fusion = Concatenate(F cnn , F lstm ) ∈ R 192 ;

[0057] Construct a dual-output branch:

[0058] Output branch 1: Prediction of the disease level classification, fully connected output + Softmax classification: P stage = Softmax(W s F fusion + b s ), P stage ∈ R 4 ; Final level output: Stage pred = argmax(P stage ), Stage pred ∈ {0, 1, 2, 3};

[0059] Output branch 2: Prediction of the disease evolution risk trend, fully connected output + Sigmoid probability regression:

[0060] S trend = σ(W t F fusion + b t ), S trend ∈ [0, 1];

[0061] The loss function is designed as the joint loss of the two outputs: Loss total = αLoss stage + βLoss trend , where, Loss trend = (Ytrend -S trend ) 2 where α and β represent the weight coefficients of the losses of two subtasks represents the actual disease level represents the probability value that the model outputs a prediction for the i-th class level, Y trend represents the actual trend label, S trend represents the probability or score value of the trend prediction output by the model

[0062] Optimization: Use the Adam optimizer with an initial value of 0.001, and gradually decay it later

[0063] Training: Initialize the model weight parameters, set the batch size to 16 - 32, and use the mini-batch gradient descent method: within each epoch, randomly batch-sample data for training, update the model parameters for each batch, and after each round of epoch training, calculate the loss function on the validation set. If the validation set loss does not decrease for 5 consecutive epochs, stop training, and finally select the model parameters with the lowest validation set loss

[0064] The model is verified on the validation set to complete the verification

[0065] In a second aspect, an embodiment of the present invention provides a computer-readable and writable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the above method are implemented

[0066] One of the above technical solutions has the following beneficial effects

[0067] In the method of the embodiment of the present invention, an evaluation method for diabetic foot based on wound image recognition is proposed. This method realizes multi-modal medical image fusion analysis, improves the accuracy of wound area localization and pathological state expression, and introduces a blood flow feature model for image correction to improve the accuracy of lesion area recognition. By using the CNN-LSTM deep learning model and integrating the patient's historical time-series data, it realizes personalized dynamic assessment of the disease development, significantly improves the accuracy and timeliness of diagnosis and prediction, and realizes intelligent recognition, quantitative assessment, and evolution trend prediction of the diabetic foot wound state. Therefore, the present invention can predict the risk of disease aggravation in advance, assist clinical decision-making, reduce the risk of serious complications such as patient amputation, and improve the patient's prognosis and quality of life

Description of the Drawings

[0068] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0069] Figure 1 is a schematic flowchart of the evaluation method S1 - S4 for diabetic foot based on wound image recognition provided by the embodiments of the present invention;

[0070] Figure 2 is a schematic flowchart of the method S11 - S16 provided by the embodiments of the present invention.

Specific Embodiments

[0071] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention in conjunction with specific embodiments and the corresponding accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0072] Please refer to Figure 1 , which is a schematic flowchart of an evaluation method S1 - S4 for diabetic foot based on wound image recognition provided by the embodiments of the present invention. As Figure 1 shown, the method includes the following steps:

[0073] S1. Obtain the infrared thermal imaging image and color image of the foot of the target diabetic foot patient, and perform pre - processing on each of them. Process the infrared thermal imaging image of the foot to generate a temperature anomaly map;

[0074] S2. Remove interference factors from the color image of the foot, mark the real wound of the diabetic foot to generate a foot wound image, and fuse the foot wound image with the temperature anomaly map to generate a fused image;

[0075] S3. Construct a foot blood flow analysis model, perform feature analysis through the foot blood flow analysis model, extract blood flow influencing factors, and correct the fused image with the blood flow influencing factors;

[0076] S4. Construct a joint evaluation model, use the corrected fused image and the historical data of the diabetic foot patient as the input of the joint evaluation model, predict the risk assessment level of the diabetic foot, and output the disease evaluation result of the diabetic foot patient according to the risk assessment level.

[0077] The present invention collects infrared thermal imaging images and color images of the foot, constructs a basic multi-modal image information source, converts the infrared image into a temperature anomaly map, realizes the quantitative expression of the thermal distribution of the foot skin, provides a basis for judging local inflammation, infection, and ischemia, and is beneficial to the potential lesion areas that have not yet ulcerated in the early stage; identifies the true wound area of diabetic foot, and at the same time excludes interference factors such as artifacts, scabs, and shadows. Through image fusion, the wound information and temperature anomaly information are fused. The fused image can provide double features of "heat + shape", effectively enhancing the discriminant ability of the system for early wounds and infections, and improving the information density and diagnostic value of the input image of the subsequent analysis model; uses the blood perfusion intensity to reflect the tissue activity and healing potential of the wound area, provides a supplementary index beyond visual information, and further corrects the physiological characteristics of the fused image through the establishment of a correction mechanism by comparing healthy feet and diseased feet, improving the generalization and stability of the subsequent model in actual clinical data; through the combined evaluation model, the corrected fused image and the patient's historical data are comprehensively analyzed to realize multi-level intelligent classification of the risk of diabetic foot, and at the same time have the ability to predict the disease trend, providing a predictive auxiliary judgment tool for doctors to assist in early intervention and personalized treatment; significantly improves the objectivity and automation level of the evaluation, and avoids misjudgment and delay caused by different doctors' experiences.

[0078] Please refer to Figure 2 In a preferred embodiment of the present invention, the processing of the infrared thermal imaging image of the foot to generate a temperature anomaly map specifically includes:

[0079] S11. Convert the original gray value of the infrared thermal imaging image of the foot into an actual temperature value to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x,y) = a·I(x,y) + b, where T(x,y) represents the actual temperature value at pixel (x,y), I(x,y) represents the original gray value at pixel (x,y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager;

[0080] S12. Perform Gaussian filtering on the converted temperature matrix to remove noise;

[0081] S13. Use the color image of the foot to extract the foot area in the infrared thermal imaging image of the foot;

[0082] S14. Mirror-align the foot areas of the left and right feet through the image coordinate system. The alignment formula is defined as: x′ R = 2·X c - x L y′ R = y L where x L and y L represent the pixel coordinates in the left foot area, x′R and y' R represent the right - foot mirror pixel coordinates corresponding to the left foot, where X c represents the abscissa of the left - right foot symmetric central axis;

[0083] S15. Calculate the temperature difference between symmetric positions of the two feet to obtain a temperature - difference matrix, and binarize the temperature - difference matrix according to a preset temperature - difference threshold to obtain an abnormal - area mask; where the temperature - difference matrix is defined as: ΔT(x L , y L ) = T L (x L , y L ) - T R (x', R , y' R ), where ΔT(x L , y L ) represents the temperature difference between corresponding positions of the left and right feet, T L (x L , y L ) represents the temperature value at the left - foot position, and T R (x', R , y' R ) represents the temperature value at the corresponding mirror position of the right foot; the calculation formula for the abnormal - area mask is: where A(x, y) represents the binary mask of the abnormal area, 1 represents abnormal, 0 represents normal, ΔT(x, y) represents the temperature difference of pixels in the corresponding areas of the left and right feet, and ΔT thr represents the temperature - difference threshold;

[0084] S16. Map the abnormal - area mask back to the foot infrared thermal - imaging image to generate a temperature - anomaly map I temp (x, y).

[0085] In the embodiments of the present invention, by physically quantifying the image information, an accurate and standardized data basis is provided for subsequent temperature - difference calculation and thermal - anomaly recognition. Gaussian filtering and noise reduction reduce the interference of thermal - imaging random noise on temperature - difference recognition. By comparing the temperature differences of symmetric parts of the left and right feet through the mirror - registration method, the single - foot thermal anomaly that appears in the early stage of diabetic foot can be effectively detected. By setting the temperature - difference threshold to generate an abnormal - area mask, the potential lesion area can be efficiently located, significantly improving the sensitivity of the system detection.

[0086] In a preferred embodiment of the present invention, S13 specifically includes:

[0087] S131. Perform color - balance processing on the foot color image, convert it to the HSV color space, and segment the hue and saturation using a preset threshold to obtain a preliminary foot area: Among them, H(x, y) and S(x, y) respectively represent the hue and saturation in the HSV color space, and M C (x, y) represents the preliminary generated foot mask of the color image, and H min and H max represent the set hue range of the foot skin color in the HSV space, and S min represents the lowest saturation threshold, indicating;

[0088] S132. Perform morphological processing on M C (x, y) to repair the edge discontinuity and obtain a smooth and continuous foot contour mask, that is, M foot (x, y);

[0089] S133. Use Canny edge detection E C (x, y) on the foot color image I C (x, y) = Canny(I C (x, y)) for processing to obtain the edge map E C (x, y), and perform the operation C C (x, y) = E foot (x, y) · M foot (x, y) on the edge map E C (x, y) and M foot (x, y) to obtain the final foot contour map C foot (x, y);

[0090] S134. Perform morphological closing operation on the final foot contour map C foot (x, y) to generate a continuous and closed contour. Among them, first perform the dilation operation: Then perform the erosion operation: Among them, S represents the structural element, and M final (x, y) represents the finally generated binary mask of the foot contour;

[0091] S135. Align the foot infrared thermal imaging image with the foot contour mask M final (x, y) obtained from the foot color image, and apply affine transformation for correction, M aligned (u, v) = T affine (M final (x, y)), where T affine represents the affine transformation used to achieve geometric registration of the color and infrared images, and M aligned (u, v) represents the binary mask of the foot after affine transformation;

[0092] S136. Use the corrected foot mask to perform region extraction on the foot infrared thermal imaging image to obtain the foot temperature map T foot(u, v), T foot (u, v) = I T (u, v) × M aligned (u, v), where I T (u, v) represents the infrared image after affine transformation, T foot (u, v) represents the infrared temperature map of the extracted foot region.

[0093] Embodiments of the present invention involve multiple sub-steps such as color space conversion, image morphology operations, edge detection, image registration, and region extraction, enabling precise extraction of the foot region, image registration, and construction of a region heat map, providing a high-quality spatial basis for infrared thermal imaging data analysis. Among them, the multi-stage image processing link (i.e., color balance, HSV space, morphology operations, edge detection, and affine correction) significantly improves the accuracy and robustness of image registration. The morphological closing operation makes the extraction result smoother and more closed, ensuring complete recognition of the foot contour. The affine transformation automatically corrects deviations and can effectively solve image misalignment caused by patient posture, shooting angle, or equipment errors.

[0094] In a preferred embodiment of the present invention, in S2, fusing the foot wound image and the temperature anomaly map to generate a fused image specifically includes:

[0095] S21. Using the color segmentation method in the HSV color space to extract the suspected wound area of the foot color image, and defining a threshold function: where H(x, y), S(x, y), and V(x, y) respectively represent the hue, saturation, and brightness of the pixel in the HSV color space, H L , H H , S min , and V max respectively represent preset thresholds for initially distinguishing the normal skin area from the suspected wound area;

[0096] S22. Performing texture feature analysis on the suspected wound area M initial (x, y) to distinguish diabetic foot wounds from interference factors, specifically calculating texture features using a gray-level co-occurrence matrix, where I gray (x, y) represents the gray-scale conversion value of the foot color image, d represents the pixel distance, θ represents the analysis direction, and W and H respectively represent the width and height of the image; calculating the contrast and homogeneity of each pixel neighborhood;

[0097] S23. Setting a texture feature threshold function to remove the interference of obvious texture areas: where Co(x, y) represents the contrast, Ho(x, y) represents the homogeneity, T c , and T hIt represents the texture feature thresholds respectively, which are used to distinguish real wounds from natural skin textures and scabbed tissues;

[0098] S24. Define the edge irregularity coefficient I of the wound area irregular , and set the irregularity threshold T irregular , remove the ordinary wound areas with higher regularity, and obtain the final diabetic foot wound mask M wound (x, y) is as follows: Among them, P represents the perimeter of the suspected wound area, and A represents the area of the area.

[0099] In the embodiment of the present invention, through hierarchical screening from color (HSV), texture (gray-level co-occurrence matrix), and structural shape (irregularity), a complete chain of "real diabetic foot wound recognition" is constructed. Through triple screening, misrecognition and missed recognition are significantly reduced, the recognition result is closer to the clinical reality, and false positive areas caused by blood scabs, skin structure, shadows, indentations, etc. are effectively filtered out. This method does not depend on image brightness or color temperature, can adapt to various actual shooting environments such as skin color differences and lighting changes, and significantly reduces the misjudgment rate through multi-dimensional intelligent analysis.

[0100] In the preferred embodiment of the present invention, in S2, removing interference factors from the foot color mouth image specifically includes:

[0101] S25. Perform image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x, y), I wound (x, y) = I C (x, y) × M wound (x, y) × γ, where I C (x, y) represents the original foot color image, and γ represents the enhancement coefficient;

[0102] S26. Fuse the foot wound image with the temperature anomaly map to generate a fused image, where the fusion coefficient is defined as: Define the fused image: I fused (x, y) = w(x, y) × I temp (x, y) + [l - w(x, y)] × I wound (x, y), where w(x, y) represents the fusion coefficient, I fused (x, y) represents the fused image, I temp (x, y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0 < λ < 1, and ε represents a very small positive number.

[0103] In the embodiments of the present invention, image enhancement is used to improve image contrast and detail presentation, which helps to enhance the model's learning of wound boundaries and tissue features. By introducing a fusion coefficient through information fusion to control the image synthesis process, thermal anomaly information and structural information are effectively combined to generate an image rich in multi-source features, enhancing image clarity and fusing multi-modal data, improving the input quality of the overall evaluation model, and thus enhancing the classification accuracy.

[0104] In a preferred embodiment of the present invention, step S3 specifically includes:

[0105] S31. Obtain the blood perfusion images of diabetic foot patients and non-diabetic foot patients collected in advance, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain the denoised image. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′) - median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighboring pixels within the window, median represents the median function, T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0106] S32. Calculate the first average blood perfusion intensity of the wound area of diabetic foot patients Calculate the second average blood perfusion intensity of the standard measurement area of non-diabetic foot patients And according to the first average blood perfusion intensity and the second average blood perfusion intensity calculate the blood flow influence factor K F , where F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the set of pixels in the wound area of the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of the diabetic foot patient group, ΩN,standard represents the set of pixels in the standard measurement area of the non-diabetic foot healthy control group, N N,standard represents the total number of pixels in the standard measurement area of the non-diabetic foot healthy control group, and ε represents a very small positive number;

[0107] S33. Use the blood flow influence factor to correct the fused image, and define the formula for correcting the fused image:

[0108] I corr (x,y) = I fused(x,y) × (1 + β × (1 - K F )),I corr (x,y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0109] In the embodiments of the present invention, using the blood perfusion images of the diabetic foot group and the healthy control group, after denoising by the median absolute deviation algorithm, the difference in the average perfusion intensity of the group is calculated. Based on this difference, a blood flow impact factor is constructed to correct the fused image, introducing the physiological attribute of blood flow in addition to the structural and temperature characteristics of the image, improving the physiological rationality of image evaluation; the entire method does not rely on the perfusion image of the target patient and is applicable to telemedicine and the deployment of a simplified screening model.

[0110] Among them, a non-contact laser Doppler imaging or a similar microcirculation imaging technique is used to obtain the blood perfusion image.

[0111] In the preferred embodiment of the present invention, S3 specifically includes:

[0112] S31. Obtain the pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, obtain the blood perfusion image of the target diabetic foot patient, and use the median absolute deviation MAD to remove the local abnormal blood flow values to obtain the denoised image. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′) - median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove the outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighboring pixels within the window, median represents the median function, T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0113] S32. Calculate the first average blood perfusion intensity of the wound area of the diabetic foot patient Calculate the second average blood perfusion intensity of the standard measurement area of the non-diabetic foot patient And according to the first average blood perfusion intensity and the second average blood perfusion intensity calculate the blood flow impact factor K F , calculate the individual offset factor K of the target diabetic foot patient T , according to the blood flow impact factor K F and the individual offset factor K T calculate the final correction factor K final ; among them, F′ D (x, y) represents the denoised blood perfusion image of a diabetic foot patient, and F′ N (x, y) represents the denoised blood perfusion image of a non - diabetic foot patient. ΩD,wound represents the set of pixel areas of the wound region in the diabetic foot patient group, and N D,wound represents the total number of pixels in the wound region of the diabetic foot patient group. ΩN,standard represents the set of pixel areas of the standard measurement region in the non - diabetic foot healthy control group, and N N,standard represents the total number of pixels in the standard measurement region of the non - diabetic foot healthy control group. ε represents a very small positive number. represents the average blood flow intensity of the wound region of the blood perfusion image of the target diabetic foot patient. represents the average blood flow intensity of the healthy foot region of the blood perfusion image of the target diabetic foot patient. represents an adjustable parameter.

[0114] S33. Through the final correction factor K final correct the fused image, and define the corrected fused image formula:

[0115] I corr (x, y) = I fused (x, y)×(1 + β×(1 - K final ))), I corr (x, y) represents the fused image corrected by blood flow, and I fused (x, y) represents the fused image, and β represents the image correction intensity coefficient.

[0116] In the embodiment of the present invention, the foot perfusion image of the currently to - be - evaluated diabetic foot patient is collected and subjected to MAD denoising processing to obtain high - quality individual perfusion data. Based on the difference in the average blood perfusion intensity between the wound region and the healthy foot region of the target patient's foot, an individual offset factor is constructed. Then, by weighted fusion of the population influence factor and the individual offset factor, a final image correction factor is obtained, and the final correction factor is applied to the fused image to achieve personalized image physiological correction for the target patient. On the basis of constructing the blood flow influence factor based on population data, the individual blood perfusion image of the target diabetic foot patient is further introduced to construct an individual offset factor, forming a final correction factor to perform personalized correction on the fused image; this solution not only retains the stability, generality, and low - cost characteristics of the population model, but also realizes a personalized image optimization mechanism that highly adapts to the current patient's state through introducing individual feature calculation.

[0117] In the preferred embodiment of the present invention, S4 specifically includes:

[0118] Construct and train a joint evaluation model combining CNN and LSTM;

[0119] Use the corrected fused image I corr (x, y) and the historical data of diabetic foot patients with a preset time length as the input of the joint evaluation model, and define the input data matrix: X image = I corr (x, y), where X image represents the image input data, with a unified size of 128×128×3, X t = [HbA1c t , blood glucose t , wound area t , number of infections t , BMI t , where X history and T represent the historical data, and T represents the time length of the historical data;

[0120] Based on the model, output the risk assessment level and evolution risk level of diabetic foot, and output the disease assessment result of diabetic foot patients according to the risk assessment level and evolution risk level.

[0121] It should be noted that the reference risk assessment levels for diabetic foot are: grade 0 (no wound or mild swelling, no ulcer), grade 1 (superficial ulcer or local infection), grade 2 (deep ulcer, infection spreading but controllable), and grade 3 (osteomyelitis or pregangrene). The specific setting criteria can be adjusted according to actual needs.

[0122] The reference evolution risk levels are: <0.3 (low risk), 0.3 - 0.6 (medium risk), and >0.6 (high risk). The specific setting criteria can be adjusted according to actual needs.

[0123] In the embodiments of the present invention, by constructing a joint model combining a convolutional neural network and a long short-term memory network, and jointly inputting the corrected fused image and the historical condition data of diabetic foot patients, the risk level judgment and disease evolution trend prediction of diabetic foot patients are realized; by fusing and inputting these two heterogeneous data, namely the image input and the historical data input sequence, into the model, the expression ability and reasoning depth of the model are significantly enhanced. CNN is used to process static image features and is good at identifying local spatial structures, boundary shapes, color contrasts, etc. LSTM is suitable for processing time series data and can capture the fluctuation patterns of patients' conditions over a period of time. The joint architecture of CNN + LSTM has the ability to jointly judge from the "current state" and "past development", constructing a "space + time" two-dimensional disease assessment logic. The dual-output mechanism is more clinically practical than traditional classification models and can be used in an integrated intelligent system for state assessment + treatment trend prediction.

[0124] In a preferred embodiment of the present invention, the specific steps of constructing and training the combined evaluation model of CNN and LSTM include:

[0125] Collect medical data of diabetic foot patients and non-diabetic foot patients, and form a data set after preprocessing the processed fusion images and the historical index data of the patients;

[0126] Randomly divide the data set into a training set and a validation set, with 80% of the patient data as the training set and 20% of the patient data as the validation set;

[0127] The spatial feature extraction adopts the CNN module structure:

[0128] The first convolutional layer Conv1: the convolutional kernel is 3×3×32, and the ReLU activation function; the first pooling layer MaxPool: the pooling size is 2×2, and the stride is 2; the second convolutional layer Conv2: the convolutional kernel is 3×3×64, and the ReLU activation function; the second pooling layer MaxPool2: the pooling size is 2×2, and the stride is 2; feature flattening: flatten the two-dimensional feature map into a one-dimensional feature vector; fully connected layer: output the image feature vector F cnn ∈R 128 ;

[0129] Among them, the formula of the CNN module is expressed as: F cnn =f cnn (X image ; θ cnn ), θ cnn represents the set of trainable parameters of CNN;

[0130] The temporal feature extraction adopts the LSTM module structure:

[0131] Construct an LSTM network: input dimension: X history ∈R T×d , d represents each temporal feature dimension, output dimension: the hidden layer feature vector F at the last moment lstm ∈R 64 ;

[0132] The formula of the LSTM unit is as follows: input gate: i t =σ(W xi X t +W hi h t-1 +b i ), forget gate: f t =σ(W xf X t +W hf h t-1 +b f ), output gate: o t =σ(W xoX t +W ho h t-1 +b o )), cell state: c t = f t ⊙ c t-1 + i t ⊙ tanh(W xc X t +W hc h t-1 +b c )), hidden state: h t = o t ⊙ tanh(c t )), final temporal feature output: F lstm = h T , σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, i t represents the input gate, controlling the update degree of input features to the cell state, f t represents the forget gate, controlling the retention or discard degree of past information, o t represents the output gate, controlling the hidden state of the final output, c t represents the cell state, used for long-term memory information, h t represents the hidden state, the expression of the final time series feature, W ** and b * represent the trainable parameters of LSTM, where, W ** represents the weight, b * represents the bias;

[0133] CNN feature and LSTM feature fusion: F fusion = Concatenate(F cnn , F lstm ) ∈ R 192 ;

[0134] Construct a dual-output branch:

[0135] Output branch 1: Prediction of disease level classification, fully connected output + Softmax classification: P stage = Softmax(W s F fusion +b s ), P stage ∈ R 4 ; Final level output: Stage pred = argmax(P stage ), Stage pred ∈ {0, 1, 2, 3};

[0136] Output Branch 2: Prediction of the risk trend of disease evolution, fully connected output + Sigmoid probability regression:

[0137] S trend = σ(W t F fusion + b t ), S trend ∈ [0, 1];

[0138] The loss function is designed as the joint loss of the two outputs: Loss total = αLoss stage + βLoss trend , where Loss trend = (Y trend - S trend ) 2 , α and β represent the weight coefficients of the losses of the two subtasks, represents the actual disease level, represents the probability value that the model output predicts the i-th class level, Y trend represents the actual trend label, S trend represents the probability or score value of the model output trend prediction;

[0139] Optimization: Use the Adam optimizer with an initial value of 0.001 and gradually decay later;

[0140] Training: Initialize the model weight parameters, set the batch size to 16 - 32, and use the mini-batch gradient descent method: In each epoch, randomly batch extract data for training, update the model parameters for each batch, and after each round of epoch training, calculate the loss function on the validation set. If the validation set loss does not decrease for 5 consecutive epochs, stop training, and finally select the model parameters with the lowest validation set loss;

[0141] The model is verified on the validation set to complete the verification.

[0142] The CNN network structure of the present invention adopts a structure of two-layer convolution + two-layer pooling + flattening, which can effectively capture edge textures, color distributions, structural integrity, etc. The convolution kernels are reasonably designed and the ReLU activation improves the non-linear expression, enhancing the ability to recognize complex images. Finally, an image feature vector is output, which is fused in parallel with the LSTM features to form a deep feature semantic expression. The LSTM adjusts the information flow through the input gate, forget gate, and output gate, and can retain the key features in the long time series. It is suitable for processing information such as patient follow-up data and daily indicator records, revealing the "changing trend" rather than the static state of the condition, helping to predict the potential deterioration risk and intervene in advance to improve the timeliness of treatment. The fusion of CNN and LSTM realizes the feature combination with semantic enhancement. The image and time series features are fused by splicing to construct a multi-level high-dimensional representation, realizing the fusion judgment of spatial information (wound structure) and time information (condition trend), significantly enhancing the comprehensive recognition ability of the model. The dual-output structure + joint loss function enhances the coordination of the training objectives. The two sub-tasks adopt a joint loss function, which enables the model to balance the objectives of the two tasks during training, avoiding the overall degradation caused by overfitting of one side. The weight coefficient is adjustable to adapt to the primary and secondary task priorities in different clinical scenarios. The Adam optimizer can automatically adjust the learning rate to improve the convergence speed. The mini-batch gradient descent method combined with the Early-stopping mechanism avoids overfitting, and the separation of the training set and the validation set ensures the generalization ability of the model on new samples.

[0143] The embodiment of the present invention further provides an apparatus embodiment for implementing each step and method in the above method embodiment.

[0144] In a preferred embodiment of the present invention, the embodiment of the present invention discloses a device, including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned evaluation method for diabetic foot based on wound image recognition.

[0145] In a preferred embodiment of the present invention, the embodiment of the present invention discloses a computer-readable and writable storage medium, characterized in that the medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the above-mentioned evaluation method for diabetic foot based on wound image recognition are realized.

[0146] Since each unit module in this embodiment can execute Figure 1 the method shown, for the parts not described in detail in this embodiment, reference can be made to the relevant description of Figure 1 it.

[0147] At the hardware level, the device may include a processor, and optionally also an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory, etc. Of course, the device may also include other hardware required for other services.

[0148] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0149] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include internal memory and non-volatile memory, and provide instructions and data to the processor.

[0150] The steps of the method disclosed in combination with the embodiments of the present invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0151] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or implemented by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0152] For the convenience of description, when describing the above device, it is divided into various units or modules according to functions and described separately. Of course, when implementing the present invention, the functions of each unit or module can be implemented in the same or multiple software and / or hardware.

[0153] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0154] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the processes Figure 1 or blocks or combinations of blocks.

[0155] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the processes Figure 1 or blocks or combinations of blocks.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the processes Figure 1 or blocks or combinations of blocks.

[0157] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0158] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0159] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0161] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0163] Each embodiment in the present invention is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0164] The above are only the embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and changes can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. An evaluation method for diabetic foot based on wound image recognition, characterized in that, The method includes: S1. Obtain the foot infrared thermal imaging image and the foot color image of the target diabetic foot patient, and perform preprocessing on each of them. Process the foot infrared thermal imaging image to generate a temperature anomaly map; S2. Remove interference factors from the foot color image, mark the true wound of the diabetic foot to generate a foot wound image, and fuse the foot wound image with the temperature anomaly map to generate a fused image; S3. Construct a foot blood flow analysis model, perform feature analysis through the foot blood flow analysis model, extract blood flow influencing factors, and correct the fused image based on the blood flow influencing factors; S4. Construct a joint evaluation model, use the corrected fused image and the historical data of the diabetic foot patient as the input of the joint evaluation model, predict the risk assessment level and the evolution risk level of the diabetic foot, and output the disease assessment result of the diabetic foot patient according to the risk assessment level and the evolution risk level.

2. The method according to claim 1, wherein The processing of the foot infrared thermal imaging image to generate a temperature anomaly map specifically includes: S11. Convert the original gray value of the foot infrared thermal imaging image into an actual temperature value to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x,y) = a·I(x,y) + b, where T(x,y) represents the actual temperature value at pixel (x,y), I(x,y) represents the original gray value at pixel (x,y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager; S12. Perform Gaussian filtering on the converted temperature matrix to remove noise; S13. Use the foot color image to extract the foot area in the foot infrared thermal imaging image; S14. Mirror-align the foot regions of the left and right feet through the image coordinate system. The alignment formula is defined as: x' R = 2·X c - x L , y' R = y L , where x L and y L represent the pixel coordinates within the left foot region, x' R and y' R represent the mirror pixel coordinates of the right foot corresponding to the left foot, and X c represents the abscissa of the symmetry axis of the left and right feet; S15. Calculate the temperature difference between the symmetric positions of the two feet to obtain a temperature difference matrix, and perform binarization processing on the temperature difference matrix according to a preset temperature difference threshold to obtain an abnormal area mask; wherein, the temperature difference matrix is defined as: ΔT(x L ,y L ) = T L (x L ,y L ) - T R (x′ R ,y′ R ), where ΔT(x L ,y L ) represents the temperature difference between the corresponding positions of the left and right feet, T L (x L ,y L ) represents the temperature value at the position of the left foot, and T R (x′ R ,y′ R ) represents the temperature value at the corresponding mirror position of the right foot; the calculation formula for the abnormal area mask is: where A(x, y) represents the binary mask of the abnormal area, 1 represents abnormal, 0 represents normal, ΔT(x, y) represents the temperature difference between the pixels in the corresponding areas of the left and right feet, and ΔT thr represents the temperature difference threshold; S16. Mask the abnormal area and map it back to the foot infrared thermal imaging image to generate the temperature anomaly map I temp (x, y).

3. The method according to claim 2, wherein The specific content of S13 includes: S131. Perform color balance processing on the foot color image, convert it to the HSV color space, and segment the hue and saturation using a preset threshold to obtain a preliminary foot region: where H(x, y) and S(x, y) represent the hue and saturation in the HSV color space respectively, and M C (x, y) represents the foot mask of the preliminarily generated color image, and H min and H max represent the preset hue range of the foot skin color in the HSV space, and S min represents the lowest saturation threshold. S132. Process M C Perform morphological processing on (x, y) to repair edge discontinuities and obtain a smooth and continuous foot contour mask, i.e., M foot (x, y); S133. Perform Canny edge detection E on the color image I of the foot C (x, y) using Canny edge detection E C (x, y) = Canny(I C (x, y)) to obtain the edge map E C (x, y), and multiply the edge map E C (x, y) with M foot (x, y) to perform C foot (x, y) = E C (x, y) · M foot (x, y) operation to obtain the final foot contour map C foot (x, y); S134. Perform morphological closing operation on the final foot contour map C foot (x, y) to generate a continuous and closed contour. Among them, first perform the dilation operation: Then perform the erosion operation: Among them, S represents the structural element, and M final (x, y) represents the finally generated binary mask of the foot contour; S135. Align the foot contour mask M obtained from the foot infrared thermal imaging image with the foot color image, and perform correction using an affine transformation. M final (x, y) is aligned, and the affine transformation is applied for correction. M aligned (u, v) = T affine (M final (x, y)), where T affine represents the affine transformation for realizing the geometric registration of the color and infrared images, and M aligned (u, v) represents the binary mask of the foot after the affine transformation; S136. Region extraction is performed on the infrared thermal imaging image of the foot using the corrected foot mask to obtain the foot temperature map T foot (u, v), T foot (u, v) = I T (u, v) × M aligned (u, v), where I T (u, v) represents the infrared image after affine transformation, and T foot (u, v) represents the infrared temperature map of the extracted foot region.

4. The method according to claim 1, characterized in that, The specific content of fusing the foot wound image with the temperature anomaly map to generate a fused image in S2 includes: S21. Use the color segmentation method in the HSV color space to extract the suspected wound area of the foot color image, and define the threshold function: where H(x, y), S(x, y), and V(x, y) respectively represent the hue, saturation, and brightness of the pixel in the HSV color space, and H L , H H , S min , and V max respectively represent the preset thresholds for initially distinguishing the normal skin area from the suspected wound area; S22. Perform texture feature analysis on the suspected wound area M initial (x, y) to distinguish diabetic foot wounds from interfering factors, specifically using a gray-level co-occurrence matrix to calculate texture features, where I gray (x, y) represents the gray-scale conversion value of the foot color image, d represents the pixel distance, θ represents the analysis direction, and W and H respectively represent the width and height of the image; calculate the contrast and homogeneity of each pixel neighborhood; S23. Set the texture feature threshold function to remove the interference of obvious texture areas: Among them, Co(x, y) represents contrast, Ho(x, y) represents homogeneity, and T c and T h respectively represent texture feature thresholds, which are used to distinguish real wounds from natural skin textures and scab tissues; S24. Define the edge irregularity coefficient I of the wound area irregular , and set the irregularity threshold T irregular . Remove the common wound areas with higher regularity to obtain the final diabetic foot wound mask M wound (x, y) is as follows: Where P represents the perimeter of the suspected wound area and A represents the area of the region.

5. The method according to claim 4, characterized in that, The specific content of removing interference factors from the foot color image in S2 includes: S25. Perform image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x,y), I wound (x,y) = I C (x,y) × M wound (x,y) × γ, where I C (x,y) represents the original foot color image, and γ represents the enhancement coefficient; S26. Generate a fused image by fusing the foot wound image and the temperature anomaly map. Among them, define the fusion coefficient: Define the fused image: I fused (x,y) = w(x,y) × I temp (x,y) + [1 - w(x,y)] × I wound (x,y), where w(x,y) represents the fusion coefficient, I fused (x,y) represents the fused image, I temp (x,y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0 < λ < 1, and ε represents a very small positive number.

6. The method according to claim 1, characterized in that The specific content of S3 includes: S31. Obtain the blood perfusion images of diabetic foot patients and non-diabetic foot patients collected in advance, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain the denoised images. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′) - median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighboring pixels within the window, median represents the median function, T MAD represents the denoising threshold, and F′(x,y) represents the denoised image; S32. Calculate the first average blood perfusion intensity of the wound area of diabetic foot patients Calculate the second average blood perfusion intensity of the standard measurement area of non-diabetic foot patients And based on the first average blood perfusion intensity And the second average blood perfusion intensity Calculate the blood flow influence factor K F , where F D ′(x,y) represents the denoised blood perfusion image of diabetic foot patients, F N ′(x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the set of pixels in the wound area of the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of the diabetic foot patient group, ΩN,standard represents the set of pixels in the standard measurement area of the non-diabetic foot healthy control group, N N,standard represents the total number of pixels in the standard measurement area of the non-diabetic foot healthy control group, and ε represents a very small positive number; S33. Correct the fused image with the blood flow influencing factors, and define the formula for correcting the fused image: I corr (x,y) = I fused (x,y) × (1 + β × (1 - K F )),I corr (x,y) represents the fusion image corrected by blood flow, I fused (x,y) represents the fusion image, and β represents the image correction intensity coefficient.

7. The method according to claim 1, characterized in that The specific content of S3 includes: S31. Obtain the blood perfusion images of diabetic foot patients and non-diabetic foot patients collected in advance, obtain the blood perfusion image of the target diabetic foot patient, and use the median absolute deviation (MAD) to remove local abnormal blood flow values to obtain the denoised image. Among them, define the MAD within the local window: MAD(x,y) = median{|F(x′,y′) - median(F(x′,y′))|}, (x′,y′) ∈ W(x,y), and use MAD to remove outliers: W(x,y) represents the local window centered on the pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image; S32. Calculate the first average blood perfusion intensity of the wound area of diabetic foot patients Calculate the second average blood perfusion intensity of the standard measurement area of non-diabetic foot patients And based on the first average blood perfusion intensity And the second average blood perfusion intensity Calculate the blood flow influence factor K F , calculate the individual offset factor K of the target diabetic foot patient T , according to the blood flow influence factor K F And the individual offset factor K T Calculate the final correction factor K final ; where F D ′(x, y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x, y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the set of pixels in the wound area of the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of the diabetic foot patient group, ΩN,standard represents the set of pixels in the standard measurement area of the non-diabetic foot healthy control group, N N,standard represents the total number of pixels in the standard measurement area of the non-diabetic foot healthy control group, ε represents a very small positive number, represents the average blood flow intensity of the wound area of the blood perfusion image of the target diabetic foot patient, represents the average blood flow intensity of the healthy foot area of the blood perfusion image of the target diabetic foot patient, represents an adjustable parameter, S33. Correct the fused image by the final correction factor K final Correct the fused image and define the formula for the corrected fused image as follows: I corr (x,y) = I fused (x,y) × (1 + β × (1 - K final )),I corr (x,y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

8. The method according to claim 1, wherein The specific content of S4 includes: Construct and train a joint evaluation model combining CNN and LSTM; The corrected fused image I corr (x, y) and the historical data of diabetic foot patients with a preset time length are used as the input of the joint evaluation model, and the input data matrix is defined: X image = I corr (x, y), X image represents the image input data, and the size is uniformly 128×128×3, X t = [HbA1c t , blood glucose t , wound area t , number of infections t , BMI t , X history represents the historical data, and T represents the time length of the historical data; Output the risk assessment level and the evolution risk level of the diabetic foot according to the model, and output the disease assessment result of the diabetic foot patient according to the risk assessment level and the evolution risk level.

9. The method according to claim 8, wherein The specific content of constructing and training a joint evaluation model combining CNN and LSTM specifically includes: Collect the medical data of diabetic foot patients and non-diabetic foot patients, and form a data set after preprocessing according to the processed fused image and the historical index data of the patients; Randomly divide the data set into a training set and a validation set. 80% of the patient data is the training set, and 20% of the patient data is the validation set; Use the CNN module structure for spatial feature extraction: The first convolutional layer Conv1: a convolutional kernel of 3×3×32, with a ReLU activation function; the first pooling layer MaxPool: a pooling size of 2×2 and a stride of 2; the second convolutional layer Conv2: a convolutional kernel of 3×3×64, with a ReLU activation function; the second pooling layer MaxPool2: a pooling size of 2×2 and a stride of 2; feature flattening: flattening the two-dimensional feature map into a one-dimensional feature vector; a fully connected layer: outputting the image feature vector F cnn ∈R 128 ; Among them, the formula of the CNN module is expressed as: F cnn = f cnn (X image ; θ cnn ), where θ cnn represents the set of trainable parameters of the CNN; Use the LSTM module structure for temporal feature extraction: Constructing the LSTM network: Input dimension: X history ∈R T×d , where d represents the dimensionality of each time-series feature, and the output dimension is the hidden-layer feature vector F at the last time step lstm ∈R 64 ; The formulas for the LSTM unit are as follows: Input gate: i t = σ(W xi X t + W hi h t-1 + b i ), Forget gate: f t = σ(W xf X t + W hf h t-1 + b f ), Output gate: o t = σ(W xo X t + W ho h t-1 + b o ), Cell state: c t = f t ⊙ c t-1 + i t ⊙ tanh(W xc X t + W hc h t-1 + b c ), Hidden state: h t = o t ⊙ tanh(c t ), Final temporal feature output: F lstm = h T , σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, i t represents the input gate, controlling the degree of update of the input features to the cell state, f t represents the forget gate, controlling the degree of retention or discard of past information, o t represents the output gate, controlling the final output hidden state, c t represents the cell state, used for long-term memory of information, h t represents the hidden state, the expression of the final time series features, W ** and b * represent the trainable parameters of the LSTM, where, W ** represents the weight, b * represents the bias; CNN Feature and LSTM Feature Fusion: F fusion = Concatenate(F cnn , F lstm ) ∈ R 192 ; Construct a dual-output branch: Output Branch 1: Prediction of disease level classification, fully connected output + Softmax classification: P stage = Softmax(W s F fusion + b s ), P stage ∈ R 4 ; Final level output: Stage pred = argmax(P stage ), Stage pred ∈ {0, 1, 2, 3}; Output branch 2: Prediction of the risk trend of disease evolution, fully connected output + Sigmoid probability regression: S trend = σ(W t F fusion + b t ), S trend ∈ [0, 1]; The loss function is designed as the joint loss of two outputs: Loss total = αLoss stage + βLoss trend , where Loss trend = (Y trend - S trend ) 2 , α and β represent the weight coefficients of the losses of two subtasks, represents the actual disease level, represents the probability value that the model output predicts the i-th class level, Y trend represents the actual trend label, S trend represents the probability or score value of the trend prediction output by the model; Optimization: Use the Adam optimizer, with an initial value of 0.001 and subsequent gradual attenuation; Training: Initialize the model weight parameters, set the batch size to 16 - 32, and adopt mini-batch gradient descent: within each epoch, randomly batch-sample data for training, update the model parameters for each batch. After each round of epoch training, calculate the loss function on the validation set. If the validation set loss does not decrease for 5 consecutive epochs, stop the training, and finally select the model parameters with the lowest validation set loss; The model is verified on the validation set to complete the verification.

10. A computer-readable and writable storage medium, characterized in that, The medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 - 9 are implemented.

Images