A diabetic foot assessment method based on wound image recognition

Abstract

The present invention discloses a method for evaluating diabetic foot based on wound image recognition, the method comprising: processing a foot infrared thermal imaging image to generate a temperature anomaly map; fusing the foot wound image with the temperature anomaly map to generate a fused image; constructing a foot blood flow analysis model, extracting blood flow influencing factors, and correcting the fused image using the blood flow influencing factors; constructing a joint evaluation model, using the corrected fused image and historical data of diabetic foot patients as inputs to the joint evaluation model, predicting the risk assessment level of diabetic foot, and outputting a disease assessment result for diabetic foot patients based on the risk assessment level. The present invention implements multimodal medical image fusion analysis, improves the accuracy of wound area positioning and pathological state expression, improves the accuracy of lesion area identification, can predict the risk of disease aggravation in advance, assists clinical decision-making, reduces the risk of severe complications such as amputation, and improves patient prognosis and quality of life.

Description

Technical field

[0001] The present invention relates to the technical field of image processing, and in particular to a diabetic foot evaluation method based on wound surface image recognition. [Background Technology]

[0002] Diabetic foot (DF) is one of the serious complications of late-stage diabetes. It causes high disability and mortality rates, is difficult to treat, and has a long treatment cycle, which places a serious burden on patients.

[0003] Traditional diabetic foot assessment mostly relies on manual visual judgment and basic medical indicators, which have problems such as strong subjectivity, insufficient accuracy and delayed diagnosis, making it difficult to achieve early and accurate diagnosis and dynamic prediction of the disease.

[0004] Therefore, there is an urgent need for a more accurate diabetic foot assessment method that integrates multimodal data and has the ability to predict personalized risk trends, so as to improve the timeliness and accuracy of diagnosis and treatment of diabetic foot patients. [Summary of the invention]

[0005] In view of this, an embodiment of the present invention provides a method for evaluating diabetic foot based on wound surface image recognition.

[0006] In a first aspect, an embodiment of the present invention provides a method for evaluating diabetic foot based on wound surface image recognition, the method comprising:

[0007] S1. Obtain an infrared thermal imaging image and a color image of the foot of a target diabetic foot patient, perform preprocessing on each image, and process the infrared thermal imaging image of the foot to generate a temperature anomaly map;

[0008] S2, removing interference factors from the foot color wound image, marking the real diabetic foot wound to generate a foot wound image, and fusing the foot wound image with the temperature anomaly map to generate a fused image;

[0009] S3. Constructing a foot blood flow analysis model, performing feature analysis based on the foot blood flow analysis model, extracting blood flow influencing factors, and correcting the fused image based on the blood flow influencing factors;

[0010] S4. Construct a joint evaluation model, use the corrected fusion image and historical data of diabetic foot patients as input of the joint evaluation model, predict the risk assessment level of diabetic foot, and output the disease assessment results of diabetic foot patients based on the risk assessment level.

[0011] According to the above aspects and any possible implementation, a further implementation is provided, wherein processing the foot infrared thermal imaging image to generate a temperature anomaly map specifically includes:

[0012] S11. Convert the original grayscale values ​​of the infrared thermal imaging image of the foot into actual temperature values ​​to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x,y)=a·I(x,y)+b, where T(x,y) represents the actual temperature value at the pixel (x,y), I(x,y) represents the original grayscale value at the pixel (x,y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager.

[0013] S12, performing Gaussian filtering on the converted temperature matrix to remove noise;

[0014] S13, extracting a foot region from the foot infrared thermal imaging image using the foot color image;

[0015] S14. Mirror-align the foot regions of the left and right feet using the image coordinate system. The alignment formula is defined as: R =2·X c -x L , y′ R =y L , where x L and y L represents the pixel coordinates in the left foot area, x′ R and y′ R Indicates the pixel coordinates of the right foot mirror image corresponding to the left foot, X c Indicates the horizontal coordinate of the symmetrical midline of the left and right feet;

[0016] S15, calculate the temperature difference of the symmetrical positions of the two feet to obtain a temperature difference matrix, and perform binarization processing on the temperature difference matrix according to the preset temperature difference threshold to obtain an abnormal area mask; wherein the temperature difference matrix is ​​defined as: ΔT(x L ,y L )=T L (x L ,y L )-T R (x′ R ,y′ R ), where ΔT(x L ,y L ) represents the temperature difference between the left and right foot positions, T L (x L ,y L ) represents the temperature value of the left foot position, T R (x′ R ,y′ R ) represents the temperature value of the mirror image position corresponding to the right foot; the calculation formula of the abnormal area mask is: Among them, A(x,y) represents the binary mask of the abnormal area, 1 represents abnormality, 0 represents normality, ΔT(x,y) represents the temperature difference of the pixels in the corresponding areas of the left and right feet, ΔT thrIndicates the temperature difference threshold;

[0017] S16, mapping the abnormal area mask back to the foot infrared thermal imaging image to generate a temperature anomaly map I temp (x,y).

[0018] According to the above aspects and any possible implementation, an implementation is further provided, wherein S13 specifically includes:

[0019] S131. Perform color balancing on the foot color image and convert it to the HSV color space. Then, use a pre-set threshold to segment the hue and saturation to obtain a preliminary foot region: Among them, H(x,y) and S(x,y) represent the hue and saturation in the HSV color space respectively, and M C (x,y) represents the initial generated color image foot mask, H min and H max Indicates the hue range of the foot skin color in the HSV space, S min Indicates the minimum saturation threshold, indicating;

[0020] S132, to M C (x,y) uses morphological processing to repair edge discontinuities and obtain a smooth and coherent foot contour mask, namely M foot (x,y);

[0021] S133, foot color image I C (x,y) using Canny edge detection E C (x,y)=Canny(I C (x,y)) is processed to obtain the edge graph E C (x,y), the edge graph E C (x,y) and M foot (x,y) performs C foot (x,y)=E C (x,y)·M foot (x,y) operation to obtain the final foot contour map C foot (x,y);

[0022] S134, final foot contour diagram C foot (x, y) performs morphological closing operation to generate a continuous closed contour, where the expansion operation is performed first: Then perform the corrosion operation: Among them, S represents the structural element, M final (x,y) represents the final generated foot contour binary mask;

[0023] S135, combining the foot infrared thermal imaging image and the foot color image to obtain the foot contour mask M final (x,y) alignment, and apply affine transformation for correction, M aligned (u,v)=T affine (M final (x,y)), where T affine represents the affine transformation used to achieve geometric registration of color and infrared images, M aligned (u,v) represents the binary mask of the foot after affine transformation;

[0024] S136: Using the corrected foot mask, perform region extraction on the foot infrared thermal imaging image to obtain a foot temperature map T foot (u,v),T foot (u,v)=I T (u,v)×M aligned (u,v), where, I T (u,v) represents the infrared image after affine transformation, T foot (u,v) represents the extracted infrared temperature map of the foot area.

[0025] According to the above aspects and any possible implementation, an implementation is further provided, wherein the step of fusing the foot wound image with the temperature anomaly map to generate a fused image in S2 specifically includes:

[0026] S21. Use the color segmentation method in the HSV color space to extract the suspected wound area in the foot color wound image and define the threshold function: Among them, H(x,y), S(x,y) and V(x,y) represent the hue, saturation and brightness of the pixel in the HSV color space respectively. L 、H H 、S min and V max They represent preset thresholds, respectively, used to preliminarily distinguish normal skin areas from suspected wound areas;

[0027] S22, suspected wound area M initial (x,y) is used to perform texture feature analysis to distinguish diabetic foot wounds from interference factors. Specifically, the gray level co-occurrence matrix is ​​used to calculate the texture features. Among them, I gray (x,y) represents the grayscale conversion value of the foot color image, d represents the pixel distance, θ represents the analysis direction, W and H represent the width and height of the image respectively; calculate the contrast and homogeneity of each pixel neighborhood;

[0028] S23. Set the texture feature threshold function to remove interference from the obvious texture area: Among them, Co(x,y) represents contrast, Ho(x,y) represents homogeneity, T c and T h Represents the texture feature threshold, which is used to distinguish real wounds from natural skin texture and scab tissue;

[0029] S24. Define the edge irregularity coefficient I of the wound area irregular , set the irregularity threshold T irregular , remove the common wound area with higher regularity, and obtain the final diabetic foot wound mask M wound (x,y) is as follows: Where P represents the perimeter of the suspected wound area, and A represents the area of ​​the area.

[0030] According to the above aspects and any possible implementation, a further implementation is provided, wherein the interference factors of the foot color mouth image are removed in S2, specifically including:

[0031] S25, performing image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x,y),I wound (x,y)=I C (x,y)×M wound (x,y)×γ, where I C (x, y) represents the original foot color image, γ represents the enhancement coefficient;

[0032] S26. Fusing the foot wound image with the temperature anomaly map to generate a fused image, wherein a fusion coefficient is defined as: Define the fused image: I fused (x,y)=w(x,y)×I temp (x,y)+[1-w(x,y)]×I wound (x,y), where w(x,y) represents the fusion coefficient, I fused (x,y) represents the fused image, I temp (x, y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0<λ<1, and ε represents a very small positive number.

[0033] According to the above aspects and any possible implementation, an implementation is further provided, wherein S3 specifically includes:

[0034] S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0035] S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F ,in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,standard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, and N N,standard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, and ε represents a very small positive number;

[0036] S33. The blood flow influencing factor is used to correct the fused image, and the correction formula of the fused image is defined as follows:

[0037] I corr (x,y)=I fused (x,y)×(1+β×(1-K F )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0038] According to the above aspects and any possible implementation, an implementation is further provided, wherein S3 specifically includes:

[0039] S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, obtain a blood perfusion image of a target diabetic foot patient, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0040] S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F , calculate the target sugar diabetic foot patient's individual deviation factor K T , according to the blood flow influencing factor K F and individual offset factor K T Calculate the final correction factor K final ;in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,standard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, and N N,standard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, ε represents a very small positive number, represents the average blood flow intensity in the wound area of ​​the blood perfusion image of the target diabetic foot patient, represents the average blood flow intensity of the healthy foot area of ​​the target diabetic foot patient's blood perfusion image, Indicates adjustable parameters,

[0041] S33, through the final correction factor K finalCorrect the fused image and define the corrected fused image formula:

[0042] I corr (x,y)=I fused (x,y)×(1+β×(1-K final )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0043] According to the above aspects and any possible implementation, an implementation is further provided, wherein S4 specifically includes:

[0044] Build and train a joint evaluation model combining CNN and LSTM;

[0045] The corrected fused image I corr (x, y) and historical data of diabetic foot patients of a preset time length are used as the input of the joint evaluation model, and the input data matrix is ​​defined as: image =I corr (x,y), X image Represents image input data, with a uniform size of 128×128×3. X t =[HbA1c t ,blood sugar t , wound area t , number of infections t , BMI t ], X history represents historical data, and T represents the time length of historical data;

[0046] The risk assessment level and evolutionary risk level of diabetic foot are output according to the model, and the disease assessment results of diabetic foot patients are output according to the risk assessment level and evolutionary risk level.

[0047] Regarding the above aspects and any possible implementation, a further implementation is provided, wherein the constructing and training of a joint evaluation model combining CNN and LSTM specifically includes:

[0048] Medical data of diabetic and non-diabetic foot patients were collected and pre-processed based on the processed fusion images and patient historical indicator data to form a data set;

[0049] The dataset was randomly divided into training and validation sets, with 80% of the patient data used as the training set and 20% of the patient data used as the validation set;

[0050] Spatial feature extraction uses the CNN module structure:

[0051] The first convolutional layer Conv1: convolution kernel 3×3×32, ReLU activation function; the first pooling layer MaxPool: pooling size 2×2, step size 2; the second convolutional layer Conv2: convolution kernel 3×3×64, ReLU activation function; the second pooling layer MaxPool2: pooling size 2×2, step size 2; feature flattening: flatten the two-dimensional feature map into a one-dimensional feature vector; fully connected layer: output image feature vector F cnn ∈R 128 ;

[0052] Among them, the CNN module formula is expressed as: F cnn =f cnn (X image θ cnn ),θ cnn Represents the set of CNN trainable parameters;

[0053] Time series feature extraction uses the LSTM module structure:

[0054] Construct LSTM network: Input dimension: X history ∈R T×d , d represents each time series feature dimension, output dimension: the last moment hidden layer feature vector F lstm ∈R 64 ;

[0055] The formula of LSTM unit is as follows: Input gate: i t =σ(W xi X t +W hi h t-1 +b i ), forget gate: f t =σ(W xf X t +W hf h t-1 +b f ), output gate: o t =σ(W xo X t +W ho h t-1 +b o ), cell state: c t =f t ⊙c t-1 +i t ⊙tanh(W xc X t +W hc h t-1 +b c ), hidden state: h t =o t ⊙tanh(c t), the final time series feature output: F lstm =h T , σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, i t Represents the input gate, which controls the degree of update of the cell state by the input feature, f t Represents the forget gate, which controls the degree of retention or discarding of past information. t Represents the output gate, which controls the hidden state of the final output, c t Represents the cell state, used for long-term memory information, h t Represents the hidden state, the expression of the final time series feature, W ** and b * Represents the LSTM trainable parameters, where W ** represents the weight, b * Indicates bias;

[0056] Fusion of CNN features and LSTM features: F fusion =Concatenate(F cnn ,F lstm )∈R 192 ;

[0057] Build a dual-output branch:

[0058] Output branch 1: disease level classification prediction, full connection output + Softmax classification: P stage =Softmax(W s F fusion +b s ),P stage ∈R 4 ;Final level output: Stage pred =argmax(P stage ),Stage pred ∈{0, 1, 2, 3};

[0059] Output branch 2: disease evolution risk trend prediction, fully connected output + Sigmoid probability regression:

[0060] S trend =σ(W t F fusion +b t ),S trend ∈[0,1];

[0061] The loss function is designed as the joint loss of two outputs: Loss total =αLoss stage +βLoss trend ,in, Loss trend =(Ytrend -S trend ) 2 , α and β represent the weight coefficients of the two subtask losses, Indicates the actual disease level, Indicates the probability value of the model output prediction being the i-th category level, Y trend Indicates the actual trend label, S trend Indicates the probability or score value of the model output trend prediction;

[0062] Optimization: Adam optimizer is used with an initial value of 0.001 and subsequent gradual decay;

[0063] Training: Initialize the model weight parameters, set the batch size to 16-32, and use the mini-batch gradient descent method: within each epoch, randomly extract data in batches for training, update the model parameters in each batch, and calculate the loss function on the validation set after each training epoch. If the validation set loss does not decrease for 5 consecutive epochs, stop training and finally select the model parameters with the lowest validation set loss;

[0064] The model is validated on the validation set to complete the validation.

[0065] In a second aspect, an embodiment of the present invention provides a computer-readable and writable storage medium, wherein the medium stores computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the above method.

[0066] One of the above technical solutions has the following beneficial effects:

[0067] The method of the embodiment of the present invention proposes a diabetic foot assessment method based on wound image recognition. The method realizes multimodal medical image fusion analysis, improves the accuracy of wound area positioning and pathological state expression, and introduces a blood flow feature model for image correction, thereby improving the accuracy of lesion area identification. The CNN-LSTM deep learning model is adopted to integrate the patient's historical time series data to realize personalized dynamic assessment of disease progression, significantly improve the accuracy and timeliness of diagnostic prediction, and realize intelligent identification, quantitative evaluation and evolution trend prediction of diabetic foot wound state. Therefore, the present invention can predict the risk of disease aggravation in advance, assist clinical decision-making, reduce the risk of serious complications such as amputation in patients, and improve the patient's prognosis and quality of life.

Brief Description of the Drawings

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0069] Figure 1 is a flow chart of S1-S4 of a diabetic foot assessment method based on wound surface image recognition provided by an embodiment of the present invention;

[0070] Figure 2 This is a flow chart of the method S11-S16 provided in an embodiment of the present invention. [Specific implementation method]

[0071] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] Please refer to Figure 1 , which is a flow chart of S1-S4 of a diabetic foot assessment method based on wound surface image recognition provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps:

[0073] S1. Obtain an infrared thermal imaging image and a color image of the foot of a target diabetic foot patient, perform preprocessing on each image, and process the infrared thermal imaging image of the foot to generate a temperature anomaly map;

[0074] S2, removing interference factors from the foot color wound image, marking the real diabetic foot wound to generate a foot wound image, and fusing the foot wound image with the temperature anomaly map to generate a fused image;

[0075] S3. Constructing a foot blood flow analysis model, performing feature analysis based on the foot blood flow analysis model, extracting blood flow influencing factors, and correcting the fused image based on the blood flow influencing factors;

[0076] S4. Construct a joint evaluation model, use the corrected fusion image and historical data of diabetic foot patients as input of the joint evaluation model, predict the risk assessment level of diabetic foot, and output the disease assessment results of diabetic foot patients based on the risk assessment level.

[0077] The present invention collects infrared thermal imaging images and color images of the foot to construct a basic multimodal image information source, and converts the infrared image into a temperature anomaly map to achieve a quantitative expression of the thermal distribution of the foot skin, providing a basis for judging local inflammation, infection, and ischemia, and is beneficial for the early potential lesion area that has not yet ruptured; identifies the real wound area of ​​diabetic foot, and at the same time eliminates interference factors such as artifacts, scabs, and shadows, and fuses the wound information with the temperature anomaly information through image fusion. The fused image can provide "heat + shape" dual features, effectively enhancing the system's ability to distinguish early wounds and infections, and improving the information density and diagnostic value of the input image of the subsequent analysis model; uses blood The fluid perfusion intensity reflects the tissue activity and healing potential of the wound area, providing a supplementary indicator beyond visual information. By establishing a correction mechanism for comparing healthy feet with affected feet, the fused image is further corrected for physiological characteristics, thereby improving the generalization and stability of subsequent models in actual clinical data. Through a joint evaluation model, the corrected fused image and the patient's historical data are comprehensively analyzed to achieve multi-level intelligent classification of diabetic foot risks. At the same time, it has the ability to predict disease trends, providing doctors with predictive auxiliary judgment tools to facilitate early intervention and personalized treatment. It significantly improves the objectivity and automation level of the evaluation, avoiding misjudgments and delays caused by different doctors' experience.

[0078] Please refer to Figure 2 In a preferred embodiment of the present invention, the step of processing the foot infrared thermal imaging image to generate a temperature anomaly map specifically includes:

[0079] S11. Convert the original grayscale values ​​of the infrared thermal imaging image of the foot into actual temperature values ​​to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x,y)=a·I(x,y)+b, where T(x,y) represents the actual temperature value at the pixel (x,y), I(x,y) represents the original grayscale value at the pixel (x,y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager.

[0080] S12, performing Gaussian filtering on the converted temperature matrix to remove noise;

[0081] S13, extracting a foot region from the foot infrared thermal imaging image using the foot color image;

[0082] S14. Mirror-align the foot regions of the left and right feet using the image coordinate system. The alignment formula is defined as: R =2·X c -x L , y′ R =y L , where x L and y L represents the pixel coordinates in the left foot area, x′R and y′ R Indicates the pixel coordinates of the right foot mirror image corresponding to the left foot, X c Indicates the horizontal coordinate of the symmetrical midline of the left and right feet;

[0083] S15, calculate the temperature difference of the symmetrical positions of the two feet to obtain a temperature difference matrix, and perform binarization processing on the temperature difference matrix according to the preset temperature difference threshold to obtain an abnormal area mask; wherein the temperature difference matrix is ​​defined as: ΔT(x L ,y L )=T L (x L ,y L )-T R (x′ R ,y′ R ), where ΔT(x L ,y L ) represents the temperature difference between the left and right foot positions, T L (x L ,y L ) represents the temperature value of the left foot position, T R (x′ R ,y′ R ) represents the temperature value of the mirror image position corresponding to the right foot; the calculation formula of the abnormal area mask is: Among them, A(x,y) represents the binary mask of the abnormal area, 1 represents abnormality, 0 represents normality, ΔT(x,y) represents the temperature difference of the pixels in the corresponding areas of the left and right feet, ΔT thr Indicates the temperature difference threshold;

[0084] S16, mapping the abnormal area mask back to the foot infrared thermal imaging image to generate a temperature anomaly map I temp (x,y).

[0085] The embodiments of the present invention provide an accurate and standardized data basis for subsequent temperature difference calculation and thermal anomaly identification by physically quantifying image information. Gaussian filtering noise reduction reduces the interference of random noise in thermal imaging on temperature difference identification. By comparing the temperature difference of symmetrical parts of the left and right feet through mirror registration, it can effectively detect single-foot thermal anomalies in the early stage of diabetic foot. By setting the temperature difference threshold to generate an abnormal area mask, potential lesion areas can be efficiently located, significantly improving the sensitivity of system detection.

[0086] In a preferred embodiment of the present invention, the S13 specifically includes:

[0087] S131. Perform color balancing on the foot color image and convert it to the HSV color space. Then, use a pre-set threshold to segment the hue and saturation to obtain a preliminary foot region: Among them, H(x,y) and S(x,y) represent the hue and saturation in the HSV color space respectively, and M C (x,y) represents the initial generated color image foot mask, H min and H max Indicates the hue range of the foot skin color in the HSV space, S min Indicates the minimum saturation threshold, indicating;

[0088] S132, to M C (x,y) uses morphological processing to repair edge discontinuities and obtain a smooth and coherent foot contour mask, namely M foot (x,y);

[0089] S133, foot color image I C (x,y) using Canny edge detection E C (x,y)=Canny(I C (x,y)) is processed to obtain the edge graph E C (x,y), the edge graph E C (x,y) and M foot (x,y) performs C foot (x,y)=E C (x,y)·M foot (x,y) operation to obtain the final foot contour map C foot (x,y);

[0090] S134, final foot contour diagram C foot (x, y) performs morphological closing operation to generate a continuous closed contour, where the expansion operation is performed first: Then perform the corrosion operation: Among them, S represents the structural element, M final (x,y) represents the final generated foot contour binary mask;

[0091] S135, combining the foot infrared thermal imaging image and the foot color image to obtain the foot contour mask M final (x,y) alignment, and apply affine transformation for correction, M aligned (u,v)=T affine (M final (x,y)), where T affine represents the affine transformation used to achieve geometric registration of color and infrared images, M aligned (u,v) represents the binary mask of the foot after affine transformation;

[0092] S136: Using the corrected foot mask, perform region extraction on the foot infrared thermal imaging image to obtain a foot temperature map T foot(u,v),T foot (u,v)=I T (u,v)×M aligned (u,v), where, I T (u,v) represents the infrared image after affine transformation, T foot (u,v) represents the extracted infrared temperature map of the foot area.

[0093] The embodiments of the present invention, through multiple sub-steps involving color space conversion, image morphological operations, edge detection, image registration, and region extraction, can achieve accurate extraction of foot regions, image registration, and regional heat map construction, providing a high-quality spatial foundation for infrared thermal imaging data analysis. Specifically, the multi-stage image processing chain (i.e., color balancing, HSV space, morphological operations, edge detection, and affine correction) significantly improves the accuracy and robustness of image registration. The morphological closing operation makes the extraction results smoother and more closed, ensuring complete foot contour recognition. The affine transformation automatically corrects deviations and effectively resolves image misalignment caused by patient posture, shooting angle, or equipment errors.

[0094] In a preferred embodiment of the present invention, the step S2 of fusing the foot wound image with the temperature anomaly map to generate a fused image specifically includes:

[0095] S21. Use the color segmentation method in the HSV color space to extract the suspected wound area in the foot color wound image and define the threshold function: Among them, H(x,y), S(x,y) and V(x,y) represent the hue, saturation and brightness of the pixel in the HSV color space respectively. L 、H H 、S min and V max They represent preset thresholds, respectively, used to preliminarily distinguish normal skin areas from suspected wound areas;

[0096] S22, suspected wound area M initial (x,y) is used to perform texture feature analysis to distinguish diabetic foot wounds from interference factors. Specifically, the gray level co-occurrence matrix is ​​used to calculate the texture features. Among them, I gray (x,y) represents the grayscale conversion value of the foot color image, d represents the pixel distance, θ represents the analysis direction, W and H represent the width and height of the image respectively; calculate the contrast and homogeneity of each pixel neighborhood;

[0097] S23. Set the texture feature threshold function to remove interference from the obvious texture area: Among them, Co(x,y) represents contrast, Ho(x,y) represents homogeneity, T c and T hRepresents the texture feature threshold, which is used to distinguish real wounds from natural skin texture and scab tissue;

[0098] S24. Define the edge irregularity coefficient I of the wound area irregular , set the irregularity threshold T irregular , remove the common wound area with higher regularity, and obtain the final diabetic foot wound mask M wound (x,y) is as follows: Where P represents the perimeter of the suspected wound area, and A represents the area of ​​the area.

[0099] The embodiment of the present invention constructs a complete chain of "real diabetic foot wound identification" by step-by-step screening from color (HSV), texture (grayscale co-occurrence matrix) and structural shape (irregularity). Through triple screening, it significantly reduces misidentification and missed identification, and the identification results are closer to clinical reality. It effectively filters out false positive areas caused by blood scabs, skin structure, shadows, indentations, etc. This method does not rely on image brightness or color temperature, and can adapt to various actual shooting environments such as skin color differences and lighting changes. It significantly reduces the misjudgment rate through multi-dimensional intelligent analysis.

[0100] In a preferred embodiment of the present invention, removing interference factors from the foot color mouth image in S2 specifically includes:

[0101] S25, performing image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x,y),I wound (x,y)=I C (x,y)×M wound (x,y)×γ, where I C (x, y) represents the original foot color image, γ represents the enhancement coefficient;

[0102] S26. Fusing the foot wound image with the temperature anomaly image to generate a fused image, wherein a fusion coefficient is defined as: Define the fused image: I fused (x,y)=w(x,y)×I temp (x,y)+[1-w(x,y)]×I wound (x,y), where w(x,y) represents the fusion coefficient, I fused (x,y) represents the fused image, I temp (x, y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0<λ<1, and ε represents a very small positive number.

[0103] The embodiments of the present invention improve image contrast and detail presentation through image enhancement, which helps to enhance the model's learning of wound boundaries and tissue features. By introducing a fusion coefficient through information fusion to control the image synthesis process, it effectively combines thermal anomaly information and structural information to generate images rich in multi-source features, enhance image clarity and fuse multimodal data, thereby improving the input quality of the overall evaluation model and improving classification accuracy.

[0104] In a preferred embodiment of the present invention, S3 specifically includes:

[0105] S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0106] S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F ,in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,standard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, and N N,standard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, and ε represents a very small positive number;

[0107] S33. The blood flow influencing factor is used to correct the fused image, and the correction formula of the fused image is defined as follows:

[0108] I corr (x,y)=I fused(x,y)×(1+β×(1-K F )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0109] The embodiment of the present invention uses blood perfusion images of a diabetic foot group and a healthy control group, and after denoising through a median absolute deviation algorithm, calculates the difference in average perfusion intensity between the groups. Based on this difference, a blood flow influencing factor is constructed, and the fused image is corrected to introduce blood flow physiological properties in addition to structural and temperature characteristics into the image, thereby improving the physiological rationality of image assessment. The entire method does not need to rely on the perfusion images of the target patients and is suitable for the deployment of telemedicine and simplified screening models.

[0110] Among them, non-contact laser Doppler imaging or similar microcirculation imaging technology is used to obtain blood perfusion images.

[0111] In a preferred embodiment of the present invention, S3 specifically includes:

[0112] S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, obtain a blood perfusion image of a target diabetic foot patient, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image;

[0113] S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F , calculate the target sugar diabetic foot patient's individual deviation factor K T , according to the blood flow influencing factor K F and individual offset factor K T Calculate the final correction factor K final ;in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,standard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, and N N,standard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, ε represents a very small positive number, represents the average blood flow intensity in the wound area of ​​the blood perfusion image of the target diabetic foot patient, represents the average blood flow intensity of the healthy foot area of ​​the target diabetic foot patient's blood perfusion image, Indicates adjustable parameters,

[0114] S33, through the final correction factor K final Correct the fused image and define the corrected fused image formula:

[0115] I corr (x,y)=I fused (x,y)×(1+β×(1-K final )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

[0116] The embodiment of the present invention collects the foot perfusion images of the diabetic foot patient to be evaluated, and performs MAD denoising processing to obtain high-quality individual perfusion data. Based on the difference in average blood perfusion intensity between the target patient's foot wound area and the healthy foot area, an individual offset factor is constructed, and then the group influence factor is weightedly fused with the individual offset factor to obtain the final image correction factor. The final correction factor is applied to the fused image to achieve personalized image physiological correction for the target patient. On the basis of constructing the blood flow influence factor based on group data, the individual blood perfusion image of the target diabetic foot patient is further introduced to construct an individual offset factor to form a final correction factor and perform personalized correction on the fused image. This solution not only retains the stability, versatility, and low-cost characteristics of the group model, but also realizes a personalized image optimization mechanism that is highly adapted to the current patient status by introducing individual feature calculation.

[0117] In a preferred embodiment of the present invention, the S4 specifically includes:

[0118] Build and train a joint evaluation model combining CNN and LSTM;

[0119] The corrected fused image I corr (x, y) and historical data of diabetic foot patients of a preset time length are used as the input of the joint evaluation model, and the input data matrix is ​​defined as: image =I corr (x,y), X image Represents image input data, with a uniform size of 128×128×3. X t =[HbA1c t ,blood sugar t , wound area t , number of infections t , BMI t ], X history represents historical data, and T represents the time length of historical data;

[0120] The risk assessment level and evolutionary risk level of diabetic foot are output according to the model, and the disease assessment results of diabetic foot patients are output according to the risk assessment level and evolutionary risk level.

[0121] It should be noted that the reference risk assessment levels for diabetic foot are: Level 0 (no wound or mild redness and swelling, no ulcer), Level 1 (superficial ulcer or local infection), Level 2 (deep ulcer, spreading but controllable infection), and Level 3 (osteomyelitis or pre-gangrene). The specific setting standards can be adjusted according to actual needs.

[0122] The reference evolution risk levels are: <0.3 (low risk), 0.3-0.6 (medium risk), and >0.6 (high risk). The specific setting standards can be adjusted according to actual needs.

[0123] The embodiment of the present invention constructs a joint model combining convolutional neural networks and long short-term memory networks, and jointly inputs the corrected fused image and historical disease data of diabetic foot patients, thereby realizing risk level judgment and disease evolution trend prediction of diabetic foot patients; by fusing two heterogeneous data types, image input and historical data input sequence, into the model, the expression ability and reasoning depth of the model are significantly enhanced. CNN is used to process static image features and is good at identifying local spatial structures, boundary shapes, color contrast, etc. LSTM is suitable for processing time series data and can capture the fluctuation pattern of the patient's condition over a period of time. The joint architecture of CNN+LSTM has the ability to jointly judge from the "current state" and "past development", and constructs a "space + time" two-dimensional disease assessment logic. The dual-output mechanism is more clinically practical than the traditional classification model, and can be used for an integrated intelligent system of state assessment + treatment trend prediction.

[0124] In a preferred embodiment of the present invention, the construction and training of a joint evaluation model combining CNN and LSTM specifically includes:

[0125] Medical data of diabetic and non-diabetic foot patients were collected and pre-processed based on the processed fusion images and patient historical indicator data to form a data set;

[0126] The dataset was randomly divided into training and validation sets, with 80% of the patient data used as the training set and 20% of the patient data used as the validation set;

[0127] Spatial feature extraction uses the CNN module structure:

[0128] The first convolutional layer Conv1: convolution kernel 3×3×32, ReLU activation function; the first pooling layer MaxPool: pooling size 2×2, step size 2; the second convolutional layer Conv2: convolution kernel 3×3×64, ReLU activation function; the second pooling layer MaxPool2: pooling size 2×2, step size 2; feature flattening: flatten the two-dimensional feature map into a one-dimensional feature vector; fully connected layer: output image feature vector F cnn ∈R 128 ;

[0129] Among them, the CNN module formula is expressed as: F cnn =f cnn (X image θ cnn ),θ cnn Represents the set of CNN trainable parameters;

[0130] Time series feature extraction uses the LSTM module structure:

[0131] Construct LSTM network: Input dimension: X history ∈R T×d , d represents each time series feature dimension, output dimension: the hidden layer feature vector F at the last moment lstm ∈R 64 ;

[0132] The formula of LSTM unit is as follows: Input gate: i t =σ(W xi X t +W hi h t-1 +b i ), forget gate: f t =σ(W xf X t +W hf h t-1 +b f ), output gate: o t =σ(W xoX t +W ho h t-1 +b o ), cell state: c t =f t ⊙c t-1 +i t ⊙tanh(W xc X t +W hc h t-1 +b c ), hidden state: h t =o t ⊙tanh(c t ), the final time series feature output: F lstm =h T , σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, i t Represents the input gate, which controls the degree of update of the cell state by the input feature, f t Represents the forget gate, which controls the degree of retention or discarding of past information. t Represents the output gate, which controls the hidden state of the final output, c t Represents the cell state, used for long-term memory information, h t Represents the hidden state, the expression of the final time series feature, W ** and b * Represents the LSTM trainable parameters, where W ** represents the weight, b * Indicates bias;

[0133] Fusion of CNN features and LSTM features: F fusion =Concatenate(F cnn ,F lstm )∈R 192 ;

[0134] Build a dual-output branch:

[0135] Output branch 1: disease level classification prediction, full connection output + Softmax classification: P stage =Softmax(W s F fusion +b s ),P stage ∈R 4 ;Final level output: Stage pred =argmax(P stage ),Stage pred ∈{0, 1, 2, 3};

[0136] Output branch 2: disease evolution risk trend prediction, fully connected output + Sigmoid probability regression:

[0137] S trend =σ(W t F fusion +b t ),S trend ∈[0,1];

[0138] The loss function is designed as the joint loss of two outputs: Loss total =αLoss stage +βLoss trend ,in, Loss trend =(Y trend -S trend ) 2 , α and β represent the weight coefficients of the two subtask losses, Indicates the actual disease level, Indicates the probability value of the model output prediction being the i-th category level, Y trend Indicates the actual trend label, S trend Indicates the probability or score value of the model output trend prediction;

[0139] Optimization: Adam optimizer is used with an initial value of 0.001 and subsequent gradual decay;

[0140] Training: Initialize the model weight parameters, set the batch size to 16-32, and use the mini-batch gradient descent method: within each epoch, randomly extract data in batches for training, update the model parameters in each batch, and calculate the loss function on the validation set after each training epoch. If the validation set loss does not decrease for 5 consecutive epochs, stop training and finally select the model parameters with the lowest validation set loss;

[0141] The model is validated on the validation set to complete the validation.

[0142] The CNN network structure of the present invention adopts a two-layer convolution + two-layer pooling + flattening structure, which can effectively capture edge texture, color distribution, structural integrity, etc. The convolution kernel is reasonably designed and ReLU activation improves nonlinear expression, enhances the ability to recognize complex images, and finally outputs image feature vectors, which are fused in parallel with LSTM features to form a deep feature semantic expression; LSTM regulates the information flow through input gate, forget gate, and output gate, and can retain key features in long time series, which is suitable for processing patient follow-up data, daily indicator records and other information, revealing the "changing trend" of the disease rather than the static state, which helps to predict potential deterioration risks, intervene in advance, and improve the timeliness of treatment; and the fusion of CNN and LSTM realizes the semantic enhancement of feature combination, which combines image and Time series features are fused through splicing to construct a multi-level high-dimensional representation, realizing the fusion judgment of spatial information (wound structure) and temporal information (disease trend), significantly improving the comprehensive recognition ability of the model; the dual-output structure + joint loss function enhances the synergy of training objectives, and the two subtasks use a joint loss function. This mechanism enables the model to balance the objectives of the two tasks during training, avoiding overfitting of one side and causing overall degradation. The weight coefficient is adjustable to adapt to the priority of primary and secondary tasks in different clinical scenarios; the Adam optimizer can automatically adjust the learning rate to improve the convergence speed, and the small batch gradient descent method is combined with the early-stopping mechanism to avoid overfitting. The separation of the training set and the validation set ensures the generalization ability of the model on new samples.

[0143] The embodiments of the present invention further provide device embodiments for implementing the steps and methods in the above method embodiments.

[0144] In a preferred embodiment of the present invention, the embodiment of the present invention discloses a device, including: one or more processors; a storage device for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned diabetic foot assessment method based on wound image recognition.

[0145] In a preferred embodiment of the present invention, the embodiment of the present invention discloses a computer-readable and writable storage medium, characterized in that the medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the above-mentioned diabetic foot assessment method based on wound image recognition are implemented.

[0146] Since each unit module in this embodiment can execute Figure 1 For the method shown in the embodiment, the part not described in detail in this embodiment can be referred to Figure 1 Related instructions.

[0147] At the hardware level, the device may include a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), and may also include non-volatile memory, such as at least one disk drive. Of course, the device may also include other hardware required for the service.

[0148] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0149] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0150] The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the method described above.

[0151] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0152] For the convenience of description, the above device is described as being divided into various units or modules according to their functions. Of course, when implementing the present invention, the functions of each unit or module can be implemented in the same or multiple software and / or hardware.

[0153] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0158] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0159] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0161] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0163] The various embodiments of the present invention are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0164] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A diabetic foot assessment method based on wound surface image recognition, characterized in that: The method comprises: S1. Obtain an infrared thermal imaging image and a color image of the foot of a target diabetic foot patient, perform preprocessing on each image, and process the infrared thermal imaging image of the foot to generate a temperature anomaly map; S2, removing interference factors from the color foot wound image, marking the real diabetic foot wound to generate a foot wound image, and fusing the foot wound image with the temperature anomaly map to generate a fused image; S3. Constructing a foot blood flow analysis model, performing feature analysis based on the foot blood flow analysis model, extracting blood flow influencing factors, and correcting the fused image based on the blood flow influencing factors; S4. Construct a joint assessment model, use the corrected fusion image and historical data of diabetic foot patients as inputs of the joint assessment model, predict the risk assessment level and evolutionary risk level of diabetic foot, and output the disease assessment results of diabetic foot patients based on the risk assessment level and evolutionary risk level; The S3 specifically includes: S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(F(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image; S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F ,in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,standard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, N N,s tan dard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, and ε represents a very small positive number; S33. The blood flow influencing factor is used to correct the fused image, and the correction formula of the fused image is defined as follows: I corr (x,y)=I fused (x,y)×(1+β×(1-K F )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

2. The method according to claim 1, characterized in that The step of processing the foot infrared thermal imaging image to generate a temperature anomaly map specifically includes: S11. Convert the original grayscale values ​​of the infrared thermal imaging image of the foot into actual temperature values ​​to obtain a temperature matrix covering the foot area. The conversion formula is defined as: T(x,y) = a·I(x,y)+b, where T(x,y) represents the actual temperature value at pixel (x,y), I(x,y) represents the original grayscale value at pixel (x,y), a represents the temperature calibration slope of the infrared thermal imager, and b represents the temperature calibration intercept of the infrared thermal imager. S12, performing Gaussian filtering on the converted temperature matrix to remove noise; S13, extracting a foot region from the foot infrared thermal imaging image using the foot color image; S14. Mirror-align the foot regions of the left and right feet using the image coordinate system. The alignment formula is defined as: R =2·X c -x L , y′ R =y L , where x L and y L represents the pixel coordinates in the left foot area, x′ R and y′ R Indicates the pixel coordinates of the right foot mirror image corresponding to the left foot, X c Indicates the horizontal coordinate of the symmetrical midline of the left and right feet; S15, calculate the temperature difference of the symmetrical positions of the two feet to obtain a temperature difference matrix, and perform binarization processing on the temperature difference matrix according to the preset temperature difference threshold to obtain an abnormal area mask; wherein the temperature difference matrix is ​​defined as: ΔT(x L ,y L )=T L (x L ,y L )-T R (x′ R ,y′ R ), where ΔT(x L ,y L ) represents the temperature difference between the left and right foot positions, T L (x L ,y L ) represents the temperature value of the left foot position, T R (x′ R ,y′ R ) represents the temperature value of the mirror image position corresponding to the right foot; the calculation formula of the abnormal area mask is: Among them, A(x,y) represents the binary mask of the abnormal area, 1 represents abnormality, 0 represents normality, ΔT(x,y) represents the temperature difference of the pixels in the corresponding areas of the left and right feet, ΔT thr Indicates the temperature difference threshold; S16, mapping the abnormal area mask back to the foot infrared thermal imaging image to generate a temperature anomaly map I temp (x,y).

3. The method according to claim 2, characterized in that The S13 specifically includes: S131. Perform color balancing on the foot color image and convert it to the HSV color space. Then, use a pre-set threshold to segment the hue and saturation to obtain a preliminary foot region: Among them, H(x,y) and S(x,y) represent the hue and saturation in the HSV color space respectively, and M C (x,y) represents the initial generated color image foot mask, H min and H max Indicates the hue range of the foot skin color in the HSV space, S min Indicates the minimum saturation threshold, indicating; S132, to M C (x,y) uses morphological processing to repair edge discontinuities and obtain a smooth and coherent foot contour mask, namely M foot (x,y); S133, foot color image I C (x,y) using Canny edge detection E C (x,y)=Canny(I C (x,y)) is processed to obtain the edge graph E C (x,y), the edge graph E C (x,y) and M foot (x,y) performs C foot (x,y)=E C (x,y)·M foot (x,y) operation to obtain the final foot contour map C foot (x,y); S134, final foot contour diagram C foot (x, y) performs morphological closing operation to generate a continuous closed contour, where the expansion operation is performed first: Then perform the corrosion operation: Among them, S represents the structural element, M final (x,y) represents the final generated foot contour binary mask; S135, combining the foot infrared thermal imaging image and the foot color image to obtain the foot contour mask M final (x,y) alignment, and apply affine transformation for correction, M aligned (u,v)=T affine (M final (x,y)), where T affine represents the affine transformation used to achieve geometric registration of color and infrared images, M aligned (u,v) represents the binary mask of the foot after affine transformation; S136: Using the corrected foot mask, perform region extraction on the foot infrared thermal imaging image to obtain a foot temperature map T foot (u,v),t foot (u,v)=I T (u,v)×M aligned (u,v), where, I T (u,v) represents the infrared image after affine transformation, T foot (u,v) represents the extracted infrared temperature map of the foot area.

4. The method according to claim 1, wherein The step S2 of fusing the foot wound image with the temperature anomaly map to generate a fused image specifically includes: S21. Use the color segmentation method in the HSV color space to extract the suspected wound area of ​​the foot color wound image and define the threshold function: Among them, H(x,y), S(x,y) and V(x,y) represent the hue, saturation and brightness of the pixel in the HSV color space respectively. L 、H H 、S min and V max They represent preset thresholds, respectively, used to preliminarily distinguish normal skin areas from suspected wound areas; S22, suspected wound area M initial (x,y) is used to perform texture feature analysis to distinguish diabetic foot wounds from interference factors. Specifically, the gray level co-occurrence matrix is ​​used to calculate the texture features. Among them, I gray (x,y) represents the grayscale conversion value of the foot color wound image, d represents the pixel distance, θ represents the analysis direction, W and H represent the width and height of the image respectively; the contrast and homogeneity of each pixel neighborhood are calculated; S23. Set the texture feature threshold function to remove interference from the obvious texture area: Among them, Co(x,y) represents contrast, Ho(x,y) represents homogeneity, T c and T h Represents the texture feature threshold, which is used to distinguish real wounds from natural skin texture and scab tissue; S24. Define the edge irregularity coefficient I of the wound area irregular , set the irregularity threshold T irregular , remove the common wound area with higher regularity, and obtain the final diabetic foot wound mask M wound (x,y) is as follows: Where P represents the perimeter of the suspected wound area, and A represents the area of ​​the area.

5. The method according to claim 4, characterized in that In the step S2, interference factors are removed from the color wound image of the foot, specifically including: S25, performing image enhancement processing on the confirmed diabetic foot wound area to obtain a clear foot wound image I wound (x,y),I wound (x,y)=I C (x,y)×M wound (x,y)×γ, where I C (x, y) represents the original foot color image, γ represents the enhancement coefficient; S26. Fusing the foot wound image with the temperature anomaly image to generate a fused image, wherein a fusion coefficient is defined as: Define the fused image: I fused (x,y)=w(x,y)×I temp (x,y)+[1-w(x,y)]×I wound (x,y), where w(x,y) represents the fusion coefficient, I fused (x,y) represents the fused image, I temp (x,y) represents the temperature anomaly map, λ represents the fusion weight parameter and 0<λ<1, and ε represents a very small positive number.

6. The method according to claim 1, characterized in that The S3 specifically includes: S31. Obtain pre-collected blood perfusion images of diabetic foot patients and non-diabetic foot patients, obtain a blood perfusion image of a target diabetic foot patient, use the median absolute deviation (MAD) to remove local abnormal blood flow values, and obtain a denoised image, wherein the MAD within the local window is defined as: MAD(x,y)=median{|F(x′,y′)-median(x′,y′))|}, (x′,y′)∈W(x,y), and use MAD to remove abnormal values: W(x,y) represents the local window centered at pixel (x,y), F(x′,y′) represents the blood flow value of the neighborhood pixels within the window, median represents the median function, and T MAD represents the denoising threshold, and F′(x,y) represents the denoised image; S32. Calculate the first average blood perfusion intensity of the wound area of ​​diabetic foot patients Calculation of the second mean blood perfusion intensity in the standard measurement area of ​​non-diabetic diabetic foot patients And according to the first mean blood perfusion intensity and the second mean blood perfusion intensity Calculate blood flow impact factor K F , calculate the target sugar diabetic foot patient's individual deviation factor K T , according to the blood flow influencing factor K F and individual offset factor K T Calculate the final correction factor K final ;in, F′ D (x,y) represents the denoised blood perfusion image of diabetic foot patients, F′ N (x,y) represents the denoised blood perfusion image of non-diabetic foot patients, ΩD,wound represents the pixel set of the wound area in the diabetic foot patient group, N D,wound represents the total number of pixels in the wound area of ​​the diabetic foot patient group, ΩN,s tan d ard represents the pixel set of the standard measurement area of ​​the non-diabetic foot healthy control group, N N,stan dard represents the total number of pixels in the standard measurement area of ​​the non-diabetic foot healthy control group, ε represents a very small positive number, represents the average blood flow intensity in the wound area of ​​the blood perfusion image of the target diabetic foot patient, represents the average blood flow intensity of the healthy foot area of ​​the target diabetic foot patient's blood perfusion image, Indicates adjustable parameters, S33, through the final correction factor K final Correct the fused image and define the corrected fused image formula: I corr (x,y)=I fused (x,y)×(1+β×(1-K final )), I corr (x, y) represents the fused image after blood flow correction, I fused (x,y) represents the fused image, and β represents the image correction intensity coefficient.

7. The method according to claim 1, characterized in that The S4 specifically includes: Build and train a joint evaluation model combining CNN and LSTM; The corrected fused image I corr (x, y) and historical data of diabetic foot patients of a preset time length are used as the input of the joint evaluation model, and the input data matrix is ​​defined as: image =I corr (x,y), X image Represents image input data, with a uniform size of 128×128×3. X t =[HbA1c t , blood sugar t, wound area t, number of infections t , BMI t ], X history represents historical data, and T represents the time length of historical data; The risk assessment level and evolutionary risk level of diabetic foot are output according to the model, and the disease assessment results of diabetic foot patients are output according to the risk assessment level and evolutionary risk level.

8. The method according to claim 7, characterized in that The construction and training of the joint evaluation model combining CNN and LSTM specifically includes: Medical data of diabetic and non-diabetic foot patients were collected and pre-processed based on the processed fusion images and patient historical indicator data to form a data set; The dataset was randomly divided into training and validation sets, with 80% of the patient data used as the training set and 20% of the patient data used as the validation set; Spatial feature extraction uses the CNN module structure: The first convolutional layer Conv1: convolution kernel 3×3×32, ReLU activation function; the first pooling layer MaxPool: pooling size 2×2, step size 2; the second convolutional layer Conv2: convolution kernel 3×3×64, ReLU activation function; the second pooling layer MaxPool2: pooling size 2×2, step size 2; feature flattening: flatten the two-dimensional feature map into a one-dimensional feature vector; fully connected layer: output image feature vector F cnn ∈R 128 ; Among them, the CNN module formula is expressed as: F cnn =f cnn (X image θ cnn ),θ cnn Represents the set of CNN trainable parameters; Time series feature extraction uses the LSTM module structure: Construct LSTM network: Input dimension: X history ∈R T×d , d represents each time series feature dimension, output dimension: the hidden layer feature vector F at the last moment lstm ∈R 64 ; The formula of LSTM unit is as follows: Input gate: i t =σ(W xi X t +W hi h t-1 +b i ), forget gate: f t =σ(W xf X t +W hf h t-1 +b f ), output gate: o t =σ(W xo X t +W ho h t-1 +b o ), cell state: c t =f t ⊙c t-1 +i t ⊙tanh(W xc X t +W hc h t-1 +b c ), hidden state: h t =o t ⊙tanh(c t ), the final time series feature output: F lstm =h T , σ represents the sigmoid activation function, ⊙ represents element-wise multiplication, i t Represents the input gate, which controls the degree of update of the cell state by the input feature, f t Represents the forget gate, which controls the degree of retention or discarding of past information. t Represents the output gate, which controls the hidden state of the final output, c t Represents the cell state, used for long-term memory information, h t Represents the hidden state, the expression of the final time series feature, W ** and b * Represents the LSTM trainable parameters, where W ** represents the weight, b * Indicates bias; Fusion of CNN features and LSTM features: F fusion =Concatenate(F cnn ,F lstm )∈R 192 ; Build a dual-output branch: Output branch 1: disease level classification prediction, full connection output + Softmax classification: P stage =Softmax(W s F fusion +b s ),P stage ∈R 4 ;Final level output: Stage pred =argmax(P stage ),Stage pred ∈{0, 1, 2, 3}; Output branch 2: disease evolution risk trend prediction, fully connected output + Sigmoid probability regression: S trend =σ(W t F fusion +b t ),S trend ∈[0,1]; The loss function is designed as the joint loss of two outputs: Loss total =αLoss stage +βLoss trend ,in, Loss trend =(Y trend -S trend ) 2 , α and β represent the weight coefficients of the two subtask losses, Indicates the actual disease level, Indicates the probability value of the model output prediction being the i-th category level, Y trend Indicates the actual trend label, S trend Indicates the probability or score value of the model output trend prediction; Optimization: Adam optimizer is used with an initial value of 0.001 and subsequent gradual decay; Training: Initialize the model weight parameters, set the batch size to 16-32, and use the mini-batch gradient descent method: within each epoch, randomly extract data in batches for training, update the model parameters in each batch, and calculate the loss function on the validation set after each training epoch. If the validation set loss does not decrease for 5 consecutive epochs, stop training and finally select the model parameters with the lowest validation set loss; The model is validated on the validation set to complete the validation.

9. A computer-readable and writable storage medium, characterized in that: The medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images