A wound healing degree evaluation method and system based on multiple models

Abstract

The present invention discloses a wound healing degree evaluation method and system based on multiple models, which relates to the field of medical treatment technology. The present invention combines the wound edge, wound color, wound type and outer dressing color of the patient, uses multiple neural network models to perform comprehensive detection of the wound, realizes intelligent detection of the patient's wound, assists medical staff in the treatment and management of the patient's wound, reduces the burden on medical staff, and reduces the infection risk of the patient's wound.

Description

Technical Field

[0001] The present invention relates to the field of medical treatment technology, and in particular to a wound healing degree evaluation method and system based on multiple models. Background Art

[0002] At present, medical staff mainly rely on manual inspection to detect the degree of wound healing of patients. They use a ruler to measure the size of the wound and visually judge whether the patient's wound has inflammation or necrosis. This traditional method has shortcomings. First, the detection efficiency is low. Medical staff need to carefully observe and measure the patient's wounds one by one, which takes a long time. Especially when facing a large number of patients, the workload is huge and it is difficult to quickly complete the wound assessment. Second, the accuracy is not high. Due to the influence of human factors, such as the experience and subjective judgment of medical staff, the measurement results may be biased, and the assessment of the degree of wound healing of patients is not accurate enough; for example, when dealing with irregular-shaped wounds, it is difficult to measure the area or volume using medical measuring rulers and medical measuring rods in traditional measurement methods, which is easy to cause errors. Third, there is an infection risk. During the manual measurement process, medical staff need to come into direct contact with the patient's wound, which may increase the risk of wound infection in the patient and have an adverse effect on the patient's recovery. Fourth, the medical consumables are large. When measuring the wound volume, a medical measuring rod needs to be used for depth measurement. The medical measuring rod is a disposable item. Each measurement needs to be reused, which requires a large number of medical measuring rods. Therefore, an intelligent method is needed to accurately judge the patient's wound, further reduce the workload of medical staff, improve detection accuracy, and reduce the risk of infection of the patient's wound. Summary of the invention

[0003] The purpose of the present invention is to provide a wound healing degree evaluation method and system based on multiple models to improve the above technical problems.

[0004] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0005] A wound healing degree evaluation method based on multiple models includes:

[0006] S1, obtaining the patient's wound image and outer dressing image;

[0007] S2, performing image enhancement on the wound image and the outer layer dressing image to obtain an enhanced wound image and an enhanced outer layer dressing image;

[0008] S3, respectively constructing a wound color detection model, a wound edge detection model, a wound type detection model and an outer dressing color detection model;

[0009] S4, inputting the enhanced outer dressing image into the outer dressing color detection model, and outputting the outer dressing color detection result; wherein the outer dressing color detection result is the color ratio of the patient's exudate on the outer dressing;

[0010] S5. Input the enhanced wound image to the wound color detection model, the wound edge detection model, and the wound type detection model respectively, and output corresponding wound color detection results, wound edge detection results, and wound type detection results; wherein the wound color detection result is the color proportion of the patient's wound; the wound edge detection result is that the patient's wound edge is flat or the patient's wound edge is curled; the wound type detection result is that the patient's wound is normal, red and swollen, or suppurated;

[0011] S6. Determine the wound healing degree evaluation result based on the wound color detection result, wound edge detection result, wound type detection result and outer dressing color detection result.

[0012] Further, S2 comprises the following steps:

[0013] S2-1, denoising and feature enhancing the wound image to obtain a processed wound image;

[0014] S2-2, performing color correction, contrast enhancement and denoising on the outer layer dressing image to obtain a processed outer layer dressing image;

[0015] S2-3, drawing a weight map corresponding to the processed outer dressing image;

[0016] S2-4, based on the weight map, removing artifacts from the processed outer layer dressing image to obtain a corresponding outer layer dressing image after the artifacts are removed;

[0017] S2-5. Perform image enhancement on the processed wound image and the image of the outer dressing after removal by using a Transformer-based image enhancement method to obtain an enhanced wound image and an enhanced outer dressing image.

[0018] Furthermore, the outer dressing color detection model uses a first convolutional neural network layer, a fusion multi-scale feature layer, a first Conv layer and an output layer connected in series in sequence; wherein the first convolutional neural network layer uses a MobileNetV2 neural network, and the fusion multi-scale feature layer uses a DeepLabv3 model; the output layer uses a softmax function;

[0019] The wound color detection model includes a second convolutional neural network layer, a CBAM attention layer, a second Conv layer, and a Softmax layer connected in series in sequence; wherein the second convolutional neural network layer adopts a MobileNetV2 neural network or a ShuffleNet neural network;

[0020] The wound edge detection model includes a wound edge detection submodule and a flatness calculation module connected in series; the wound edge detection submodule includes a first editor, a second editor, a third editor, a fourth editor, an SFA layer and a fusion layer connected in series in sequence; the first editor, the second editor, the third editor and the fourth editor all include a third Conv layer, a DWconv layer, an EFA layer and a maximum pooling layer connected in series in sequence; the flatness calculation module adopts an edge detection algorithm;

[0021] The wound type detection model includes a wound type feature extraction module and a classification module connected in series; the wound type feature extraction module adopts the ResNet-18 model; and the classification module adopts a classifier.

[0022] Furthermore, the training process of the outer dressing color detection model includes:

[0023] S4-1, obtaining a training outer layer dressing image; processing the training outer layer dressing image using the same method as S2 to obtain an initial training outer layer dressing image;

[0024] S4-2, performing color conversion and normalization on the initial training outer layer dressing image to obtain a processed training outer layer dressing image;

[0025] S4-3, inputting the processed training outer layer dressing image into the first convolutional neural network layer, and outputting a multi-level feature map of the training outer layer dressing;

[0026] S4-4, inputting the multi-level feature map of the training outer dressing into the fusion multi-scale feature layer, and outputting the multi-scale feature map of the training outer dressing;

[0027] S4-5, inputting the multi-scale feature map of the training outer dressing into the first Conv layer and the output layer in sequence, and outputting the initial training outer dressing color detection result;

[0028] S4-6. Calculating a training outer dressing color detection result based on the initial training outer dressing color detection result;

[0029] S4-7. Based on the training results of outer dressing color detection, adjust the weight parameters of the outer dressing color detection model.

[0030] Furthermore, the training process of the wound color detection model includes:

[0031] S5-11, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain an initial training wound image;

[0032] S5-12, normalizing the initial training wound image to obtain a normalized training wound image;

[0033] S5-13, inputting the normalized training wound image into the second convolutional neural network layer, and outputting an initial feature map of the training wound color;

[0034] S5-14, input the initial feature map of the training wound color into the CBAM attention layer, and output the key feature map of the training wound color;

[0035] S5-15, inputting the training wound color key feature map into the second Conv layer, and outputting the convolved training wound color key feature map;

[0036] S5-16, input the convolved training wound color key feature map into the Softmax layer, and output the training wound color detection result;

[0037] S5-17. Based on the training wound color detection results, adjust the weight parameters of the wound color detection model.

[0038] Furthermore, the training process of the wound edge detection submodule includes:

[0039] S5-21, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain a processed training wound image;

[0040] S5-22, inputting the processed training wound image into a first editor, and outputting a first-scale wound feature map;

[0041] S5-23, inputting the processed training wound image into a second editor, and outputting a second-scale wound feature map;

[0042] S5-24, inputting the processed training wound image into a third editor, and outputting a third-scale wound feature map;

[0043] S5-25, inputting the processed training wound image into a fourth editor, and outputting a fourth-scale wound feature map;

[0044] S5-26, respectively inputting the first scale wound feature map, the second scale wound feature map, the third scale wound feature map and the fourth scale wound feature map into the SFA layer, and outputting the corresponding first scale wound key feature map, second scale wound key feature map, third scale wound key feature map and fourth scale wound key feature map;

[0045] S5-27, inputting the first scale wound key feature map, the second scale wound key feature map, the third scale wound key feature map and the fourth scale wound key feature map into the fusion layer to obtain training wound edge features;

[0046] S5-28. Based on the training wound edge features, adjust the weight parameters of the wound edge detection submodule.

[0047] Further, S5-22 includes:

[0048] S5-22-1, input the processed training wound image to the third Conv layer, and output a first training wound edge feature map;

[0049] S5-22-2, inputting the first training wound edge feature map into the DWConv layer, and outputting a second training wound edge feature map;

[0050] S5-22-3, inputting the second training wound edge feature map into the EFA layer, and outputting a training wound edge key feature map;

[0051] S5-22-4, input the training wound edge key feature map into the maximum pooling layer, and output the first scale wound feature map;

[0052] The second to fourth editors use the same method as S5-22-1 to S5-22-4.

[0053] Furthermore, the training process of the wound type detection model includes:

[0054] S5-31, inputting the normalized training wound image into a wound type feature extraction module, and outputting a training wound type feature map;

[0055] S5-32, inputting the training wound type feature map, the training outer dressing color detection result, and the training wound color detection result into the classification module, and outputting the training wound type detection result;

[0056] S5-33. Based on the training wound type detection results, adjust the weight parameters of the wound type detection model.

[0057] Further, S6 includes the following steps:

[0058] S6-1. Obtain wound edge features through a wound edge detection model, and calculate the corresponding wound size or wound volume;

[0059] S6-2, obtaining the wound size or wound volume change value of the previous round of wound healing degree evaluation results, and calculating the wound size change value or wound volume change value;

[0060] S6-3, obtaining a color detection result based on the wound color detection result and the outer dressing color detection result;

[0061] S6-4, determine whether the absolute value of the wound size change value or the wound volume change value is greater than 0; if so, proceed to S6-5; otherwise, proceed to S6-6;

[0062] S6-5, determining whether the wound type detection result shows that the patient's wound is normal and whether the wound edge detection result shows that the edge is smooth; if so, determining that the patient's wound healing degree is first-grade healing; otherwise, proceeding to S6-6;

[0063] S6-6. Determine the wound healing degree evaluation result based on the wound type detection result, wound edge detection result and color detection result.

[0064] A wound healing degree evaluation system based on multiple models includes a patient data acquisition module, an image enhancement module, an outer dressing color detection module, a wound edge detection module, a wound color detection module, a wound type detection module and a wound healing degree evaluation module; wherein:

[0065] A patient data acquisition module, used to obtain the patient's wound image and outer dressing image;

[0066] An image enhancement module is used to enhance the wound image and the outer dressing image to obtain an enhanced wound image and an enhanced outer dressing image;

[0067] An outer dressing color detection module is used to input the enhanced outer dressing image into the outer dressing color detection model and output the outer dressing color detection result;

[0068] A wound type detection module is used to input the enhanced wound image into the wound type detection model and output the corresponding wound type detection result;

[0069] A wound edge detection module is used to input the enhanced wound image into the wound edge detection model and output the corresponding wound edge detection result;

[0070] The wound color detection module is used to input the enhanced wound image into the wound color detection model and output the corresponding wound color detection result;

[0071] The wound healing degree evaluation module is used to determine the wound healing degree evaluation result based on the wound color detection result, the wound edge detection result, the wound type detection result and the outer dressing color detection result.

[0072] The beneficial effects of the present invention are:

[0073] This method combines the patient's wound edge, wound color, wound type and outer dressing color, and uses multiple neural network models to perform comprehensive detection of the wound, thereby realizing intelligent detection of the patient's wound and assisting medical staff in the treatment and management of the patient's wound. Through the wound edge detection model and image processing technology, the area or volume of the patient's wound is calculated, which can reduce the use of medical measuring sticks, reduce consumables, reduce the burden on medical staff, and reduce the risk of infection of the patient's wound.

[0074] This system constructs multiple network models to detect the color, edge, type and outer dressing color of the patient's wound, extract different wound characteristics, and combine multiple factors to accurately identify the degree of wound healing, solving the problem that existing technologies cannot intelligently identify patient wounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 A flow chart of a method in an embodiment of the present invention;

[0077] Figure 2 This is a schematic diagram of the structure of the outer dressing color detection model in an embodiment of the present invention;

[0078] Figure 3 Schematic diagram of the structure of the wound color detection model in an embodiment of the present invention;

[0079] Figure 4 Schematic diagram of the structure of the wound edge detection model in an embodiment of the present invention;

[0080] Figure 5 Schematic diagram of the structure of the wound type detection model in an embodiment of the present invention;

[0081] Figure 6 1 is a system structure diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0083] See also Figure 1 This embodiment provides a wound healing degree evaluation method based on multiple models, which includes:

[0084] S1, obtaining the patient's wound image and outer dressing image;

[0085] S2, performing image enhancement on the wound image and the outer layer dressing image to obtain an enhanced wound image and an enhanced outer layer dressing image;

[0086] Due to insufficient light in the treatment room or ward, the patient's wound image and outer dressing image need to be enhanced, so S2 includes the following steps:

[0087] S2-1, denoising and feature enhancing the wound image to obtain a processed wound image;

[0088] S2-2, color correction, contrast enhancement and denoising are performed on the outer layer dressing image to obtain a processed outer layer dressing image; contrast enhancement can be performed by using operations such as histogram equalization, contrast stretching or exponential transformation.

[0089] S2-3, drawing a weight map corresponding to the processed outer dressing image;

[0090] S2-4, based on the weight map, removing artifacts from the processed outer layer dressing image to obtain a corresponding outer layer dressing image after the artifacts are removed;

[0091] S2-5. The processed wound image and the removed outer dressing image are enhanced by a Transformer-based image enhancement method to obtain an enhanced wound image and an enhanced outer dressing image. The Transformer-based image enhancement method is Vision Transformer, which can solve the problems of uneven light, noise interference and detail loss of wound images and outer dressing images, reduce the difficulty of image processing by subsequent models, and further reduce processing time.

[0092] S3, respectively constructing a wound color detection model, a wound edge detection model, a wound type detection model and an outer dressing color detection model;

[0093] S4. Input the enhanced outer dressing image into the outer dressing color detection model, and output the outer dressing color detection result; wherein, the outer dressing color detection result is the color ratio of the patient's exudate on the outer dressing. Because patients will generate exudate during the healing process, different exudates can show the current state of the wound; exudate is divided into yellow pus, green-yellow pus, blood, clear water-like transparent liquid and brown. Because it is difficult to distinguish clear water-like transparent liquid during color detection, the outer dressing color detection result can be yellow ratio, green-yellow ratio, brown ratio and red ratio.

[0094] like Figure 2 As shown, the outer dressing color detection model uses the first convolutional neural network layer, the fusion multi-scale feature layer, the first Conv layer and the output layer connected in series in sequence; among them, the first convolutional neural network layer uses the MobileNetV2 neural network, and the fusion multi-scale feature layer uses the DeepLabv3 model; the output layer uses the softmax function;

[0095] The training process of the outer dressing color detection model includes:

[0096] S4-1, obtaining a training outer layer dressing image; processing the training outer layer dressing image using the same method as S2 to obtain an initial training outer layer dressing image;

[0097] S4-2, performing color conversion (converting to HSV or LAB) and normalization on the initial training outer layer dressing image to obtain a processed training outer layer dressing image;

[0098] S4-3, inputting the processed training outer layer dressing image into the first convolutional neural network layer, and outputting a multi-level feature map of the training outer layer dressing;

[0099] S4-4, inputting the multi-level feature map of the training outer dressing into the fusion multi-scale feature layer, and outputting the multi-scale feature map of the training outer dressing;

[0100] S4-5, inputting the multi-scale feature map of the training outer dressing into the first Conv layer and the output layer in sequence, and outputting the initial training outer dressing color detection result, that is, obtaining the category probability map corresponding to the color proportion in the training outer dressing image;

[0101] S4-6. Based on the initial training outer dressing color detection result, calculate the training outer dressing color detection result, perform threshold processing on the category probability map, and obtain the corresponding color proportion (training outer dressing color detection result);

[0102] S4-7. Based on the training results of outer dressing color detection, adjust the weight parameters of the outer dressing color detection model.

[0103] Therefore, the formula corresponding to S4 is:

[0104] ;

[0105] ;

[0106] ;

[0107] in, Represents the multi-level feature map of the training outer dressing, Represents the multi-scale feature fusion module in the DeepLabv3 model, represents the MobileNetV2 neural network, represents the activation function, Represents the convolution operation corresponding to the first Conv layer, represents the processed training outer dressing image, represents the multi-scale feature fusion (ASPP module) in the DeepLabv3 model, Represents the training outer dressing color detection results.

[0108] S5. Input the enhanced wound image to the wound color detection model, the wound edge detection model, and the wound type detection model respectively, and output corresponding wound color detection results, wound edge detection results, and wound type detection results; wherein the wound color detection result is the color proportion of the patient's wound; the wound edge detection result is that the patient's wound edge is flat or the patient's wound edge is curled; the wound type detection result is that the patient's wound is normal, red and swollen, or suppurated;

[0109] like Figure 3 As shown in the figure, the wound color detection model includes a second convolutional neural network layer, a CBAM attention layer, a second Conv layer, and a Softmax layer connected in series; the second convolutional neural network layer uses a MobileNetV2 neural network or a ShuffleNet neural network. The use of a lightweight MobileNetV2 neural network or ShuffleNet neural network can reduce the structural complexity of the wound color detection model and is suitable for a variety of scenarios; the use of a CBAM attention layer to capture the deep features of the wound color makes it easier to judge the patient's wound healing.

[0110] The training process of the wound color detection model includes:

[0111] S5-11, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain an initial training wound image;

[0112] S5-12, normalizing the initial training wound image to obtain a normalized training wound image;

[0113] S5-13, inputting the normalized training wound image into the second convolutional neural network layer, and outputting an initial feature map of the training wound color;

[0114] S5-14, input the initial feature map of the training wound color into the CBAM attention layer, and output the key feature map of the training wound color;

[0115] S5-15, inputting the training wound color key feature map into the second Conv layer, and outputting the convolved training wound color key feature map;

[0116] S5-16. Input the convolutional training wound color key feature map into the Softmax layer, and output the training wound color detection result; the training wound color detection result includes the black proportion, yellow proportion, pink proportion and red proportion. When the tissue at the patient's wound is black, the tissue can be regarded as necrosis; when the tissue at the patient's wound is yellow, the tissue can be regarded as slough; when the tissue at the patient's wound is pink, the tissue can be regarded as new epidermis; when the tissue at the patient's wound is red, the tissue can be regarded as granulation.

[0117] S5-17. Based on the training wound color detection results, adjust the weight parameters of the wound color detection model.

[0118] Therefore, the formulas corresponding to S5-11 to S5-17 are:

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] in, Represents the initial feature map of wound color for training, represents the normalized training wound image, represents the second convolutional neural network layer, Represents the convolution operation corresponding to the second Conv layer, represents a multi-layer perceptron, represents the sigmoid activation function, represents the global average pooling operation, Represents the initial key feature map of wound color training, Represents the key feature map of wound color training, Represents the training wound color detection results, Represents a convolution operation.

[0124] like Figure 4 As shown, the wound edge detection model includes a wound edge detection submodule and a flatness calculation module connected in series; the wound edge detection submodule includes a first editor, a second editor, a third editor, a fourth editor, an SFA layer and a fusion layer connected in series in sequence; the first editor, the second editor, the third editor and the fourth editor all include a third Conv layer, a DWconv layer, an EFA layer and a maximum pooling layer connected in series in sequence; the flatness calculation module adopts an edge detection algorithm.

[0125] Because when detecting the edge of a patient's wound, it is only necessary to determine whether the edge is flat or curled, thereby reducing the number of encoders and introducing the EFA layer. Even if the number of encoders is reduced, the detailed features of the wound edge can be accurately extracted and the model processing time is reduced. The SFA layer is introduced to process the output of each encoder through the attention mechanism, which further improves the accuracy of edge detection and prevents the problem of being unable to fully extract details due to the reduction in the number of encoders and the DWConv layer.

[0126] The training process of the wound edge detection submodule includes:

[0127] S5-21, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain a processed training wound image;

[0128] S5-22, input the processed training wound image into the first editor, and output a first-scale wound feature map. The corresponding process is:

[0129] S5-22-1, input the processed training wound image to the third Conv layer, and output a first training wound edge feature map;

[0130] S5-22-2, inputting the first training wound edge feature map into the DWConv layer, and outputting a second training wound edge feature map;

[0131] S5-22-3, inputting the second training wound edge feature map into the EFA layer, and outputting a training wound edge key feature map;

[0132] S5-22-4, input the training wound edge key feature map into the maximum pooling layer, and output the first scale wound feature map;

[0133] The second to fourth editors use the same method as S5-22-1 to S5-22-4.

[0134] S5-23, inputting the processed training wound image into a second editor, and outputting a second-scale wound feature map;

[0135] S5-24, inputting the processed training wound image into a third editor, and outputting a third-scale wound feature map;

[0136] S5-25, inputting the processed training wound image into a fourth editor, and outputting a fourth-scale wound feature map;

[0137] Therefore, the formulas corresponding to S5-22 to S5-25 are:

[0138] ;

[0139] ;

[0140] ;

[0141] ;

[0142] ;

[0143] in, Indicates The editor's input data, Indicates The first training wound edge feature map output by the Conv layer in the editor, , Respectively represent The convolution kernel and bias term of the editor's Conv layer, Indicates The convolution kernel of the depthwise convolution of the editor’s DWConv layer, Indicates The output data of the channel-by-channel convolution of the editor's DWConv layer, Indicates The second training wound edge feature map output by the DWConv layer in the editor, represents the channel-by-channel convolution operation, , Respectively represent The convolution kernel and bias term of the point-by-point convolution of the editor's DWConv layer, Indicates Key feature map of the training wound edge output by the EFA layer in the editor, represents the convolution operation in the EFA layer, Indicates The editor corresponds to the output data, Represents a max pooling operation.

[0144] S5-26, respectively inputting the first scale wound feature map, the second scale wound feature map, the third scale wound feature map and the fourth scale wound feature map into the SFA layer, and outputting the corresponding first scale wound key feature map, second scale wound key feature map, third scale wound key feature map and fourth scale wound key feature map;

[0145] S5-27, inputting the first scale wound key feature map, the second scale wound key feature map, the third scale wound key feature map and the fourth scale wound key feature map into the fusion layer to obtain training wound edge features;

[0146] S5-28. Based on the training wound edge features, adjust the weight parameters of the wound edge detection submodule.

[0147] The processing process of the flatness calculation module is:

[0148] The wound edge features output by the wound edge detection submodule are processed using an edge detection algorithm (such as Canny, Sobel or Laplacian operator) to obtain the corresponding final wound edge features; based on the final wound edge features, the corresponding edge flatness is calculated. The edge flatness can be selected from the curvature change value or straightness of the edge, etc. The corresponding calculation process is a prior art, so it is not described in detail. It is determined whether the edge flatness is greater than the flatness threshold. If so, the wound edge detection result is determined to be a flat wound edge of the patient; otherwise, the wound edge detection result is determined to be a curled wound edge of the patient.

[0149] In addition, when performing edge detection, in addition to determining whether the wound edge is curled or flat, it can also determine whether the wound edge is immersed, further improving the accuracy of the patient's wound healing degree and assisting medical staff in selecting or mixing dressings.

[0150] like Figure 5 As shown, the wound type detection model includes a wound type feature extraction module and a classification module connected in series; the wound type feature extraction module adopts the ResNet-18 model; and the classification module adopts a classifier.

[0151] The convolution kernels of the first and second Conv layers are both 1×1; the convolution kernels of the third Conv layer are both 3×3; and the stride of the maximum pooling layer is 2.

[0152] The training process of the wound type detection model includes:

[0153] S5-31, inputting the normalized training wound image into a wound type feature extraction module, and outputting a training wound type feature map;

[0154] S5-32, inputting the training wound type feature map, the training outer dressing color detection result, and the training wound color detection result into the classification module, and outputting the training wound type detection result;

[0155] Thus, the formula corresponding to the classifier is:

[0156] ;

[0157] ;

[0158] ;

[0159] in, represents the training wound type feature map, represents the pooled training wound type feature map, Represents the wound fusion characteristic map, , denote the weight matrix and bias term respectively, Represents the training wound type detection results.

[0160] S5-33. Based on the training wound type detection results, adjust the weight parameters of the wound type detection model.

[0161] S6. Determine the wound healing degree evaluation result based on the wound color detection result, wound edge detection result, wound type detection result and outer dressing color detection result.

[0162] The healing evaluation results are divided into first-degree healing (the patient's wound has recovered well), second-degree healing (the patient's wound has recovered poorly), third-degree healing (the patient's wound has recovered poorly), and fourth-degree healing (the patient's wound has recovered poorly). S6 includes the following steps:

[0163] S6-1. Obtain wound edge features through a wound edge detection model and calculate the corresponding wound size or wound volume, that is, process the wound edge features using image processing technology, count the corresponding pixel values, and obtain the wound area;

[0164] S6-2. Obtain the wound size or wound volume change value of the previous round of wound healing evaluation results, and calculate the wound size change value or wound volume change value; when measuring the wound volume, three-dimensional modeling technology can also be used for measurement to obtain more precise wound volume change values, which is more conducive to the judgment of the wound healing degree.

[0165] S6-3. Based on the wound color detection result and the outer dressing color detection result, obtain the color detection result, that is, count the color information of the wound color detection result and the outer dressing color detection result. Because the yellow pus and green-yellow pus of the exudate are produced by wound suppuration, the color proportions corresponding to the two are averaged with the color proportion corresponding to the yellow in the wound color detection result to obtain the yellow-color proportion; the color proportion corresponding to the blood water in the outer dressing color detection result is averaged with the color proportion corresponding to the red in the wound color detection result to obtain the red-color proportion. The color proportions corresponding to black and pink in the wound color detection result are used as the black-color proportion and the pink-color proportion. Thus, the color detection results include the yellow-color proportion, the red-color proportion, the black-color proportion, and the pink-color proportion.

[0166] S6-4, determine whether the absolute value of the wound size change value or the wound volume change value is greater than 0; if so, proceed to S6-5; otherwise, proceed to S6-6;

[0167] S6-5, determining whether the wound type detection result shows that the patient's wound is normal and whether the wound edge detection result shows that the edge is smooth; if so, determining that the patient's wound healing degree is first-grade healing; otherwise, proceeding to S6-6;

[0168] S6-6. Based on the wound type detection results, the wound edge detection results and the color detection results, the wound healing degree evaluation results are determined. The corresponding specific process is as follows:

[0169] When the absolute value of the patient's wound size change or wound volume change is less than 0:

[0170] If the following two conditions are met: the wound type test result is that the patient's wound is purulent; the wound type test result is that the patient's wound is red and swollen and the wound edge test result is curled; then the patient's wound healing degree is determined to be level 4 healing;

[0171] If the wound type test result is that the patient's wound is red and swollen and the wound edge test result is that the edge is flat, the patient's wound healing degree is determined to be level three healing;

[0172] When the absolute value of the patient's wound size change or wound volume change is greater than 0:

[0173] Under the condition that the wound type test result is that the patient's wound is normal, if the wound edge test result is that the edge is curled, the patient's wound healing degree is determined to be secondary healing;

[0174] Under the condition that the wound type detection result is that the patient's wound is red and swollen, if the wound edge detection result is that the edge is flat, and the pink-color ratio or the red-color ratio is the maximum value, the patient's wound healing degree is determined to be secondary healing; otherwise, the patient's wound healing degree is determined to be tertiary healing;

[0175] Under the condition that the wound type test result is that the patient's wound is suppurated, if the wound edge test result is that the edge is flat, and the pink-color proportion or the red-color proportion is the maximum value, the patient's wound healing degree is determined to be grade 3 healing; otherwise, in all other cases, the patient's wound healing degree is determined to be grade 4 healing.

[0176] In addition, the weight change value of the outer dressing can be obtained. The outer dressing absorbs the patient's exudate. If the weight change value of the outer dressing exceeds the threshold, it means that the patient's wound healing is poor, and the type or ratio of the outer dressing and the patient's medication need to be adjusted.

[0177] In summary, the present invention combines the patient's wound edge, wound color, wound type and outer dressing color, and uses multiple neural network models to perform comprehensive detection of the wound, thereby realizing intelligent detection of the patient's wound and assisting medical staff in the treatment and management of the patient's wound; through the wound edge detection model and image processing technology, the area or volume of the patient's wound is calculated, which can reduce the use of medical measuring sticks, reduce consumables, reduce the burden on medical staff, and reduce the risk of infection of the patient's wound.

[0178] like Figure 6 As shown, a wound healing degree evaluation system based on multiple models includes a patient data acquisition module, an image enhancement module, an outer dressing color detection module, a wound edge detection module, a wound color detection module, a wound type detection module and a wound healing degree evaluation module; wherein:

[0179] A patient data acquisition module, used to obtain the patient's wound image and outer dressing image;

[0180] An image enhancement module is used to enhance the wound image and the outer dressing image to obtain an enhanced wound image and an enhanced outer dressing image;

[0181] An outer dressing color detection module is used to input the enhanced outer dressing image into the outer dressing color detection model and output the outer dressing color detection result;

[0182] A wound type detection module is used to input the enhanced wound image into the wound type detection model and output the corresponding wound type detection result;

[0183] A wound edge detection module is used to input the enhanced wound image into the wound edge detection model and output the corresponding wound edge detection result;

[0184] The wound color detection module is used to input the enhanced wound image into the wound color detection model and output the corresponding wound color detection result;

[0185] The wound healing degree evaluation module is used to determine the wound healing degree evaluation result based on the wound color detection result, the wound edge detection result, the wound type detection result and the outer dressing color detection result.

[0186] In summary, this system constructs multiple network models to detect the color, edge, type and outer dressing color of the patient's wound, extract different wound characteristics, and combine multiple factors to accurately identify the degree of wound healing, solving the problem that existing technologies cannot intelligently identify patient wounds.

[0187] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A wound healing degree evaluation method based on multiple models, characterized in that: include: S1, obtaining the patient's wound image and outer dressing image; S2, performing image enhancement on the wound image and the outer layer dressing image to obtain an enhanced wound image and an enhanced outer layer dressing image; S3, respectively constructing a wound color detection model, a wound edge detection model, a wound type detection model and an outer dressing color detection model; S4, inputting the enhanced outer dressing image into the outer dressing color detection model, and outputting an outer dressing color detection result; wherein the outer dressing color detection result is the color proportion of the patient's exudate on the outer dressing; S5, respectively inputting the enhanced wound image into the wound color detection model, the wound edge detection model, and the wound type detection model, and outputting corresponding wound color detection results, wound edge detection results, and wound type detection results; wherein the wound color detection result is the color proportion of the patient's wound; the wound edge detection result is that the patient's wound edge is flat or the patient's wound edge is curled; the wound type detection result is that the patient's wound is normal, red and swollen, or suppurated; S6. Determine the wound healing degree evaluation result based on the wound color detection result, the wound edge detection result, the wound type detection result and the outer dressing color detection result; The outer dressing color detection model adopts a first convolutional neural network layer, a fusion multi-scale feature layer, a first Conv layer and an output layer connected in series in sequence; wherein the first convolutional neural network layer adopts a MobileNetV2 neural network, the fusion multi-scale feature layer adopts a DeepLabv3 model; the output layer adopts a softmax function; The wound color detection model includes a second convolutional neural network layer, a CBAM attention layer, a second Conv layer and a Softmax layer connected in series in sequence; wherein the second convolutional neural network layer adopts a MobileNetV2 neural network or a ShuffleNet neural network; The wound edge detection model includes a wound edge detection submodule and a flatness calculation module connected in series; the wound edge detection submodule includes a first editor, a second editor, a third editor, a fourth editor, an SFA layer and a fusion layer connected in series in sequence; the first editor, the second editor, the third editor and the fourth editor all include a third Conv layer, a DWconv layer, an EFA layer and a maximum pooling layer connected in series in sequence; the flatness calculation module adopts an edge detection algorithm; The wound type detection model includes a wound type feature extraction module and a classification module connected in series; the wound type feature extraction module adopts a ResNet-18 model; and the classification module adopts a classifier.

2. The wound healing degree evaluation method based on multiple models according to claim 1, characterized in that: The S2 comprises the following steps: S2-1, denoising and feature enhancing the wound image to obtain a processed wound image; S2-2, performing color correction, contrast enhancement and denoising on the outer layer dressing image to obtain a processed outer layer dressing image; S2-3, drawing a weight map corresponding to the processed outer dressing image; S2-4, based on the weight map, removing artifacts from the processed outer layer dressing image to obtain a corresponding outer layer dressing image after the artifacts are removed; S2-5. Perform image enhancement on the processed wound image and the removed outer dressing image by using a Transformer-based image enhancement method to obtain the enhanced wound image and the enhanced outer dressing image.

3. The wound healing degree evaluation method based on multiple models according to claim 1, characterized in that: The training process of the outer dressing color detection model includes: S4-1, obtaining a training outer layer dressing image; processing the training outer layer dressing image using the same method as S2 to obtain an initial training outer layer dressing image; S4-2, performing color conversion and normalization on the initial training outer layer dressing image to obtain a processed training outer layer dressing image; S4-3, inputting the processed training outer layer dressing image into the first convolutional neural network layer, and outputting a multi-level feature map of the training outer layer dressing; S4-4, inputting the training outer dressing multi-level feature map into the fused multi-scale feature layer, and outputting a training outer dressing multi-scale feature map; S4-5, inputting the multi-scale feature map of the training outer dressing into the first Conv layer and the output layer in sequence, and outputting an initial training outer dressing color detection result; S4-6. Calculating a training outer dressing color detection result based on the initial training outer dressing color detection result; S4-7. Based on the training outer dressing color detection result, adjust the weight parameters of the outer dressing color detection model.

4. The wound healing degree evaluation method based on multiple models according to claim 1, characterized in that: The training process of the wound color detection model includes: S5-11, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain an initial training wound image; S5-12, normalizing the initial training wound image to obtain a normalized training wound image; S5-13, inputting the normalized training wound image into the second convolutional neural network layer, and outputting an initial feature map of the training wound color; S5-14, inputting the training wound color initial feature map into the CBAM attention layer, and outputting a training wound color key feature map; S5-15, inputting the training wound color key feature map into the second Conv layer, and outputting the convolved training wound color key feature map; S5-16, inputting the convolved training wound color key feature map into the Softmax layer, and outputting the training wound color detection result; S5-17. Based on the training wound color detection result, adjust the weight parameters of the wound color detection model.

5. The wound healing degree evaluation method based on multiple models according to claim 1, characterized in that: The training process of the wound edge detection submodule includes: S5-21, obtaining a training wound image; processing the training wound image using the same method as S2 to obtain a processed training wound image; S5-22, inputting the processed training wound image into the first editor, and outputting a first-scale wound feature map; S5-23, inputting the processed training wound image into the second editor, and outputting a second-scale wound feature map; S5-24, inputting the processed training wound image into the third editor, and outputting a third-scale wound feature map; S5-25, inputting the processed training wound image into the fourth editor, and outputting a fourth-scale wound feature map; S5-26, respectively inputting the first-scale wound feature map, the second-scale wound feature map, the third-scale wound feature map, and the fourth-scale wound feature map into the SFA layer, and outputting corresponding first-scale wound key feature map, second-scale wound key feature map, third-scale wound key feature map, and fourth-scale wound key feature map; S5-27, inputting the first scale wound key feature map, the second scale wound key feature map, the third scale wound key feature map and the fourth scale wound key feature map into the fusion layer to obtain training wound edge features; S5-28. Based on the training wound edge features, adjust the weight parameters of the wound edge detection submodule.

6. The wound healing degree evaluation method based on multiple models according to claim 5, characterized in that: The S5-22 includes: S5-22-1, inputting the processed training wound image into the third Conv layer, and outputting a first training wound edge feature map; S5-22-2, inputting the first training wound edge feature map into the DWConv layer, and outputting a second training wound edge feature map; S5-22-3, inputting the second training wound edge feature map into the EFA layer, and outputting a training wound edge key feature map; S5-22-4, inputting the training wound edge key feature map into the maximum pooling layer, and outputting the first scale wound feature map; The second to fourth editors adopt the same method as S5-22-1 to S5-22-4.

7. The wound healing degree evaluation method based on multiple models according to claim 3 or 4, characterized in that: The training process of the wound type detection model includes: S5-31, inputting the normalized training wound image into the wound type feature extraction module, and outputting a training wound type feature map; S5-32, inputting the training wound type feature map, the training outer dressing color detection result, and the training wound color detection result into the classification module, and outputting the training wound type detection result; S5-33. Based on the training wound type detection result, adjust the weight parameters of the wound type detection model.

8. The wound healing degree evaluation method based on multiple models according to claim 5, characterized in that: The S6 comprises the following steps: S6-1, obtaining wound edge features through the wound edge detection model, and calculating the corresponding wound size or wound volume; S6-2, obtaining the wound size or wound volume change value of the previous round of wound healing degree evaluation results, and calculating the wound size change value or wound volume change value; S6-3, obtaining a color detection result based on the wound color detection result and the outer dressing color detection result; S6-4, determining whether the absolute value of the wound size change value or the wound volume change value is greater than 0; if so, proceeding to S6-5; otherwise, proceeding to S6-6; S6-5, determining whether the wound type detection result indicates that the patient's wound is normal and whether the wound edge detection result indicates that the edge is smooth; if so, determining that the patient's wound healing degree is primary healing; otherwise, proceeding to S6-6; S6-6. Determine a wound healing degree evaluation result based on the wound type detection result, the wound edge detection result and the color detection result.

9. A wound healing degree evaluation system based on multiple models, used to implement a wound healing degree evaluation method based on multiple models as claimed in any one of claims 1 to 8, characterized in that: It includes a patient data acquisition module, an image enhancement module, an outer dressing color detection module, a wound edge detection module, a wound color detection module, a wound type detection module and a wound healing degree evaluation module; among which: A patient data acquisition module, used to obtain the patient's wound image and outer dressing image; An image enhancement module is used to enhance the wound image and the outer dressing image to obtain an enhanced wound image and an enhanced outer dressing image; An outer dressing color detection module is used to input the enhanced outer dressing image into the outer dressing color detection model and output the outer dressing color detection result; A wound type detection module is used to input the enhanced wound image into the wound type detection model and output the corresponding wound type detection result; A wound edge detection module is used to input the enhanced wound image into the wound edge detection model and output the corresponding wound edge detection result; The wound color detection module is used to input the enhanced wound image into the wound color detection model and output the corresponding wound color detection result; The wound healing degree evaluation module is used to determine the wound healing degree evaluation result based on the wound color detection result, the wound edge detection result, the wound type detection result and the outer dressing color detection result.

Images