Assessment method and device for diabetic foot ulcer, storage medium and equipment

Abstract

The invention provides an evaluation method and device for diabetic foot ulcers, a storage medium and equipment, and relates to the technical field of smart medical treatment.The method comprises the steps that foot image data to be processed are acquired; carrying out segmentation identification processing on the foot image data by using a pre-trained image segmentation processing model, and determining wound feature information representing a diabetic foot ulcer condition in the image; evaluating the grade of the diabetic foot ulcer based on the wound feature information, and outputting an evaluation result; wherein the types of the wound feature information comprise a wound area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene and a periinvasive red and swollen area. By applying the method provided by the invention, an auxiliary basis can be effectively provided for referral evaluation of the DFU patient, and the problem of medical treatment delay is avoided.

Description

Technical Field

[0001] The present application relates to the field of smart medical technology, and specifically to an evaluation method, device, storage medium and equipment for diabetic foot ulcers. Background Art

[0002] Diabetic foot ulcer (DFU) is one of the most serious and most expensive chronic complications of diabetes. Approximately 25% to 34% of diabetic patients will experience at least one DFU in their lifetime. If diabetic foot ulcer is not treated promptly and appropriately, severe cases may lead to amputation or even death.

[0003] The median delay in referral of patients with diabetic foot ulcers by primary care specialists is 7 to 31 days, and the median time for final treatment is 6.2 to 56 days. In reality, due to the uneven regional development, some primary medical institutions in some regions lack professional medical staff and lack experience in wound assessment. Many DFU patients have delayed medical treatment, which leads to delayed treatment and serious consequences. As the number of diabetic patients continues to expand, this problem has become increasingly prominent.

[0004] With the development of artificial intelligence (AI) technology, it has become possible to apply AI in the medical field to solve such problems.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] In order to overcome the problems existing in the related art to at least a certain extent, the embodiments of the present application provide a method, device and equipment for evaluating diabetic foot ulcers, which adopt artificial intelligence technology to specifically evaluate diabetic foot ulcers, so as to solve the problem of insufficient evaluation and processing capabilities caused by the shortage of professional medical staff in existing practices.

[0007] First aspect Some embodiments of the present application provide a method for evaluating diabetic foot ulcers, the method comprising: Acquiring foot image data to be processed; Using a pre-trained image segmentation processing model to perform segmentation and recognition processing on the foot image data, and determine wound surface feature information representing the diabetic foot ulcer in the image; Performing a grade assessment of the diabetic foot ulcer based on the wound surface characteristic information, and outputting the assessment result; The categories of the wound surface characteristic information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area.

[0008] In some possible implementations, the grade assessment of diabetic foot ulcer based on the wound surface feature information is specifically as follows: According to the category label information of the wound surface feature information, a preset grading rule is used to perform a grade evaluation on the foot image data.

[0009] In some possible implementations, the preset classification rules include: When there is a wound area category label, and there is no bone tissue category label, tendon category label, or gangrene category label, the grade classification is w1-2; When there is a wound area category label, and at least one of the bone tissue category label and the tendon category label exists, and there is no gangrene category label, the level classification is w3; When the gangrene class label is present, the grade classification is w4.

[0010] In some possible implementations, the evaluation method further includes outputting the sub-image and category label corresponding to the wound surface feature information.

[0011] In some possible implementations, the process of obtaining the foot image data to be processed includes: Receiving raw image data output by the shooting component; Detect and judge the original image data based on the target detection algorithm to determine whether there is a human foot image in the image; Preprocessing is performed on the original image data that has passed the detection to obtain the foot image data.

[0012] In some possible implementations, the pre-training process of the image segmentation processing model includes: Preprocessing is performed based on the collected foot wound images of diabetic foot patients to obtain intermediate image data; Manually annotating the intermediate image data to construct a training data set, wherein the annotated type labels include wound area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area; Dividing the data set into a training set and a test set, and training a plurality of initial image segmentation models based on the training set to obtain corresponding trained models to be evaluated; A horizontal comparative evaluation is performed on the models to be evaluated based on the test set, and the model to be evaluated with the best performance is selected as the image segmentation processing model.

[0013] In some possible implementations, the types of the initial image segmentation model include Mask2former model, Deeplabv3plus model and Swin-Transformer model.

[0014] Second aspect Some embodiments of the present application provide an evaluation device for diabetic foot ulcers, the evaluation device comprising: An acquisition unit, used for acquiring foot image data to be processed; A segmentation processing unit, used to perform segmentation and recognition processing on the foot image data using a pre-trained image segmentation processing model, and determine wound surface feature information representing the diabetic foot ulcer in the image; An evaluation processing unit, used to evaluate the grade of the diabetic foot ulcer based on the wound surface characteristic information, and output the evaluation result; The categories of the wound surface characteristic information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area.

[0015] The third aspect Some embodiments of the present application provide a computer-readable medium having program code stored thereon, and when the program code is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0016] The fourth aspect Some embodiments of the present application provide a device, including: one or more processors; a storage device, on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect above.

[0017] The method for evaluating diabetic foot ulcers provided in an embodiment of the present application includes: acquiring foot image data to be processed; performing segmentation and recognition processing on the foot image data using a pre-trained image segmentation processing model to determine wound surface feature information in the image that characterizes the condition of the diabetic foot ulcer; performing a grade assessment of the diabetic foot ulcer based on the wound surface feature information, and outputting the assessment result; wherein the categories of the wound surface feature information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and red and swollen area around the wound. In the technical solution of the present application, in the evaluation application of diabetic foot ulcers, a pre-trained image segmentation processing model is used to segment and identify foot image data, and automatic evaluation and grading are performed based on the identified wound feature information. The application of the technical solution of the present application in scenarios with insufficient medical resources can effectively provide auxiliary basis for the referral evaluation of DFU patients and avoid the problem of delayed medical treatment; and in the technical solution of the present application, compared with the implementation of direct image classification and evaluation using AI, it uses image segmentation and recognition technology to accurately segment and define the scope of the wound area, and deeply identify the tissue components in the wound, such as necrotic tissue, tendon, bone, gangrene, etc., and then perform graded evaluation based on the segmentation results. This method is more in line with the actual medical scenario and is more conducive to ensuring the accuracy of grading.

[0018] Other advantages, objectives, and features of the present application will be described in part in the following description, and in part will be apparent to those skilled in the art based on the following examination and study, or may be taught from the practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the technical solution of the present application or the prior art, and constitute a part of the specification. Among them, the accompanying drawings expressing the embodiments of the present application are used together with the embodiments of the present application to explain the technical solution of the present application, but do not constitute a limitation on the technical solution of the present application.

[0020] Figure 1 It is a schematic diagram of an application scenario of a method for evaluating diabetic foot ulcers in some embodiments of the present application; Figure 2 This is a schematic flow chart of the method for evaluating diabetic foot ulcers of the present application in some embodiments; Figure 3 This is a schematic diagram illustrating the overall process of training and determining an image segmentation processing model in some embodiments of the present application; Figure 4 A schematic diagram for comparing segmentation tasks of different segmentation models in some embodiments of the present application; Figure 5 This is a schematic diagram of the structure of the device for evaluating diabetic foot ulcers in some embodiments of the present application; Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0022] It should also be noted that, for ease of description, only parts related to the relevant embodiments are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0023] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0027] Figure 1 It is a schematic diagram of an application scenario of a method for evaluating diabetic foot ulcers according to some embodiments of the present application.

[0028] like Figure 1 As shown, the user can send the foot image data to be processed to the server 104 through the network 103 at the terminal devices 101 and 102. The server 104 receives the foot image data to be processed; the server 104 can use the pre-trained image segmentation processing model to perform segmentation and recognition processing on the foot image data, determine the wound surface feature information representing the diabetic foot ulcer in the image, and perform a grade assessment of the diabetic foot ulcer based on the wound surface feature information to generate an assessment result; the server 104 then feeds back the assessment result to the corresponding terminal device, and the terminal device outputs the assessment result so that the user can use it as a reference.

[0029] It should be understood that Figure 1 The number of terminal devices and servers in the embodiment is only for illustration. Any number of terminal devices and servers may be provided according to implementation requirements.

[0030] Continue to refer Figure 2 , Figure 2 The flowchart 200 of some embodiments of the method for evaluating diabetic foot ulcer according to the present application is shown. The method for evaluating diabetic foot ulcer comprises the following steps:

[0031] Step S201, obtaining foot image data to be processed; For example, a standardized photographing system device may be set up in community medical stations and other similar places. The photographing system device has a photographing detection table, a camera component, and a lighting component. Medical staff use the device to obtain corresponding image data based on photographing the feet of DFU patients.

[0032] Step S202, after obtaining the foot image data to be processed, the pre-trained image segmentation processing model can be used to perform segmentation and recognition processing on the foot image data to determine wound surface feature information representing the diabetic foot ulcer in the image; It should be noted that, in the technical solution of the present application, the categories of wound surface feature information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound red and swollen area. That is, for a piece of image data, the above-mentioned multiple target objects can be identified and segmented using a pre-trained image segmentation processing model. For example, for image A, the wound surface feature information obtained includes sub-image A1 with a type label of wound surface area and sub-image A2 with a type label of gangrene. In this embodiment, the image segmentation processing model here is implemented based on the Mask2Former model framework.

[0033] Then, step S203 is performed to evaluate the grade of the diabetic foot ulcer based on the wound surface characteristic information, and the evaluation result is output; In the analysis and evaluation of diabetic foot ulcers, the Wagner grading method is a commonly used clinical grading standard, which evaluates the severity of the wound according to the location of the wound and whether there is infection or gangrene. When clear wound feature information is obtained in step S202, relevant primary care medical staff can make grading judgments based on it; In practice, if the grading rules are clear, in order to further reduce the difficulty of implementation and avoid errors caused by human factors, this step can be implemented based on specific software settings and through software judgment; specifically, in this embodiment, the grade of diabetic foot ulcer is assessed based on wound surface feature information, specifically: According to the category label information of the wound surface feature information, the preset classification rules are used to perform the grade evaluation of the foot image data; Furthermore, in actual implementation, for the sake of overall method efficiency, logical operators can be used to construct corresponding conditional expressions based on the correspondence between the presence or absence of category labels and the corresponding levels. Conditional expressions can be used to match and judge the category label information of the wound feature information to generate and output grade evaluation results.

[0034] On the other hand, in the process of realizing the present invention, the inventors found that in the Wagner grading method, the ulcer conditions in both grade 1 and grade 2 involve the skin and soft tissue levels, and there are similarities in the treatment strategies. Considering the actual needs of referral evaluation in practice, Wagner grade 1 and grade 2 can be merged into one grade to configure the preset grading rules in the above steps, which can more accurately reflect the wound surface characteristics and the degree of tissue damage, while ensuring the clarity of the grading and classification tasks in the overall technical solution; Specifically, in one embodiment, the preset classification rules include: When there is a wound area category label, and there are no bone tissue category labels, tendon category labels, and gangrene category labels, the level classification is w1-2; when there is a wound area category label, and there is at least one of the bone tissue category label and the tendon category label, and there is no gangrene category label, the level classification is w3; when there is a gangrene category label, the level classification is w4.

[0035] In the assessment of diabetic foot ulcers, the technical solution of the present application is applied to segment and identify foot image data using a pre-trained image segmentation processing model, and automatic assessment and grading are performed based on the identified wound surface feature information. In scenarios where medical resources are insufficient, it can effectively provide auxiliary basis for the referral assessment of DFU patients and avoid delays in medical treatment. On the other hand, compared with the implementation of direct image classification and evaluation using AI in the technical solution of the present application, the technical idea of ​​the present application is to use image segmentation and recognition technology to accurately segment and define the wound area, and to deeply identify necrotic tissue, tendon, bone, gangrene and other tissue components in the wound, and then perform a graded evaluation based on the segmentation results. This method is more in line with the actual medical scenarios, that is, the grading rules can be clearly established in practice without the need for machine learning; and because the AI ​​model used does not need to consider the implementation of the classification function, under certain computing power and energy efficiency conditions, it can focus on improving the segmentation accuracy, which is beneficial to ensuring the grading accuracy.

[0036] In the above embodiment, the foot image data to be processed is realized by photographing using a dedicated photographing device. The patient still needs to go to a place where the photographing device is installed to take a photographic evaluation. In practice, DFU patients generally have mobility problems, and the photographing performance of existing smart terminals including smart phones is becoming more and more powerful. Based on the settings of the relevant App, the patient's family can take original images based on such devices, and directly transmit the photographed data to the back-end server as a basis for subsequent evaluation and processing; this method is conducive to improving the convenience of patients' medical treatment, but in this method, since the data collection operation is performed by non-professionals, it is uncontrollable. For example, if image data that is not a foot image is transmitted, such data will bring unnecessary processing pressure to the back-end server; therefore, for this potential defect, relevant technical measures need to be taken to carry out corresponding processing. Based on the above embodiment, in some implementation methods, the process of obtaining the foot image data to be processed includes: Receive the original image data output by the shooting component. For example, the shooting component here is a smart phone device. Based on the settings of a dedicated App on the mobile phone, the captured image is uploaded and output, and the server receives the original image data; On the server side, the received raw image data is detected and judged based on the target detection algorithm to determine whether there is a human foot image in the image. For example, due to the limitation of computing power and efficiency, the target detection algorithm here adopts the YOLO series algorithm. In actual application, the corresponding pre-trained model can be used for simple training and application deployment; Then, the original image data that has passed the detection, that is, the image data containing the content of the human foot, is preprocessed to obtain the foot image data. It should be noted that the preprocessing method here should be consistent with the data preprocessing method in the training data construction stage during the pre-training process of the image segmentation processing model, so as to ensure the segmentation and recognition effect.

[0037] To facilitate understanding of the technical solution of this application, Figure 3 As shown, the pre-training process of the image segmentation processing model in this application is introduced.

[0038] In one embodiment, the pre-training process of the image segmentation processing model of the present application includes: First, we collected images. In actual implementation, we collected 671 images of foot wounds of patients with diabetic foot who visited the Air Force Special Medical Center from January 2015 to December 2023. Among them, the usual diagnostic criteria for diabetic foot are that patients with newly diagnosed diabetes or a history of diabetes have infection, ulcers or tissue destruction on the feet, usually accompanied by lower limb neuropathy and / or peripheral arterial disease. Based on this, the above images in this application are images of Wagner 1-4 diabetic foot ulcers confirmed by professional medical personnel, and the images clearly show the foot. The images are collected from different ulcer areas with sufficient resolution to ensure accurate distinction of tissue features such as granulation, necrosis, tendon, bone and gangrene in the wound surface; these images come from different shooting equipment and conditions to enhance the robustness of model training; during the collection process, the exclusion criteria used include: poor image quality, such as blur, underexposure or overexposure, and obstruction of key information; the cause of the ulcer is not due to diabetes, or the nature of the ulcer is unknown; the image information is incomplete and key information is missing; images of unknown origin or involving copyright disputes; and any image data that does not meet the research ethics and legal requirements.

[0039] Afterwards, technical means are used to preprocess the collected foot wound images of diabetic foot patients to obtain intermediate image data; specifically, image preprocessing is an important step in image analysis and recognition, which can improve the efficiency and accuracy of image processing; for example, the preprocessing here includes resolution adjustment processing. In practice, the sizes of patient images collected vary, with a maximum of 3864*5152 and a minimum of 1080*1920. In this application, the image size is uniformly fixed to 1024*1024. In actual implementation, relevant tools in OpenCV can be used to perform this processing.

[0040] After that, the intermediate image data is manually annotated to construct a training data set, where the annotated type labels include wound area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area; In this stage, in actual implementation, the labeling work was performed by an expert team from the Endocrinology Department of the Air Force Special Medical Center. Labelme software was used to accurately mark the intermediate image data. The labeling content included seven categories: wound range (or wound area), granulation tissue, necrotic tissue, tendon, bone (bone tissue), gangrene, and infection (or periwound redness and swelling area). Among the intermediate image data, 631 involved granulation tissue, 458 involved necrotic tissue, 130 involved tendons, 140 involved bone, 122 involved gangrene, and 264 involved infection.

[0041] In terms of the construction of the initial image segmentation model, there are many deep learning models that can realize image segmentation in the existing related technologies. In order to facilitate rapid deployment in practice, this application adopts a method of comparing and evaluating multiple existing deep learning models and selecting the best one to determine the model framework used in this application; in actual implementation, a training set and a test set are divided from the data set, and multiple initial image segmentation models are trained based on the training set to obtain corresponding trained models to be evaluated; a horizontal comparison and evaluation of the models to be evaluated is performed based on the test set, and the model to be evaluated with the best performance is selected as the image segmentation processing model; Specifically, based on the analysis of the segmentation and recognition requirements in the technical scenarios of this application and the characteristics of the existing model frameworks in practice, deeplabv3Plus, swin-transformer and mask2former are determined as the model frameworks to be evaluated and selected; among them, deeplabv3Plus is a mature model based on convolutional neural networks, which is widely used in the field of semantic segmentation; swin-transformer uses the Transformer architecture to process long-distance dependencies and is suitable for identifying complex features; and mask2former combines the global modeling capabilities of Transformer and the local perception capabilities of convolutional neural networks; deeplabv3Plus uses the convolutional neural network model ResNet101 as the backbone network, swin-transformer and mask2former use the Transformer model based on the self-attention mechanism as the backbone network, and the convolutional neural network has translation invariance and local connectivity, and mainly extracts local features in the image through convolution operations and pooling operations. The Transformer network model uses a self-attention mechanism and a multi-layer perception structure to obtain global feature representations through complex spatial transformations and long-distance feature dependencies; In actual research, in order to improve development efficiency, the above-mentioned multiple initial image segmentation models are all pre-trained models, which are pre-trained in public data sets, such as using the ImageNet data set for pre-training, and then using the pre-trained model parameters as the initialization parameters of the model; and in the actual training process, the data set of diabetic foot wound images constructed in this application is used for retraining, and the loss function value, accuracy change curve, etc. of the training set and the test set are monitored during the training process until the loss value becomes small enough and the accuracy curve tends to be stable; The training server configuration used in the above process is specifically: centos linux release 7.4.1708 operating system, Intel(R) Xeon(R) CPU, NvidiaNVIDIA GeForce RTX 3090 GPU, CUDA Version: 11.4, the artificial intelligence framework adopts the Pytorch framework, opencv: 4.3.0.36.

[0042] In order to quantitatively evaluate the performance of the proposed model and make a horizontal comparison, the dataset was divided into two subsets with a ratio of 8:2 in the actual implementation, which were used for model training and testing respectively. Two key metrics were used: Intersection over Union (IOU) and Dice coefficient to comprehensively evaluate the recognition and segmentation performance of the three deep learning models on diabetic foot ulcer images. Both indicators are used to quantify the degree of overlap between the wound area predicted by the model and the true area (GroundTruth) annotated by experts. A perfect match will result in an IOU and Dice coefficient of 1, which means that the predicted area is completely consistent with the true area; and these two indicators are 0, which means that there is no overlap at all.

[0043] In the actual comparison process, in terms of training and inference efficiency, in the learning cycle of 100 epochs, the DeeplabV3Plus model showed a faster training speed, taking about 11 hours to complete the entire training process, and the inference time of a single image was stable at around 0.19 seconds; the training time of the Swin-transformer model was slightly longer, taking about 14 hours and 30 minutes to complete 100 epochs of training, and its inference time was 0.42 seconds; the Mask2former model training cycle was also longer, at 13 hours and 10 minutes, which should be due to its complex model architecture and high attention to details, however, its inference time was only 0.39 seconds.

[0044] In terms of wound segmentation performance, the segmentation and recognition results of the three models on diabetic foot wounds are shown in Table 1. Figure 4 As shown, Figure 4In the figure, red indicates granulation tissue, yellow indicates necrotic tissue, green indicates bone tissue, cyan indicates tendon, blue indicates infection, and purple indicates gangrene. In the wound segmentation task, the IOU value of the Mask2former model is 85.88%, which is close to 85.9% of Swin-Transformer. However, in multi-label comprehensive recognition, Mask2former performs better. For the recognition of different tissues such as granulation tissue, necrotic tissue, tendon, bone, and gangrene, its IOU values ​​are 78.78%, 59.53%, and 4. 8.72%, 53.45% and 77.14%; in terms of the mean intersection over union (mIOU) on the test set, Mask2former reached 65.79%, while Swin-Transformer's mIOU was 65.31% and DeeplabV3Plus's mIOU was 52.86%; this shows that although Mask2former and Swin-Transformer perform similarly in single wound segmentation, Mask2former has a clear advantage in multi-label recognition tasks.

[0045] Table 1-Segmentation and recognition results of three models on diabetic foot wounds In terms of infection recognition capability, as shown in Table 3, the DeeplabV3Plus model showed a sensitivity of 0.8868 in infection status recognition, but was slightly weaker in specificity, which was 0.7317, with an overall accuracy of 0.7926 and an AUC value of 0.8093; the Swin-Transformer model showed a relatively balanced performance in the recognition of infection and non-infection states, with a sensitivity of 0.7925 and a specificity of 0.8780, respectively, an accuracy of 0.8444, and an AUC of 0.8353; the Mask2Former model performed best in all evaluation indicators, especially in accuracy and AUC, which were 0.8519 and 0.8353, respectively.

[0046] Table 2- Identification results of three models on whether diabetic foot wounds are infected Therefore, considering the above comparison results comprehensively, it is obvious that mask2former has obvious advantages in average accuracy, segmentation accuracy, etc., and can be used as a priority in practice.

[0047] Based on the above introduction to the relevant embodiments, it is obvious that the technical solution proposed in the present application can not only complete the precise segmentation and scope definition of the wound surface, but also deeply identify the tissue components in the necrotic tissue, tendon, bone, and gangrenous wound surface; detailed wound segmentation and tissue identification are key steps in the management of diabetic foot ulcers; by accurately distinguishing the granulation tissue, necrotic tissue, tendon, bone tissue, gangrene and the range of redness and swelling around the wound surface, it can provide an intuitive basis for the severity classification of the ulcer, and can also provide accurate information for prognosis evaluation and treatment plan formulation; based on this, in some embodiments, the evaluation method of the present application also includes outputting the sub-image and category label corresponding to the wound surface feature information for reference by relevant personnel.

[0048] In summary, in the technical solution of this application, the deep learning model can identify different tissue types (such as granulation tissue, necrotic tissue, etc.) in the wound surface, which helps doctors to more accurately assess the severity of the wound surface and thus formulate a more accurate treatment plan. In addition, by identifying and segmenting wound surface features, the model can assist doctors in grading wound surfaces according to the Wagner grading standard, quickly determine the severity of the wound surface, and adjust the treatment plan in a timely manner; make up for the shortage of professional medical staff and improve the management efficiency of diabetic foot ulcers. Even in the absence of experienced doctors, AI technology can provide preliminary diagnostic suggestions and reduce the occurrence of medical delays.

[0049] Considering the actual needs, in some implementation scenarios, the foot image data also includes scale mark information. For example, in the actual shooting implementation, a marker of a standard size can be set in the camera field of view of the detection platform to leave the scale mark information in the image data formed by shooting. Accordingly, in this implementation, the evaluation method also includes: Based on the scale mark information, the area of ​​the sub-image corresponding to the wound feature information is calculated. For example, the actual area of ​​each pixel is calculated according to the scale mark information, and then the area measurement is achieved by calculating the total number of pixels in the sub-image. For example, the number of pixels can be measured based on a related edge detection algorithm to obtain the area information of the corresponding sub-image. The obtained area information is output, which is beneficial for medical staff to quantitatively perceive the size of the patient's ulcer wound and facilitate the implementation of related treatment and nursing measures.

[0050] Furthermore, in some implementation scenarios, the foot image data also includes identification information of the patient. Accordingly, in this implementation, the evaluation method further includes: According to the identification information, historical evaluation data corresponding to the identification information is called; the corresponding sub-images in each of the historical evaluation data are arranged in chronological order, and based on edge detection and tracking of the corresponding sub-images, morphological change information of the corresponding sub-images is generated; the morphological change information is output. In the technical scenario of the present application, the morphological change information here represents the recovery of the ulcer wound, and can provide relevant reference information for actual treatment and care, such as adjusting the treatment plan based on the wound recovery condition.

[0051] Further references Figure 5 As an implementation of the above methods, the present application also provides a diabetic foot ulcer assessment device 500, which is similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices; like Figure 5 As shown, in some embodiments, a device 500 for evaluating diabetic foot ulcers includes: An acquisition unit 501 is used to acquire foot image data to be processed; The segmentation processing unit 502 is used to perform segmentation and recognition processing on the foot image data using a pre-trained image segmentation processing model to determine wound surface feature information representing the diabetic foot ulcer in the image; An evaluation processing unit 503 is used to evaluate the grade of the diabetic foot ulcer based on the wound surface characteristic information and output the evaluation result; The categories of the wound surface characteristic information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area.

[0052] It is understood that the units described in the device 500 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 500 and the units included therein, and will not be described in detail here.

[0053] In some embodiments, the present application further provides a computer-readable storage medium having program code stored thereon, and when the program code is executed by a processor, the method described in any one of the above embodiments is implemented.

[0054] In some embodiments, the present application also provides a device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the above embodiments.

[0055] like Figure 6As shown, the (electronic) device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0056] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0057] In particular, according to some embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0058] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0059] In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0060] In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP (Hypertext Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0061] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the method proposed in this application.

[0062] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a unit, a program segment or a part of a code, and the unit, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0064] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0065] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for evaluating diabetic foot ulcers, characterized in that: include: Acquiring foot image data to be processed; Using a pre-trained image segmentation processing model to perform segmentation and recognition processing on the foot image data, and determine wound surface feature information representing the diabetic foot ulcer in the image; Performing a grade assessment of the diabetic foot ulcer based on the wound surface characteristic information, and outputting the assessment result; The categories of the wound surface characteristic information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area.

2. The evaluation method according to claim 1, wherein: The grade assessment of diabetic foot ulcer based on the wound surface characteristic information is specifically as follows: According to the category label information of the wound surface feature information, a preset grading rule is used to perform a grade evaluation on the foot image data.

3. The evaluation method according to claim 2, wherein: The preset classification rules include: When there is a wound area category label, and there is no bone tissue category label, tendon category label, or gangrene category label, the grade classification is w1-2; When there is a wound area category label, and at least one of the bone tissue category label and the tendon category label exists, and there is no gangrene category label, the level classification is w3; When the gangrene class label is present, the grade classification is w4.

4. The evaluation method according to claim 1, wherein: The evaluation method further includes outputting the sub-image and category label corresponding to the wound surface feature information.

5. The evaluation method according to claim 1, wherein: The process of obtaining the foot image data to be processed includes: Receiving raw image data output by the shooting component; Detect and judge the original image data based on the target detection algorithm to determine whether there is a human foot image in the image; Preprocessing is performed on the original image data that has passed the detection to obtain the foot image data.

6. The evaluation method according to claim 1, wherein: The pre-training process of the image segmentation processing model includes: Preprocessing the collected foot wound images of diabetic foot patients to obtain intermediate image data; Manually annotating the intermediate image data to construct a training data set, wherein the annotated type labels include wound area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area; Dividing the data set into a training set and a test set, and training a plurality of initial image segmentation models based on the training set to obtain corresponding trained models to be evaluated; A horizontal comparative evaluation is performed on the models to be evaluated based on the test set, and the model to be evaluated with the best performance is selected as the image segmentation processing model.

7. The evaluation method according to claim 6, wherein: The types of the initial image segmentation models include Mask2former model, Deeplabv3plus model and Swin-Transformer model.

8. An evaluation device for diabetic foot ulcers, characterized in that: include: An acquisition unit, used for acquiring foot image data to be processed; A segmentation processing unit, used to perform segmentation and recognition processing on the foot image data using a pre-trained image segmentation processing model, and determine wound surface feature information representing the diabetic foot ulcer in the image; An evaluation processing unit, used to evaluate the grade of diabetic foot ulcer based on the wound surface characteristic information, and output the evaluation result; The categories of the wound surface characteristic information include: wound surface area, granulation tissue, necrotic tissue, tendon, bone tissue, gangrene, and periwound redness and swelling area.

9. A computer-readable storage medium having program code stored thereon, characterized in that: When the program code is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images