A digital wound assessment method, apparatus and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI PROVINCIAL HOSPITAL
- Filing Date
- 2026-04-30
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, wound assessment relies on manual measurement, which is cumbersome and prone to cross-infection. It cannot accurately determine the depth and volume changes of the wound, leading to delays in treatment plans.
By acquiring color and depth images of the wound, the area and volume of the wound are calculated using binarized segmentation maps and depth information. The wound is then evaluated by fusing physiological data with a multimodal large model. A wound region cutting plane correction mechanism is introduced to correct the depth information under non-perpendicular imaging.
It achieves accuracy and stability in wound assessment under different shooting angles, improves the practicality and clinical reliability of wound analysis, and provides comprehensive three-dimensional quantitative indicator support.
Smart Images
Figure CN122434893A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically to a digital wound assessment method, apparatus, and storage medium. Background Technology
[0002] Currently, clinical assessment of traumatic wounds is necessary. Visual inspection alone cannot accurately determine whether a wound is healing normally, but changes in data can directly reflect trends. For example, a continuously shrinking area and shallowing depth indicate successful healing; if the area remains unchanged or even expands and the depth increases, it suggests potential problems such as infection or tissue necrosis, requiring timely adjustments to the treatment plan. Wounds of different sizes and depths require entirely different treatment methods. For instance, shallow, large epidermal wounds may only require ordinary dressings; deep, voluminous sinus tract wounds require specialized filling dressings to promote granulation tissue growth; excessively large wounds may also require assessment to determine if skin grafting is necessary. These decisions all rely on specific measurement data. Changes in depth and volume help doctors identify potential risks. For example, a sudden increase in wound depth may indicate further necrosis of underlying tissue; increased volume accompanied by increased exudation may indicate worsening infection. These data signals allow medical staff to intervene earlier, preventing the condition from deteriorating.
[0003] Current technology relies on visual inspection or the use of external tools such as rulers and sterile cotton balls to measure the length, width, area, and depth of the wound. Based on these measurements, the wound is assessed, and the necessary dressings are selected for treatment at the current stage. This manual measurement method is extremely cumbersome and involves direct contact with the patient's affected area, increasing the risk of cross-infection. Summary of the Invention
[0004] This application is made to address the aforementioned problems. According to one aspect of this application, a digital wound assessment method is provided, the method comprising: Acquire a color image and a depth image of the wound, wherein the depth image includes the depth information; A binarized segmentation map is obtained based on the color image; The wound area is calculated based on the binarized segmentation map and the depth information; A three-dimensional surface map of the wound is drawn based on the binarized segmentation map and the depth information; The outer contour pixels of the wound are obtained based on the binarized segmentation map, and the wound cutting plane is fitted by combining the outer contour pixels of the wound with the depth information. Calculate the distance between the pixels of the binarized segmentation map and the wound cutting plane, and correct the three-dimensional surface map according to the distance to generate a wound depth map; Calculate the wound volume based on the distance and depth information; The multimodal large model obtains wound assessment results based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound.
[0005] In one embodiment of this application, the depth information includes the depth value corresponding to the pixel and the camera depth scaling factor.
[0006] In one embodiment of this application, the binarized segmentation map includes pixels in the wound region and pixels in the non-wound region, and the step of calculating the wound area based on the binarized segmentation map and the depth information includes: The area corresponding to each pixel in the wound region is calculated based on the depth value corresponding to the pixel in the wound region and the camera depth scaling factor. The area of the wound is obtained by adding up the areas corresponding to each pixel in the wound region.
[0007] In one embodiment of this application, the step of drawing a three-dimensional surface map of the wound based on the binarized segmentation map and the depth information includes: A three-dimensional surface map of the wound is drawn based on the pixels of the wound area and the corresponding depth values.
[0008] In one embodiment of this application, obtaining the outer contour pixels of the wound based on the binarized segmentation map, and fitting the wound cutting plane by combining the outer contour pixels of the wound with the depth information, includes: The outer contour pixels of the wound are obtained based on the pixels in the wound area, and the horizontal and vertical coordinates of the outer contour pixels of the wound are recorded. By combining the horizontal and vertical coordinates of the outer contour pixels of the wound with the depth value corresponding to the outer contour pixels of the wound, the three-dimensional coordinates of the outer contour pixels of the wound are obtained. The tangential plane of the wound is obtained by performing planar fitting based on the three-dimensional coordinates of the pixels of the outer contour of the wound.
[0009] In one embodiment of this application, the step of calculating the distance between the pixels of the binarized segmentation map and the wound cutting plane, and correcting the three-dimensional surface map based on the distance to generate a wound depth map includes: Calculate the distance between the pixels in the wound region and the cutting plane of the wound to obtain the distance between each pixel in the wound region and the cutting plane of the wound; The depth values corresponding to the pixels in the wound region of the three-dimensional surface map are corrected to distance to generate a wound depth map.
[0010] In one embodiment of this application, calculating the wound volume based on the distance and depth information includes: The area corresponding to each pixel in the wound region is calculated based on the depth value corresponding to the pixel in the wound region and the camera depth scaling factor. The volume corresponding to each pixel in the wound region is obtained by multiplying the distance between each pixel in the wound region and the cutting plane of the wound region with the area corresponding to each pixel in the wound region. The volume of the wound is obtained by adding the volumes corresponding to each pixel in the wound area.
[0011] In one embodiment of this application, the multimodal large model obtains wound assessment results based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound, including: Convert the wound depth map into geometric features; The color image and the binarized segmentation map are converted into visual features; The wound area, wound volume, and physiological data of the wound are converted into physiological characteristics; The clinical data of the wound were converted into clinical features; The geometric features, visual features, physiological features and clinical features are fused together to obtain fused features; The multimodal large model obtains wound assessment results based on the fusion features.
[0012] According to another aspect of this application, a digital wound assessment device is provided, including a scanner, a data processing terminal, a display screen, and a data interface: A scanner for acquiring color and depth images of a wound, wherein the depth image includes the depth information; A data processing terminal is used to obtain a binarized segmentation map from the color image; calculate the wound area based on the binarized segmentation map and the depth information; draw a three-dimensional surface map of the wound based on the binarized segmentation map and the depth information; obtain the outer contour pixels of the wound based on the binarized segmentation map, and fit a wound tangent plane by combining the outer contour pixels of the wound with the depth information; calculate the distance between the pixels of the binarized segmentation map and the wound tangent plane, correct the three-dimensional surface map based on the distance, and generate a wound depth map; calculate the wound volume based on the distance and the depth information; and a multimodal large model obtains a wound assessment result based on the color image, the binarized segmentation map, the wound area, the wound depth map, the wound volume, and the physiological and clinical data of the wound. A display screen is used to show the wound area, wound volume, wound depth map, and wound assessment results; A data interface is provided for transmitting the wound area, wound volume, wound depth map, and wound assessment results to a hospital system. According to another aspect of this application, a storage medium is provided storing a computer program that, when executed, performs the aforementioned digital wound assessment method.
[0013] This application achieves wound area detection by fusing color and depth image information. It introduces an automatic correction mechanism based on the tangent plane of the wound region, which can geometrically correct depth information under non-perpendicular shooting conditions. This maintains the accuracy and consistency of depth estimation and volume calculation under different shooting angles, greatly improving the practicality and stability of wound assessment. By fusing multimodal data through a multimodal large model, the wound assessment results are obtained, effectively improving the accuracy and clinical reliability of wound analysis results. Attached Figure Description
[0014] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0015] Figure 1 A schematic flowchart illustrating a digital wound assessment method according to an embodiment of this application is shown.
[0016] Figure 2 A color image is shown according to an embodiment of this application.
[0017] Figure 3 A depth image is shown according to an embodiment of this application.
[0018] Figure 4 A binarized segmentation map according to an embodiment of this application is shown.
[0019] Figure 5 A color map of the wound area according to an embodiment of this application is shown.
[0020] Figure 6 A three-dimensional surface view of a wound is shown according to an embodiment of this application.
[0021] Figure 7 A wound incision plane diagram according to an embodiment of this application is shown.
[0022] Figure 8 A wound depth diagram is shown according to an embodiment of this application.
[0023] Figure 9 A schematic structural diagram of a digital wound assessment device according to an embodiment of this application is shown.
[0024] Reference numerals: Scanner 91, Data processing terminal 92, Display screen 93, Data interface 94, Lead wire 95, Wireless transmission signal transmitter 96. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of this application, and not all of the embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.
[0026] First, refer to Figure 1 This describes a digital wound assessment method for implementing embodiments of the present invention.
[0027] In step S100, a color image and a depth image of the wound are acquired, wherein the depth image includes depth information.
[0028] In step S200, a binarized segmentation map is obtained based on the color image.
[0029] In step S300, the wound area is calculated based on the binarized segmentation map and depth information.
[0030] In step S400, a three-dimensional surface map of the wound is drawn based on the binarized segmentation map and depth information.
[0031] In step S500, the outer contour pixels of the wound are obtained based on the binarized segmentation map, and the tangent plane of the wound is fitted by combining the outer contour pixels of the wound with the depth information.
[0032] In step S600, the distance between the pixels of the binarized segmentation map and the cutting plane of the wound is calculated, and the three-dimensional surface map is corrected according to the distance to generate a wound depth map.
[0033] In step S700, the wound volume is calculated based on the distance and depth information.
[0034] In step S800, the multimodal large model obtains wound assessment results based on color images, binarized segmentation maps, wound area, wound depth maps, wound volume, and physiological and clinical data of the wound.
[0035] This application achieves wound area detection by fusing color and depth image information. It introduces an automatic correction mechanism based on the tangent plane of the wound region, which can geometrically correct depth information under non-perpendicular shooting conditions. This ensures that the accuracy and consistency of depth estimation and volume calculation are maintained under different shooting angles, greatly improving the practicality and stability of the assessment. By fusing multimodal data through a multimodal large model, the wound assessment results are obtained, effectively improving the accuracy and clinical reliability of wound analysis results.
[0036] In one embodiment, step S100 involves acquiring a color image and a depth image of the wound, the depth image including depth information. For example... Figure 2 , 3 As shown, a color image captured by a regular camera. Depth images acquired by a binocular structured light camera A depth image acquired by a binocular structured light camera is a special type of image, where each pixel value represents the depth information of that location relative to the camera in the real world. Figure 3 Different colors correspond to different distances, with the redder the color, the closer it is.
[0037] In one embodiment, depth information includes the depth value corresponding to each pixel and the camera depth scaling factor. The actual physical area corresponding to each pixel in the captured color image needs to be obtained by combining depth information. The depth value corresponding to each pixel: This is the raw data directly output by the binocular structured light camera and stored at each pixel location in the depth image. It is usually an integer, such as a 16-bit or 32-bit value, and directly corresponds to the physical distance (e.g., meters or millimeters) from the object at the pixel to the lens. The camera depth scaling factor: This is a fixed parameter determined before the camera leaves the factory or during calibration. Essentially, it is the conversion ratio between "pixel size" and "actual physical size." The scaling factor may differ between different cameras. To obtain the actual physical area of a pixel, it needs to be calculated using the formula "(pixel depth value / depth scaling factor)^2".
[0038] In one embodiment, a binarized segmentation map is obtained from the color image in step S200. For example... Figure 4As shown, the UNet-Lite algorithm is used to process color images to extract the wound region, resulting in a binarized segmentation map M. UNet-Lite is a lightweight version of the standard UNet network. It reduces the number of convolutional layers and uses lightweight convolutions (such as depthwise separable convolutions) to reduce the number of parameters, while retaining the core "encoder-decoder-skip connection" structure. It is suitable for wound segmentation tasks on color images (3 channels). The specific logic is as follows: Encoder (downsampling): The input 3-channel color image is progressively downsampled, and low-level features (such as edges and textures) and high-level features (such as overall contours) of the wound are extracted through convolution and pooling operations. Skip connection: Feature maps from different levels of the encoder are directly passed to the corresponding levels of the decoder to supplement the details lost during downsampling and help to accurately locate the wound boundary. Decoder (upsampling): The feature map output from the encoder is progressively upsampled, and combined with the detailed features of the skip connections, a single-channel feature map with the same size as the input image is finally output. Binarization: The single-channel feature map output from the decoder is segmented using a threshold (e.g., 0.5)—pixels greater than the threshold are considered wounds (set to 1), and pixels less than the threshold are considered background (set to 0), resulting in a binarized segmentation map M. The binarized segmentation map M is compared with the depth image. The dimensions are consistent to ensure that pixels (i,j) correspond to the same physical location.
[0039] like Figure 5 As shown, overlaying the wound region from the binarized segmentation image onto a color image yields a color image of the wound region. The segmentation image M is a single-channel binary image (0 for background, 1 for wound). It needs to be converted to a three-channel mask (with the same number of channels as the color image), and the transparency can be adjusted to control the overlay effect. The mask color for the wound region can be changed to red (or another contrasting color) instead of white, making the wound easier to distinguish after overlay (simply replace `mask` with the red channel mask in the formula). Figure 5 In the image, the red area represents the identified wound area, clearly showing the positional correspondence between the segmented area and the original image.
[0040] In one embodiment, the wound area is calculated in step S300 based on the binarized segmentation map and depth information.
[0041] In one embodiment, the binarized segmentation map includes pixels in the wound region and pixels in the non-wound region. Calculating the wound area based on the binarized segmentation map and depth information includes: calculating the area corresponding to each pixel in the wound region based on the depth value corresponding to the pixels in the wound region and the camera depth scaling factor; and adding the areas corresponding to each pixel in the wound region to obtain the wound area.
[0042] Let the number of pixels in the wound area be... The set of corresponding x and y coordinates for ; That is, in the binarized segmentation image In the diagram, the x and y coordinates of all pixels with a value of 1. The set that constitutes the wound area. A pixel value of 1 represents the wound region, then the wound area... It can be represented as: ; In the depth image, the horizontal and vertical coordinates are The depth value of the pixel, This is the camera depth scaling factor. The area of the wound region is calculated by iterating through all pixels belonging to the wound region. Each pixel is calculated based on its corresponding depth value. And the camera's depth scaling factor Calculate the area corresponding to a single pixel. By summing the areas of all pixels, the wound area can be obtained in square millimeters.
[0043] In one embodiment, in step S400, a three-dimensional surface map of the wound is drawn based on the binarized segmentation map and depth information.
[0044] In one embodiment, drawing a three-dimensional surface map of the wound based on a binarized segmentation map and depth information includes: drawing a three-dimensional surface map of the wound based on the pixels of the wound region and their corresponding depth values. Using the binary segmentation map M of the wound region as a mask, the depth values of the wound region corresponding to the depth image are extracted and drawn as a three-dimensional surface of the wound region, such as... Figure 6 As shown.
[0045] First, data filtering is performed. In the binary mask M, white areas (pixel value of 1) correspond to the wound area, and black areas (pixel value of 0) represent the background. A bitwise AND operation is performed on the depth image using the mask, retaining only the depth values of the wound area and setting the background area as invalid. Second, a mesh is constructed. Based on the resolution (width W, height H) of the depth image, a two-dimensional mesh of pixel coordinates is generated (X-axis for horizontal pixel indices, Y-axis for vertical pixel indices). Finally, 3D rendering is performed. The filtered depth values are used as Z-axis data, combined with the X and Y meshes, to create a surface plot representing the 3D shape of the wound.
[0046] Since the lens may not be perpendicular to the wound surface during the shooting process, the maximum depth and volume of the wound cannot be directly calculated from the depth image, and further geometric correction is required.
[0047] In one embodiment, in step S500, the outer contour pixels of the wound are obtained based on the binarized segmentation map, and the tangential plane of the wound is fitted by combining the outer contour pixels of the wound with depth information.
[0048] In one embodiment, obtaining the outer contour pixels of the wound based on the binarized segmentation map, and fitting the wound cutting plane by combining the outer contour pixels with depth information, includes: obtaining the outer contour pixels of the wound based on the pixels in the wound region, and recording the horizontal and vertical coordinates of the outer contour pixels; obtaining the three-dimensional coordinates of the outer contour pixels by combining the horizontal and vertical coordinates of the outer contour pixels with the depth value corresponding to the outer contour pixels; and performing plane fitting based on the three-dimensional coordinates of the outer contour pixels to obtain the wound cutting plane.
[0049] The outer contour of the white region in the binarized segmentation image M is detected, and the horizontal and vertical coordinates of the corresponding pixels are recorded. Let the number of pixels corresponding to the outer contour be... The corresponding set of x and y coordinates for: ; Combined with depth images Construct a set of three-dimensional coordinates for the outer contour pixels: ; in That is, the horizontal and vertical coordinates of the outer contour pixels and their corresponding depth values. The three-dimensional coordinates of all pixels on the outer contour. Perform planar fitting; the fitted wound tangent plane, such as... Figure 7 As shown.
[0050] Extract the outer contour of the binary image. Using OpenCV, extract the outer contour of the white region from the binary segmentation image M, retaining only the outermost contour to eliminate interference from internal holes. Obtain the contour pixel coordinates. Parse the extracted contour data and convert it into a list of pixel coordinates in (x, y) format, ensuring that each coordinate corresponds to the edge of the white region in M. Construct 3D coordinates by matching depth values. Based on the (x, y) coordinates of the contour, read the corresponding depth value z from the depth image and combine them into a 3D coordinate set P in (x, y, z) form. Fit the 3D point cloud to a plane. Use the least squares method to fit the 3D coordinates P to obtain the plane equation:
[0051] That is, the cutting plane of the wound, where a, b, c, d are plane parameters, and x, y, z are coordinate variables in three-dimensional space.
[0052] In one embodiment, in step S600, the distance between the pixels of the binarized segmentation map and the wound cutting plane is calculated, and the three-dimensional surface map is corrected based on the distance to generate a wound depth map.
[0053] In one embodiment, calculating the distance between pixels in the binarized segmentation map and the wound cutting plane, and correcting the three-dimensional surface map based on the distance to generate a wound depth map includes: calculating the distance between pixels in the wound region and the wound cutting plane to obtain the distance between each pixel in the wound region and the wound cutting plane; correcting the depth values corresponding to pixels in the wound region in the three-dimensional surface map to distance, and generating a wound depth map.
[0054] Calculate each pixel Distance to the cutting plane of the wound And use the calculated distance The depth image of the wound area is automatically corrected to generate a wound depth map. The depth value of the wound depth map in the wound area satisfies... The corrected depth map can be viewed as a depth map obtained by shooting the wound perpendicularly, and the corresponding depth value is no longer the distance from the wound to the camera, but rather the depth of the depression after the wound is damaged, such as... Figure 8 As shown, the algorithm introduces a correction mechanism based on the fitting of the tangent plane of the wound region, which can geometrically correct the depth information under non-perpendicular shooting conditions, thereby maintaining the accuracy and consistency of depth estimation under different shooting angles, and greatly improving the practicality and stability of the system.
[0055] In one embodiment, the wound volume is calculated based on distance and depth information in step S700.
[0056] In one embodiment, calculating the wound volume based on distance and depth information includes: calculating the area corresponding to each pixel in the wound region based on the depth value corresponding to the pixel in the wound region and the camera depth scaling factor; calculating the product of the distance between each pixel in the wound region and the wound cutting plane and the area corresponding to each pixel in the wound region to obtain the volume corresponding to each pixel in the wound region; and adding the volumes corresponding to each pixel in the wound region to obtain the wound volume.
[0057] At this point, the maximum depth of the depression in the wound area can be determined. Furthermore, the volume (pixel area x distance) corresponding to each pixel in the wound region is calculated. Then, by summing up all the pixels, the volume of the wound can be obtained. : ; In area and volume calculations, the method not only utilizes the camera's depth scaling factor. It is also based on the actual depth Dynamic scaling is performed to automatically adjust the actual area of pixels at near and far distances. This depth-adaptive scaling correction avoids the geometric distortion under the assumption of traditional planar projection, ensuring consistency and accuracy in wound area and volume calculations at different shooting distances.
[0058] In one embodiment, the multimodal large model in step S800 obtains wound assessment results based on color images, binarized segmentation maps, wound area, wound depth maps, wound volume, and physiological and clinical data of the wound.
[0059] Using the algorithm described above, we can obtain a color image, binarized segmentation map, wound depth map, wound area, and wound volume of the patient's wound region at a certain time (denoted as time t). The physiological and clinical data of the wound are obtained from the hospital system and correspond to the physiological and clinical data of the patient's wound.
[0060] Physiological data of a wound include local tissue temperature, the pH value of the wound microenvironment, and chemical indicators of the exudate. The local temperature of a normally healing wound is close to or slightly higher than that of the surrounding healthy tissue (due to the initial inflammatory response). However, if the temperature continues to rise and is accompanied by increased redness, swelling, and pain, it usually indicates wound infection (over-activation of the inflammatory response). Healthy skin is slightly acidic (pH 4.0-6.0). The pH value of the wound microenvironment changes dynamically during wound healing: initially, the pH may rise due to the alkaline content of the inflammatory exudate, and gradually returns to acidity as granulation tissue grows. Chemical indicators of the exudate include: protein content (excessive protein in the exudate may indicate severe inflammation or infection, while excessively low protein may be due to malnutrition affecting repair), glucose levels (elevated glucose in the wound exudate of diabetic patients indicates poor blood sugar control and decreased healing ability), white blood cell count and differential (an elevated proportion of neutrophils indicates bacterial infection), and enzymes such as lactate dehydrogenase (reflecting the degree of tissue damage).
[0061] Clinical data on wounds includes medical history, medication, and nursing records. Physiological data directly reflects the local microenvironment of the wound and is the core basis for judging the wound healing process and identifying abnormalities. Clinical data focuses on tracking the individual patient background and intervention process, providing "context" for interpreting physiological data and ensuring targeted treatment and nursing care. Medical history includes the history of underlying diseases, wound etiology, and previous wound history. Recording the type, dosage, and course of medication allows for timely adjustments to the medication regimen, reducing adverse effects on wound healing. Nursing records typically include: wound cleaning methods (e.g., saline irrigation, iodine disinfection), dressing type and change frequency (e.g., absorbent dressings for wounds with excessive exudation), changes in wound appearance (e.g., granulation tissue color, changes in wound size / depth), and the patient's subjective symptoms (pain, itching, etc.).
[0062] In one embodiment, the multimodal large model obtains a wound assessment result based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound, including: converting the wound depth map into geometric features; converting the color image and binarized segmentation map into visual features; converting the wound area, wound volume, and physiological data of the wound into physiological features; converting the clinical data of the wound into clinical features; fusing the geometric features, visual features, physiological features, and clinical features to obtain fused features; and the multimodal large model obtains the wound assessment result based on the fused features.
[0063] Wound depth maps represent geometric data, reflecting the physical morphology modality. Color images and binarized segmentation maps represent visual data, reflecting the tissue appearance modality. Wound area, wound volume, and physiological data represent physiological indicators, reflecting the wound environmental state modality. Clinical data represents clinical textual data, reflecting the diagnostic prior modality.
[0064] For each of the four modalities mentioned above, a corresponding encoder is designed. The encoders for each modality can map the data of that modality into high-dimensional feature vectors (all with the same dimension), denoted as follows: . Geometric features As a visual feature, Physiological characteristics This is a clinical feature.
[0065] Next, the feature vectors corresponding to all modalities are fused using the following formula: ; in , , , The weights are respectively for geometric features, visual features, physiological features, and clinical features. The weights are summed to one. Weight selection is automatic: when a modality undergoes a sudden change (i.e., the feature vector from the previous time step differs significantly from the current time step), the corresponding weight increases. Furthermore, when a modality is missing, the corresponding weight approaches zero. Through this weight selection method, adaptive fusion of multimodal information can be achieved.
[0066] To enhance data security and system deployment flexibility, the geometric encoder, visual encoder, physiological indicator encoder, text encoder, and multimodal fusion module are all deployed in a locally embedded data acquisition device. This device performs feature extraction and fusion of wound multimodal data locally, generating a unified feature vector representing the patient at time t. and will Uploaded to the cloud, the multimodal large model is used to assist in diagnosis.
[0067] because The abstract semantic features, derived from high-dimensional transformations using deep learning models, cannot be directly reconstructed from the original images or physiological data. Therefore, patient privacy and security are effectively protected even during network transmission or cloud storage. Simultaneously, all color images, binarized segmentation maps, wound area, wound depth maps, wound volume, and physiological and clinical data of the wound are stored locally on an embedded device, ensuring privacy and compliance from the outset.
[0068] Considering the patient's feature vector sequence over time This information will accumulate continuously, and uploading and storing it all would impose a heavy computational and communication burden. Therefore, this application performs special fine-tuning on the cloud-based multimodal large model to enable it to compress historical information and maintain its state. Specifically, the input of the multimodal large model at time t includes: the feature vector at time t. The historical state vector at time t-1 , represents the model's summary of past time-series information. When t=0, this vector is initialized as an all-zero vector. User consultation text is used for interactive and clinical question-and-answer support.
[0069] The output consists of two parts: the updated historical state vector. It automatically incorporates trend memory based on current features; natural language response text is used to indicate risks, stage changes, and care recommendations.
[0070] That is, the model updates the state vector recursively. It maintains an understanding of wound evolution trends without uploading all historical data. Therefore, at any given time, the system input remains essentially constant, while the model can still accurately determine the healing direction and trend through a state memory mechanism, effectively reducing communication and storage pressure and improving inference efficiency. When the trend worsens or uncertainty increases, a follow-up reminder is triggered; dressing / nursing strategies or referral suggestions are generated.
[0071] This method extracts deep features from geometric morphology modalities (wound depth map), visual appearance modalities (color image and binarized segmentation map), physiological state modalities (such as wound area, wound volume, temperature, pH, and exudate indicators), and clinical text modalities (medical history, medication, and nursing records) on a local embedded device and maps them to a unified semantic space. By introducing an adaptive weight allocation strategy, the feature contributions of different modalities can be dynamically adjusted based on the change amplitude across time, enabling the information fusion process to automatically focus on key modalities of sudden changes in wound state, while maintaining the robustness of the system when modalities are missing or abnormal. This fusion mechanism effectively improves the accuracy and clinical reliability of wound analysis results.
[0072] To address the challenges of massive amounts of data, limited network bandwidth, and privacy concerns in long-term wound follow-up, this method modifies the cloud-based multimodal large-scale model with a temporal recursive structure. This compresses historical healing trends into fixed-dimensional state vectors, which are recursively updated in each inference iteration. This historical state preservation mechanism allows the system to complete trend understanding and risk assessment simply by uploading the current feature vector and the state vector from the previous moment, eliminating the need to store and transmit the complete time series. Compared to traditional centralized data accumulation methods, this approach significantly reduces communication and storage costs while continuously capturing changes in wound healing direction and stages, achieving highly efficient long-term wound monitoring capabilities.
[0073] Furthermore, since all raw images and physiological data are stored locally, the uploaded unified feature vector is a deeply abstract representation, making it impossible to reverse engineer the original images or sensitive physiological information. This invention, without sacrificing diagnostic intelligence, establishes end-to-end privacy protection capabilities from data acquisition to inference, possessing superior medical data security and compliance, and is suitable for deployment in various scenarios such as home care, primary hospitals, and remote follow-up.
[0074] This method detects wound area by fusing color and depth image information. Besides accurately calculating the wound surface area, the algorithm can further estimate the maximum depth and total volume of the wound, providing more comprehensive three-dimensional quantitative indicators for wound healing assessment. An automatic correction mechanism based on the wound region's tangential plane is introduced, enabling geometric correction of depth information under non-perpendicular shooting conditions. This maintains the accuracy and consistency of depth estimation and volume calculation across different shooting angles, significantly improving the practicality and stability of wound assessment. By fusing multimodal data through a multimodal large model, the accuracy and clinical reliability of wound analysis results are effectively improved.
[0075] This application also provides a digital wound assessment device, such as Figure 9As shown, the system includes a scanner 91, a data processing terminal 92, a display screen 93, and a data interface 94: The scanner 91 is used to acquire color and depth images of the wound, with the depth image including depth information; the data processing terminal 92 is used to obtain a binarized segmentation map from the color image; calculate the wound area based on the binarized segmentation map and depth information; draw a three-dimensional surface map of the wound based on the binarized segmentation map and depth information; obtain the outer contour pixels of the wound based on the binarized segmentation map, and fit the wound tangent plane by combining the outer contour pixels of the wound with the depth information; calculate the distance between the pixels of the binarized segmentation map and the tangent plane of the wound, correct the three-dimensional surface map based on the distance, and generate a wound depth map; calculate the wound volume based on the distance and depth information; the multimodal large model obtains the wound assessment result based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound; the display screen 93 is used to display the wound area, wound volume, wound depth map, and wound assessment result; and the data interface 94 is used to transmit the wound area, wound volume, wound depth map, and wound assessment result to the hospital system.
[0076] The data processing terminal 92 is equipped with a display screen 93 and a data interface 94. The data processing terminal 92 obtains physiological and clinical data of the wound from the hospital system.
[0077] For example, the scanner 91 and the data processing terminal 92 are connected via a connecting cable 95. Alternatively, the scanner 91 and the data processing terminal 92 are wirelessly connected. In the case of a wireless connection, the data processing terminal 92 is equipped with a wireless transmission signal transmitter 96, and the scanner 91 and the data processing terminal 92 transmit information through the wireless transmission signal transmitter 96.
[0078] Furthermore, this application also provides a storage medium storing a computer program thereon, which, when run by a processor, causes the processor to execute the digital wound assessment method according to the embodiments of this application described above. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
[0079] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules according to the embodiments of this application. This application can also be implemented as a program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0080] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0081] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.
Claims
1. A method of digital wound assessment, characterized in that, The method includes: Acquire a color image and a depth image of the wound, wherein the depth image includes the depth information; A binarized segmentation map is obtained based on the color image; The wound area is calculated based on the binarized segmentation map and the depth information; A three-dimensional surface map of the wound is drawn based on the binarized segmentation map and the depth information; The outer contour pixels of the wound are obtained based on the binarized segmentation map, and the wound cutting plane is fitted by combining the outer contour pixels of the wound with the depth information. Calculate the distance between the pixels of the binarized segmentation map and the wound cutting plane, and correct the three-dimensional surface map according to the distance to generate a wound depth map; Calculate the wound volume based on the distance and depth information; The multimodal large model obtains wound assessment results based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound.
2. The digital wound assessment method of claim 1, wherein, The depth information includes the depth value corresponding to each pixel and the camera depth scaling factor.
3. The digital wound assessment method as described in claim 2, characterized in that, The binarized segmentation map includes pixels in the wound region and pixels in the non-wound region. Calculating the wound area based on the binarized segmentation map and the depth information includes: The area corresponding to each pixel in the wound region is calculated based on the depth value corresponding to the pixel in the wound region and the camera depth scaling factor. The area of the wound is obtained by adding up the areas corresponding to each pixel in the wound region.
4. The digital wound assessment method as described in claim 3, characterized in that, The step of drawing a three-dimensional surface map of the wound based on the binarized segmentation map and the depth information includes: A three-dimensional surface map of the wound is drawn based on the pixels of the wound area and the corresponding depth values.
5. The digital wound assessment method as described in claim 4, characterized in that, The step of obtaining the outer contour pixels of the wound based on the binarized segmentation map, and fitting the wound cutting plane by combining the outer contour pixels of the wound with the depth information, includes: The outer contour pixels of the wound are obtained based on the pixels in the wound area, and the horizontal and vertical coordinates of the outer contour pixels of the wound are recorded. By combining the horizontal and vertical coordinates of the outer contour pixels of the wound with the depth value corresponding to the outer contour pixels of the wound, the three-dimensional coordinates of the outer contour pixels of the wound are obtained. The tangential plane of the wound is obtained by performing planar fitting based on the three-dimensional coordinates of the pixels of the outer contour of the wound.
6. The digital wound assessment method as described in claim 5, characterized in that, The step of calculating the distance between the pixels of the binarized segmentation map and the wound cutting plane, and correcting the three-dimensional surface map based on the distance to generate a wound depth map includes: Calculate the distance between the pixels in the wound region and the cutting plane of the wound to obtain the distance between each pixel in the wound region and the cutting plane of the wound; The depth values corresponding to the pixels in the wound region of the three-dimensional surface map are corrected to distance to generate a wound depth map.
7. The digital wound assessment method as described in claim 6, characterized in that, The step of calculating the wound volume based on the distance and depth information includes: The area corresponding to each pixel in the wound region is calculated based on the depth value corresponding to the pixel in the wound region and the camera depth scaling factor. The volume corresponding to each pixel in the wound region is obtained by multiplying the distance between each pixel in the wound region and the cutting plane of the wound region with the area corresponding to each pixel in the wound region. The volume of the wound is obtained by adding the volumes corresponding to each pixel in the wound area.
8. The digital wound assessment method as described in claim 1, characterized in that, The multimodal large model obtains wound assessment results based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound, including: Convert the wound depth map into geometric features; The color image and the binarized segmentation map are converted into visual features; The wound area, wound volume, and physiological data of the wound are converted into physiological characteristics; The clinical data of the wound were converted into clinical features; The geometric features, visual features, physiological features and clinical features are fused together to obtain fused features; The multimodal large model obtains wound assessment results based on the fusion features.
9. A digital wound assessment device, characterized in that, Includes scanner, data processing terminal, display screen and data interface: A scanner for acquiring color and depth images of a wound, wherein the depth image includes the depth information; A data processing terminal is used to obtain a binarized segmentation map from the color image; and to calculate the wound area based on the binarized segmentation map and the depth information. A three-dimensional surface map of the wound is drawn based on the binarized segmentation map and the depth information; The outer contour pixels of the wound are obtained based on the binarized segmentation map. The wound tangent plane is fitted by combining the outer contour pixels with the depth information. The distance between the pixels of the binarized segmentation map and the wound tangent plane is calculated. The three-dimensional surface map is corrected based on the distance to generate a wound depth map. The wound volume is calculated based on the distance and the depth information. The multimodal large model obtains the wound assessment result based on the color image, binarized segmentation map, wound area, wound depth map, wound volume, and physiological and clinical data of the wound. A display screen is used to show the wound area, wound volume, wound depth map, and wound assessment results; The data interface is used to transmit the wound area, wound volume, wound depth map, and wound assessment results to the hospital system.
10. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the digital wound assessment method as described in any one of claims 1-8.