Artificial intelligence technology for generating future images of trauma prediction

Abstract

An exemplary system includes a processor that obtains image capture data of a sequence of one or more images representing the appearance of a wound at corresponding image capture times, each image being separated by a sampling time interval between the image and the next image, and passes the image capture data of a sequence of images, where a prediction time interval between a future time and the capture time of the last image of the sequence of images is greater than each sampling time interval, to a machine learning model trained to generate image data representing one or more predicted images of the future appearance of the wound at the corresponding future time, and is configured to output image data representing one or more predicted images of the future appearance of the wound.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 351,954, filed on June 14, 2022, which is hereby incorporated by reference in its entirety.

Background Art

[0002] Many real - world processes, especially those of a chemical and biological nature, progress slowly over time. For example, a wound may take several weeks to fully heal depending on the nature of the wound, the size of the wound, and the treatment used during the healing period.

Summary of the Invention

[0003] The present disclosure generally describes techniques for generating a predicted image showing the future appearance of a wound. More specifically, the present disclosure describes exemplary techniques for generating a predicted image of the future appearance of a wound based on applying a machine - learning model to a time - series of actual images of the wound. The predicted image can represent the estimated appearance of the wound in days or weeks in the future. This can enable medical practitioners to make an early determination of the treatment or treatment parameters to be used to treat a wound based on a reasonably accurate prediction of the possible future appearance of the wound when given a particular treatment or treatment parameters.

[0004] As described herein, a prediction system receives time series images of a wound during image capture, before treatment, and / or during a preliminary stage of treatment, and can generate a predicted image of the wound that will appear at a given future time point for a particular treatment or treatment parameter. A processing unit of the prediction device receives the image data and provides the image data to a machine learning model that trains based on the image data, treatment method, and / or treatment parameter to generate a predicted image of the future appearance of the wound. In various examples described herein, the prediction system receives time series images from an initial period, e.g., an initial image of the wound before treatment and an image of the wound captured after treatment has begun, and processes the time series images to generate a predicted image of the wound that is likely to appear after several days or weeks of treatment.

[0005] Existing methods of wound treatment typically lead to treatment decisions that depend on the current state of the wound. The techniques of the present disclosure may provide at least one technical advantage over existing methods. For example, the actual application of the techniques disclosed herein can be used to generate a predicted image of the future appearance of the wound that can be used to guide decisions regarding treatment and / or treatment parameters that will result in improved outcomes with respect to wound healing and wound appearance. As treatment progresses, a prediction system using the techniques disclosed herein can receive additional image captures of the wound and generate new predicted images of the future appearance of the wound. These new predicted images can be used to determine whether the current treatment plan is optimal or whether the treatment of the wound needs to be modified or replaced with a new treatment.

[0006] In one example, the present disclosure describes a system that includes a memory and a processing unit having one or more processors coupled to the memory, and the one or more processors cause the processing unit to obtain image capture data of one or more image sequences representing the appearance of a wound at corresponding image capture times, wherein each image before the last image of the image sequence is separated by a sampling time interval between the image and the next image; pass the image capture data of the image sequence to a machine learning model trained to generate image data representing one or more predicted images of the future appearance of the wound, wherein each of the one or more predicted images represents the future appearance of the wound at a corresponding future time, and the machine learning model is trained using historical image data, the historical image data including one or more historical image data sets, each historical image data set of the one or more historical image data sets including image data of a historical image sequence of the appearance of a corresponding historical wound, and a prediction time interval between a future time and the capture time of the last image of the image sequence is greater than each sampling time interval; and output image data representing one or more predicted images of the future appearance of the wound.

[0007] In another example, the present disclosure describes a method that includes obtaining, by a processing unit including one or more processors, image capture data of one or more image sequences representing the appearance of a wound at corresponding image capture times, wherein each image prior to the last image of the image sequence is separated by a sampling time interval between the image and the next image; passing the image capture data of the image sequence to a machine learning model trained to generate image data representing one or more predicted images of a future appearance of the wound, wherein each of the one or more predicted images represents the future appearance of the wound at a corresponding future time, and the machine learning model is trained using historical image data, the historical image data including one or more historical image data sets, each historical image data set of the one or more historical image data sets including image data of a historical image sequence of the appearance of a corresponding historical wound, and a predicted time interval between the future time and the capture time of the last image of the image sequence is greater than each sampling time interval; and outputting image data representing one or more predicted images of the future appearance of the wound.

[0008] In another example, the present disclosure describes a method that includes receiving historical image data, the historical image data including a plurality of historical image data sets, each historical image data set of the historical image data sets including image data of a historical image sequence of a corresponding wound, wherein each image of the historical image sequence prior to the last image of the historical image sequence is separated by a sampling time interval between the image and the next image; for each historical image data set of the plurality of historical image data sets, training a machine learning model to generate one or more predicted images of a future appearance of the wound, wherein each image corresponds to a future time from the historical image sequence, and a predicted time interval between the future time and the capture time of the last image of the historical image sequence is greater than each sampling time interval; and adjusting weights within a layer of the machine learning model based on a difference between the one or more predicted images and one or more target images associated with the wound.

[0009] In a further example, the present disclosure describes a system comprising means for obtaining image capture data of one or more image sequences representing the appearance of a wound at corresponding image capture times, wherein each image prior to the last image of the image sequence is separated by a sampling time interval between the image and the next image; means for passing the image capture data of the image sequence to a machine learning model trained to generate image data representing one or more predicted images of the future appearance of the wound, wherein each of the one or more predicted images represents the future appearance of the wound at a corresponding future time, and the machine learning model is trained using historical image data, the historical image data comprising one or more historical image data sets, each historical image data set of the one or more historical image data sets comprising image data of a historical image sequence of the appearance of a corresponding historical wound, and a prediction time interval between the future time and the capture time of the last image of the image sequence is greater than each sampling time interval; and means for outputting image data representing one or more predicted images of the future appearance of the wound.

[0010] In yet a further example, the present disclosure describes a system comprising means for receiving historical image data, the historical image data comprising a plurality of historical image data sets, each historical image data set of the plurality of historical image data sets comprising image data of a historical image sequence of a corresponding wound, and each image of the historical image sequence prior to the last image of the historical image sequence being separated by a sampling time interval between the image and the next image; means for training a machine learning model for each historical image data set of the plurality of historical image data sets to generate one or more predicted images of the future appearance of the wound, each image corresponding to a future time from the historical image sequence, and a prediction time interval between the future time and the capture time of the last image of the historical image sequence being greater than each sampling time interval; and means for adjusting weights within a layer of the machine learning model based on a difference between the one or more predicted images and one or more target images associated with the wound.

[0011] Details of at least one example of the present disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present disclosure will become apparent from the description and drawings, and from the claims.

Brief Description of the Drawings

[0012]

Fig. 1A

Fig. 1B

Fig. 1C

Fig. 2A

Fig. 2B

Fig. 3

Fig. 4A

Fig. 4B

Fig. 4C

Fig. 5

Fig. 6

Fig. 7

[0013] Systems and techniques are described for generating a predicted image of a future image of a wound based on a current image capture of the wound and metadata associated with the image capture. The prediction system can receive an image capture of a wound taken by a patient using the patient's own image capture device (e.g., a smartphone camera, a digital camera, etc.). The image capture can be a time series of samples over an initial sample period. The prediction system can generate a predicted image of the future appearance of the wound that will appear at some point in the future, perhaps in the next few days or weeks. The difference between the future point in time and the last sample of the time series may be referred to as the prediction interval. The sampling period can be made relatively much shorter than the entire treatment period of the wound, i.e., the sampling period may be much shorter than the prediction interval.

[0014] Figure 1A is a block diagram showing a system for generating a predictive image of the future appearance of a wound according to at least one exemplary technique described in the present disclosure. In some aspects, system 100 includes a prediction system 102 and a client device 132. The prediction system 102 and the client device 132 may be communicatively coupled to each other via a network 130. The network 130 can be any type of network, including a local area network, a wide area network, or a network that is part of the Internet.

[0015] The client device 132 can be any type of computing device having an image capture device 110. In some aspects, the client device 132 can be a smartphone, for example, a smartphone owned by a patient or another person associated with the patient. In some aspects, the client device 132 can be a camera in a clinic, a hospital, or other medical facility.

[0016] The image capture device 110 acquires one or more images 103 of the wound. In the example shown in Figure 1A, the image capture device 110 captures three images at different times after the patient is injured. In this example, the image capture device 110 captures an image of the wound 109A on the first day after the patient is injured, an image of the wound 109B three days after the patient is injured, and an image of the wound 109C seven days after the patient is injured.

[0017] The image capture device 110 can be a camera or other component configured to capture image data representing a wound. The image capture device 110 can include components capable of capturing image data, such as a video recorder, an infrared camera, a CCD (charge-coupled device) array, or a laser scanner. Although one image capture device 110 is shown in FIG. 1A, there may be multiple image capture devices 110. The images 109 may all be captured from the same device or from different devices. For example, an image of the wound 109A may be captured by the client device 132 in a medical facility treating or diagnosing the patient's wound, while images of the wounds 109B and 109C may be captured by the client device 132 owned by the patient.

[0018] In some embodiments, the image 103 may be represented as a two-dimensional image. In some embodiments, the image 103 can be a three-dimensional (3D) volume of the image. For example, the image may be represented as a 3D volume of image data recorded over a relatively short period of time. As an example, the 3D volume can be a video recording. The three dimensions of the volume can be the x-dimension, the y-dimension, and the time dimension. Thus, capturing image data can refer to capturing a 2D image or recording image data of multiple frames as a 3D volume over a period of time.

[0019] The image capture device 110 may store the captured image 103 in the storage unit 107 of the client device 132. The client device 132 may send the captured image 103 to the prediction system 102 via the network 130. In some embodiments, the client device 132 may send the captured image 103 to the prediction system 102 individually, i.e., before another image of the wound 109 is captured. In some embodiments, the client device 132 may store multiple images 103 of the wound in the storage unit 107 and send the multiple images to the prediction system 102 together.

[0020] The prediction system 102 can receive the image 103 transmitted by the client device 132 and store the received image in the storage unit 105 as part of the wound image sequence 112. In some embodiments, the wound image sequence can be a single image of the wound 109. In some embodiments, the wound image sequence 112 can be a time-series image of the wound 109 (e.g., an image in which wounds 109A-109C are captured over a period of time). The prediction system 102 can store one or more timestamps as part of the metadata 114 associated with the wound sequence 112. The timestamp can be a timestamp obtained from the image data 103 indicating when the image was captured. A timestamp indicating when the image was received may be generated by the prediction system. The timestamp may be stored with the image or as part of the metadata 114. The metadata 114 may also include data such as the type of treatment used to treat the wound (e.g., wound closure and healing treatment), the product used to clean the skin and / or the wound, and the product used to bandage the wound. As an example, the metadata 114 may include one or more parameters for controlling a Negative-Pressure Wound Therapy (NPWT) system. The NPWT system can be configured to control the fluid at the wound site based on one or more input parameters that control wound cleaning and / or infusion. Details of the NPWT system are described in U.S. Provisional Patent Application No. 63 / 201,319, filed April 23, 2021, entitled "Wound Therapy System," which is hereby incorporated by reference in its entirety.

[0021] Metadata 114 may also include characteristics of the wound itself, such as the size of the wound, the location of the wound, the type of tissue affected, signs and / or symptoms of infection and exudate (secretion) associated with the wound, or other data associated with the wound. Metadata 114 may also include demographic information about the patient (age, gender, ethnicity, etc.), or other data associated with the patient. For example, Metadata 114 may include data from patient records such as weight, body mass index, personal and / or family medical history and co-morbidities, past and current diagnoses, such as diabetes, obesity, cardiovascular disease, cholesterol, blood pressure, etc., prescription medications including dosage and frequency of use, blood test results and values, genetic test results, allergies and allergy test results, and any other appropriate patient health record data.

[0022] The processing unit 104 of the prediction system 102 can read and process the wound image sequence 112. For example, in response to receiving a command from a user via the user interface 111, the prediction system 102 can read and process the wound sequence 112. The processing unit 104 can utilize the artificial intelligence (AI) engine 106 and the machine learning model 108 to process the image data of the wound image sequence 112 and optionally the metadata 114, and generate the predicted wound image data 116 and optionally the predicted metadata 117. In some embodiments, the AI engine 106 and the machine learning model 108 can implement a neural network. For example, the machine learning model 108 can define the layers of a neural network trained using the techniques described herein, receive the wound image sequence 112 as an input, and generate the predicted wound image data 116 as an output. In some embodiments, the predicted wound image data 116 is in the same format as the image data of the images in the wound image sequence 112. For example, if the images in the wound image sequence 112 are 2D images, the predicted wound image data 116 can represent 2D images. Similarly, if the images in the wound image sequence 112 are 3D volumes, the predicted wound image data 116 represents 3D volumes. In some embodiments, the predicted wound image data 116 can have a format different from the image data of the images in the wound image sequence 112. For example, the images in the wound image sequence 112 can be 3D volumes. The prediction system 102 can generate the predicted wound image data 116 as a 2D image.

[0023] In some embodiments, the prediction system can extract image processing features (e.g., differences from an initial image over time, gradient-based images, etc.) and use such features as additional inputs to the prediction system 102 and the machine learning model 108.

[0024] In some embodiments, the predicted wound image data 116 can be the data of a single predicted wound image. In some embodiments, the predicted wound image data 116 can be an image sequence (e.g., a 2D image or a 3D volume). In some embodiments, the image sequence can be a time sequence of predicted images having a temporal order. For example, the first image of the predicted wound image data 116 can be the earliest predicted image, and the last image can be the predicted image at the farthest time point in the sequence.

[0025] In the example shown in FIG. 1A, the wound image sequence 112 is presumed to be a sequence of a plurality of images. In some embodiments, the wound image sequence 112 can be a single image (e.g., a single 2D image or a single 3D volume), and the machine learning model 108 may be trained to generate the predicted wound image data 116 from a single input image.

[0026] In some examples, the user interface 111 enables a user to control the system 100. The user interface 111 can include any combination of a display screen, a touch screen, buttons, voice input, or voice output. In some examples, the user interface 111 is configured to power on or off any combination of elements of the system 100, provide configuration information and other inputs to the prediction system 102 and / or the processing unit 104, and display the output from the prediction system 102.

[0027] FIG. 1B is a block diagram showing another system for generating a predicted image of the future appearance of a wound according to at least one exemplary technique described in the present disclosure. In the example shown in FIG. 1B, the prediction system 102 includes a preprocessor 136 that can process the wound image 103 before the prediction system 102 generates the predicted wound image data 116. As described above, the wound image 103 may be generated using the image capture device 110 of the client device 132. In some embodiments, the client device 132 can be a smartphone or other handheld device. Images of the wound captured over time may be captured under different conditions. For example, images of the wound may be captured at different distances, different angles, and different lighting conditions. The images may also include various amounts of background elements in the image. These images in their pre-processed form are referred to as the unregistered wound image sequence 142. The preprocessor 136 can use image segmentation techniques to segment the wound from the image data and exclude non-wound elements such as background elements and unaffected body parts. The preprocessor 136 can then align the segmented wound image with respect to scale and angle. The segmented and aligned wound image may be referred to as the registered wound image sequence 144. In some embodiments, the processing unit 132 can use the aligned wound image sequence 142 as an input to generate the predicted wound image data 116. In the example shown in FIG. 1B, the prediction system 132 generates three predicted images 140A-140C of the future appearance of the wound, representing the appearance of the wound in one week (140A), two weeks (140B), and three weeks (140C) in the future.

[0028] In some embodiments, the prediction system 102 may generate prediction metadata 117 in addition to, or instead of, the prediction image 116. The prediction metadata 117 can include predicted future wound characteristics such as the predicted wound shape (e.g., wound area, wound depth, wound location), wound healing stage, etc. In the example shown in FIG. 1B, the prediction system 132 generates three sets of prediction metadata, metadata 118A - 118C, associated with the wound. These sets of metadata represent the wound characteristics at one week in the future (metadata 118A), two weeks in the future (metadata 118B), and three weeks in the future (metadata 118C).

[0029] FIG. 1C is a block diagram showing input image data of a system that generates a predicted image of the future appearance of a wound by at least one exemplary technique described in the present disclosure. In the example shown in FIG. 1C, the image capture device 110 captured image data of images 112A - 112C of an exemplary wound sequence 112 at various time points over the input time interval 120. In this example, the time interval between the capture of image 112A and image 112B can be m days. The time interval between the capture of image 112B and image 112C is m ± k days. The predicted time interval 122 between the time when image 112C was captured and the time when the predicted wound image data 116 is generated is m + h days. The predicted time interval 122 does not represent the amount of time that has actually elapsed. Instead, the predicted time interval m + h represents the simulated time interval between image 112C and the predicted wound image data 116. The actual time interval between image 112C and the generation of the predicted wound image data 116 may simply be the amount of time it takes for the prediction system 102 to generate the predicted wound image data 116. The predicted time interval 122 can be much longer than the input time interval 120. For example, the predicted time interval 122 can be longer than twice the input time interval 120, and in some examples, even much longer, e.g., several weeks longer. However, the predicted time interval 122 is not limited in this regard and may in some cases be equal to, or approximately equal to, the input time interval.

[0030] As an example, the image capture device 110 can create a wound image sequence 112 by capturing an image 103 of the wound 109 (FIG. 1A) every other day over a period of five days. The prediction system 102 can process the wound image sequence 112 to generate prediction wound image data 116 representing a predicted image of the future appearance of the wound at a future point in time, for example, in the next two weeks. Using the prediction wound image data 116, it can be determined whether the current treatment plan for the wound is acceptable to the physician and / or patient, or whether a different or modified treatment plan should be considered. Using the techniques described herein, a user (or user system) can reach a conclusion regarding wound treatment much earlier than would be possible using currently existing methods, using the predicted image of the future appearance of the wound. In the example described above, the user can reach a conclusion regarding wound treatment several days or weeks earlier than with current methods.

[0031] The example shown in FIG. 1C shows some aspects of the wound image sequence 112 and the prediction wound image data 116. The first aspect is that there can be long sampling intervals. Each input image can be several days apart from each other. Within these intervals, there can be many changes governed by a potentially non-linear process involved in wound healing. Therefore, it is difficult to generate an accurate future image or 3D volume of the wound.

[0032] The second aspect is that the time intervals between image captures can be inconsistent and non-uniform. As shown in FIG. 1C, the first two samples can be m days apart, while the time between the next two samples can be longer or shorter than m days (i.e., m ± k days). Also, the expected intervals can be relatively large due to data dropout or corruption. Therefore, k can be even higher when the wound image capture is missing or corrupted.

[0033] A third aspect is that the predicted time interval (e.g., m+h) associated with the predicted wound image data 116 can be very long compared to the interval between image captures of the wound. For example, the predicted time interval 122 between the image (e.g., image 112C) captured last in the wound image sequence 112 and the predicted wound image data 116 can be several days to several weeks apart.

[0034] A user, such as a clinician or healthcare provider, can operate the prediction system 102 to determine the possible effects of different treatments or different treatment parameters (e.g., NPWT parameters) on wound healing and the future appearance of the predicted wound. For example, a set of one or more current images of the wound, along with metadata describing the wound, may be provided to the prediction system 102. The user can provide, as input to the prediction system 102, data describing a treatment method, or additional metadata such as parameters of a wound treatment system. The prediction system 102 can generate a predicted future image of the wound based on the current image of the wound and the input metadata. The user can change different input parameters, such as the proposed treatment and / or treatment system parameters, and select the treatment and / or treatment parameters that produce the desired result for the predicted image of the future appearance of the wound and apply them to the wound.

[0035] FIG. 2A is a block diagram showing training data for a training system, such as the training system described below with reference to FIG. 2B. In some embodiments, the training data includes a plurality of historical wound image sequences. The historical wound image sequence 212 includes images 232A-232N that include the image data of the corresponding wound image sequences captured at different times. The images 232A-232M of the historical wound image sequence can be the images captured during the sampling period 210. The sampling period 210 can include the images captured before the completion of the treatment of the wound, which can include the images captured several days or weeks before the expected completion of the treatment. As an example, the image 232A may be captured before the start of the treatment when the patient first visits a doctor seeking treatment for his wound. Generally speaking, the images 232B-232M may be captured at any time before the completion of the treatment, for example, during the initial stage of the treatment of the wound. The images 232M+1-232N can be the images captured during the later stage of the wound treatment period 208. The image 232N, which is the final image of the sequence, can be the target image of the sequence. That is, the final image 232N may be used as the ground truth regarding the appearance of the corresponding wound undergoing a given treatment at a desired time in the treatment of the wound, for example, at the end of the treatment or at a time after the treatment is completed.

[0036] FIG. 2B is a block diagram illustrating a training system according to at least one exemplary technique described in the present disclosure. The training system 202 can include a machine learning framework 204 that includes a machine learning engine 206. The machine learning framework 204 can receive training data 203 and process the training data to generate a machine learning model 224. In some aspects, the machine learning framework 204 can include a machine learning engine 206 that can use supervised or unsupervised machine learning techniques to train the machine learning model 224. In some aspects, the machine learning engine 206 can be a deep learning engine that implements a convolutional neural network (CNN). In some aspects, the machine learning engine 206 can be, for example, a generative adversarial network (GAN). As an example, the machine learning engine can be a T-Adversarial GAN. In some aspects, the machine learning engine 206 can be a U-Net-based machine learning engine that includes a U-Net 2D architecture and a U-Net 3D architecture. The U-Net architecture can be used to preserve content such as spatial information in the training data. The U-Net architecture typically has a contracting path and an expanding path and can be used in combination with skip connections within the layers to link corresponding feature maps on the encoder and decoder. Linking of the feature maps can facilitate reuse of features in the encoder, thereby reducing information loss. Additionally, the U-Net architecture is computationally efficient and can be trained with a relatively small dataset.

[0037] In some embodiments, the machine learning framework 204 can implement multiple machine learning techniques that can be applied together when training the machine learning model 224. For example, the machine learning engine 206 can be a U-Net engine, and the machine learning framework can apply a cyclic learning technique using the machine learning engine 206. Further details regarding the machine learning framework and cyclic learning are provided below with respect to FIGS. 4A-4C.

[0038] The training data 203 can include historical wound image sequences 212A-212N (collectively referred to as the historical wound image sequence 212). Each historical wound image sequence 212 within the training data is a time-series image sequence of a specific wound captured or recorded over a period of time prior to training the machine learning model 224. For example, the historical wound image sequence 212A can be the image data of an image sequence showing the appearance of a first wound over time, the historical wound image sequence 212B can be the image data of an image sequence showing the appearance of a second wound over time, and the historical wound image sequence 212C can be the image data of an image sequence showing the appearance of a third wound over time.

[0039] Each historical wound image sequence 212A, 212N within the training data 203 can have a corresponding target image 220A, 220N. The target image of the image sequence is the "ground truth" final image, e.g., the actual image of the wound associated with the image sequence captured at the end of the treatment period.

[0040] The training data 203 may also include metadata 214 that can be used to train the machine learning model 224. The metadata 214 can include a timestamp indicating when the images in the historical wound image sequence 212 were captured. The metadata 214 can include patient demographic information, such as data from patient records such as weight, body mass index, personal and / or family medical history and co-morbidities, past and current diagnoses, such as diabetes, obesity, cardiovascular disease, cholesterol, blood pressure, etc., prescription medications including dosage and frequency of use, blood test results and values, genetic test results, allergies and allergy test results, and any other appropriate patient health record data. The metadata 214 can include wound information such as wound shape (e.g., wound location, wound depth, wound size, etc.), affected tissue type, signs or symptoms of infection, and / or healing stage. The metadata 214 may also include data such as the type of treatment used to treat the wound (e.g., wound closure and healing treatment), products used to clean the skin and / or the wound, and products used to bandage the wound. As an example, the metadata 214 can include one or more parameter values of the NPWT system used to treat the wound. The metadata 214 may also include the size of the wound, the location of the wound, the affected tissue type, signs and / or symptoms of infection, and exudate (secretion) associated with the wound at the time the historical image was captured. The metadata 214 may also include demographic information about the patient (age, gender, etc.), or other data associated with the patient or the wound at the time the historical image was captured. In some embodiments, the metadata 214 may be added to the training system 202 (or the prediction system 102 of FIG. 1) by padding metadata information on the boundaries of the image. In some embodiments, the metadata 214 may be added as a vector to a latent feature vector generated in an intermediate layer of the machine learning model 224.

[0041] Training system 202 provides training data 203 to a machine learning framework 204 for processing by a machine learning engine 206. The machine learning engine 206 processes a sequence of historical wound images 212 to generate predicted image data 218. The predicted image data 218 can include a sequence of predicted images of the future appearance of the wound, each having an associated future time. The machine learning framework 204 can compare the predicted images to a target image 220 associated with the sequence of historical wound images 212 to determine the difference between the predicted images and the target image 220. The difference between the predicted images and the target image 220 is used to update the training weights in a machine learning model 224 to improve the model's ability to generate accurate predicted images of the future appearance of the wound. In some embodiments, the weights in the machine learning model 224 can be adjusted using a loss function such as a reconstruction loss or a GAN loss.

[0042] In some embodiments, the machine learning framework 204 can also train the machine learning model 224 to generate predicted metadata 217 using historical metadata information (e.g., metadata 214) associated with the sequence of historical wound images 212. The metadata 214 can be a historical sequence of metadata and target metadata corresponding to the sequence of historical wound images 212 and the target image 220. The machine learning framework can compare the predicted metadata 217 to the target metadata and adjust the machine learning model based on the difference between the predicted metadata 217 and the target metadata.

[0043] After the training system 202 trains the machine learning model 224, the model may be deployed to a prediction system 216. The prediction system 216 can be an implementation of the prediction system 102 of FIG. 1. The prediction system 216 can receive a sequence of wound images 112 and process the image sequence using an AI engine 222 and the deployed machine learning model 224 to generate predicted wound image data 116 and / or predicted metadata 117.

[0044] As shown in FIG. 2B, the machine learning framework 204 can generate prediction image data 218 that can include a sequence of S images (e.g., 2D images or 3D volumes). In some embodiments, the machine learning framework 204 can generate prediction image data 218 that can be a single 2D image or 3D volume. Additionally, the historical wound image sequence 212 can be a sequence of multiple images as shown in FIG. 2B. In some embodiments, the wound image sequence 212 can be a single image (e.g., a single 2D image or a single 3D volume), and the machine learning framework 204 can train the machine learning model 224 to generate the prediction image data 218 from a single input image.

[0045] In some embodiments, the machine learning engine 206 can execute a weighted loss function that assigns different weights to the images in the prediction image data 218. For example, the weighted loss function can assign a greater weight to images that are later in the image sequence than to images that are earlier in the sequence. In other words, a first predicted future image associated with a predicted future time earlier than a second predicted future time associated with a second predicted future image will have a smaller weight than the second predicted future image. This can be beneficial because a more accurate and temporally later predicted future image within a sequence of prediction images may be more valuable to an end user than another predicted image predicted for an earlier future time within the sequence. In some examples, these weights can also be learned from the data. For example, the machine learning model can automatically learn the relevance and importance of each image data in the input.

[0046] FIG. 3 is a block diagram showing a further aspect of a training system according to at least one exemplary technique described in the present disclosure. In the example shown in FIG. 3, the training system 300 includes a loading and formatting unit 302, a data splitting unit 304, a spatial expansion unit 306, a temporal expansion unit 308, a sampling unit 310, a batch processing unit 312, a preprocessing unit 313, a machine learning framework 314, a test unit 320, and a result visualization unit 322. The loading and formatting unit 302, the data splitting unit 304, the spatial expansion unit 306, the temporal expansion unit 308, the sampling unit 310, the batch processing unit 312, the preprocessing unit 315, the machine learning framework 314, the test unit 320, and the result visualization unit 322 can be implemented as a configurable pipeline for processing the candidate image dataset 301 into batches of image datasets used by the machine learning framework 314 to train the machine learning model 319.

[0047] The loading and formatting unit 302 can process the candidate image dataset 301 to format the image sequences in the candidate image dataset 301 into a form that can be processed by the training system. For example, the images may be resized, sized, trimmed, etc. to be in a form compatible with the machine learning framework 314.

[0048] The data splitting unit 304 can split the candidate image dataset 301 into training data, test data, and / or validation data. For example, the input parameters may specify the ratio of the dataset to be used as training data, test data, and / or validation data.

[0049] The spatial expansion unit 306 can increase the amount of training data by converting existing images into one or more additional training images. For example, the image may be converted by cropping a part of the image, moving the part along the left, right, and diagonal axes, rotating the image, mirroring the image, etc. to create new images that can be included in the training data.

[0050] The temporal expansion unit 308 can control the selection of images from the candidate image dataset 301 based on the temporal aspects of the candidate training data. The temporal expansion unit 308 can select an image sequence based on where the image is located on the time axis. As an example, the temporal expansion unit 308 can select an image based on a start time and an end time.

[0051] The sampling unit 310 can select images from the training data according to the skip factor 311. For example, instead of including all images in the candidate image dataset 301, the sampling unit 310 can select a subset of the images in the candidate dataset. The skip factor 311 may be used to control how the images are selected. For example, if the skip factor is 4, the sampling unit 310 may skip four images in the candidate dataset before selecting the next image to include in the training data.

[0052] The configuration data 324 can include data that determines data sources, hyperparameters, machine learning parameters, types of machine learning, etc. for use by the machine learning framework 314.

[0053] The batch processing unit 312 creates and controls batches of training data to be processed as a unit. For example, the first batch of training data may be used to train the first machine learning model 319, and the second batch of training data may be used to train the second machine learning model 319. The batch processing unit 312 may use the configuration data 324 to determine which data source should be used for a batch of training data. The batch processing unit 312 may also use the configuration data 324 to specify the configuration parameters that the machine learning framework 314 should use when training the machine learning model 319 using the corresponding batch of training data.

[0054] The batch processing unit 312 can provide a batch of training data to the machine learning framework 314 for use in training the machine learning model 319. In some aspects, the batch processing unit 312 can provide the training data to the preprocessing unit 313. The preprocessing unit 313 can apply image segmentation techniques to each wound image in the training set of images to segment the wound from other image data and exclude non-wound elements such as background elements and / or unaffected body parts. The preprocessor 136 can then align the segmented wound images with respect to scale and angle. The segmented and aligned wound images may be referred to as an aligned wound image sequence 315.

[0055] The machine learning framework 314 can include a machine learning engine 316. In some aspects, the machine learning framework 314 and / or the machine learning engine 316 can be implemented in the form of the machine learning framework 204 and / or the machine learning engine 206 of FIG. 2B. As described above, the machine learning engine 316 can be a deep learning engine that implements a U-Net-based machine learning engine including CNN, GAN, U-Net 2D architecture, and U-Net 3D architecture. The machine learning framework 314 can use the techniques described herein to train a machine learning model 319 to generate a predicted image of the future appearance of a wound.

[0056] The test unit 320 can test the machine learning model 319 to determine the accuracy of the predicted future wound image generated using the machine learning model 319. As described above, the candidate image dataset 301 can be split into training data and test data. The machine learning model 319 is trained using the training data. The test unit 320 can receive a sequence of historical wound images in the test data as an input and generate a predicted wound image as an output using the machine learning model 319.

[0057] As an example, the test data may include a historical sequence of wound images, where the first portion of the images in the sequence is captured during sampling period 210 and the other images in the sequence are captured after sampling period 210. The last image in the sequence can be the target wound image. The test unit 320 can apply the machine learning model 319 to the first portion of the historical sequence of images to generate a predicted wound image. The test unit 320 can then compare the predicted wound image to the target wound image and, based on the comparison, determine the accuracy of the predicted wound image. The test unit 320 can determine various measurements of the performance of the machine learning model 319 and compare those measurements to other machine learning models that may have been generated using different training parameters and / or training data. The results of the comparison can be used to determine the machine learning model 319 that generates a better (e.g., more accurate) predicted wound image.

[0058] The result visualization unit 322 can provide feedback to the user regarding the training of the machine learning model 319. For example, the result visualization unit 322 can output statistics regarding the accuracy of the predicted images generated by the machine learning model 319. In some aspects, the result visualization unit 322 can output examples of the input wound image sequences and the predicted wound images generated by the machine learning model 319. The user can utilize the output of the result visualization unit 322 to determine whether any adjustments need to be made regarding the training of the machine learning model 319. For example, the user can adjust hyperparameters, prediction time intervals, or other configuration data 324 and the signal batching unit 312 to initiate the provision of another batch of training data for training a new machine learning model 319. The result visualization unit 322 can provide an output that can be used to compare the performance of the machine learning model 319 to other machine learning models.

[0059] The training system 300 does not have to include all of the components shown in FIG. 3, and in various implementations, the training system 300 may include various combinations of one or more of the components shown in FIG. 3.

[0060] FIGS. 4A-4C are block diagrams showing an exemplary bidirectional machine learning framework used in the training systems shown in FIGS. 2 and 3 according to at least one exemplary technique described in the present disclosure. In the examples shown in FIGS. 4A-4C, the machine learning model is trained to generate a predicted image of the future appearance of a wound using a sequence 406 of input wound images. In the examples shown in FIGS. 4A-4C, the image sequence 406 is a sequence of k images IMG1-IMG k where IMG1 is the first image in the sequence and IMG k is the last image in the sequence. The input image sequence 406 can be images of the wound captured during a sampling period. IMG out is a predicted future image generated by the machine learning framework using image data selected from the images 406. IMG label is an image captured at the end of or after treatment. IMG label can be an image captured several days or weeks after the input image 406 was captured. IMG label represents the "ground truth" image, also referred to as the target image.

[0061] FIG. 4A is a block diagram showing a machine learning framework 402 that trains a machine learning model by executing two paths that pass an image sequence and predicts a future image of the appearance of a wound. The machine learning framework 402 can be an implementation of the machine learning framework 204 of FIG. 2B and / or the machine learning framework 314 of FIG. 3. The machine learning framework 402 includes two deep learning architectures 404A and 404B. The deep learning architecture 404A is used to train a machine learning model 405 to generate a predicted future image from a sequence of past images, and the deep learning architecture 404B is used to train the machine learning model 405 to reconstruct a past image from a later image and the predicted future image. The deep learning architectures 404A and 404B can each be a CNN (including U-Net 2D and U-Net 3D), a GAN, a T-Adversarial GAN, a Time Cyclic GAN, or a GAN that uses privileged information. In some embodiments, the deep learning architectures 404A and 404B share layers within the machine learning model 405. The shared layers provide a learning constraint that past information is linked to future information and the past image can be reconstructed using the predicted future image. In the example shown in FIG. 4A, the goal of the first path is to use the deep learning architecture 404A to generate a predicted future image IMG label similar or identical to the ground truth image IMG out . IMG out is compared with IMG label , and the difference between IMG out and IMG label is used to update the training weights in the deep learning architecture 404A to attempt to improve the generated predicted future image IMG out . In some embodiments, the weights in the machine learning model 405 can be adjusted using a loss function such as a reconstruction loss or a GAN loss. The process described above is generally the same as that used for one-way training in some implementations.

[0062] In the second pass, the secondary goal is to use the deep learning architecture 404B to generate a reconstructed first image IMG1' within the sequence that is the same as or similar to the actual first image IMG1 in the sequence, using the subsequent images IMG2 to IMG k and the predicted future image IMG out as inputs to the deep learning architecture 404B. IMG1' is compared to IMG1 (which is now considered the target image), and the difference is used to adjust the weights within the layers of the machine learning model 405. This second pass can make the layer weights more robust and avoid the machine learning model from overfitting to the training data.

[0063] FIG. 4B is a block diagram showing a machine learning framework 410 that executes a two-stage bidirectional pass to train a machine learning model and predict a future image of the appearance of a wound. The machine learning framework 410 includes deep learning architectures 411A, 411B, and 411C (collectively referred to as "deep learning architecture 411"). Each of the deep learning architectures 411 can be a CNN (including U-Net 2D and U-Net 3D), GAN, T-Adversarial GAN, Time Cyclic GAN, or a GAN that uses privileged information.

[0064] In some embodiments, the deep learning architecture 411A is implemented in the same manner as the deep learning architectures 404A and 404B described above with reference to FIG. 4A. That is, the deep learning architecture 411A can be bidirectional and can execute two passes over an image sequence, i.e., the first pass generates a predicted future image IMG out from the initial image in the image sequence 406, and the second pass uses the predicted future image IMG outAnd generate a first reconstructed image IMG1 based on the first image in the image sequence and subsequent images. In some embodiments, the deep learning architecture 411A is different from the deep learning architectures 404A and 404B of FIG. 4A in that when training the machine learning model, it includes image data with a longer time range. For example, the deep learning architecture 411A may also include images 408 collected during the later stages of wound treatment in addition to the input image 406 collected before wound treatment and the images at the initial stage of treatment.

[0065] In the example shown in FIG. 4B, the first stage of generating the machine learning model 415 is to train the machine learning model 415 using the deep learning architecture 411A to generate a predicted future image IMG label that is the same as or similar to the ground truth image IMG out . The machine learning framework 410 compares IMG out with IMG label and uses the difference between IMG out and IMG label to update the training weights in the machine learning model 415 and attempt to improve the generated predicted future image IMG out . In some embodiments, the weights in the machine learning model 415 can be adjusted using a loss function such as reconstruction loss or GAN loss. Further, the deep learning architecture 411A trains the machine learning model 415 to reconstruct the first image IMG1 from the images following IMG out and IMG1. As described above, in the example shown in FIG. 4B, the input to the deep learning architecture 411A in the first stage of machine learning can include images captured before treatment (e.g., IMG1~IMG k ) and images captured after treatment has started but before treatment is completed (e.g., images IMG k+n , IMG k+n+1 , IMG k+n+2 , etc.). Thus, in the first stage of training, the machine learning framework 410 utilizes data with a longer time frame to improve the accuracy of the machine learning model 415.

[0066] To improve the accuracy of predicted future images, it may be advantageous to use a longer time frame. However, one aspect of the techniques disclosed herein is a machine learning model that can generate a predicted future image shape using images captured before treatment of the wound and images captured during earlier stages of treatment of the wound without relying on images captured during stages after treatment of the wound. Thus, in the example shown in FIG. 4B, during the second stage of machine learning, the machine learning framework 410 uses the images 406 captured before and during the initial stage of treatment of the wound to continue training the machine learning model 415'. Similar to the first stage, the second stage of training can be two-way, and the deep learning architecture 411B shares the layers of the machine learning model 415' with the deep learning architecture 411C. For example, the deep learning architecture 411B trains the machine learning model 415' to generate the predicted future image IMG k ) using the input image 406 (e.g., IMG1 to IMG out , and determines an adjustment to the weights of the machine learning model 415' by comparing IMG out with IMG label . Further, the deep learning architecture 411C trains the machine learning model 415' to generate the reconstructed image IMG1' using IMG out and IMG k - IMG2 as inputs, and determines an adjustment to the weights of the machine learning model 415' by comparing IMG1' with IMG1.

[0067] The machine learning framework 410 can impose constraints 412 on the training of the machine learning model 415'. For example, the machine learning framework 410 can enforce a constraint that a particular layer of the machine learning model 415' matches the weights of the corresponding layer of the machine learning model 415. In some aspects, the constraint can be that the weights of the final layer of the machine learning model 415' match the weights of the final layer of the machine learning model 415. In some aspects, the constraint can be that the weights of the intermediate layers of the machine learning model 415' match the weights of the corresponding intermediate layers of the machine learning model 415.

[0068] In addition to the aspects described above, a further aspect of the present disclosure shown in FIG. 4B is that the machine learning model 415' can be trained using additional data to be acquired in the future. For example, the machine learning model 415 may be trained using an initial set of training data. As additional data becomes available at a future time, the machine learning model 415' may be trained as described above to potentially improve the accuracy of the predicted images.

[0069] In the example shown in FIG. 4B, the machine learning framework 410 performs bidirectional training (i.e., cyclic training) to train both the machine learning model 415 and the machine learning model 415'. However, bidirectional training is not a requirement, and in some aspects, the machine learning framework 410 can train either or both of the machine learning models 415 and 415' using a single direction.

[0070] FIG. 4C is a block diagram showing another machine learning framework 420 that executes a two-stage bidirectional pass to train a machine learning model to predict the future appearance of a wound. The machine learning framework 420 includes deep learning architectures 422A, 422B, and 422C (collectively "deep learning architecture 422"). Each of the deep learning architectures 422 can be a CNN (including U-Net 2D and U-Net 3D), GAN, T-Adversarial GAN, Time Cyclic GAN, or a GAN that uses privileged information.

[0071] In some embodiments, the deep learning architecture 425A is implemented in the same manner as the deep learning architectures 404A and 404B described above in FIG. 4A, and the deep learning architecture 411A described above in FIG. 4B. That is, the deep learning architecture 422A can be bidirectional and can execute two passes over an image sequence. For example, the first pass generates a predicted future image IMG out from an initial image within the image sequence 406, and the second pass generates a first reconstructed image IMG1 based on the predicted future image IMG out and the images following the first image in the image sequence. Similar to the deep learning architecture 411A, the deep learning architecture 422A includes more training data than the example shown in FIG. 4A. However, in the example of FIG. 4C, the additional captured training data can include more images from the images captured before the treatment of the wound. In the example shown in FIG. 4C, the deep learning architecture 422A first trains the machine learning model 425 using the images IMG1-IMG6. Thus, in the first stage of training, the deep learning architecture 422A utilizes more image samples to improve the accuracy of the machine learning model 425.

[0072] During the second stage, the deep learning architecture 422B trains the machine learning model 425’ using fewer images from the image 406. In the example shown in FIG. 4C, the deep learning architectures 422B and 422C use half the number of images (e.g., IMG1, IMG3, and IMG5). Similar to the first stage, the second stage of training can be bidirectional, and the deep learning architecture 422B shares the layers of the machine learning model 425’ with the deep learning architecture 422C. For example, the deep learning architecture 422B uses the input images 406 (e.g., IMG1, IMG3, and IMG5) to train the machine learning model 425’ to generate a predicted future image IMG out and determines an adjustment to the weights of the machine learning model 425’ by comparing IMG out to IMG label . Further, the deep learning architecture 422C trains the machine learning model 425’ to generate a reconstructed image IMG1’ using IMG out , IMG5, and IMG3 as inputs, and determines an adjustment to the weights of the machine learning model 425’ by comparing IMG1’ to IMG1.

[0073] The machine learning framework 420 can impose constraints 424 on the training of the machine learning model 425’. For example, the machine learning framework 420 can enforce a constraint that a particular layer of the machine learning model 425’ matches the weights of the corresponding layer of the machine learning model 425. In some aspects, the constraint can be that the weights of the final layer of the machine learning model 425’ match the weights of the final layer of the machine learning model 425. In some aspects, the constraint can be that the weights of the intermediate layers of the machine learning model 425’ match the weights of the corresponding intermediate layers of the machine learning model 425.

[0074] In addition to the aspects described above, a further aspect of the present disclosure shown in FIG. 4C is that, even though fewer samples are utilized during the initial stages of training (e.g., the test stage), the machine learning framework 420 can still generate a machine learning model 425' that has the same or similar accuracy as a machine learning model trained using more samples.

[0075] Similar to the example shown in FIG. 4B, in the example shown in FIG. 4C, the machine learning framework 420 performs bidirectional training (i.e., cyclic training) to train both the machine learning model 425 and the machine learning model 425'. However, bidirectional training is not a requirement, and in some aspects, the machine learning framework 410 can train either or both of the machine learning models 415 and 415' using a single direction.

[0076] FIG. 5 is a block diagram of an exemplary processing unit of a system for generating a predicted future image of the appearance of a wound according to at least one exemplary technique described in the present disclosure. FIG. 5 is a block diagram showing an exemplary processing unit 500 according to at least one exemplary technique described in the present disclosure. The processing unit 500 can be an example or alternative implementation of the processing unit 104 of FIGS. 1A and 1B. The architecture of the processing unit 500 shown in FIG. 5 is shown for illustrative purposes only. The processing unit 500 should not be limited to the illustrated exemplary architecture. In other examples, the processing unit 500 may be configured in various ways. In the example shown in FIG. 5, the processing unit 500 includes a prediction unit 510 configured to generate a predicted wound image based on a sequence of input wound images. The prediction unit 510 can include an AI engine 512 configured to process the wound image sequence using a machine learning model 514 to generate a predicted wound image as an output.

[0077] In some embodiments, the machine learning model 514 can include data that defines a CNN. In some embodiments, the machine learning model 514 can include data that defines a U-Net, including a Generative Adversarial Network (GAN), a T-Adversarial GAN, a U-Net 2D, and a U-Net 3D.

[0078] The processing unit 500 may be implemented as any suitable computing system (e.g., at least one server computer, workstation, mainframe, appliance, cloud computing system, and / or other computing system) that can perform the operations and / or functions described according to at least one embodiment of the present disclosure. In some examples, the processing unit 500 represents a cloud computing system, server farm, and / or server cluster (or a portion thereof) configured to connect to the system 100 via a wired or wireless connection. In other examples, the processing unit 500 may represent or be implemented through at least one virtualized compute instance (e.g., a virtual machine or container) of a data center, cloud computing system, server farm, and / or server cluster. In some examples, the processing unit 500 includes at least one computing device, and each computing device has a memory and at least one processor.

[0079] As shown in the example of FIG. 5, the processing unit 500 includes a processing circuit 502, at least one interface 504, and at least one storage unit 506. The prediction unit 510 including the AI engine 512 may be implemented as program instructions and / or data stored in the storage unit 506 and executable by the processing circuit 502. The storage unit 506 may store a machine learning model 514. The storage unit 506 of the processing unit 500 may also store an operating system (not shown) executable by the processing circuit 502 to control the operation of the components of the processing unit 500. The components, units, or modules of the processing unit 500 may be coupled (physically, communicatively, and / or operably) using communication channels for inter-component communication. In some examples, the communication channels include a system bus, a network connection, an inter-process communication data structure, or any other means for communicating data.

[0080] In one example, the processing circuit 502 may include at least one processor configured to execute functions and / or process instructions for execution within the processing unit 500. For example, the processing circuit 502 may be capable of processing instructions stored by the storage unit 506. The processing circuit 502 may include, for example, a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry, or any combination of the foregoing devices or circuits.

[0081] To facilitate parallel processing of inspection operations, multiple instances of processing circuit 502 may exist within processing unit 500. The multiple instances may be of the same type, such as a multiprocessor system or a multi-core processor. The multiple instances may also be of different types, such as a multi-core processor having a plurality of associated graphics processor units (GPUs).

[0082] Processing unit 500 can communicate with an external system via at least one network using interface 504. In some examples, interface 504 includes an electrical interface configured to electrically couple processing unit 500 to prediction system 102. In other examples, interface 504 can be a network interface (e.g., an Ethernet interface, an optical transceiver, a radio frequency (RF) transceiver, Wi-Fi, or any other type of device capable of transmitting and receiving information via use of the wireless technology of the trademark “BLUETOOTH®”), a telephone interface, or. In some examples, processing unit 500 wirelessly communicates with an external system using interface 504.

[0083] Storage unit 506 may be configured to store information within processing unit 500 during operation. The storage unit 506 may include a computer-readable storage medium or a computer-readable storage device. In some examples, the storage unit 506 includes at least short-term memory or long-term memory. The storage unit 506 may include, for example, random access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), magnetic disk, optical disk, flash memory, magnetic disk, optical disk, flash memory, or may include forms such as electrically programmable memory (EPROM) or electrically erasable and programmable memory (EEPROM). In some examples, the storage unit 506 is used to store program instructions for execution by processing circuit 502. The storage unit 506 may be used by software or an application operating on processing unit 500 to temporarily store information during program execution.

[0084] FIG. 6 is a flow diagram showing an exemplary operation of a prediction system according to one or more techniques of the present disclosure. The prediction system can receive image data of an image sequence representing the appearance of a wound by a processing unit comprising one or more processors, the images being captured at a plurality of times, each image prior to the last image of the image sequence being separated by a sampling time interval between the image and the next image (605). Next, the prediction system can pass the image data of the image sequence to a machine learning model trained to generate image data representing a predicted image of the future appearance of the wound at a future time, the machine learning model being trained using historical image data, the historical image data including a plurality of historical image data sets, each historical image data set of the historical image data sets including image data of a historical image sequence of a corresponding historical wound, and the prediction time interval between the future time and the capture time of the last image of the image sequence being greater than each sampling time interval (610). Next, the prediction system can output image data representing a predicted image of the future appearance of the wound (615).

[0085] FIG. 7 is a flowchart illustrating an exemplary operation of a training system according to one or more techniques of the present disclosure. The training system can receive historical image data, which includes a plurality of historical image data sets, where each historical image data set of the historical image data sets includes image data of a corresponding historical image sequence of a wound, and each image of the historical image sequence prior to the last image of the historical image sequence is separated by a sampling time interval between the image and the next image (705). In some embodiments, the sampling time interval can be variable. That is, the sampling time interval between images within the historical sequence need not be uniform across all of the images, and for some images, the sampling interval may be different from the sampling interval for other images. Next, the training system can train a machine learning model to generate a predicted image of the future appearance of the wound at a future time from the historical image sequence, where the predicted time interval between the future time and the capture time of the last image of the historical image sequence is greater than each of the sampling time intervals (710). Next, the training system can adjust the weights within the layers of the machine learning model based on the difference between the predicted image and a target image associated with the wound (715).

[0086] The above discussion has been presented in the context of predicting future images of a wound based on images taken before and / or during the initial stages of treatment of the wound. However, the techniques discussed herein may be applied to the prediction of other characteristics of wounds and wound treatment. For example, the machine learning model may be trained to predict wound area, wound depth, and / or stage of healing based on input image data, input metadata, or a combination of the two.

[0087] Furthermore, the techniques described herein can be readily applied to other fields. For example, the technique may be applied to images of microbial growth to generate predicted future images of a microbial colony based on an image sequence of the microbial colony.

[0088] The techniques of the present disclosure may also be applied to agriculture. Plant growth behavior, such as bacterial colony growth and wound healing, may have a slow and long progression. Using the techniques described herein, new cultivars and field areas that are most resistant to diseases can be predicted using image sequences of the field.

[0089] The techniques described in the present disclosure may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented within at least one processor that includes at least one microprocessor, DSP, ASIC, FPGA, and / or any other equivalent integrated logic circuit or discrete logic circuit, and any combination of such components. The term "processor" or "processing circuit" may generally refer to any of the foregoing logic circuits, alone or in combination with other logic circuits, or any other equivalent circuit. A control unit including hardware may also execute at least one of the techniques of the present disclosure.

[0090] Such hardware, software, and firmware may be implemented within the same device or in separate devices to support the various operations and functions described in the present disclosure. Additionally, any of the units, modules, or components described may be implemented together or separately as discrete but interoperable logic devices. The description of different features as modules or units is intended to emphasize different functional aspects, and does not necessarily mean that such modules or units must be implemented by separate hardware or software components. Rather, the functions associated with at least one module and / or unit may be performed by separate hardware or software components, or may be integrated within common or separate hardware or software components.

[0091] The techniques described in this disclosure may also be embodied or encoded in a computer-readable medium such as a non-transitory computer-readable medium or a computer-readable storage medium that includes instructions. The instructions embedded or encoded in the computer-readable medium may cause a programmable processor or other processor to execute a method (e.g., when the instructions are executed). The computer-readable storage medium may include RAM, read only memory (ROM), programmable read only memory (PROM), EPROM, EEPROM, flash memory, a hard disk, a CD-ROM, a floppy disk, a cassette, a magnetic medium, an optical medium, or other computer-readable storage media. The term "computer-readable storage medium" refers to a physical storage medium, not a signal or a carrier wave, but the term "computer-readable medium" may include a transient medium such as a signal in addition to the physical storage medium.

Claims

1. It is a system, Memory and The processing unit includes having one or more processors coupled to the memory, and the one or more processors are provided to the processing unit. A sequence of one or more images representing the appearance of a wound at a corresponding image capture time, wherein each of the images preceding the last image in the sequence is separated by a sampling time interval between the image and the next image, thereby obtaining image capture data for a sequence of one or more images. A machine learning model is trained to generate image data representing one or more predicted images of the future appearance of the wound, wherein each of the one or more predicted images represents the future appearance of the wound at a corresponding future time, the machine learning model is trained using historical image data, the historical image data comprises one or more historical image datasets, each of the one or more historical image datasets comprises image data of a historical image sequence of the corresponding historical wound appearance, and the predicted time interval between the future time and the capture time of the last image in the image sequence is greater than each of the sampling time intervals, the machine learning model is passed A system configured to output image data representing one or more predictive images of the future appearance of the wound.

2. The system according to claim 1, wherein the image capture data includes metadata that identifies a treatment method or one or more parameters of a treatment method.

3. The system according to claim 2, wherein the treatment method parameters include negative pressure wound therapy (NPWT) parameters.

4. The system according to claim 1, wherein the machine learning model is trained bidirectionally, the first direction of training is to train the machine learning model to generate one or more predicted future images from the historical image sequence, and the second direction of training is to train the machine learning model to generate one or more predicted images and the first image reconstructed from images in the historical image sequence following the first image.

5. The system according to claim 4, wherein the layers within the machine learning model are shared by the first direction of training and the second direction of training.

6. The aforementioned machine learning model includes a second machine learning model, The first machine learning model is trained prior to the second machine learning model using a first training image dataset which includes a first subset of images of the historical image sequence captured during a sampling period associated with the images of the historical wound and a second subset of images captured after the sampling period. The second machine learning model is constrained to include one or more layers of the first machine learning model. The system according to claim 4.

7. The system according to claim 6, wherein the first machine learning model is trained bidirectionally.

8. The system according to claim 6, wherein the one or more layers include the last layer, the second to last layer, or one or more intermediate level layers.

9. The aforementioned machine learning model includes a second machine learning model, The first machine learning model is trained prior to the second machine learning model using a first training image dataset which includes a first image subset of the historical image sequence captured during a sampling period associated with the historical wound, and a second image subset captured during the sampling period, wherein the number of images in the first image subset is greater than the number of images in the second image subset. The second machine learning model is constrained to use one or more layers of the first machine learning model. The system according to claim 1.

10. The system according to claim 9, wherein the first machine learning model is trained bidirectionally.

Images