Machine learning systems and techniques for multispectral amputation site analysis

Abstract

Certain aspects of the present disclosure relate to devices and techniques for noninvasive and noncontact optical imaging that acquire multiple images corresponding to different times and different frequencies. Additionally, the options described herein are used with various tissue classification applications, including assessing the presence and severity of tissue conditions such as necrosis and small vessel disease at a potential or determined amputation site.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 246,956, filed March 2, 2017, entitled “Systems and Machine Learning Techniques for Amputation Site Analysis,” pursuant to 35 USC § 119(e), the entire contents of which are incorporated herein by reference.

[0003] Statement on Federal Government-Sponsored Research and Development

[0004] Part of the work described in this disclosure was carried out with the support of the United States government, under the Office of the Assistant Secretary for Preparedness and Response at the U.S. Department of Health and Human Services, pursuant to Agreement No. HHSO100201300022C, which was awarded by the Biomedical Advanced Research and Development Authority (BARDA). The U.S. government may hold certain rights to this invention. Technical Field

[0005] The systems and methods disclosed in this article relate to non-invasive clinical imaging, and more specifically, to non-invasive imaging of subcutaneous blood flow, diffuse reflectance spectroscopy, and computer-aided diagnosis. Background Technology

[0006] Optical imaging is a promising emerging technology for improving disease prevention, diagnosis, and treatment in emergency situations, clinics, clinical settings, or operating rooms. Optical imaging techniques enable non-invasive differentiation between tissues and between native tissues and tissues labeled with endogenous or exogenous contrast agents by measuring the different distributions of photon absorption or scattering at different wavelengths. These differences in photon absorption and scattering hold great promise for providing specific tissue contrasts and enabling the study of functional and molecular-level activities that underlie health and disease. Summary of the Invention

[0007] The aspects of the invention described herein relate to apparatus and methods for classifying tissue regions at or near amputation sites using non-contact, non-invasive, and non-radiative optical imaging. For example, such apparatus and methods can identify tissue regions corresponding to different tissue health classifications involving small vessel disease and can output a map of the identified regions for clinicians to determine the level or location of resection. In some alternatives, such apparatus and methods can identify areas of recommended amputation levels or predicted optimal amputation sites and, for example, use tissue classification mapping to provide recommendations. There has long been a need for a non-invasive imaging technique that can provide physicians with quantitative information for selecting appropriate amputation levels.

[0008] Approximately 185,000 lower limb amputations are performed annually in the United States, affecting over two million American adults. Regardless of diabetes (DM) status, the most significant risk factor for amputation is severe peripheral artery disease (PAD), known as severe limb ischemia (CLI), which accounts for more than half of all amputations and is called vascular abnormality amputations. Individuals with diabetes have a 10-fold increased risk of lower limb amputation compared to the general population, with over 60,000 amputations annually due to diabetic lower limb ulcers. About 30 out of every 100,000 people require amputations secondary to microvascular disease each year, and this incidence is projected to increase by 50% in the next decade due to the aging U.S. population.

[0009] The cost and financial burden of limb amputations in the US healthcare system are enormous each year. In a separate study by the Veterans Affairs (VA) system, the cost burden associated with diabetic limb loss exceeded $200 million in one year (fiscal year 2010) ($60,647 per patient). In the US, total hospital-related costs for all lower limb amputations in fiscal year 2009 amounted to $8.3 billion, with the lifetime cost of major amputations, including rehabilitation and prosthetics, estimated at approximately $500,000 per patient. Beyond the heavy financial burden of limb amputation, patients experience significant impairments and reduced quality of life due to amputation. Most importantly, these patients face functional challenges; only 25% of those with major lower limb CLI amputations are able to walk outside their homes using prostheses. With progressively increasing proximal levels of amputation, the likelihood of successfully regaining mobility decreases due to increased energy expenditure caused by increased tissue loss.

[0010] While there is a clear priority to preserve as much limb tissue as possible during amputation, physicians must weigh the likelihood of initial wound healing at a given level of amputation (LOA), which tends to decrease with more distal amputations. The appropriate LOA is initially determined by the surgeon's clinical judgment (considering clinical factors such as diabetes, smoking, and nutritional status, using patient history and physical examination, including skin color, temperature, peripheral pulse, and wound bleeding during treatment), possibly combined with various non-invasive tests designed to quantify tissue blood flow and / or oxygenation (brachial index (ABI), transcutaneous oxygenation measurement (TCOM), or skin perfusion pressure (SPP)). However, one study showed that 30% of patients with a normal ABI required repeat amputation after the initial forefoot amputation. This suggests that assessing large vessel blood flow alone is insufficient to predict amputation success. In the same study, despite significant efforts in distal limb revascularization, nearly 50% of patients who underwent concurrent revascularization also required repeat amputation. This again suggests that assessing large vessel blood flow alone is insufficient to predict amputation success. Therefore, several tests (i.e., transcutaneous oxygenation measurement [TCOM]) have been developed to assess microcirculation in local tissues and have been clinically tested to guide the selection of the lobe of perfusion (LOA). Although TCOM initially showed promise in identifying initial wound healing after amputation, its use remains controversial due to a lack of sufficiently large and authoritative studies to clarify its role in clinical application. Furthermore, TCOM measurements are affected by physiological conditions such as body temperature, and TCOM electrodes can only analyze small areas of skin. Therefore, despite decades of use, TCOM has not yet been incorporated into routine clinical practice. The limited success of these methods stems from the fact that the sensitivity of perfusion assessment has never been proven to independently replace clinical judgment.

[0011] Given the challenge of balancing maximal tissue preservation with minimizing the risk of initial wound non-healing, and the initial reliance on clinical judgment to determine the appropriate level of attachment (LOA), published repeat amputation rates are by no means optimal. Repeat amputation rates vary depending on the initial level of amputation, ranging from approximately 10% of above-knee (AKA) amputations to 35% of foot amputations, and further to more proximal levels. Limited data on the direct costs of repeat amputations are currently available, but it is clear that a major portion of the billions of dollars spent annually on care related to CLI amputations is attributable to costs associated with repeat amputations, readmissions, and essentially ineffective wound care between the initial surgery and repeat amputation. Delayed and failed initial healing increases patient risk, including infection, for morbidity and mortality. Furthermore, delayed and failed initial wound healing following primary amputation significantly impacts patients' quality of life. Patients requiring amputation repair experience delays in physical rehabilitation and in obtaining prostheses to return to a walking state. These patients also have increased contact with the healthcare system, often undergoing additional wound care treatments before repair, efforts that could be avoided if an appropriate LOA (Lower Orifice of Artery) is chosen initially. Finally, while repeat amputation rates are clearly reported, studies on the extent to which physicians' awareness of the risks of repeat amputation leads to overly aggressive choices of LOA at more proximal levels are not published. In reality, it is feasible for some patients to receive amputations closer to the necessary extent because their surgeons cannot confidently predict the likelihood of healing at more distal levels. Therefore, devices to guide decisions regarding LOA have the potential to reduce repeat amputation rates and conserve tissue for patients facing major amputations.

[0012] Currently, there is no gold standard test to determine the initial wound healing capacity of patients with CLI after amputation. Attempts have been made to find such a gold standard through local assessment of tissue microcirculation alone. In this context, several instruments known to accurately assess skin tissue perfusion and oxygenation have been tested, including TCOM, SPP, and laser Doppler. However, when choosing the LOA (Lower Orifice), microcirculation assessment alone is insufficient to accurately assess tissue healing capacity as a substitute for clinical judgment. Therefore, characterizing local skin perfusion and oxygenation clearly does not provide enough information to quantify tissue healing potential. The predictive results of these techniques also fail to include the systemic effects of comorbidities that also influence wound healing potential. In fact, about two decades ago, in a review of factors influencing wound healing after major amputations in CLI, regarding the selection of the appropriate level of amputation, one author concluded: “There is no ‘gold standard test’ to predict the likelihood of healing after major amputations because wound healing is not only related to tissue blood flow. All other factors mentioned in that review (smoking, nutritional status, diabetes, and infection) are likely also important. Therefore, a combination of clinical judgment and various tests will be the most common approach.”

[0013] Although the authors have predicted this, the disclosed technology includes an imaging device capable of integrating information collected from targeted tests characterizing tissue blood flow physiology with important patient health indicators. In some embodiments, the aforementioned problem is addressed by the machine learning techniques of this disclosure (also known as artificial intelligence, computer vision, and pattern recognition techniques), which combine optical microcirculation assessments with the patient's overall health indicators to generate predictive information. Using this method, the disclosed device can provide a quantitative assessment of wound healing potential, whereas current clinical practice standards only offer qualitative assessments. The disclosed technology can accurately identify tissue healing potential at amputation sites, helping physicians select the optimal LOA (Longest Area of ​​Arrest).

[0014] Therefore, one aspect relates to a tissue classification system comprising: at least one emitter configured to sequentially emit light at each of a plurality of wavelengths to illuminate a tissue region, each of the at least one emitter being configured to emit spatially uniform light; at least one photodetector configured to collect light emitted from the at least one emitter and reflected from the tissue region; and one or more processors communicating with the at least one emitter and the at least one photodetector, and configured to: select a classifier from a plurality of classifiers based on at least one patient health indicator value; train each classifier from different subsets of a training dataset, the training data for the selected classifier including data from other patients having the at least one patient health indicator value; control the at least one emitter to sequentially emit light at each of a plurality of wavelengths; and receive a plurality of signals from the at least one photodetector. A first subset of the plurality of signals represents light emitted sequentially at the plurality of wavelengths and reflected from the tissue region, and a second subset of the plurality of signals represents light of the same wavelength reflected from the tissue region at multiple different times; an image having a plurality of pixels depicting the tissue region is generated based on at least a portion of the plurality of signals; for each pixel of the plurality of pixels depicting the tissue region: a reflection intensity value of the pixel at each of the plurality of wavelengths is determined based on the first subset of the plurality of signals, a PPG amplitude value of the pixel is determined based on the second subset of the plurality of signals, and a classification of the pixel is determined by inputting the reflection intensity value and the PPG amplitude value into a selected classifier, the classification associating the pixel with one of a plurality of tissue categories; and a mapping of the plurality of tissue categories is generated on the plurality of pixels depicting the tissue region based on the classification of each pixel.

[0015] In some embodiments, the one or more processors are configured to output a visual representation of the mapping for display to a user. In some embodiments, the visual representation includes the image having pixels displayed in colors based on a classification selection of the pixels, wherein pixels associated with each of the plurality of tissue categories are displayed in a different color.

[0016] In some embodiments, the plurality of tissue categories includes a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease. In some embodiments, generating the map includes identifying regions of the plurality of pixels that depict tissue regions associated with at least one of the live tissue category, the necrotic tissue category, and the tissue category with small vessel disease. In some embodiments, the one or more processors are configured to output an image for display, wherein the plurality of pixels depicting the tissue regions are displayed in different colors corresponding to the identified regions. In some embodiments, the one or more processors are configured to determine a recommended amputation location based on the identified regions.

[0017] In some embodiments, the one or more processors are configured to determine a melanin index of the tissue region based on the plurality of signals. In some embodiments, the one or more processors are configured to select the classifier based at least on the melanin index.

[0018] Another aspect relates to a tissue classification method, the tissue classification method comprising: selecting a classifier from a plurality of classifiers based on at least one patient health indicator value; training each classifier from different subsets of a training dataset, the training data for the selected classifier including data from other patients having the at least one patient health indicator value; receiving a plurality of signals from at least one photodetector element, the at least one photodetector element being positioned to receive light reflected from a tissue region, a first subset of the plurality of signals representing light emitted sequentially at the plurality of wavelengths and reflected from the tissue region, and a second subset of the plurality of signals representing light of the same wavelength reflected from the tissue region at multiple different times; based on From at least a portion of the plurality of signals, an image having a plurality of pixels depicting the tissue region is generated; for each pixel of the plurality of pixels depicting the tissue region: based on a first subset of the plurality of signals, a reflectance intensity value of the pixel at each of the plurality of wavelengths is determined; based on a second subset of the plurality of signals, a PPG amplitude value of the pixel is determined; and a classification of the pixel is determined by inputting the reflectance intensity value and the PPG amplitude value into a selected classifier, the classification associating the pixel with one of a plurality of tissue categories; and based on the classification of each pixel, a mapping of the plurality of tissue categories is generated on the plurality of pixels depicting the tissue region.

[0019] Some implementations also include determining a melanin index of the tissue region based on the plurality of signals. Some implementations also include selecting the classifier based at least on the melanin index.

[0020] In some embodiments, generating the map includes identifying regions of the plurality of pixels that depict tissue regions associated with at least one of a viable tissue category, a necrotic tissue category, and a tissue category with small vessel disease. Some embodiments also include outputting an image for display, wherein the plurality of pixels depicting the tissue regions are displayed in different colors, patterns, or other applicable visual representations corresponding to the identified regions, based on the associated tissue category.

[0021] On the other hand, a method for identifying a recommended location for amputation is provided, the method comprising: selecting a patient having a tissue region requiring amputation; programmatically controlling an imaging system via one or more hardware processors to capture data representing multiple images of the tissue region, the data representing the multiple images comprising a first subset and a second subset, each of the multiple images being captured using light of a different wavelength among a plurality of different wavelengths reflected from the tissue region, and the second subset being captured sequentially at multiple times; generating an image having a plurality of pixels depicting the tissue region based on at least one of the multiple images; for each of the plurality of pixels depicting the tissue region: determining a reflectance intensity value of the pixel at each of the plurality of wavelengths based on the first subset of the data representing the multiple images, determining a PPG amplitude value of the pixel based on the second subset of the data representing the multiple images, and determining a classification of the pixel at least by inputting the reflectance intensity value and the PPG amplitude value into a classifier, the classification associating the pixel with one of a plurality of tissue categories; and identifying a recommended location for amputation within the tissue region based on the classification of each pixel.

[0022] Some implementations also include identifying at least one patient health indicator value for the patient. Some implementations also include inputting the at least one patient health indicator value into the classifier to determine the classification of each pixel. Some implementations further include: selecting a classifier from a plurality of classifiers based on the at least one patient health indicator value, training each classifier from different subsets of a training dataset, wherein the training data for the selected classifier includes data from other patients having the same value as the at least one patient health indicator.

[0023] Some implementations also include generating a mapping of the plurality of tissue categories across the plurality of pixels depicting the tissue region based on a classification of each pixel. Some implementations also include outputting a visual representation of the mapping to a user, wherein the recommended location for amputation identification is based at least in part on the visual representation of the mapping.

[0024] In some implementations, the identification of the recommended location for amputation is performed programmatically by the one or more hardware processors.

[0025] Another aspect relates to a method for training a convolutional neural network to classify tissue regions of an amputation site, the method comprising: receiving training data representing a plurality of images of the amputation site, the data representing the plurality of images including a first subset and a second subset, each of the plurality of images being captured using light of a different wavelength among a plurality of different wavelengths reflected from the amputation site, and the second subset being captured sequentially at multiple times at the same wavelength; feeding the training data as a three-dimensional volume into the input layer of the convolutional neural network, the height and width of the three-dimensional volume corresponding to the number of pixels in the height and width of each of the plurality of images, and the depth corresponding to the number of images in the plurality of images; and performing convolutional stages of a plurality of encoders in the convolutional neural network and Multiple convolutions are performed at multiple decoder convolution stages, wherein a first encoder convolution stage of the multiple encoder convolution stages includes an input layer as a first convolutional layer; the output of the last decoder convolution stage of the multiple decoder convolution stages is fed to a normalized exponential function layer of the convolutional neural network; based on the output of the normalized exponential function layer, classification values ​​for each pixel are generated across the height and width of the multiple images; the classification values ​​of each pixel are compared with ground truth classifications of the pixels in a ground truth image, wherein the ground truth classifications are based on physician analysis of the amputation site; based on the comparison results, any errors in the classification values ​​of the pixels are identified; and at least one weight of the multiple convolutions is adjusted, at least in part, based on errors backpropagated through the convolutional neural network.

[0026] In some implementations, generating the classification includes classifying each pixel as one of background, healthy tissue, diseased tissue, or necrotic tissue.

[0027] In some implementations, a first subset of the multiple images comprises eight images captured using light of different wavelengths from eight different wavelengths, and a second subset of the multiple images comprises hundreds of images captured sequentially at the same wavelength at a rate of 30 frames per second.

[0028] In some implementations, each of the plurality of encoder convolutional stages and each of the plurality of decoder convolutional stages includes at least two convolutional layers, each followed by a rectified linear unit layer.

[0029] In some implementations, performing multiple convolutions includes: at each of the plurality of encoder convolution stages: performing at least a first encoder convolution, feeding the output of the first encoder convolution to a rectified linear unit layer, and downsampling the output of the rectified linear unit layer using a max-pooling layer; and at each of the plurality of decoder convolution stages: receiving a pooling mask from a max-pooling layer of the corresponding one of the plurality of encoder convolution stages, and performing at least a first decoder convolution at least in part based on the pooling mask.

[0030] Another aspect relates to a method for classifying tissue regions of a potential amputation site using a convolutional neural network, the method comprising: receiving data representing a plurality of images of a potential amputation site, the data representing the plurality of images including a first subset and a second subset, each of the plurality of images being captured using light of a different wavelength among a plurality of different wavelengths reflected from the potential amputation site, and the second subset being captured sequentially at multiple times at the same wavelength; feeding the data as a three-dimensional volume to an input layer of the convolutional neural network, the height and width of the three-dimensional volume corresponding to the number of pixels at the height and width of each of the plurality of images, and the depth corresponding to the number of images in the plurality of images; performing at least one convolution on the three-dimensional volume; feeding the output of the at least one convolution to a normalized exponential function layer of the convolutional neural network; generating classification values ​​for each pixel across the height and width of the plurality of images based on the output of the normalized exponential function layer; and generating a mapping of multiple tissue classifications of the tissue at the potential amputation site based on the classification values ​​for each pixel.

[0031] In some implementations, generating the classification includes classifying each pixel as one of background, healthy tissue, diseased tissue, or necrotic tissue.

[0032] In some implementations, a first subset of the multiple images comprises eight images captured using light of different wavelengths from eight different wavelengths, and a second subset of the multiple images comprises hundreds of images captured sequentially at the same wavelength at a rate of 30 frames per second.

[0033] In some embodiments, performing at least one convolution includes performing multiple convolutions at multiple encoder convolutional stages and multiple decoder convolutional stages of the convolutional neural network, wherein a first encoder convolutional stage of the multiple encoder convolutional stages includes an input layer as a first convolutional layer. In some embodiments, performing multiple convolutions includes: at each of the multiple encoder convolutional stages: performing at least a first encoder convolution, feeding the output of the first encoder convolution to a rectified linear unit (RCU) layer, and downsampling the output of the RCU layer using a max-pooling layer; and at each of the multiple decoder convolutional stages: receiving a pooling mask from a max-pooling layer of a corresponding one of the multiple encoder convolutional stages, and performing at least a first decoder convolution at least partially based on the pooling mask. In some embodiments, each of the multiple encoder convolutional stages and each of the multiple decoder convolutional stages includes at least two convolutional layers, each convolutional layer followed by a RCU layer.

[0034] In some implementations, the above method is used to train the convolutional neural network. Attached Figure Description

[0035] The disclosed aspects will be described below in conjunction with the accompanying drawings and appendices, which are provided for illustrative purposes and not for limiting the disclosed aspects, wherein similar reference numerals denote similar elements. The patent or application document includes at least one color drawing. Upon request and payment of the necessary fees, the official authority will provide a copy of the disclosed text of the patent or application document with color drawings.

[0036] Figure 1A An example component of an imager is shown, which is imaging a subject.

[0037] Figure 1B This is an illustration of an exemplary movement of an exemplary imaging detector.

[0038] Figure 2 This is an illustration of an exemplary user interface for acquiring images.

[0039] Figure 3 This is an example illustration of a high-resolution multispectral camera used in some of the alternative solutions described in this article, and the data that can be obtained.

[0040] Figure 4 This is an example diagram illustrating how some of the alternatives described in this article interact with remote computing centers (i.e., cloud computing environments) used for data storage and processing.

[0041] Figure 5A , Figure 5B and Figure 6An exemplary fiber optic system that can be used to obtain the image data described herein is shown.

[0042] Figure 7 The reflection mode and components of a 2-D PPG imaging system are shown (left). Monochromatic light incident on the tissue surface is scattered within the tissue as it interacts with molecular structures. A small portion of the light returns to the camera. As measurements are taken over a period of time, the intensity variation of the backscattered light produces a PPG waveform. Each pixel in the raw data cube contains a unique PPG waveform, which can be analyzed to generate a single blood flow image of the tissue (right).

[0043] Figure 8 The components of a multispectral imager are shown, including a broadband illumination source, a digital camera, and a rotating filter wheel equipped with various optical filters that separate light of predetermined wavelengths reflected from a target surface (left). The system rapidly acquires images at various locations on the filter wheel to generate a spectral data cube (right). Each pixel in the data cube represents the low-spectral-resolution diffuse reflectance spectrum of the tissue.

[0044] Figure 9 This is an example flowchart illustrating the steps used for organizing categories in some of the alternative schemes described herein.

[0045] Figure 10 The time-resolved PPG signal extraction is shown.

[0046] Figure 11 A sample block diagram of PPG output preprocessing is shown.

[0047] Figure 12 A high-level graphical overview of two optical imaging techniques, according to this disclosure, that can be combined with patient health indicators to generate predictive information: photoplethysmography (PPG imaging) and multispectral imaging (MSI).

[0048] Figure 13 An example view of a device designed to combine optical imaging techniques such as photoplethysmography (PPG imaging) and multispectral imaging (MSI) is shown.

[0049] Figure 14 An example of combining a PPG imager, an MSI camera, and target patient health indicator inputs is shown.

[0050] Figure 15A , Figure 15B and Figure 15C An example process is shown for training a machine learning diagnostic tool and generating a classifier model for amputation levels.

[0051] Figure 16A graphic example of the tissues involved in a traditional amputation surgery is shown.

[0052] Figure 17A and Figure 17B A sample clinical study flowchart is shown.

[0053] Figure 17C An example training study flowchart is shown.

[0054] Figure 17D A flowchart of an example validation study is shown.

[0055] Figure 18 The results of an example statistical sample size analysis are shown.

[0056] Figure 19 An example graphical representation of the tissue classification process for amputation sites described in this article is shown.

[0057] Figure 20A Example images of amputation sites marked by physicians are shown.

[0058] Figure 20B It shows the basis Figure 20A Example image of tissue classification mapping generated from the image.

[0059] Figure 21 The cross-validation results of the disclosed amputation machine learning technique are shown.

[0060] Figure 22 Example images are shown after training the publicly available amputation machine learning model.

[0061] Figure 23A and Figure 23B An example categorical data stream and classifier structure for generating the organization mapping described in this paper are shown. Detailed Implementation

[0062] introduction

[0063] Various aspects of this disclosure relate to a non-contact, non-invasive, non-radiation imaging device, for example, for classifying and / or quantitatively selecting the appropriate level of amputation (LOA) or amputation region at the amputation site in patients with peripheral artery disease (PAD). In the United States, more than 150,000 patients undergo lower limb amputation annually due to peripheral artery disease (PAD). Surgeons prefer to salvage as much limb tissue as possible during amputation to increase patient mobility while reducing morbidity and mortality. However, clinicians must weigh this preference against the likelihood of initial wound healing at a given LOA, which decreases with increasing distal amputation. As used herein, “distal” can refer to a tissue area distant from the LOA to the area of ​​tissue to be amputated from the patient, while “proximal” can refer to a tissue area at or closer to the LOA than to tissue not to be amputated from the patient. There is no gold standard test for selecting the LOA in patients with PAD; therefore, re-amputation rates are reported to be very high in current practice. Approximately 10% of above-knee amputations and about 35% of foot amputations require repair to a more proximal level. Furthermore, in some cases, physicians aware of the risk of re-amputation may over-aggressively select a more proximal level for LOA (Location of Arrival). In fact, some patients can accept a more proximal amputation level than necessary because their surgeons cannot confidently predict the high probability of healing at a more distal level. In current practice, LOA selection is qualitatively determined by the surgeon through clinical judgment using patient history and physical examination. Therefore, a system is needed that provides physicians with quantitative information about patient tissue health and healing potential during the LOA selection process. The disclosed device and technology provide a quantitative assessment of LOA selection by integrating photoplethysmography, multispectral imaging, and / or patient-specific health indicators to classify tissue based on microvascular blood flow using machine learning models.

[0064] The diagnostic device described herein provides a point-of-care perfusion imaging system that delivers diagnostic images derived from optical measurements of tissue perfusion. The device is non-contact, non-invasive, non-laser, and non-radioactive. It can perform two optical imaging methods simultaneously or sequentially to obtain blood flow assessment: photoplethysmography (PPG) imaging and multispectral imaging (MSI). These optical measurements are integrated using a machine learning model, and in some embodiments, patient health indicators are also integrated to quantify the likelihood of healing at a given LOA (Location of Arrest). Caregivers can be easily trained to perform imaging tests on this user-friendly device. Imaging multiple limb levels using this device may take approximately ten minutes in total. Results are immediately available and stored electronically for later review by a physician.

[0065] If this sensitivity and specificity were used for routine assessment of patients prior to amputation, the disclosed diagnostic device and technique could reduce the repeat amputation rate by approximately 67%, resulting in a reduction of 10,000 repeat amputations annually. This would both improve the quality of life for amputees and reduce healthcare costs associated with their care. For example, using the disclosed technique, the quality of life for amputees could be improved by reducing the rate of secondary amputations due to healing failure and / or by selecting a LOA location that provides improved healing time compared to a qualitatively assessed LOA location.

[0066] The disclosed apparatus and techniques enable PPG imaging to capture over one million unique photoplethysmography (PPG) signals over large areas of tissue. PPG signals can be generated by measuring the dynamic interaction of light with vascularized tissue. Vascularized tissue expands and contracts by approximately 1-2% of its volume with each incoming systolic pressure fluctuation at the frequency of the cardiac cycle. The inflow of blood increases the tissue volume and introduces other hemoglobins that strongly absorb light. Therefore, the total light absorption within the tissue oscillates with each heartbeat. To generate an image from the PPG signals detected by the disclosed imaging apparatus, the techniques described herein utilize the path of light through the tissue. For example, a small portion of light incident on the tissue surface is scattered into the tissue. A portion of this scattered light leaves the tissue from the same surface it initially entered. Using a sensitive digital camera, this backscattered light can be collected over the tissue area such that each pixel in the image data includes a unique PPG waveform determined by variations in the intensity of the scattered light. To generate a two-dimensional visual mapping of the tissue blood flow, the average amplitude of each unique waveform can be measured relative to a number of heartbeat samples. Therefore, the main advantage of this technology is that it allows for the measurement and mapping of microvascular blood flow, which is crucial for wound healing.

[0067] The disclosed apparatus and techniques utilize multispectral imaging (MSI) to measure the reflectance of selected wavelengths of visible and near-infrared (NIR) light (400 nm–1100 nm) from tissue surfaces. MSI can effectively quantify key tissue characteristics relevant to many pathologies, such as amputations, burns, diabetic ulcers, and skin cancers (e.g., melanoma, squamous cell carcinoma, and basal cell carcinoma), because it can quantify the volume fraction of hemoglobin and the presence of oxyhemoglobin, as well as other tissue features. The wavelength of light used by the disclosed apparatus can be selected based on recognized light-tissue interaction characteristics. For example, melanin in the stratum corneum and epidermis primarily absorbs UV and visible wavelengths. NIR wavelengths (700 nm–5000 nm) are absorbed least by melanin and are found to penetrate the dermis most effectively to determine its depth. Blood vessels penetrating the dermis contain a large amount of hemoglobin, and the concentration of hemoglobin determines the degree of dermal absorption at wavelengths greater than 320 nm. The light absorption of hemoglobin pairs also varies depending on whether the molecules are in an oxygenated or deoxygenated state. Because the concentrations of melanin, hemoglobin, and oxyhemoglobin in tissues change with disease states, microscopy (MSI) can detect changes in the resulting reflectance spectra. Abnormal skin tissue can be identified by these changes compared to healthy tissue. Therefore, MSI can quantitatively distinguish and map active and inactive skin tissue, which is important for healing at amputation or resection sites.

[0068] The alternatives disclosed herein relate to systems and techniques for identifying, assessing, and / or classifying tissues of a subject. Some alternatives relate to apparatus and methods for classifying tissues, wherein such apparatus includes optical imaging components. Some alternatives described herein include reflectance-mode multispectral time-resolved optical imaging software and hardware, which, when implemented, are applicable to several tissue classification methods provided herein.

[0069] The options described in this article allow for the automated or semi-automated assessment and classification of tissue areas in subjects who may require amputation, and can also provide treatment recommendations. Some of the options described are particularly well-suited for amputation level assessment because they enable physicians to quickly and quantitatively assess the tissue condition around the amputation site, allowing for rapid and accurate decisions regarding the amputation level. Some options can also help surgeons select nearby healthy tissue to cover the amputation site.

[0070] Throughout this specification, one or more terms "wound" are used. It should be understood that the term "wound" is interpreted broadly to include both open and closed wounds in which the skin is torn, cut, punctured, or diseased, or in which trauma results in contusions, superficial lesions, or a condition or defect on the skin of a subject (e.g., a human or animal, particularly a mammal). "Wound" is also intended to include any area of ​​tissue damage in which fluid is produced or not produced due to injury or disease. Examples of such wounds include, but are not limited to: acute wounds, chronic wounds, surgical and other incisions, subacute and lacerations, trauma, flaps and skin grafts, abrasions, bruises, burns, diabetic ulcers, pressure ulcers, pores, surgical wounds, traumatic and venous ulcers, and in some cases, skin cancers (e.g., melanoma, squamous cell carcinoma, basal cell carcinoma), etc., are also referred to as wounds herein. It is understood that a machine learning classifier described herein can be trained based on ground truth classification of similar images in a training dataset to classify the tissue state of tissue areas that include (or are suspected of including) any of these types of wounds.

[0071] For illustrative purposes, various alternatives will be described below with reference to the accompanying drawings. It should be understood that many other embodiments of the disclosed concepts are possible, and various advantages can be achieved using the disclosed embodiments. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Many of the alternatives described herein include similar components, and therefore, these similar components can be interchanged in different aspects of the invention.

[0072] This document includes headings for reference and to help identify the various sections. These headings are not intended to limit the scope of the concepts described herein. These concepts may apply throughout the entire specification.

[0073] Overview of Example Imaging System Options

[0074] Figure 1A and Figure 1BAn example of an alternative embodiment of the invention is shown. The apparatus illustrated in these figures is particularly suitable for whole-body or partial body assessment of patients awaiting amputation. The apparatus is especially useful for amputation level assessment, in which clinical decisions related to the amputation location are made. In this example, detector 100 includes one or more light sources (in this case, four light sources 101, 104, 118, and 120) and image acquisition device 102. Light sources 101, 104, 118, and 120 illuminate a tissue area, in this case, tissue 103, which advantageously encompasses the entire body surface of the subject facing detector 100. In some alternative embodiments, the one or more light sources may be light-emitting diodes (LEDs), halogen lamps, tungsten lamps, or any other lighting technology. The one or more light sources can be selected by the user to emit white light or light falling within one or more spectral bands as needed.

[0075] Many LEDs produce narrow-bandwidth light (e.g., full width at half maximum (FWHM) of 50 nm or less), where specific LEDs can be selected to irradiate with a specific bandwidth. Typically, one or more spectral bands can be selected based on the type of photometry most relevant to the desired data and / or clinical application. One or more light sources can also be connected to one or more drivers to power and control the light source. These drivers can be part of the light source itself or separate. Multiple narrow-band or broadband light sources with optional filters (e.g., filter wheels) can be used to irradiate tissue 103 serially or simultaneously using light from multiple spectral bands. The center wavelength of the selected spectral band is typically between visible and near-infrared wavelengths, such as 400 nm to 1100 nm, for example, less than (but not zero), at least or equal to 400, 500, 600, 700, 800, 900, 1000, or 1100 nm, or a range defined by any two of the aforementioned wavelengths.

[0076] In some alternatives, the light source illuminates the tissue region with a substantially uniform intensity over the entire area of ​​the irradiated tissue region (referred to herein as "spatially uniform" light or illumination). For example, substantially uniform intensity can be achieved by using a light diffuser configured as part of light sources 101, 104, 118, and 120, which produces a substantially uniform distribution of light intensity applied to tissue 103. The light diffuser also has the additional benefit of reducing unwanted specular reflections. In some cases, a significant improvement in the signal-to-noise ratio of the signal acquired by image acquisition device 102 can be achieved by utilizing a broad-spectrum spatially uniform illumination mode with high-power LEDs, such as a cross-polarizing filter. In some cases, patterned light systems such as checkerboard pattern illumination can also be used. In some such alternatives, the field of view of the image acquisition device is directed towards tissue regions that are not directly illuminated by the light source but are adjacent to the irradiated area. For example, in the case of using substantially uniform intensity light, image acquisition devices such as image acquisition device 102 can read light from outside the irradiated area. Similarly, for example, when using chessboard pattern lighting, the acquisition device 102 can read light from the non-illuminated portion of the chessboard.

[0077] Furthermore, while substantially uniform light intensity is effective in some of the alternatives described herein, other alternatives may employ non-uniform light, where one or more beams are arranged to minimize differences in light intensity on the surface. In some cases, these differences may also be attributable to the data acquisition process, backend software, or hardware logic. For example, top-hat transformations or other image processing techniques can be used to compensate for non-uniform background illumination.

[0078] In some alternatives, the light can be polarized as desired. In some cases, the light is polarized using any method of polarization known in the art, such as reflection, selective absorption, refraction, scattering, and / or polarization. For example, polarization can be achieved using prisms (such as Nicol prisms), mirrors and / or reflective surfaces, filters (such as polarizing filters), lenses, and / or crystals. The light can be cross-polarized or co-polarized. In some alternatives, light from one or more light sources is polarized before it illuminates the subject. For example, a polarizing filter can be incorporated into light sources 101, 104, 118, and 120. In some alternatives, reflected light from tissue is polarized after reflection from the tissue. For example, a polarizing filter can be incorporated into the acquisition device 102. In other alternatives, the light is polarized both before and after illuminating the subject. For example, a polarizing filter can be incorporated into light sources 101, 104, 118, and 120 and also into the image acquisition device 102.

[0079] The type of polarization technique used can depend on factors such as the illumination angle, the reception angle, the type of illumination source used, the type of data desired (e.g., measurement of scattered, absorbed, reflected, transmitted, and / or fluorescent light), and the depth of the tissue being imaged. For example, when illuminating tissue, some light can be directly reflected off the surface of the skin as surface glare and reflection. This reflected light typically has a different polarity than the light diffused into the dermis, where it can be scattered (e.g., reflected) and its direction and polarity can change. Cross-polarization techniques can be used to minimize the amount of glare and reflection read by the acquisition device while maximizing the amount of backscattered light read. For example, polarizing filters can be configured as part of light sources 101, 104, 118, and 120 and as part of the image acquisition device 102. In this arrangement, the light is polarized before illuminating the target 103. After the light is reflected from the target 103, the reflected light can then be polarized in a direction orthogonal to the first polarization to measure the backscattered light while minimizing the amount of incident light read off the surface of the target 103.

[0080] In some cases, it is also desirable to image tissue at a specific depth. For example, tissue imaging at a specific depth may be used in assessing a specific wound at a particular depth, identifying and / or recognizing the presence of a cancerous tumor, determining the stage of a tumor or cancer progression, or any of the other therapeutic applications mentioned in this disclosure. Selective imaging of tissue at a specific depth may be performed using certain polarization techniques well known in the art, based on optical properties and / or mean free path length.

[0081] In some alternative approaches, other techniques for controlling imaging depth can be used. For example, the optical scattering properties of tissue change with temperature, while the depth of light transmission in the skin increases with cooling. Thus, imaging depth can be controlled by controlling the temperature of the tissue region being imaged. Furthermore, imaging depth can be controlled, for example, by modulating (or flashing) the light source with pulses of different frequencies. Pulsed light penetrates deeper into the skin than non-pulsed light: the wider the pulse, the deeper the light penetrates. As another example, imaging depth can also be altered by adjusting the light intensity; stronger light penetrates deeper than weaker light.

[0082] Further as Figure 1AAs shown, the image acquisition device 102 is configured to receive reflected light from tissue 103. The image acquisition device 102 can detect light from an illuminated area, a sub-region of the illuminated area, or a non-illuminated area. Further, as described below, the field of view of the image acquisition device 102 can include the entire body surface of the subject facing the detector 100. When the entire subject facing the detector is illuminated, the entire subject facing the detector is within the field of view of the image acquisition device, improving the speed and ease of classification. The image acquisition device 102 can be a two-dimensional charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) image acquisition device with suitable optical characteristics for imaging all or part of the illuminated tissue 103.

[0083] In some alternatives, module 112 may be a controller, classifier, and processor that can be connected to detector 100. Module 112 controls the detector, which may include setting parameters such as the detector's physical location, light intensity, resolution, color filter, or any parameters of the camera and / or light source described in this disclosure. Module 112 also receives and processes data acquired by the detector, as described later in this disclosure.

[0084] In some alternatives, module 112 may also be connected to module 114, where module 114 is a display and user interface (“UI”). The display and UI shows information and / or data to the user, which in some alternatives includes the presence of tissue conditions, the severity of the tissue conditions, and / or additional information about the subject, including any information mentioned herein. Module 114 receives user input, which in some alternatives includes patient information such as age, weight, height, sex, race, skin color or complexion, and / or blood pressure. Module 114 may also receive user input such as calibration information, user-selected scan sites, user-selected tissue conditions, and / or additional information for diagnosis, including any information mentioned herein. In some alternatives, some or any of the foregoing user input may be automatically sent to module 112 without requiring user login information to use module 114.

[0085] like Figure 1BAs shown, in some alternatives, detector 100 can move in any direction or a combination of directions, such as up, down, left, right, diagonally up to the right, diagonally up to the left, diagonally down to the right, diagonally down to the left, etc., or any combination of these directions. In some alternatives, the detector can also move in a direction orthogonal to the subject, wherein the detector moves closer to or further away from the subject. For example, the detector can be connected to a track or articulated arm, wherein the position is manually controlled or automatically controlled by controller 112, or a combination of both. In some alternatives, the light source or image acquisition device can be fixed; in other alternatives, the light source or image acquisition device can move independently. Some alternatives connect the image acquisition device to a motor to automate the movement of the image acquisition device, thereby allowing the camera to image different parts of the subject. The camera can also be connected to a rail, track, guide rail, and / or a driveable arm. While the image acquisition device moves, the light source can illuminate the entire tissue area 103, or, during the scanning process, the light source can be controlled to illuminate only the desired tissue portion that the camera is imaging.

[0086] exist Figure 1A In the illustrated alternatives, the subject stands in an upright position against the background 110 when acquiring images or partial images of the subject (e.g., the subject's entire body or desired tissue location). In some alternatives, the background 110 is a support structure on which the subject lies or leans against in a horizontal or inclined direction when acquiring images. Measuring plates 106 and 108 can be configured to weigh the subject when acquiring images. Additionally or alternatively, the measuring plates may be equipped with other biological readers for measuring heart rate, temperature, body composition, body mass index, body shape, blood pressure, and other physiological data.

[0087] Figure 2 An example UI 200 displayed on a monitor / UI 114 for acquiring images using the device is shown. In this alternative, the user interface displays the field of view of the image acquisition device when the light source is illuminating the tissue 103. In some alternatives, the user can position the field of view of the image acquisition device 102 to include the entire subject 202. The user can also use the zoom component 208 to adjust the image acquisition device 102 so that the subject almost fills the field of view. In some alternatives, the user can also use the user interface to obtain other information about the subject 202. For example, the user can select positions 204 and 206 to measure the subject's height. In some cases, the user can use the user interface to instruct the image acquisition device to acquire an image of the subject, such as by pressing the image acquisition button 210.

[0088] When acquiring images of a subject for tissue classification, the light source (with relevant filters, if present) and image acquisition device are controlled to acquire multiple images of the subject, as well as separated images associated with different spectral bands of reflected light and / or time-separated images. Images acquired in different spectral bands can be processed according to image processing techniques for spectral domain data (e.g., MSI techniques) to classify tissue regions, and time-separated images can be processed according to image processing techniques for temporal domain data (e.g., PPG techniques) to classify tissues. In some alternatives, both types of image sets are acquired, and these results are fused for more accurate classification, as further explained below.

[0089] For amputees, image acquisition can be performed in multiple orientations, such as front-facing, back-facing, left-facing, and right-facing. The patient stands against the background 110 in these different orientations, or, if the background 110 is a horizontally oriented support structure, the patient may lie on the background 110 in different orientations. Data from the acquired images is then used to classify different areas of the subject's skin as potentially healable after amputation, and may also classify specific tissue types within potential amputation areas.

[0090] After acquiring images from different orientations, the controller / classifier / processor 112 can process the image data from each subject's orientation. When the background 110 is a color different from that of skin tissue, the controller / classifier / processor can separate the subject from the background, classifying each pixel in each acquired image as either background or subject. As an alternative, a UI can be used to display the initial image (e.g., as shown in the image). Figure 2 The subject's outline is marked (e.g., using a stylus or mouse cursor on a touchscreen) to distinguish the background from the subject. Once pixels of the image relevant to the subject are identified, MSI and / or PPG technologies can be used to analyze these pixels to classify the subject's skin regions based on microvascular status.

[0091] The focus has now shifted to specific devices and methods for irradiating tissue, acquiring images, and analyzing image data. It is understandable that in some cases, PPG alone cannot fully classify tissue because it only performs volumetric measurements. Furthermore, multispectral imaging (MSI) has been used to identify differences in skin tissue, but this technique cannot provide complete tissue classification. Several challenges often arise with existing MSI techniques: variations due to skin type, differences in skin across different body regions, and feasible wound pretreatment. MSI alone may also fail to provide a comprehensive assessment of skin condition because it only measures the appearance or composition of the skin, without measuring dynamic variables important for skin classification, such as the availability of nutrients and oxygen to the tissue.

[0092] Some of the alternative approaches described in this article combine MSI and PPG to improve the speed, reliability, and accuracy of skin classification. For example, these alternative approaches can use image data to measure the effects of blood, water, collagen, melanin, and other markers, thus providing a more accurate picture of skin structure and normal functioning relative to diseased or traumatized skin, as in a normal state. Additionally, these alternative approaches detect changes in light reflected from the skin over time, allowing for the acquisition of important physiological information that enables clinicians to rapidly assess tissue viability and characteristics, such as blood perfusion and oxygenation at tissue sites.

[0093] Figure 3 A system is shown that can (but is not required to) be used in some alternative configurations as detector 100, controller / classifier / processor 112, and display / UI 114. The following describes the integration with MSI and PPG technologies. Figure 3 The system can also be used to analyze and classify smaller tissue areas with higher accuracy than previous techniques, and is not necessarily used only in conjunction with the whole-body analysis systems and methods described above.

[0094] exist Figure 3 In the system, detector 408 includes one or more light sources and one or more high-resolution multispectral cameras, which record multiple images of a target tissue region 409 while maintaining temporal, spectral, and spatial resolution to perform high-precision tissue classification. Detector 408 may include multiple cameras and imagers, prisms, beam splitters, photodetectors and filters, and light sources with multiple spectral bands. The cameras can measure the scattering, absorption, reflection, transmission, and / or fluorescence of light of different wavelengths from the tissue region over time. The system also includes a display / UI 414 and a controller / classifier / processor 412, which controls the operation of detector 408, receives input from the user, controls the display output, and performs analysis and classification of image pixels.

[0095] Data set 410 is an example output of detector 408, including data on reflected light at different wavelengths and times regarding the imaging spatial location. Data subset 404 shows an example of data on light at different wavelengths regarding the imaging spatial location. Data subset 404 may include multiple images of a tissue region, each measuring light reflected from the tissue region at a different selected frequency band. The multiple images of data subset 404 may be acquired simultaneously or substantially simultaneously, where substantially simultaneously means within one second of each other. Data subset 402 shows an example of data on light reflected from a tissue region at different times regarding the imaging spatial location. Data subset 402 includes multiple images acquired at different times within a time period exceeding one second, typically exceeding two seconds. The multiple images of data subset 402 may be acquired in a single selected frequency band. In some cases, the multiple images of data subset 404 may be acquired over a time period exceeding one second, and the multiple images of data subset 402 may be acquired at multiple frequency bands. However, the combined dataset including subsets 404 and 402 includes images acquired corresponding to different times and different frequency bands.

[0096] To collect images of data subset 404, in some alternatives, one or more cameras are connected to filter wheels of multiple filters with different passbands. As one or more cameras acquire images of a tissue region of the subject, the filter wheels rotate, allowing the one or more cameras to record the subject in different spectral bands by acquiring images synchronized with the filter position of the filter wheel during rotation. In this way, the cameras receive light reflected at different frequency bands at each pixel in the tissue region. Furthermore, in many cases, the filters allow the apparatus described herein to analyze light in spectra that are not visible to the naked eye. In many cases, the amount of reflected and / or absorbed light from these different spectra can provide clues about the chemical and physical composition of the subject's tissue or specific regions of the tissue. In some cases, the data obtained using the filters forms a three-dimensional data array, where the data array has one spectral dimension and two spatial dimensions. Each pixel in the two spatial dimensions can be characterized by the spectral features defined by the intensity of reflected light in each acquired spectral band. Since different components differentially scatter, absorb, reflect, transmit light of different frequencies and / or emit fluorescence of different frequencies, the intensity of light at different wavelengths provides information about the target component. By measuring these different wavelengths of light, detector 408 captures the component information at each spatial location corresponding to each image pixel.

[0097] In some alternatives, to acquire images for dataset 404 in a multispectral band, one or more cameras include a hyperspectral line-scan imager. The hyperspectral line-scanner has continuous spectral bands, rather than separate bands for individual filters in a filter wheel. The filters of the hyperspectral line-scanner can be integrated with a CMOS image sensor. In some cases, the filters are monolithically integrated optical interference filters, where multiple filters are arranged in a stepped line. In some cases, the filters are wedge-shaped and / or stepped. In some cases, there are tens to hundreds of spectral bands corresponding to wavelengths between 400 nm and 1100 nm, for example, wavelengths of 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, 900 nm, 1000 nm, or 1100 nm, or any wavelength range defined between any two of the aforementioned wavelengths. The imager uses the individual filters to line-scan the tissue, sensing the light reflected from the tissue after passing through each of these filters.

[0098] In other alternatives, there are other filtering systems that can filter light in different spectral bands. For example, some alternatives use Fabry-Perot filters. Other filter arrangements include, for example, arranging the filters in a tile structure, or arranging the filters directly on the image sensor (CMOS, CCD, etc.) in patterns such as Bayer arrays or multi-sensor arrays.

[0099] In any case, the passband of the filter is selected based on the type of information sought. For example, using wavelengths between 400 nm and 1100 nm (such as 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, 900 nm, 1000 nm, or 1100 nm, or any wavelength range defined between any two of the aforementioned wavelengths), the amputation site can be imaged to capture the contributions of blood, water, collagen, and melanin from the amputation site and surrounding tissues. In some embodiments, absorption spectra at approximately or at wavelengths of 515 nm, 750 nm, and / or 972 nm can be used, while in other embodiments, absorption spectra at approximately or at wavelengths of 542 nm, 669 nm, and / or 960 nm can be used to distinguish between tissue classifications.

[0100] Healthy skin can include areas of skin without damage associated with vascular disease or microcirculatory problems. Congestion corresponds to areas of high perfusion, typically tissue expected to heal without treatment. Graftable or tissue flap classifications can correspond to skin with a threshold level greater than the microvascular activity threshold. Vascular disease categories can correspond to areas of localized ischemia in tissue with reduced perfusion but potentially salvageable tissue. Necrosis classifications can correspond to areas of irreversible tissue loss where amputation may be expected.

[0101] The optional approach disclosed herein is used to measure light reflected from a tissue sample at various wavelengths (e.g., 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, 900 nm, 1000 nm, or 1100 nm, or any wavelength range defined between any two of the aforementioned wavelengths) in order to determine which set of wavelengths provides a greater amount of variation among the light reflected from different types of tissue. This variation can be used to effectively distinguish tissue categories, at least classifying them as healthy tissue, congestion, suitable flaps, vascular lesions (e.g., small vessel lesions), and necrosis. Sometimes, the optimal set of wavelengths can be identified as including the most correlated wavelength with the least redundancy. In this regard, maximum correlation is sometimes derived when a wavelength can effectively distinguish a particular tissue category from other categories. Minimum redundancy is sometimes derived by including only one wavelength among multiple wavelengths that measure the same information. After classifying tissue samples using wavelength sets, practitioners compare the classifications to accurately evaluate the tissue samples.

[0102] Data was split across different experiments to test classification accuracy. In the first set of experiments, wavelengths of 475, 515, 532, 560, 578, 860, 601, and 940 nm were measured. In the second set of experiments, wavelengths of 420, 542, 581, 726, 800, 860, 972, and 1064 nm were measured. In the third set of experiments, wavelengths of 420, 542, 581, 601, 726, 800, 972, and 860 nm were measured. Furthermore, in the fourth set of experiments, wavelengths of 620, 669, 680, 780, 820, 839, 900, and 860 nm were measured.

[0103] To classify tissues with 83% accuracy, wavelengths providing optimal variability for tissue classification based on the first and second experimental groups were used. These wavelengths (in order of relative weight) were 726 nm, 420 nm, 515 nm, 560 nm, 800 nm, 1064 nm, 972 nm, and 581 nm. Similarly, to classify tissues with 74% accuracy, wavelengths providing optimal variability for tissue classification based on the third and fourth experimental groups were used. These wavelengths (in order of relative weight) were 581 nm, 420 nm, 620 nm, 860 nm, 601 nm, 680 nm, 669 nm, and 972 nm. Furthermore, it is noteworthy that the 860 nm wavelength is particularly effective for MSI and PPG analyses and is therefore used in the combined apparatus. These experimental groups demonstrate that wavelengths in the 400 nm to 1100 nm range (e.g., 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, 900 nm, 1000 nm, or 1100 nm, or any wavelength range defined between any two of the aforementioned wavelengths) can be used for effective tissue classification. As mentioned above, other wavelength groups can also be effective. For example, the effective wavelength groups in the experiments minimize redundancy. Thus, other wavelengths can also be used to effectively classify certain aspects of tissue. Furthermore, using the above experiments, other wavelengths can be identified for effective classification of necrotic tissue and / or any other tissue conditions described in this disclosure.

[0104] In summary, the experiments described above demonstrate that wavelengths in the range of 400 nm to 900 nm (including 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, or 900 nm, or any wavelength range between any two of the aforementioned wavelengths) are particularly effective for imaging amputation sites. More specifically, within this range, a set of wavelengths can be constructed for imaging amputation sites, wherein: at least one (1) wavelength is less than 500 nm; and at least two (2) wavelengths are between 500 and 650 nm; and at least three (3) wavelengths are between 700 and 900 nm. This set of wavelengths is effective for imaging amputation sites and for classifying the imaged amputation sites into several categories.

[0105] Furthermore, based on the above experiments, the following order table lists the various test wavelengths, in order of their significance in the classification:

[0106] Table 1

[0107]

[0108] To collect images of data subset 402, one or more cameras are also configured to acquire a selected number of images, with time intervals short enough to measure temporal changes in reflected light intensity due to movement of tissue areas corresponding to physiological events or conditions of the patient. In some cases, data obtained from multiple temporally separated images form a three-dimensional data array, which has one temporal dimension and two spatial dimensions. Individual pixels in the three-dimensional array can be distinguished by temporal variations in reflected light intensity. This temporal signal has different energies at different frequency components relating to blood pressure, heart rate, vascular resistance, nerve stimulation, cardiovascular health, respiration, body temperature, and / or blood volume. In some alternatives, filters can also be used to filter out noise. For example, an 860 nm bandpass filter can be used to filter out light waves corresponding to the dominant wavelength spectrum of indoor ambient light, so that the acquired images correspond to reflected light originating from the light source in detector 408. This can reduce and / or prevent interference from ambient light fluctuations, such as the 60 Hz fluctuations in ambient light caused by AC power line frequencies.

[0109] Figure 4 An example of a dynamic library is illustrated. In the figure, an example imaging device 1000 is connected to an example cloud 1002. The example imaging device 1000 can be the device described herein, or it can be any other computer or user device similarly connected to the dynamic library. In some cases, the cloud 1002 may include a program execution server (PES) comprising multiple data centers, each data center including one or more physical computing systems configured to execute one or more virtual desktop instances, each virtual desktop instance being associated with a computing environment including an operating system configured to execute one or more applications, each virtual desktop instance being accessible via a network by computing devices of users of the PES. The cloud may also include other methods for synchronizing computation and storage.

[0110] Data path 1004 illustrates a bidirectional connection between imaging device 1000 and cloud 1002. Cloud 1002 itself has a processing unit 1006, which is where cloud 1002 receives signals, processes data, performs classification, and generates metadata. This processing unit indicates whether the dynamic library is synchronized with one or more computing devices.

[0111] In some alternatives, data analysis and classification are performed in the cloud. This analysis may involve collecting data from sample signals for comparison with the acquired signals. Such sampling can be used to generate or refine one or more machine learning models for classifying tissue regions in the acquired signals, for example, using the machine learning techniques described herein. In other alternatives, the processing unit may be located on the imaging device 1000 to perform local processing at the data collection site. Other alternatives may split processing requirements between the cloud and the imaging device 1000.

[0112] In addition to collecting and analyzing data from the dynamic library, the processing component may also include general error data and calculations. Errors can be calculated locally and aggregated in the cloud, and / or calculated in the cloud. In some cases, error thresholds can be established for specific classification models. Thresholds consider Type I and Type II errors (e.g., false positives and false negatives) and criteria for clinical reliability.

[0113] The processing unit 1006 can also analyze the data. The cloud 1002 also has a data unit 1008, which includes information about the dynamic library itself and also receives updates. The data unit 1008 and the processing unit 1006 are connected to each other.

[0114] Other sources or repositories are also connected to the cloud. In this example, entity 1012 is also connected to cloud 1002. Entity 1012 is a system 1000 connected to cloud 1002, which can provide updated and / or updated tissue classification models to any device or system to improve system functionality. Through learning and experience, the methods at each stage can be updated to reduce the overall error. Entity 1012 can simultaneously and rapidly evaluate changes to multiple classification models and provide system improvements. It can also update new datasets and models for new clinical applications. Additionally, for example, entity 1012 can update system 1000 or any device or system connected to cloud 1002 to acquire and analyze data for new medical uses, such as analyzing frostbite. This approach expands functionality and allows the system to adapt to different situations due to improvements in scientific knowledge.

[0115] Furthermore, various aspects of the alternatives described in this disclosure have been demonstrated as experimental subjects in tissue models and animal models. These experiments show that the alternatives of this disclosure are effective in treating at least the amputation site. For illustrative purposes, non-limiting examples are given below. The examples provide further details of the experiments conducted.

[0116] Figure 5A , Figure 5B and Figure 6 An example fiber optic system that can be used to obtain the image data described herein is shown. Figure 5AAs shown, the fiber optic detector 7000 may include multiple light-emitting fibers 7005 surrounding a light-collecting fiber 7010. Each light-emitting fiber 7005 can illuminate one of a plurality of overlapping regions 7015, and the light emitted from the light-emitting fiber 7005 can be reflected from the subject's tissue and collected by the light-collecting fiber 7010 from a uniformly irradiated area 7020. In some embodiments, the light-emitting fibers may be controlled to sequentially emit one of 1000 different wavelengths between 400 nm and 1100 nm, and the signal received by the light-collecting fiber 7010 can be used to generate an image of the irradiated tissue at the emitted wavelength.

[0117] In some implementations, the detector 7000 may be a fiber optic spectrophotometer equipped with a coaxial light source for reflection and backscatter measurements. The detector may be configured to block ambient light using a cover (not shown), allowing imaging of tissue using only the emitted wavelength, which results in more accurate classification compared to tissue illuminated by both ambient light and selectively emitted wavelengths.

[0118] like Figure 6 As shown, detector 7100 may include a first optical cable 7105 having a light-emitting and detection end 7110. The light-emitting and detection end 7110 may include multiple light-emitting fibers 7115 surrounding a light-collecting fiber 7125. The light-emitting fibers 7115 may pass through the first optical cable 7105 and split into a second optical cable 7140, the cross-section 7145 of which is shown includes the light-emitting fibers 7115 surrounding a core 7120. This multi-fiber second optical cable 7140 may be connected to a light source, supplying light of a desired wavelength to the light-emitting and detection end 7110 of the first optical cable 7105 via the second optical cable 7140. The light-detecting fiber 7125 may pass through the first optical cable 7105 and split into a third optical cable 7130, the cross-section 7135 of which is shown includes only the light-detecting fiber 7125. The single-fiber third optical cable 7130 can supply signals from the optical detection fiber 7125 to an image sensor (e.g., a CMOS or CCD image sensor) configured to capture image data or to a spectrometer. The fiber core size can range from 200 μm to 600 μm, such as 200 μm, 250 μm, 300 μm, 350 μm, 400 μm, 450 μm, 500 μm, 550 μm, or 600 μm, or within the range defined by any two of the aforementioned wavelengths.

[0119] exist Figure 7 Example components of an implementation of a PPG imaging system are shown in the figure. Figure 8 The diagram illustrates example components of an implementation of an MSI system. In some alternatives, these can be physically separate imaging systems, while in others they can be integrated into a single monolithic imaging system.

[0120] PPG imaging systems can include a 10-bit monochrome CMOS camera (e.g., Nocturn XL, Photonis USA), which offers low dark noise and high dynamic range. A 10-bit ADC resolution can provide a signal-to-noise ratio of 60 dB. The imager's resolution can be set to 1280. 1040 (5:4 aspect ratio). This camera can be mounted vertically, facing downwards at the target surface. Typically 20. A 16cm field of view (FOV) can be controlled for inter-system comparisons. The camera's exposure time can be calibrated using a 95% reflectivity reference standard (e.g., Spectralon SG3151; LabSphere Inc.; North Sutton, NH). For tissue illumination, four monochromatic high-power LED emitters (SFH 4740, OSRAM) can be used at 2... 2. The array is mounted in the same plane as the sensor. The LED emitter array and the camera can be arranged at a distance of 15 cm from the target surface. In some alternatives, LED emitters are preferred because they can uniformly illuminate the tissue within the camera's field of view (FOV) (i.e., spatial intensity variation less than 15%). The camera's FOV can be controlled by an optical lens and can be slightly narrower than the illumination area.

[0121] Noise introduced into the PPG signal by the subject's movement during respiration can make initial analysis of PPG imaging difficult. The disclosed technique, using a signal processing method known as envelope extraction, can reduce the influence of respiratory motion. For each pixel in the image, the signal is flattened using a low-pass filter to extract the envelope of the noise signal. The noise signal can then be segmented by its envelope to remove motion spikes of interest. The remaining clear signal reflects the information that can then be processed into a PPG image. PPG image data can be generated based on the number of frames captured over a time interval of at least 10 seconds, preferably between 5 and 30 seconds.

[0122] A PPG system may include three functional modules: illumination; a sensor (CMOS camera); and an imaging target. The illumination and sensing modules may be positioned on the same side relative to the target, for example, in reflective mode. A light beam incident on the object is scattered into the target, and the backscattered light signal is then captured by the camera. The imaging target, embedded in an opaque medium, changes over time (e.g., changes in blood vessel volume due to pulsating blood flow), causing the backscattered light to have modulation in intensity. In some embodiments, differences in signal noise from spatially different tissue regions can be used to classify tissue regions into healthy tissue, vascularized tissue, necrotic tissue, or other tissue classifications.

[0123] like Figure 8 As shown, in some alternatives, multispectral images can be acquired using a filter wheel camera (e.g., SpectroCam, Pixelteq, Largo, FL) via a staring method. This camera is equipped with eight unique optical bandpass filters with wavelengths between 400 nm and 1100 nm. Wavelength filters with the following peak transmission values ​​can be used: 581, 420, 620, 860, 601, 680, 669, and / or 972 nm (filter widths can be ±10 nm; e.g., Ocean ThinFilms; Largo, FL). Other suitable wavelengths can be used in other alternatives. The system can be calibrated using a 95% square reflectance standard (e.g., Spectralon SG3151; LabSphere Inc.; North Sutton, NH) to compensate for different spectral responses of the imaging sensor. The light source used can be a 250 W halogen tungsten lamp (e.g., LowePro) equipped with a frosted glass diffuser to produce a more uniform illumination surface within the imager's field of view. Figure 8 The system can use telescopic lenses (e.g., Distagon T* 2.8 / 25 ZF-IR; Zeiss Inc.; USA).

[0124] In some implementations, multispectral image data can be acquired using a multispectral image acquisition system designed according to the following parameters: The light source and image capture module can be placed at a distance of 60 cm from the target surface in reflective mode. A tungsten lamp (e.g., ViP Pro-light, Lowel Inc.) can provide a broad spectral projection onto the target surface in DC mode. A frosted glass (e.g., iP-50, Lowel Inc.) can be mounted in front of the tungsten lamp to diffuse the light and improve the uniformity of spatial illumination. Some incident light can penetrate the target surface, while any backscattered light signals can be collected by the image capture module. The image capture module may include a high-performance IR-enhancing optical lens (example model: Distagon T* F-2.8 / 25 mm, Zeiss), an eight-slot filter wheel, and a 12-bit monochrome camera (BM-141GE, JAI Inc.). The optical bandpass filter can be designed and selected to separate single wavelengths of light for the camera. In some implementations, eight bandpass filters can be mounted in the filter wheel: the center wavelength (CWL) and full width at half maximum (FWHM) of the eight filters can be (CWL-FWHM, both in nm): 420-20, 542-10, 581-20, 601-13, 726-41, 800-10, 860-20 and / or 972-10. The wavelength intensity can be normalized using a Reflectance Zenith Lite Panel (e.g., SphereOptics GmbH), and the maximum pixel count can be 4098 (12-bit). At these wavelengths, where accurate tissue differences for effective classification can be obtained, the eight implemented wavelengths can be selected based on the known absorption characteristics of skin tissue. As the filter wheel rotates, the camera sequentially captures single-wavelength images through each of the eight filters. The images can be saved on a computer in an uncompressed format. Calculations and statistics are performed using MATLAB® software (version 2014b) or via a suitable signal processor.

[0125] In some implementations, the light source can be any broad-spectrum illumination source, or any illumination source that matches the desired wavelength of light required for data analysis.

[0126] Overview of the wavelength range for example implementations of MSI

[0127] In some implementations, the multispectral images described herein can be captured via an optical fiber cable having an emitter and a photodetector at the same end of the detector. Compared to previously used camera systems that used approximately eight independent wavelength options, the emitter is capable of emitting approximately 1000 different wavelengths of light between 400 nm and 1100 nm to provide a smooth range of subject illumination across different wavelengths. In some implementations, the subject is sequentially irradiated with wavelengths within a defined range, such as wavelengths between 400 nm and 500 nm (e.g., 400 nm, 425 nm, 450 nm, 475 nm, or 500 nm), and wavelengths between 720 nm and 1000 nm (e.g., 720 nm, 750 nm, 775 nm, 800 nm, 825 nm, 850 nm, 875 nm, 900 nm, 925 nm, 950 nm, 975 nm, or 1000 nm), or within any wavelength range defined by any two of the aforementioned wavelengths, and one or more images of the subject are captured at each wavelength.

[0128] The visible and near-infrared ranges can differ significantly between wound and healthy tissue, and the disclosed classification techniques contain sufficient information to distinguish clinically important tissue types. In some implementations, the highest reflectance values ​​for various tissue types may occur at approximately 625 nm. Secondary peaks may be present at 525 nm and 575 nm.

[0129] In some implementations, the most distinct values ​​between damaged and healthy skin may appear between 475 nm and 525 nm; between 450 nm and 500 nm; and between 700 nm and 925 nm. Therefore, to classify wound tissue compared to healthy skin, the multispectral imaging system described herein can use wavelengths in both low-end and high-end ranges rather than continuous ranges, for example, between 450 nm and 525 nm and between 700 nm and 925 nm, or within any range defined by any two of the aforementioned wavelengths.

[0130] In some implementations, the most significant differences between necrotic tissue or vascular lesions and healthy skin may occur between: 400 nm and 450 nm; 400 nm and 450 nm; 525 nm and 580 nm; and 610 nm or 1050 nm. Therefore, to classify wound tissue compared to healthy skin, the multispectral imaging system described herein can use wavelengths in both low-end and high-end ranges rather than a continuous range, for example, between 400 nm and 450 nm, or between 525 nm and 580 nm, or between 610 nm and 1050 nm, or within any range defined by any two of the aforementioned wavelengths.

[0131] In some implementations, multispectral imaging systems for tissue classification may use wavelengths in a low-end and high-end range rather than a continuous range, for example, between 400 nm and 500 nm, or between 720 nm and 1000 nm, or within any wavelength range defined by any two of the aforementioned wavelengths. For example, such as Figure 5A , Figure 5B and Figure 6 The detector shown can be configured to emit multiple wavelengths, which are between 400 nm and 500 nm or between 720 nm and 1000 nm, or within a range defined by any two of the aforementioned wavelengths. In some embodiments, such a wavelength range can be suitable for tissue classification of the entire range of skin pigmentation, similar to the skin pigmentation of patients from which training data is collected, and different range groups deviating from the disclosed range can be used for tissue classification of lighter or darker skin. Different range groups can be identified based on the interval between the spectrum received from healthy tissue and the spectrum received from tissue of interest (e.g., necrotic tissue or vascular lesion tissue).

[0132] In one embodiment, the multispectral image set for tissue classification of amputation sites may include eight images captured at different wavelengths. A set of wavelengths may include (listed in terms of center wavelength ± full width at half maximum): 420 nm ± 20 nm, 525 nm ± 35 nm, 581 nm ± 20 nm, 620 nm ± 20 nm, 660 nm ± 20 nm, 726 nm ± 41 nm, 820 nm ± 20 nm, or 855 nm ± 30 nm, or within a range defined by any two of the aforementioned wavelengths.

[0133] An overview of example PPG and / or MSI image processing options

[0134] Reference Figure 9 Additional details regarding the advantageous image acquisition and signal processing procedures are provided. Figure 9The process that can be performed by the imaging apparatus described herein is shown. Figure 9 Example flowchart 600 illustrates the tissue classification process used in some alternative approaches. Boxes 602 and 603 illustrate some alternative approaches that use, for example, detector 408 to acquire multispectral images and multiple temporally separated images (e.g., video). For temporally separated images, such as data subset 402, a relatively long exposure time is considered desirable to obtain a signal with less overall noise and a higher signal-to-noise ratio. In some cases, an acquisition time of twenty-seven (27) seconds is used, which is longer than the seven (7) second acquisition time of a conventional PPG imaging procedure. Therefore, in some alternatives, the desired capture time is at least 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, or 60 seconds, or any number between these, or within a range defined by any two of the aforementioned numbers. Within these capture times, the number of frames per second captured by the imager can be set. In some cases, 30 frames per second (30 fps) or 60 frames per second (60 fps) may be effective when imaging tissue. For 30 fps over 27 seconds, the imager acquires approximately 810 images. For 60 fps over 27 seconds, the imager acquires approximately 1620 images. In some alternatives, the number of images acquired can be varied depending on the resolution of the required data (e.g., to capture a human heartbeat). For example, for CMOS cameras, 20 fps to 120 fps can be used. This includes sampling rates of 20 fps, 30 fps, 40 fps, 50 fps, 60 fps, 70 fps, 80 fps, 90 fps, 100 fps, 110 fps, or 120 fps, or a range of sampling rates defined by any two of the aforementioned sampling rates.

[0135] Furthermore, in some alternatives, the arrangement of the light source is important due to the location of the irradiation point, which is the position of high-intensity light that saturates the signal and shields the pulse waveform. In some alternatives, this problem is addressed by using diffusers and other front-end hardware techniques. However, in cases where the irradiation point cannot be eliminated by front-end techniques, signal processing is used to eliminate the irradiation point in some alternatives. In fact, to form reliable images for histopathology, it is desirable to preserve and display the signal while removing noise. This processing involves removing noise associated with the irradiation point and other irrelevant signals.

[0136] At box 604, the time-resolved image sequence (e.g., data subset 402) is sent to controller / classifier / processor 412 for processing, whereby the controller / classifier / processor uses PPG analysis to calculate blood perfusion in the tissue region. This processing may include amplification, linearization, signal averaging, correlation, and / or one or more filters (e.g., bandpass, high-pass, or low-pass) to remove noise, separate portions of the signal of interest, and improve the signal-to-noise ratio. The choice of filters is important because too much filtering removes necessary data, while too little filtering makes the signal difficult to analyze. Cross-correlation and autocorrelation can also be used to remove noise. In some alternatives, the sampled signal can also be used for noise removal, which will be explained later. The signal is then converted to the frequency domain. For example, in some alternatives, a Fast Fourier Transform (FFT) is used. After performing the FFT, the signal can be analyzed by frequency. The temporal variation of reflected light intensity at each pixel across multiple time-separated image segments results in different signal energies at different frequencies. These frequencies and their corresponding physiological events indicate the presence and intensity of these physiological events at tissue locations imaged by pixels. For example, the signal strength of a pixel in the band near 1.0 Hz, close to the heart rate of a resting person, can be used to assess blood flow to and near tissue at the pixel location in an image.

[0137] In some alternatives, relevant signals can be identified by examining local maxima. For example, heart rate can be derived by examining the signal energy at the maximum peak in a nearby frequency band and assuming the peak is caused by blood pressure changes due to heartbeats. However, this method cannot identify noise signals with peaks higher than those from the actual heart rate. In this case, other alternatives employ signal processing such as computer learning and training based on examples or a reference database of noise signals, white noise signals, and other example signals. The computer analyzes examples of relevant signals and noise to learn to identify signals from noise. For example, in identifying signals related to blood flow, signals with the same frequency components as the heartbeat can be relevant. Computer learning uses example heart rate signals or a reference database of heart rate signals to identify heart rate from noise. Computer learning can also utilize these reference points and databases to analyze white noise, erroneous heart rate signals, and noise signals with peaks higher than the heart rate signal. Computer learning can identify signals based on features such as frequency, amplitude, signal-to-noise ratio, zero-crossing points, typical shape, or any other characteristics of the signal.

[0138] In some cases, other comparisons are used to identify the signal. For example, in some alternatives, a summary of manually selected clinical staging signals is generated. The manually selected clinical staging signals are then compared with the measured signals to classify the measured signals as either signals of interest or noise. Another technological advancement achieved is the elimination of edge artifacts. In some alternatives, there is blurred noise near image edges, and in some cases, the region of interest is not as clearly defined as expected. When edge artifacts are eliminated, the signal intensity of the region of interest is higher. In some alternatives, edge elimination is achieved through image processing, including averaging, dilation and erosion, as well as edge detection and enhancement.

[0139] Another technological advancement is the automatic removal of motion artifacts. Motion artifacts include movements related to patient breathing, patient movement, or any regular vibrations in or near the camera that can distort the image. To remove these motion artifacts, the signal is processed using a “window” that identifies larger and noisier temporal regions than nearby areas and identifies these regions as “motion.” These regions are then clipped from the temporal domain, resulting in a corrected signal free of motion artifacts. Other filters and selection methods can also be used to remove noise and other unwanted signal components. After this processing, the signal energy calculated at the desired frequency (e.g., typically around 1 Hz) can be categorized for tissue regions (e.g., for each two-dimensional pixel location) to define the blood perfusion at that pixel location.

[0140] Roughly concurrent with the execution of blocks 602 and 604, some alternatives also execute blocks 603 and 605. Block 603 acquires the multispectral data cube (e.g., Figure 3 The data cube contains images of a subset of data (404). The data cube includes a 2D image of each MSI spectral band. In box 605, these alternatives then apply MSI analysis to the data, and in box 614, the system assigns categories of tissue components to each tissue region (e.g., for each two-dimensional pixel location).

[0141] Next, box 616 combines blood perfusion and MSI data from boxes 603 and 604 to generate tissue classification based on MSI and PPG data.

[0142] For example, for illustrative purposes, eight bandpass filters can be used to generate eight reflectance values ​​for each imaging pixel, each corresponding to a selected spectral band. Furthermore, using filters with center wavelengths of infrared or near-infrared wavelengths (e.g., close to 840 nm–880 nm, or around 840 nm–880 nm, or within the range of 840 nm–880 nm, including wavelengths of 840 nm, 850 nm, 860 nm, 870 nm, or 880 nm, or any wavelength defined by any two of these wavelengths), 810 images can be acquired in 27 seconds at the adopted 30 frames per second. As described above, these 810 images can be analyzed in the frequency domain to generate PPG data characterizing blood perfusion at each imaging spatial location, generating perfusion values ​​for each imaging pixel. Thus, each pixel in the imaging tissue region has a measurement value corresponding to the measurement acquired through each of the eight bandpass filters, and a value corresponding to local blood flow. There are a total of nine measurements at each pixel. Using these nine measurements, the pixels can be classified (e.g., categorized) into different categories. As will be understood by those skilled in the art, each pixel can be divided by any number of measurements (e.g., 2, 10, 20, or any number of measurements between or greater than any of these measurements).

[0143] Different partitioning / classification methods can be used. Typically, a classifier is trained using a "training" dataset, where the parameters and appropriate classifications measured in the "training" dataset are known. The trained classifier is then tested against a "test" dataset, where the parameters and appropriate classifications measured in the "test" dataset are also known but not used for training. The quality of the classifier can be evaluated by how successfully it classifies the test dataset. In some alternatives, a predetermined number of categories can be used to classify pixels into predetermined categories related to the amputation site described in this paper.

[0144] In other alternatives, the number of categories is unknown; processors such as processor 112 generate classifications based on pixel grouping and features relative to each other. For example, using these measurements relative to nearby values, the processor identifies tissue regions with less blood flow and lower standard pixel intensity at certain wavelengths as associated with necrotic tissue.

[0145] In some alternatives, pixels are assigned based on preset ranges of values ​​for each category. For example, certain ranges of light reflectance values ​​may be associated with healthy skin. When data falls within these ranges, the tissue is identified as healthy skin. These preset ranges may be stored in the memory of system 412, input by the user, or determined automatically by the system through learning or an adaptive classifier. In some alternatives, classification is defined by information transmitted to the system from an external resource, such as a data uplink, the cloud (as described above in this disclosure), or any data resource. In other alternatives, the ranges of values ​​for each category are unknown, and the processor adapts the classification based on comparisons of the measurements of each pixel.

[0146] In some alternative approaches, suitable classifiers can be used to group pixels with common features and identify these groups. For example, by looking at image segmentation such as minimum cut, graph theory can be used to divide pixels into several classes. Other partitioning methods can also be used, such as thresholding, clustering (e.g., k-means, hierarchical clustering, and fuzzy clustering), watersheds, edge detection, region growing, statistical grouping, shape recognition, morphological image processing, computer training / computer vision, histogram methods, and any partitioning method well-known in the field of grouped data.

[0147] In some alternatives, historical data can be used to further supplement the segmentation. Historical data may include previously acquired patient data and / or data from other patients. In some alternatives, additional data such as skin color, ethnicity, age, weight, sex, and other physiological parameters are considered in the segmentation process. In any case, data may be uploaded, obtained from the cloud, or otherwise entered into the system, including using UI 114. In some alternatives, a dynamic library of patient data is analyzed. Statistical methods, including t-tests, f-tests, z-tests, or any other statistical methods used for comparison, can be used to compare the pre-identified images with the acquired images. Such comparisons may, for example, consider measured pixel intensity, pixel measurements relative to other pixels in the image, and pixel distribution.

[0148] In some alternatives, the dynamic library can be updated with example images of tissue conditions, such as amputation sites, to aid in tissue classification. In other alternatives, images can be assigned and identified by showing what tissue conditions are present and how those conditions are. Ideally, the dynamic library should contain a full range of images from different angles to illustrate variations in the angle, quality, and condition of the imaged skin.

[0149] Return to Figure 9The system can present various data outputs to the user. These data include PPG perfusion images 620 based on PPG data, MSI classification images based on MSI data, white light illumination images based on standard RGB data, and MSI / PPG fusion images 622 showing classifications based on combined MSI and PPG data. For example, the display output could be the combined MSI / PPG fusion classification image 622. In such an image, as described above, the subject's various tissue regions (e.g., pixels) are categorized into amputation site categories such as healthy, congested, necrotic, potential amputation flap, and vascular lesions. Alternatively, data outputs such as percentages in each category can be presented to the user.

[0150] Compared to existing technologies, there are significant advancements in the use of composition and viability data in taxonomic tissues.

[0151] Figure 10 An example of time-resolved PPG signal extraction is shown. Block diagram 1700 illustrates the intensity of image pixels (x, y) extracted sequentially from 800 contingent frames. Block diagram 1701 illustrates the processing method for quantifying the PPG signal.

[0152] In one example, a sequence of 800 images can be acquired at a frame rate of 30 frames per second and stored as an uncompressed TIFF file. PPG signal strength can be calculated pixel-by-pixel. The steps for PPG signal and image processing can be as follows: (1) de-trending, which removes DC drift; (2) downsampling in the time domain to reduce data volume; (3) signal filtering; (4) fast Fourier transform, converting the time-resolved signal into a frequency-resolved signal; (5) extracting spectral power, particularly at frequencies equivalent to heart rate; (6) calculating the ratio of the sum of the heart rate band intensity to the sum of the intensity of slightly higher frequency bands or non-target frequency bands (considered noise) as the signal-to-noise ratio (SNR); (7) outputting the PPG image using a color plot to display the PPG SNR of each pixel. Colors are plotted linearly from the lowest to the highest signal in an image. Signal processing can be performed using MATLAB or a dedicated signal processor.

[0153] Figure 11 A sample block diagram of PPG output preprocessing is shown. A PPG image is generated from 800 frames of a 27-second video from the amputation site. The PPG signal is temporally defined by individual pixels. The purpose of this preprocessing is to obtain some physiological information related to the subject's heart rate, as well as some initial features for the classification step. Figure 11 As shown below, the preprocessing is performed based on this time-domain signal (one for each pixel).

[0154] An initial spatial average is calculated; then, deconvolution is performed, where low-frequency, high-amplitude components are removed (corresponding to artificial ventilation in anesthetized subjects). Detrending of the signal is performed, along with bandpass filtering within the frequency range of the expected heart rate. A Fast Fourier Transform algorithm is applied to the time-domain signal of each pixel to compute the corresponding frequency-domain signal.

[0155] In some implementations, for each pixel, four metrics are obtained from these frequency signal sets: (1) signal-to-noise ratio (SNR), (2) maximum mean, (3) number of standard deviations from the mean, and (4) number of times the signal crosses a threshold level. These four metrics are used to establish the vascular probability in each pixel of the image. The vascular probability indicates how useful the pixel is for providing information about heart rate. For those pixels with a vascular probability > 0.9, the heart rate value corresponding to the maximum value of the frequency signal is stored. The most frequently repeated value is selected as the true heart rate of the subject in the current step. Based on this value, an improved SNR metric is calculated. Finally, depending on the degree of difference from the true heart rate, a mask is defined such that pixels whose heart rate corresponds to the calculated heart rate are set to 1, and the remaining pixels are set to values ​​between 0 and 1. The PPG output metric is the result of multiplying the improved SNR by this mask.

[0156] For the study of each pixel of the image, all six indicators provide physiological information about blood flow at a depth of approximately 0.5 cm below the subject's body surface.

[0157] These selected pixels can be extracted and processed individually to determine the intensity of the PPG signal at the indicator point. A metric used to evaluate PPG signal intensity is the power spectral density (PSD), which is a measurement of the signal energy distribution across frequencies. The PSD at the pulsating frequency provides a clear logarithmic trend, where the received PPG signal intensity increases progressively in intensity value.

[0158] An overview of the possible options for amputation site analysis.

[0159] The lack of sufficiently accurate indicators or tests to assess healing potential, along with the numerous known factors influencing the body's wound healing ability, highlights the need for a multivariate approach to improve assessment for diagnosis. The technology of this disclosure is specifically proposed to address this problem because the disclosed device is designed to process information as multiple independent variables to classify histopathological processes. The disclosed imaging device can use machine learning to combine two optical imaging techniques, photoplethysmography (PPG) and multispectral imaging (MSI), with patient health indicators such as diabetes control or smoking to generate predictive information. Figure 12 ).

[0160] Figure 12A high-level graphical overview of two optical imaging techniques, photoplethysmography (PPG imaging) and multispectral imaging (MSI), according to this disclosure, is shown, which can be combined with patient health indicators to generate predictive information. The disclosed system can utilize any of the imaging techniques described herein to implement these techniques for classifying patient tissues. The combination of MSI and PPG image data maintains the high sensitivity and specificity required to select the appropriate LOA in patients with vascular disease. Both optical imaging methods aim to infer important tissue features, including arterial perfusion and tissue oxygenation. These two measures are critical for LOA selection because wound healing in patients with peripheral artery disease (PAD) is limited by severe lack of arterial perfusion and low tissue oxygenation. The disclosed method can simultaneously assess tissue-level perfusion over a large area of ​​the leg to identify subperfusion areas of the limb. This contrasts with the guesswork involved in using clinical judgment alone, where the observer must assess the appropriate LOA based on patient history, physical examination, and vascular studies, which rarely include a comprehensive assessment of the patient's microcirculation. Furthermore, the disclosed technique also assesses patient health indicators that have a systemic impact on wound healing potential. By combining local assessments of tissue microcirculation with a holistic assessment of systemic factors influencing wound healing, the disclosed technique considers multiple factors rather than a single variable that affect wound healing.

[0161] The amputation site analysis system described in this paper can be appropriately studied using a statistical discipline called machine learning to analyze multivariate systems for predictive analysis. By combining data from local microcirculation assessments with systemic factors influencing wound healing (such as diabetes, smoking status, age, nutritional status, etc.), this method can provide important information about the overall likelihood of initial wound healing in patients, factors that are not easily observed in the microcirculation using existing techniques. Since both local and systemic factors influence the likelihood of eventual healing, considering all these factors together improves the accuracy of the amputation site analysis system.

[0162] The amputation site analysis system can predict the likelihood of initial wound healing after amputation at the research level with at least 95% sensitivity and 95% specificity. If used for routine patient assessment with this sensitivity and specificity prior to amputation, the amputation site analysis system described herein could reduce the re-amputation rate by 67%, potentially reducing re-amputations by 10,000 cases annually, while improving the quality of life for amputees and reducing healthcare costs associated with their care. Currently, the cost of an ABI test prior to amputation is approximately $150 per patient under Medicare, with the majority of the cost stemming from the time spent by technicians performing the test and interpreting the results. The proposed device does not impact the existing cost of LOA assessment because the expected cost is the same as existing vascular assessments. Unlike some existing LOA tests, the disclosed imaging system does not require disposable supplies. The routine cleaning and servicing costs of this system are similar to those of existing systems on the market.

[0163] The disclosed imaging technology is designed to combine optical imaging techniques of photoplethysmography (PPG imaging) and multispectral imaging (MSI). Figure 13 An example view of a device designed to integrate optical imaging techniques, including photoplethysmography (PPG) and multispectral imaging (MSI), is shown. Furthermore, it is capable of incorporating key patient health indicators into its assessment classifier. The disclosed technology now combines blood flow assessment (e.g., arterial pulsation amplitude) with tissue characterization (e.g., spectral analysis). When these measurements are acquired together from the tissue, they provide a more accurate assessment of the tissue compared to a single measurement.

[0164] Studies assessing the likelihood of healing at a given site of interest (LOA) have demonstrated that significant differences in tissue oxygenation levels between different sites can lead to successful vs. unsuccessful amputations. These studies used percutaneous oxygenation measurement (TCOM) to investigate tissue oxygenation. However, despite the technique's decades-long existence, TCOM has not proven superior to clinical assessment, and a clear boundary for tissue oxygenation at a given LOA predicted as a successful amputation has not been established in numerous clinical trials. According to expert assessment, TCOM is not suitable for clinical practice for several reasons. First, TCOM is limited to very small areas of interest (approximately 3 cm). Data is collected at 3 cm. TCOM treatment also requires heating the patient's skin, which can sometimes lead to skin burns, especially in patients with PAD. For example, TCOM treatment requires the placement of adhesive electrodes that heat the patient's skin to 45°C, which can sometimes lead to skin burns, especially in patients with PAD. Furthermore, in large clinical trials, a clear boundary has not been established for TCOM levels that predict successful amputation. Finally, TCOM results are affected by ambient temperature and local tissue edema, which limits the internal timing consistency of the device.

[0165] The Amputation Site Analysis System is designed to overcome the limitations of TCOM and other available devices to predict the likelihood of healing at a selected LOA. The device captures data over a large area of ​​tissue surface, enabling the characterization and mapping of tissue oxygenation and perfusion changes across the entire surface, rather than in isolated areas. The Amputation Site Analysis System is non-invasive and non-contact, and does not emit harmful radiation, thus posing no significant inherent risk of harm to the patient. The device is also unaffected by minor variations in ambient temperature. Most importantly, however, the Amputation Site Analysis System analyzes clinically important patient health indicators such as history of diabetes, presence of infection, smoking status, and nutritional status to provide end-users with a comprehensive assessment of wound healing potential, whereas previous technologies could only assess local tissue oxygenation.

[0166] The proposed imaging device encompasses non-invasive, non-contact, non-radiation optical imaging for various tissue classification applications, including the classification of tissue types at preoperative and / or postoperative amputation sites, and the confirmation of recommended optimal LOA (Location of Occurrence). The disclosed imaging system is a point-of-care perfusion imaging system that provides diagnostic images derived from measurements of tissue perfusion and patient health indicators. Caregivers can be easily trained to perform the imaging test. Limb imaging takes approximately 10 minutes, and the results are stored electronically for physician review. From the patient's perspective, the test is highly acceptable because it has no harmful side effects, does not contact the patient's skin, and causes no discomfort.

[0167] One aspect of the disclosed technology is the addition of patient health indicators to microcirculation assessments to improve the accuracy of diagnosing wound healing potential during amputation planning. As previously described, the amputation site analysis device simultaneously performs two optical imaging methods for blood flow assessment. While the disclosed amputation site imaging devices are more advanced because they capture over one million spatially unique PPG signals over large areas of tissue, the first type of PPG imaging is the same technique used in pulse oximetry to acquire vital signs including heart rate, respiratory rate, and SpO2. PPG signals are generated by measuring the interaction of light with dynamic changes in vascularized tissue. Vascularized tissue expands and contracts by approximately 1–2% of its volume with each systolic pressure fluctuation at the frequency of the cardiac cycle. This blood inflow increases the tissue volume and brings in additional light-absorbing hemoglobin. Therefore, the overall absorption of light within the tissue oscillates with each heartbeat. This information can be translated into vital signs recorded by a pulse oximetry device.

[0168] To generate an image from a volumetric plethysmogram, the disclosed system utilizes the path of light through tissue. A small portion of the light incident on the tissue surface is scattered into the tissue. A small portion of this scattered light exits the tissue from the same surface it initially entered. Using a sensitive digital camera, this backscattered light is collected over the entire tissue area, such that each pixel in the imager includes a unique PPG waveform determined by variations in the intensity of the scattered light. To generate a 2D visualization of blood flow relative to the tissue, the amplitude of each unique waveform is measured. To improve accuracy, the disclosed system can measure the average amplitude over numerous heartbeat samples.

[0169] The second optical measurement captured by the amputation site analysis system is MSI. This technique measures the reflectance of selected visible and near-infrared (NIR) wavelengths (400–1100 nm) on the tissue surface. The spectral characteristics of an object were initially used in remote sensing (e.g., satellite or flight imaging) for geological exploration or military target detection, but this technique is increasingly accepted in the medical field. This method is effective for quantifying important skin features associated with multiple pathologies, including PAD. Regarding the selection of LOA, MSI can be used to quantify the volume of hemoglobin and the presence of oxyhemoglobin.

[0170] The wavelengths of light used in MSI (Measuring Intensity Sequencing) systems can be selected based on recognized light-tissue interaction characteristics. Melanin in the stratum corneum and epidermis primarily absorbs UV and visible light wavelengths. Near-infrared wavelengths (700–5000 nm) are the least absorbed by melanin and are considered the best choice for penetrating the dermis. Blood vessels penetrating the dermis contain a large amount of hemoglobin, and the concentration of hemoglobin determines the extent to which the dermis absorbs wavelengths greater than 320 nm. The light absorption of hemoglobin also varies depending on whether the molecule is in an oxygenated or deoxygenated state. Because the concentration of tissue melanin and hemoglobin, as well as the number of oxyhemoglobin, changes occur in disease states, MSI detects changes in the resulting reflectance spectrum. Therefore, abnormal skin tissue can be identified by comparing changes in the reflectance spectrum to those of healthy tissue. Although MSI can use fewer specific wavelengths to characterize tissue compared to newer hyperspectral imagers, it still has advantages when considering spatial resolution, spectral range, image acquisition speed, and cost.

[0171] The third part of the data utilized by the amputation site analysis system consists of relevant patient health indicators collected during routine patient assessments. Various factors influencing wound healing have been identified. Many or all of these factors (including patient age, diagnosis of diabetes, smoking history, infection, obesity, medication use, and nutritional status) typically affect patients with peripheral artery disease (PAD) undergoing lower limb amputation. While these variables are currently considered holistically when clinicians assess potential LOAs, the amputation site analysis system can quantitatively assess these indicators to predict the likelihood of initial wound healing at a given LOA. The amputation site analysis system utilizes its machine learning tissue classification technology to integrate patient health indicators with optical imaging data. Practitioners can input relevant patient health indicators into the device during imaging. In some implementations, patients and / or physicians can authorize the input of patient health indicators, which can then be automatically provided from hospital information databases or electronic patient records. In some implementations, this data can be processed as an additional variable by the disclosed machine learning model, without distinction from optical data collected via PPG imaging and MSI. In other implementations, the machine learning model can use patient metrics to identify, for example, a specific tissue classifier suitable for classifying the patient's tissues from many different classifiers trained on different datasets. The machine learning model can be trained to generate a quantitative output after evaluating all data collected through the amputation site analysis system. This quantitative output can be transformed into image recognition regions of the scanned tissue surface that are likely or unlikely to heal post-amputation, generating mappings for each region of the tissue classification, and / or recommending a Level of Analysis (LOA).

[0172] Amputation site analysis systems can include a combination of PPG imagers, MSI cameras, and target patient health indicator inputs, such as... Figure 14As shown. By adjusting system settings and classifier parameters, the amputation site analysis system can be tailored to assess tissue characteristics under different pathological conditions. For LOA studies, the disclosed techniques can be used to develop specific classifiers and employ specific optics and filters suitable for measuring pulsation amplitude and tissue oxygenation to predict wound healing after the initial amputation, as detailed below.

[0173] Figure 15A An example process for training a machine learning diagnostic model is shown. Training a diagnostic classifier using a machine learning model can be accomplished using data from the population on which it will ultimately be used. Figure 15A The accuracy of a classifier can only be as accurate as the method used to identify the true state of the training data, in this case, non-healing amputation group vs. healing amputation group. To address this issue, the disclosed method generates a standardized amputation healing assessment system to track and classify outcomes. Classifier development iterates in the following manner: initial judgment of accuracy, research to improve accuracy, and then evaluation of the new accuracy. This provides evidence that combining microcirculation imaging with patient health indicators can accurately classify patient tissue at the amputation site. Although some disclosed implementations are described as using both PPG and MSI data, other implementations have successfully achieved appropriate classification accuracy using MSI features without using PPG features.

[0174] The pilot clinical study design included a study of 60 patients, which investigated the accuracy of the publicly available system in predicting initial amputation healing in patients with PAD compared to current standards of care.

[0175] The disclosed imager uses optical methods to capture images from large areas (up to 15 cm of tissue). The device collects skin blood supply spectra and PPG signals from a 20 cm segment. It is well-suited for studying microcirculation in large areas of the lower extremities. A unique aspect of this device is its ability to integrate important patient health characteristics into its diagnostic classifier to improve accuracy. This pilot study identified useful patient health indicators and confirmed that the patient health indicators included in the device's machine learning diagnostic classifier improved accuracy in individual microcirculation measurements. The disclosed technique can assess microcirculation at various conventional LOA sites by combining patient health characteristics affecting wound healing and determine their correlation with the initial wound healing potential of patients after amputation.

[0176] Each patient's lower limb to be amputated was examined and included in this study. Clinically relevant patient health information was collected by the facility's care provider. Measurements performed using the disclosed imaging equipment were conducted by hospital staff who had been previously trained to perform imaging tests.

[0177] The skin area used to cover the remaining portion of the amputation is graded as positive or negative healing capacity using a LOA tissue classifier. The technicians performing the amputation site analysis are unaware of the clinical decision-making process regarding where the amputation will be performed.

[0178] To obtain true positive (+) and negative (-) results, or for subjects who are not healing and those who are healing, the disclosed technique uses a standardized initial wound healing assessment after amputation (Table 2). This assessment includes three categories: successful amputation, successful amputation with prolonged healing, and non-healing. Successful amputation is defined as healing within 30 days with complete granulation tissue formation, requiring no further amputation. Successful amputation with prolonged healing is defined as delayed healing, with incomplete granulation tissue formation at 30 days, but eventual healing within six months, requiring no further amputation to a more proximal level. Finally, non-healing is characterized by the development of gangrene and / or necrosis, and / or the need for further amputation to a more proximal level. Additionally, the disclosed technique may consider a wound requiring vascular reconstruction for healing as non-healing.

[0179] Table 2 Standardized Wound Healing Assessment

[0180]

[0181] These healing assessments were performed 30 days post-surgery. For subjects with delayed healing, the disclosed technique allows for a second healing assessment at six months post-surgery. Subjects who had not healed at six months and had not undergone further proximal amputation were classified into the non-healed group.

[0182] Figure 15B Example steps for generating a classifier model for amputation levels are shown. After establishing initial accuracy, a set of standard methods for improving accuracy can be used to develop the classifier. Figure 15B One challenge in this process is the trade-off between bias and variance, a problem encountered by most models. In other words, the classifier is well-suited to the current research group's data but not to the general population. To address this issue, the disclosed technique performs feature selection to create a combination of microcirculation measurements and patient health data with high accuracy and minimal redundancy between variables (i.e., using covariance to eliminate information from the model). A range of classifier models can be used to classify the data, including but not limited to: linear and quadratic discriminant analysis, decision trees, clustering, neural networks, and convolutional neural networks.

[0183] The disclosed technology can predict initial healing of amputations with an accuracy comparable to current standards of care (70-90%), and further improvements in accuracy can be achieved in large-scale clinical studies.

[0184] Sometimes vascular reconstruction is performed during amputation, and this additional procedure can influence diagnostic outcomes. These cases should be documented and considered in statistical analyses to identify any interactions between these procedures and the diagnostic decision. Another potential issue is combining the delayed healing group with the healed group in the dichotomous device output. Significant differences may exist between the delayed healing and healed groups, potentially leading to their inclusion as separate categories in the diagnostic output. Conversely, the delayed healing group may have more consistent data with the non-healed group, making them difficult to distinguish. In such cases, the disclosed technique could include data from more recent images in the classifier. In this scenario, the device's clinical utility can still be valuable as a tool to identify complications of amputation, rather than simply determining success or failure.

[0185] In this study, differences in skin pigmentation could lead to variations in measurements collected from subjects. To overcome these variations, the disclosed methods could include the identification of healthy areas of patient tissue, from which wound tissue measurements could be standardized. Other alternatives could automatically identify the melanin index of the patient's skin and use the melanin index as an input to the patient's health, as a variable in a classifier, or as a factor input to select a suitable classifier from multiple different classifiers.

[0186] Another issue is that normal blood flow can be seen in the skin of patients with PAD. This could be a result of collateral vessel compensation. However, this indicates poor motor response and short-term ischemia in patients with PAD. A readily implementable modification is to test the patient's imaging signal after the blood pressure cuff inflates in the limb being measured for 3 minutes to create ischemia. PAD is known to extend the time to reach 50% of the peak reactive hyperemia, which can be measured using the same optical properties of tissues assessed by amputation site analysis systems.

[0187] Figure 15C Example steps for generating a machine learning model according to this disclosure are shown. Machine learning (ML) can be a valuable tool for simultaneously analyzing multiple variables to achieve medical diagnosis, such as PPG information, MSI information, and patient health indicator information as described herein. The methods for developing ML tissue classifiers described herein may include, for example... Figure 15C The following steps are shown: 1) Define the clinical problem, 2) Select the variables to be measured, 3) Collect data to train the classifier, 4) Measure the accuracy of the classifier, 5) Identify error sources, 6) Correct the classifier to rectify errors and improve accuracy, 7) Freeze and validate the accuracy using independent datasets. Figure 3 ).

[0188] One consideration in developing the disclosed ML classifier is the transferability of the model from a clinical research group to the general population. To ensure better transferability, careful consideration can be given to selecting the training data that represents the final population to which the classifier will be used. To this end, in some implementations, the classifier can be divided into multiple variants based on training data grouped by common values ​​of patient health indicators such as diabetes status, BMI, smoking status, and other indicators described herein. The training data can be grouped based on common values ​​of one or more patient health indicators to provide many refined classifiers specific to certain patient populations. Furthermore, the selection of clinical variables implemented in the model can be stringent. As more data is collected from the population of interest, new information can be integrated into the database and used to retrain or “smart” the classifier.

[0189] ML is well-suited for addressing the multivariate problem of predicting healing at a selected LOA. While numerous factors provide useful information in diagnosing LOA, their individual importance to the final decision currently depends on qualitative clinical judgment. In contrast, ML integrates multiple variables and assigns appropriate weights to them, resulting in a quantitative output. Tissue classifier analysis uses optical data collected via PPG imaging and MSI to quantitatively predict healing at the amputation site. The output can be converted into an image that classifies the microcirculatory health of the skin tissue at potential LOAs and / or identifies lines or regions corresponding to the predicted optimal LOA.

[0190] As part of this assessment, the disclosed technology can collect data from numerous amputees to train a diagnostic machine learning classifier to diagnose healing potential in various amputation scenarios. Imaging using the disclosed device is rapid, non-invasive, and non-contact, and therefore can be performed in routine care settings such as at the bedside or preoperatively.

[0191] Figure 16 The diagram shows a graphical example of the tissues that may be involved in the amputation process. Dashed lines indicate the location of the skin incision, and red ellipses indicate the location of active skin that must be present for successful initial healing of the amputation.

[0192] Important patient health information that can be used in the diagnostic model can be collected by clinical staff at various clinical sites. The disclosed technology will not collect any data beyond the standards of care. These patient health indicators may include, but are not limited to: indicators of diabetes control (e.g., HbA1c, glucose, and insulin), smoking history, obesity (e.g., BMI or lean circumference), nutrition (e.g., albumin, prealbumin, transferrin), infection (e.g., WBC, granulocyte status, body temperature, antibiotic use), age, impairment function, and important medications (e.g., glucocorticoids or chemotherapy). By inputting this information into the software of the amputation site analysis system, these patient health indicator values ​​(e.g., binary "yes" or "no" values ​​at a certain scale, values ​​representing the patient's health indicator level, etc.) can be added to the diagnostic classifier.

[0193] Based on clinical characteristics collected from each patient, a machine learning classifier can categorize subjects into those with unhealed (positive outcome) and those with healed (negative outcome). Some implementations can include all features in the classifier. The accuracy of the classifier can then be determined using ten-fold cross-validation as follows: First, classifier coefficients are generated using 60% of the randomly included subjects; then, the trained classifier is used to classify the remaining 40% of subjects. Using standard sensitivity and specificity methods, the accuracy of the classifier in classifying the subjects in the retained 40% group can be calculated. This can be repeated 10 times to generate a stable quantified accuracy.

[0194] Figure 17A A sample clinical study flowchart for imaging assessment of amputation sites is shown. Subjects meeting inclusion and exclusion criteria are selected. Microcirculation data for each subject can be collected by imaging the skin using a publicly available MSI and PPG imaging device. For each limb awaiting amputation, a scan of approximately 30 seconds can be performed each time. The device can image the ankle and foot areas according to conventional amputation procedures for PAD patients, including: above the knee (AKA), below the knee (BKA), above the ankle (i.e., the foot), metatarsals, or toes. Skin areas that can be selected as flaps to cover the stump can be analyzed to identify whether the flap tissue has the required level of circulation, thus contributing to successful amputation. Figure 16 PPG and MSI images can be collected from the skin region that will be used for flaps of the most distal portion of the stump at each conventional LOA site. This region of the tissue is selected because of its influence on the initial healing of the surgical site.

[0195] Figure 17BA flowchart of an example clinical study for imaging assessment of amputation sites is shown. Subjects meeting inclusion and exclusion criteria are selected. Diagnosis of amputation site healing can be performed during imaging using the disclosed imaging device. For each leg awaiting amputation, a scan of approximately 30 seconds can be performed each time. The device can image the ankle and foot areas according to conventional amputation procedures for PAD patients, including: above the knee (AKA), below the knee (BKA), above the ankle (AAA), metatarsals, or toes. Skin areas covering the stump can be selected for analysis as flaps. Figure 16 ).

[0196] Important patient health information used in the diagnostic model can be collected by clinical staff at various clinical sites or automatically extracted from electronic medical information databases such as patient electronic medical records, with authorization. The disclosed technology complies with all applicable privacy regulations and does not collect any data beyond the standards of care. Patient health indicators may include: measurements of diabetes control (e.g., HbA1c, glucose, and / or insulin), smoking history, obesity (e.g., BMI or lean circumference), nutrition (e.g., albumin, prealbumin, or transferrin), infection (e.g., WBC, granulocyte status, body temperature, or antibiotic use), age, and / or important medications (e.g., glucocorticoids or chemotherapy). By inputting this information into software on an amputation site imaging device, it can be added to a diagnostic classifier or used to select a suitable classifier trained on data from patients with the corresponding patient health indicator values.

[0197] PPG and MSI imaging measurements from five amputation sites of the affected limb can be evaluated to determine wound healing potential. Based on each limb, the disclosed technique can determine an overall healing score, and these measurements are compared with actual amputation success in the limb to obtain the overall accuracy of the assessment. This yields results in measurements of recipient operating characteristic (ROC), sensitivity, and specificity.

[0198] One possible outcome measure for graded wound healing is that the disclosed technique compares an automated amputation site diagnosis of the amputation location determined by the clinician with amputation success determined by a standardized wound healing assessment. This analysis yields the receiver operating characteristic (ROC) curve for the amputation site diagnosis classifier.

[0199] This trial established the sensitivity and specificity of the device and validated that these values ​​are superior to clinical judgment in selecting LOA. The established goal is for the amputation site analysis system to achieve 95% sensitivity and 95% specificity in diagnosing LOA, superior to the 70–90% accuracy of current clinical judgment. To establish the sample size, the disclosed technique first obtains positive predictive value (PPV) and negative predictive value (NPV), which requires knowledge of the disease prevalence. The disclosed amputation site analysis technique can identify a prevalence of re-amputation to a more proximal level of approximately 20% (baseline) in the screening population (patients aged >18 years requiring initial amputation of the affected limb due to microvascular disease). Therefore, the expected positive predictive value is 97%, and the expected negative predictive value is 93%.

[0200] The sample size analysis for testing the following hypothesis was performed using the method proposed by Steinberg et al. (2008), see "Sample size for positive and negative predictive value in diagnostic research using case-control designs," Biostatistics, Vol. 10, No. 1, pp. 94-105, 2009. The significance level (α) was 0.05, and the expected weight (β) was 0.80.

[0201] For PPV For NPV

[0202] H0: PPV 系统 = PPV 临床判断 H0: NPV 系统 = NPV 临床判断

[0203] H1: PPV 系统 PPV 临床判断 H1: NPV 系统 > NPV 临床判断

[0204] The results showed that, according to the healing assessment ( Figure 18Rejecting these null hypotheses (H0), the disclosed technique must be able to register a total of 236 lower limbs, of which 1 / 5 of the limbs are unhealed (positive outcome). However, because it is impossible to know the subject's disease status before registration, the disclosed technique cannot pre-select this proportion as 1 / 5. Therefore, the proportion may vary. If the proportion is too low, 1 / 10 of the limbs are unhealed (positive), the disclosed technique may require approximately 450 limbs in total; if the proportion is too high, 3 / 5 of the limbs are unhealed (positive), the disclosed technique may only require a total of 124 limbs.

[0205] Figure 17C An example training study flowchart for training the machine learning classifier described herein is shown. In Phase I, the disclosed imaging apparatus will be used to collect data to ultimately complete the training of the classifier in a clinical study recruiting amputees. Figure 17C As part of a non-critical risk study, imaging studies will be conducted in routine care facilities prior to amputation.

[0206] During this training study, a large dataset of training images will be obtained, on which tests will be performed to evaluate variables in specific classifier components, aiming to achieve 90% sensitivity and specificity. Standardized amputation healing can be judged based on the final outcome of the amputation (healed or unhealed). Training can be performed by collecting data that accurately represents the population that will ultimately use the classifier. Importantly, the classifier must be as accurate as the method used to identify the true state of the training data, in this case, the healed or unhealed state of the amputation site chosen by the clinician. Therefore, the disclosed technique involves generating a Standardized Amputation Healing Assessment System (SAHAS) to classify the outcomes (Table 3 below). Additionally, skin color is expected to affect at least MSI optical measurements. Therefore, for example, using data from a portable colorimeter, the amputation site analysis system can classify the subject's skin color based on the subject's melanin index, and this data can be utilized in classifier development.

[0207] Table 3

[0208]

[0209] Training data will be collected from 49 hospitals and 8 wound care centers with one of the busiest vascular surgery teams. Participants will be assessed against inclusion and exclusion criteria. Inclusion criteria include patients requesting their first amputation of a limb secondary to atherosclerosis (PAD), being able to sign informed consent, and being at least 18 years old. PAD is defined as insufficient arterial blood supply due to atherosclerosis based on one or more of the following assessments: ABI 0.9; clinically significant obstruction on duplex ultrasound; arteriography; or assessment showing median arterial calcification. Exclusion criteria include no diagnosis of PAD, previous amputation of the affected limb, or life expectancy of less than 6 months.

[0210] Patients will be evaluated 30 days post-surgery to determine, based on SAHAS, whether the initial wound on the LOA selected by the clinician has healed successfully, thus enabling the training of the classifier.

[0211] The lower limbs to be amputated will be imaged by hospital staff trained to use the publicly available imaging equipment. The study coordinator will record the subjects' melanin index, as well as SAHAS data obtained by the attending vascular surgeon at 30 days.

[0212] One hundred and sixty (160) subjects were recruited and imaged prior to amputation according to a standardized protocol. Figure 17C Imaging was performed during the same clinical talk in which the LOA had been determined and the clinician was unaware of the amputation analysis results. Postoperative wound healing in subjects was assessed using SAHAS 30 days post-amputation (Table 3). Pre-amputation image data could be “realized” based on the results of the clinical healing assessment (healed vs. non-healed) and used to train the classifier.

[0213] A classifier that categorizes unhealed and healed areas can be trained from the collected data. To train the classifier to correctly identify microcirculatory areas in the amputation site, the disclosed technique can "realize" the captured images to the true state of the microcirculatory areas in the amputation site (e.g., a ground truth mask). Figure 20B The classifier is trained based on this realignment data. To obtain the true state of the microcirculation zone at the amputation site, subjects undergo clinical evaluation at 30 days to classify the amputation site according to SAHAS. The results of the clinical healing assessment will be used to help physicians label the true area of ​​microcirculation at the amputation site on the captured images. The ground truth plus the raw image data can be included as input in the classifier training protocol.

[0214] The classifier can initially include all proposed tissue measurements (PPG and MSI). These measurements, along with patient health indicators, can be used to train a fully convolutional neural network to generate the initial data. Additionally, preprocessing and post-processing methods, as well as other classifiers, can be tested to improve accuracy. The accuracy of the classifier is determined using a 10-fold cross-validation procedure: 1) Generate classifier coefficients using a training sample consisting of 60% of randomly selected subjects without replacement (training group); 2) Classify the remaining 40% of subjects (test group) using the trained classifier; 3) Quantify the accuracy of the classifier; 4) Repeat 10 times to generate stable measurements of sensitivity and specificity. The most accurate classifier architecture can then be selected. Further analysis of these best performers involves generating receiver operating characteristic (ROC) curves.

[0215] To address the potential influence of skin color, the disclosed technique can assess the impact of skin color on raw optical measurements from different microcirculation categories. If this influence is significant, the disclosed technique can incorporate melanin index measurements from each subject into a classifier, retrain it, and evaluate the accuracy of the classifier. In some embodiments, the melanin index can be automatically identified using the disclosed imaging system, can be input by a clinician, and / or can be provided from electronic records. In some embodiments, this can provide a significant improvement in classifier performance and can be used to develop methods to obtain a subject's melanin index during imaging using the disclosed imaging device (e.g., by developing a classifier to obtain the melanin index directly from MSI measurements).

[0216] After establishing initial sensitivity and specificity, the disclosed technique can leverage a set of standard methods to refine the classifier. One challenge in this process is addressing the inherent trade-offs of complex classifiers. Complex classifiers are well-suited for data derived from them, but due to their complexity, they may not be transferable to other populations. To address this, the disclosed technique employs variable selection to create a combination of PPG and MSI measurements that provides high accuracy and minimal redundancy between measurements. Variable selection simultaneously ranks the importance of each measurement in the model to highlight which optical signals are most critical to the diagnostic accuracy of the disclosed technique. Final refinements to the classifier can then be successfully generalized to subsequent amputees. Furthermore, the disclosed methods for studying various ML techniques enable the acquisition of models of varying complexity for validation in Phase II, as detailed below. Figure 17D As mentioned above.

[0217] The group size of 160 is determined by estimating the number of subjects required to reach maximum classifier accuracy. Currently, there is no standardized method for predicting the training size for developing machine learning classifiers. However, research shows that the accuracy of a machine learning classifier converges to its maximum as the number of observations in the training dataset increases, and adding more training data does not improve performance. To identify the most likely convergence point for the disclosed amputation site tissue classifier, the training technique can use data from the preliminary study portion to simulate optical data from active and diseased microcirculatory tissues. Using this data, the disclosed technique can train four different machine learning classifiers and identify the sample size at which these classifiers will converge to their maximum accuracy. Multiple classifiers can be used to obtain a more comprehensive evaluation of the convergence point. These differ from the classifiers used in the preliminary data portion in that, unlike the complex convolutional neural networks used in the preliminary data examples, these four general classifiers can be less complex, thus expected to provide a more conservative estimate of the sample size.

[0218] Sufficient data can be collected and cross-referenced into post-amputation clinical wound healing assessment using SAHAS to train a robust classifier that can predict amputation site healing. Phase I (preliminary testing) of the classifier demonstrated accuracy exceeding the expected 90% sensitivity and specificity, leading to a high success rate in Phase II. A sample of 160 amputees based on preclinical testing was sufficient to train a robust classifier with 90% accuracy, exceeding current standards of care.

[0219] Like many predictive models, the goal is for the classifier to perform exceptionally well on the training data. However, a challenge is addressing the classifier's general applicability to data not included in the training dataset. "Overfitting," a machine learning term, refers to a model that is highly accurate on the training dataset but performs poorly on the general population. The purpose of the Phase II validation study is to assess overfitting. The success rate in Phase II can be improved in two ways. The first is to utilize a machine learning technique called regularization to reduce the complexity of the classifier and the likelihood of overfitting becoming a significant problem. The second approach could be to generate multiple classifier variables by employing various machine learning techniques for classifying the data, for example, grouping the training data into subsets based on commonalities in the patient data. These techniques can then be tested on the validation data in Phase II.

[0220] If skin color is determined to be a confounding variable for the classifier's predictions, it can be addressed in one of two ways: 1) incorporate this variable into the ML classifier and determine if the presence of this feature improves performance; or 2) divide the training data into groups based on the levels of this variable and identify the levels where the classifier is underperforming. Once these levels are identified, the published techniques can segment the data and generate separate classifiers at each level. This may involve collecting more training data from certain subgroups within the melanin index scale.

[0221] In rare cases, unhealed amputation sites may result from processes other than a lack of microcirculatory blood flow to the wound. For example, surgical site infection, poor nutritional status, or failure of the subject to adhere to wound care protocols may hinder amputation site healing independently of factors measured by the classifier. In these cases, the subject's physician may examine the subject to determine whether their data should be included for classifier training.

[0222] Figure 17D A sample validation study flowchart for Phase II is shown, which includes validation results of the trained machine learning model described herein. Phase II can be a validation study to further evaluate the sensitivity and specificity of the disclosed machine learning model for LOA selection. Upon completion of this phase, the disclosed classifier can be confirmed to have 90% sensitivity and specificity for detecting microcirculatory health of skin tissue at potential LOA sites. In addition to validating the accuracy of the classifier, the classifier's assessment of amputation sites can be compared with clinical assessments to demonstrate the potential impact of the device in a clinical setting.

[0223] Location, patient recruitment, consent, inclusion and exclusion criteria, and data collection can be the same as in Phase I.

[0224] Data from the Phase II study can be used to validate previously trained and frozen classifiers. Figure 17D Validation may require demonstrating that the classifier achieves predetermined sensitivity and specificity targets when applied to new study populations, thereby confirming the classifier's general applicability to patients outside the original training cohort. The target could be 90% sensitivity and specificity in the ability of the published classifier to correctly identify microcirculatory areas at the amputation site.

[0225] The device can record clinicians' assessments of the healing potential of tissue at the amputation site and compare them with the judgments of a publicly available classifier. After identifying the true healing status of the amputation, the validation technology can determine how many correct amputation assessments the physician made compared to those made by the classifier, thus demonstrating the potential impact of the device in a clinical setting.

[0226] The Phase II validation study workflow mirrors the Phase I training study workflow, allowing for the validation of data collected during the Phase I training study. The validation study design may include 65 participants, depending on the incidence of nonhealing at the amputation site during the Phase I training study.

[0227] To verify the classifier's ability to correctly identify microcirculatory regions in amputation sites, a validation technique compares the ML image results with the ground truth state of the microcirculatory regions at the amputation site. The ground truth mask for the microcirculatory regions at the amputation site can be obtained using the method employed in the Stage I training study. This ground truth mask can then be compared with images of the microcirculatory regions generated by the classifier. This comparison verifies the sensitivity and specificity of the machine learning classifier in identifying the microcirculatory state at the amputation site.

[0228] Clinician assessments of healing potential, compared with classifier assessments, can include the following analyses: A physician's success rate can be obtained by examining the 30-day healing status of the amputation using previously collected SAHAS data; a successful amputation is equated with the physician successfully selecting an appropriate site. To determine classifier success or failure, tissue regions can be identified in the pre-amputation output images, where the identified regions can be used to cover the amputation site. Figure 16 The percentage of identified regions or regions of interest (ROIs) classified as unhealed by the system can be calculated. FROC analysis can be used to identify a threshold percentage of non-active tissue within an ROI that most accurately predicts unhealed amputations. For each amputation site assessment, this analysis can provide a single "healed" or "unhealed" prediction. Using McNemar's repeated proportion test, amputation decisions made by the classifier based on this ROI analysis can be compared to physician amputation healing outcomes.

[0229] Phase II studies aimed to validate the sensitivity and specificity of the disclosed technique in determining the microcirculatory status of the skin at potential sites of locomotion (LOA) and to compare the accuracy of the classifier with the actual microcirculatory status. A refined method for collecting the validation dataset, which comprised 30% of the training set of 160 subjects, was used for validation. Considering potential confounding factors affecting amputation healing, an additional 10% of subjects was added, bringing the total to 64 subjects.

[0230] The power analysis of secondary outcomes was conducted from a total of 65 participants. Secondary outcomes used McNemar's comparison test to compare the classifier's assessment of healing potential based on ROI analysis with physician amputation healing outcomes. The contingency table below (Table 4) illustrates how the validation study was able to pair amputation site outcomes selected from clinicians and the published classifier:

[0231] Table 4

[0232]

[0233] The hypothesis of this test is: H0: Π 12 ≠ Π 21 , and H1: Π 12 ≠ Π 21 ; among which Π 12 This is the expected proportion of automated amputation site analysis failures when clinicians succeed. 21 This is the expected percentage of clinicians who fail when automated amputation site analysis is successful. Use the following formula:

[0234] in

[0235] In order to perform this calculation, the disclosed technique can... 12 The estimate is 0.27, and Π 21 The estimate is 0.07. These figures are based on the prevalence of amputation failure (approximately 30%) and the expected 90% sensitivity and 90% specificity of the published techniques for predicting LOA. Validation can be performed using a two-sided test with a power setting of 0.80 and α of 0.05. Therefore, a sample size of 65 subjects will contribute 80% to this secondary outcome.

[0236] Phase II studies can validate that the classifier achieves the established target of 90% sensitivity and specificity in assessing the microcirculatory status of potential amputation sites across multiple loci of interest (LOA). Furthermore, this study demonstrates that this technology represents a significant improvement over current standards of care for patients undergoing amputations secondary to peripheral arterial disease (PAD).

[0237] The research design is based on Figure 17C The stage I research is indicated.

[0238] The training and validation techniques described set a high standard for the application of the disclosed techniques in selecting an appropriate LOA. However, amputation failure is a significant clinical problem with a lack of user-friendly, quantitative, and safe solutions; and any incremental improvements in appropriate LOA delineation should result in significant improvements in patient outcomes and cost savings.

[0239] Figure 18 The results of an example statistical sample size analysis are shown. The overall sample size is based on the ratio of non-healed (+) to healed (-) amputations in the study group. The significance level (α) is 0.05, and the expected weight (β) is 0.80.

[0240] Considering the potential variation in the ratio of positive to negative subjects, the disclosed technique may include approximately 50% of the 236 subjects initially estimated. Therefore, the total sample size can be established with a total of 354 subjects. The disclosed technique monitors the research data during collection, calculates the total number of limbs studied and the ratio of unsuccessful amputations (+outcome) to successful amputations (-outcome), and stops the study once an appropriate ratio and total sample size are obtained.

[0241] To determine the correlation between the automated amputation site analysis output and initial wound healing, the disclosed technique compares the automated results with a standardized healing assessment, which categorizes subjects into healed or unhealed groups. This comparison reveals a correlation supporting high sensitivity and specificity in predicting initial healing after amputation. The ROC incorporates a decision threshold, resulting in sensitivity and specificity exceeding the requirements established by current standards of care (70–90% accuracy).

[0242] A sufficiently large sample size for training data in machine learning can enhance the significance of all non-imaging data (patient health indicators) used in the diagnostic model. For example, diabetes may be an important clinical feature, but in a small sample size, all patients may have diabetes, or the incidence of diabetes may be insufficient to study its impact. Therefore, without an adequate sample size, the presence of this comorbidity in the disclosed diagnostic classifier cannot be accounted for. Patient study groups may have many similarities in overall health status, but some of these variables may be measured at different levels and cannot be simply treated as dichotomies. For example, diabetic subjects may have a range of controls measured by HbA1c and blood glucose tests. For situations where this is not feasible, the disclosed techniques can consider continuing to collect this data by observing a larger amputee population. In some implementations, when additional data is collected from the disclosed imaging device in a central database and the machine learning classifier is improved by training from such additional data (e.g., using similar techniques as described above), the improved classifier or a set of classifiers based on different subsets of the training data can be fed to the diagnostic machine in the field.

[0243] Figure 19A sample graphical representation of the tissue classification process for amputation sites described herein is shown. PPG and MSI data can be collected simultaneously or sequentially from the amputation site. Based on the pixels of each image (approximately one million pixels in some examples), multiple independent measurements, including PPG pulse amplitude values ​​and intensity values ​​at each MSI wavelength, can be calculated. Data from each pixel can be individually fed back into a classifier previously trained using known patient data. The classifier can return a value for each pixel representing the health of the microvascular system and the potential for successful healing after amputation. These can be presented to the physician in the form of images. For example, the classification can be displayed by associating a specific value or range of values ​​with one of many tissue categories, generating a visual representation of that category for display (e.g., color or pattern filling), and displaying the pixel using the visual representation of the category to which it was classified. In this way, mappings of different tissue health categories can be overlaid onto images of the patient's amputation site. In some embodiments, additionally or alternatively, regions of recommended or potential LOAs can be overlaid onto the image based on analysis of the classified tissue regions.

[0244] Figure 20A Example images of amputation sites marked by physicians are shown. Figure 20B It shows the basis Figure 20A Example image of tissue classification mapping generated from the image.

[0245] To determine whether the classifier accurately identified the microcirculatory health of lower limb tissues during amputation assessment in subjects with PAD, images were collected from multiple subjects prior to amputation. These subjects were followed up for approximately 30 days to determine whether the collected images correlated with the final amputation site healing status. The subjects had severe PAD and were scheduled for amputation of one lower limb.

[0246] On the day of the planned amputation, just before the scheduled surgery, images of the limb to be amputated were collected from the subject. Images included the area where amputation assessment had been performed, the foot in some cases, and other areas of the limb closer to the amputation site. An image set was obtained from each imaging location. This set consisted of eight MSI images captured at different wavelengths of light, listed as (in center wavelength ± full width at half maximum): 420 nm ± 20 nm, 525 nm ± 35 nm, 581 nm ± 20 nm, 620 nm ± 20 nm, 660 nm ± 20 nm, 726 nm ± 41 nm, 820 nm ± 20 nm, and 855 nm ± 30 nm. These images were captured using appropriate wavelength bandpass filters at a frame rate of approximately 4–6 fps. The image set also included 400 time-resolved images captured using an 855 nm bandpass filter, acquired sequentially at a frame rate of 30 fps. When these images are acquired continuously, imaging takes approximately 15 seconds, with some systems capturing consecutive MSI images at intervals of about 4-6 fps, while other systems can capture all eight MSI images simultaneously. The camera orientation can be varied between images, and in some implementations, healing classification scores at each pixel can be recorded after calculation. The distance between the imaging system and the tissue site is 40 cm, and the field of view of the imaging system is 15. 20 cm. The healing or non-healing of the amputation site is assessed 30 days postoperatively.

[0247] A machine learning classifier was trained to detect three microvascular blood flow regions in the imaged tissue, including (1) active skin (tissue unaffected by PAD), (2) small vessel disease (SVD) (skin with damaged blood vessels affected by PAD), and (3) necrosis (skin and inactive subcutaneous tissue).

[0248] To train the machine learning classifier, the subject's surgeon first "realized" the captured MSI / PPG images by dividing them into three microcirculation categories on images depicting the imaging region. To improve the accuracy of the surgeons' labeling of these images, the images were further divided after obtaining the results of the 30-day amputation healing assessment. These divided images were then converted into digital images for labeling individual pixels in the subject's MSI / PPG images. Figure 20A Once each pixel is labeled, a machine learning model classifier is trained to classify the categories of tissue microcirculation. The machine learning model used is a fully convolutional neural network, and the data input to the model includes MSI and PPG images. The image matrix has the following dimensions: 1408 pixels x 2048 pixels x 9 features.

[0249] The classifier was trained using 72 images from three subjects. Before training, one image from each subject was retained in the training set to be used as a test image. Once training was complete, the classifier's accuracy was tested on this retained image to obtain the classifier's accuracy within that image.

[0250] Unlike training images, the system obtains images from a subject after amputation to determine whether the classifier can make accurate predictions on images not used for training.

[0251] Figure 21 The cross-validation results of the disclosed amputation machine learning technique are shown. The left column displays color images of the amputation site immediately before surgery; the middle column displays ground truth data determined by the surgeon 30 days post-surgery; and the right column displays the output of automated amputation site analysis after classifier training. This indicates that the classified optical data is closely correlated with the surgeon's clinical judgment of the three microvascular blood flow regions of the potential amputation site, achieving an accuracy of 97.5%, as shown in Table 5 below. Table 4 shows the confusion matrix and average efficiency of the disclosed amputation healing assessment technique. The average efficiency is calculated to be 97.5% ± 3.4%.

[0252] Table 5

[0253]

[0254] Figure 22 Example images are shown after being classified following training of a publicly available amputation machine learning classifier. The image in the upper left shows the postoperative site of a transosseous amputation where the stump failed to heal and required further surgery. This is Figure 6 The same foot is shown in the first row. The ML classifier output in the top right image shows small vessel disease and skin necrosis used to cover the stump, consistent with clinical findings. The control image (bottom left) of the healthy thigh of the subject, which was not used in the classifier training, and the ML classifier output of the control image (bottom right) demonstrate that the classifier can identify active tissue with high accuracy.

[0255] In summary, the disclosed classifier was trained and tested using images from a preliminary group of subjects with pre-amputation anterior vascular obstruction (PAD). It is understood that the disclosed technique can be applied to amputees without PAD. The classifier showed excellent correlation with the actual state of tissue microcirculation in the limb, approximately 98%. Equally impressive is the classifier's ability to process novel, unknown images and accurately identify the microvascular state of tissues, meaning that pre-amputation images can be used to predict unhealed amputation sites in subjects.

[0256] One implementation of classifying patient tissue using an amputation site classifier can be achieved using the following steps. The diagnostic system described herein can capture PPG and / or MSI image data of wound sites (e.g., tissue areas including potential amputation sites). The system can generate a map of wound sites from the image data, for example, by correcting the captured image set, such that the same pixel locations on the set of images represent approximately the same physical location of the imaged tissue area. In some implementations, the map can be segmented into pixels depicting the wound site and background pixels not depicting patient tissue. For each pixel depicting the wound site, the system can determine a PPG amplitude value and an MSI reflectance value. Before, during, or after capturing the image data, the system can receive input of patient health indicators, for example, as input from a physician or as data automatically provided by an electronic medical record.

[0257] Based on patient health indicators, one implementation of the system can input patient health indicator values ​​as variables along with PPG and MSI values ​​into a classifier. The classifier can be generated using the machine learning techniques described herein. The system can use the classifier to classify individual pixels at the wound site into one of three categories: necrotic, active, or small vessel disease, based on a weighted combination of PPG, MSI, and patient health indicator values. The system can generate a mask image that visually represents the pixel classification of the wound site and output this mask image for display to clinicians.

[0258] Based on patient health indicators, another implementation of the system can select one of several classifier variants, each variant based on a subgroup of training data representing patients with different combinations of patient health indicator values. The selected classifier can be generated from the training data, which contains patient health indicators that match those of the patients of interest. The system can input determined PPG and MSI values ​​into the selected classifier. Using the selected classifier, and based on a weighted combination of PPG and MSI values ​​at the pixel, the system can classify each pixel at the wound site as necrotic, active, or small vessel disease. The system can generate a mask image visually representing the pixel classification of the wound site and output this mask image for display to the clinician.

[0259] Understandably, some implementations can combine the aforementioned classification techniques, for example, by performing classifier selection based on certain patient health indicators and by inputting the same or other patient health indicators into the classification model. In addition to masked images, some implementations can also output automatically determined LOAs or regions of potential LOA locations that map over the wound site.

[0260] Given a large database of training data, including information such as gender, diabetes status, age, and smoking history, the disclosed ML training technique can subset the database based on those parameters that match the values ​​of these indicators for patients of interest. A classifier is trained on this subset of the overall training database, providing a more customized classifier for more accurate tissue classification and / or LOA prediction. Alternatively, these parameters can be input as variables into the classifier (e.g., encoded as 1 or 0 for smokers, 1 or 0 for diabetics, etc.), and the level of this variable will affect the usefulness of the device. This feature is referred to herein as "patient health indicators." According to one implementation, the disclosed classification technique can selectively use training data representing the final population on which the classifier will be used. This can be done in real time or automatically before imaging a specific patient, for example, based on known information about the patient, allowing individual patient outcomes to be collected from a personalized pool of training data.

[0261] As shown in the figure, the classifier's output can be an image that includes mappings of different regions of tissue classification. Such mappings can help physicians select an appropriate LOA by instructing them on the disease state of the tissue used to construct the amputation. Alternatively or additionally, other implementations can provide a visual representation of the region of the recommended LOA or a possible LOA location, for example, based on mappings of different regions of tissue classification and a machine learning model that associates the boundaries, location, and extent of the classified tissue regions with LOAs that may provide an appropriate healing outcome for the patient. For example, the disclosed system can automate the process of selecting an appropriate site by providing the imager with a larger field of view or by capturing image data using a system capable of imaging the entire limb (e.g., the leg from the toes to the hip).

[0262] Overview of Example Classifiers

[0263] Figure 23A An example classification data stream 2300 is shown for generating the organizational diagram 2315 of the classification described herein. Multiple time-resolved images I can be included. t1 -I tn and multiple multispectral images I λ1 -I λn The input 2305 is set in the machine learning classifier 2310. As described above, a series of time-resolved images I t1 -I tn This can include photoplethysmography (PPG) information, which represents pulsatile blood flow in patient tissue. In some implementations, a series of time-resolved images I t1 -I tnThis can include hundreds of images, for example, the 400 time-resolved images captured above relative to the training data using an 855 nm bandpass filter (or another filter through a suitable wavelength), acquired sequentially at a frame rate of 30 fps. As mentioned above, a series of time-resolved images I can be omitted from some implementations. t1 -I tn This can beneficially reduce the amount of time required to capture images in the input data 2305.

[0264] Multispectral Image I λ1 -I λn This includes images captured at every location in multiple different wavelengths of light. In some implementations, multispectral images I λ1 -I λn This can include eight images captured under light of different wavelengths. As an example, wavelengths can include (listed in center wavelength ± full width at half maximum): 420 nm ± 20 nm, 525 nm ± 35 nm, 581 nm ± 20 nm, 620 nm ± 20 nm, 660 nm ± 20 nm, 726 nm ± 41 nm, 820 nm ± 20 nm, and / or 855 nm ± 30 nm. These images can be captured using appropriate wavelength bandpass filters at a frame rate of approximately 4–6 fps. Time-resolved image I t1 -I tn and multispectral images I λ1 -I λn Each can have the same x The image has a pixel size and, in some embodiments, may have approximately the same object located across the image set. Other embodiments can be captured from varying viewpoints, and the images can be registered before being input into the machine learning classifier 2310. Other embodiments can be captured from different viewpoints and provided to the classifier 2310 respectively, wherein the mapping of the output pixel classification scores is registered after computation.

[0265] In one implementation, the image set of input 2305 may be two-dimensional images of the imaged tissue site. In some implementations, the images may be captured entirely from a single viewpoint. In other implementations, the image set may include multiple subsets of MSI and PPG images, wherein each subset is captured from different viewpoints around the entire circumference of the potential amputation site. In some implementations, such images may be stitched together to form a panoramic image set of the tissue site. Other implementations may generate the panoramic image set via an imaging system comprising multiple cameras that can collaboratively capture some or all of the circumference around the tissue site in a single scan. In another implementation, image information from around the tissue site may be used to construct a 3D model of the tissue site, and the disclosed classifier may operate on a set of 3D models including time-resolved data and multispectral data.

[0266] In some implementations, the machine learning classifier 2310 can be an artificial neural network, such as a convolutional neural network (CNN), as discussed above. Figure 23B This will be discussed in more detail. In other implementations, the machine learning classifier 2310 may be another type of neural network or another machine learning classifier suitable for predicting pixel-level classifications based on supervised learning. In a sense, artificial neural networks are artificial; they are computational entities, similar to the biological neural networks of animals, but implemented by computational devices. Neural networks typically include an input layer, one or more intermediate layers, and an output layer, each layer comprising multiple nodes. Nodes in each layer are connected to some or all nodes in subsequent layers, and the weights of these interconnections are typically learned from data during training. Each individual node may have a summation function that combines the values ​​of all its inputs together. Specifically, nodes in adjacent layers may be logically connected to each other, and each logical connection between nodes in adjacent layers may be associated with its respective weight. A node can be considered a computational unit that computes an output value as a function of multiple distinct input values. A node can be considered a "connection" when the input value of the function associated with the current node includes the output of the function associated with a node in the previous layer multiplied by the weights associated with the respective "connections" between the current node and the nodes in the previous layer.

[0267] A CNN is a type of feedforward artificial neural network. A CNN layer has nodes arranged in three dimensions: width, height, and depth. Nodes within a layer are connected only to a small region preceding the width and height layers, called the receptive field. In some implementations, the convolutional filters can be two-dimensional, allowing repeated convolutions (or convolutional transformations of images) to be applied to individual images within a specified subset of the input or image. In other implementations, the convolutional filters can be three-dimensional, extending through the entire depth of the nodes in the input. Different types of layers, either locally or fully connected, are stacked to form the CNN architecture. Nodes in the convolutional layers of a CNN can share weights, allowing the convolutional filters of a given layer to be replicated across the entire width and height of the input, thus reducing the total number of trainable weights and increasing the applicability of the CNN to datasets beyond the training data.

[0268] The parameters of a CNN can be set during a process called training. For example, a CNN can be trained using training data, which includes input data and the correct or preferred output of the model for each input data. By using an input matrix instead of a single input vector, a collection of single input vectors can be processed simultaneously (“micro-batch processing”). A CNN can repeatedly process the input data and modify the convolutional filters (e.g., the weight matrix), which is equivalent to a trial-and-error process until the CNN produces (or “converges”) the correct or preferred output. Modification of the weight values ​​can be performed through a process called “backpropagation.” Backpropagation involves determining the difference between the expected model output and the obtained model output, and then determining how to modify the values ​​of some or all of the model's parameters to reduce the difference between the expected and obtained model outputs.

[0269] The machine learning classifier 2310 can be implemented as computer-executable instructions representing the computational architecture of the machine learning classifier 2310 via one or more processors, such as one or more graphics processing units (GPUs) 2340. In some implementations, the input 2305 can be processed in micro-batch processing, including a subset of images from the total input image set 2305, based on the size of each image and the capabilities of the GPU 2340, for example, eight images at a time. In such implementations, training involves batch backpropagation to backpropagate the average error of the images in the micro-batch processing across the CNN.

[0270] Machine learning classifier 2310 generates per-pixel classification, where the output includes classification values ​​for each [x, y] pixel location. During training of machine learning classifier 2310, these output classification values ​​can be compared with a pre-generated tissue map 2315, and the identified error rate can be fed back into machine learning classifier 2310. As described above, the pre-generated tissue map 2315 may include a ground truth mask generated by a physician after analyzing and / or treating the imaged tissue site. Machine learning classifier 2310 can learn the weights of various node connections, such as convolutional filters in multiple convolutional layers, through this backpropagation.

[0271] During the implementation of the machine learning classifier 2310, the output classification values ​​can be used to generate a tissue map 2315 to indicate pixels corresponding to various specified categories of tissue health. As described above, the tissue map 2315 may include a number of visually different colors or patterns to indicate different pixel categories, such as background 2335, healthy 2330, diseased 2325, and necrotic 2320.

[0272] Figure 23B An example classifier architecture for a CNN implementation of machine learning classifier 2310 is shown. Input 2305 can be provided to the CNN as a three-dimensional volume [x, y, z], which stores the raw pixel values ​​of each spectrum and / or photoplethysmography (PPG) images from the input image set. Each pixel value in the input dataset can be considered as a node in the CNN input layer. As mentioned above, each pixel can have an intensity value representing the brightness of the pixel. In some implementations, this number is stored as an 8-bit integer, giving a range of possible values ​​from 0 to 255, where a zero value represents a “dark” pixel from a photodiode that has not detected light, and a value of 255 represents a “bright” pixel from a photodiode that is fully filled with light. The width (x) of the input volume can be equivalent to the number of pixels along the x-axis of the image, and the height (y) of the input volume can be equivalent to the number of pixels along the y-axis of the image. The depth (z) of the input volume can be equivalent to the number of multispectral images and temporally resolved images in the dataset.

[0273] As shown in the figure, a CNN can include a symmetric convolutional encoder-decoder architecture, which comprises multiple convolutional stages followed by normalized exponential function layers. Each convolutional stage can include one or more convolutional layers, followed by rectified linear unit ("ReLU layer") layers. The convolutional layers compute the output (based on kernel size) of nodes connected to local regions in the input, where each node computes the dot product between its weights and the values ​​of the small regions it connects to in the input. As mentioned above, this can be achieved in 3x3 of the width and height of the input. In the 3 blocks, and along the depth (z) of the input quantity, through some or all values, make the kernels of each convolutional layer (e.g., the 3 learned during training)... 3. Weighted Block Convolution. After convolutions on each convolutional layer, the ReLU layer can apply element-wise activation functions before setting the output volume to the next layer. For example, the function of the ReLU layer could be a function that replaces any negative numbers in the output of the convolutional layer with zeros and multiplies all positive numbers in the output by a slope of 1.

[0274] The convolutional stages consist of several encoder convolutional stages (convolutional stages 1-5), followed by a corresponding number of decoder convolutional stages (convolutional stages 6-10). Each encoder convolutional stage corresponds to one of the decoder convolutional stages, as indicated by the arrows representing pooling mask transmission. Following the encoder convolutional stages (convolutional stages 1-5) are pooling layers (pooling layers 1-5), which feed downsampled data to subsequent convolutional stages and provide pooling masks (pooling masks 1-5) to the upsampling layers (upsampling layers 1-5) of the corresponding decoder convolutional stages (convolutional stages 6-10). Each pooling layer 1-5 performs downsampling along the spatial dimension (x, y) of the input volume and outputs the downsampled volume to subsequent stages. The pooling mask transmits information about the downsampled data to the corresponding decoder convolutional stage. The function of the pooling layers is to gradually reduce the spatial size of the representation, thereby reducing the number of parameters and computations in the CNN, and also controlling overfitting. The function of the pooling mask is to help preserve discarded spatial information while allowing for reduced computation.

[0275] To illustrate, pooling layers 1-5 can implement max pooling. For example, max pooling can include identifying 2 along the spatial dimensions of the output volume. Find the value of the matrix 2. The maximum value of the four values ​​in the matrix is ​​taken, and this maximum value is output as the single value of the block. In some implementations, downsampling is not performed along the depth of the input. Thus, for subsequent convolutional layers, each pooling layer reduces the spatial size of the output volume by 25%. The pooling mask from the layer output stores the other three (i.e., non-maximum) pixel values ​​and feeds them to the upsampling layer of the corresponding decoder convolutional layer. Therefore, during upsampling, the actual values ​​of the corresponding encoder convolutional layer are used, instead of using zeros in the matrix of values.

[0276] The normalized exponential function layer can receive the output of the final convolutional layer of convolutional stage 10 and generate class probabilities for each [x, y] pixel location of the input image. The normalized exponential function layer can apply the normalized exponential function (softmax), which is a normalized exponential function that "compresses" an arbitrary real-valued K-dimensional vector into a K-dimensional vector of real values ​​in the range (0, 1) with a cumulative sum of 1. The output of the normalized exponential function layer can be a classification score matrix for each pixel location or an N-channel probability image, where N is the number of classification categories. In some implementations, pixels can be assigned to the category corresponding to the highest probability at that pixel.

[0277] In one example implementation of a CNN, encoder convolutional stage 1 and the corresponding decoder convolutional stage 10 each have two convolutional layers, encoder convolutional stage 2 and the corresponding decoder convolutional stage 9 each have two convolutional layers, encoder convolutional stage 3 and the corresponding decoder convolutional stage 8 each have three convolutional layers, encoder convolutional stage 4 and the corresponding decoder convolutional stage 7 each have three convolutional layers, and encoder convolutional stage 5 and the corresponding decoder convolutional stage 6 each have three convolutional layers. In some implementations, each convolutional layer can implement a 3x3 convolution with a padding of 1 and a stride of 1. 3. Kernel. Weights in the kernel (3 The values ​​in the matrix (3) can be applied to each node in the input volume. Some or all kernel weights can vary between convolutional layers or can be the same, depending on the decisions made during training.

[0278] In other examples of CNNs, the encoder and decoder convolutional stages can have more or fewer convolutional layers, each followed by a ReLU layer. Other implementations of CNNs can have more or fewer convolutional stages to give the CNN corresponding encoder and decoder convolutional stages. Other CNN implementations may not use encoder and decoder convolutional stages, but may instead have several convolutional layers without using pooling masks. Convolutional filters can have other sizes, for example, kernel sizes of 4 or 5, and can have the same size across the various convolutional layers of the CNN, or they can be of varying sizes.

[0279] Overview of Feature Set Examples

[0280] As discussed below, experimental data demonstrate the advantages of incorporating PPG and MSI features into a single classifier.

[0281] In the following discussion, feature sets include photoplethysmography (PPG), multispectral imaging (MSI), and “real-time image” (RI; i.e., structural and geometric information from the spatial domain) features. Example methods include deriving ground truth, training a classifier using the three feature sets individually or jointly, classifying images, reporting errors, and comparing the classifier against different feature sets. Currently, these features have been developed and can be used for classification. These features are divided into three feature sets: PPG, MSI, and RI. For the example below, the classifier, Quadratic Discriminant Analysis (QDA), is trained using multiple feature sets. Feature sets are combined until all 33 features are included in the model. Based on the classification errors of the classifiers, the developed classifiers (i.e., those with discriminative feature sets) are compared.

[0282] The PPG classifier features can include the following 14 features:

[0283] 1. Image Output

[0284] 2. Maximum average value

[0285] 3. Standard deviation from the mean

[0286] 4. Number of intersection points

[0287] 5. Small Neighborhood SNR

[0288] 6. Improved SNR

[0289] 7. Standardized lighting

[0290] 8. Standardized images

[0291] 9. Standard deviation

[0292] 10. Skew

[0293] 11. Kurtosis

[0294] 12. X-gradient

[0295] 13. Y-gradient

[0296] 14. Standard deviation of the gradient

[0297] The features of a real-time image classifier can include the following 11 features:

[0298] 1. Real-time images

[0299] 2. Standardized real-time images

[0300] 3. Skew

[0301] 4. Kurtosis

[0302] 5. X-gradient

[0303] 6. Y-gradient

[0304] 7. Standard deviation of the X-gradient

[0305] 8. Range within a small neighborhood

[0306] 9. Standardized scope within a small neighborhood

[0307] 10. Scope within a large neighborhood

[0308] 11. Standardized scope within a large neighborhood

[0309] The features of an MSI classifier can include the following eight features:

[0310] 1. MSI λ1

[0311] 2. MSI λ2

[0312] 3. MSI λ3

[0313] 4. MSI λ4

[0314] 5. MSI λ5

[0315] 6. MSI λ6

[0316] 7. MSI λ7

[0317] 8. MSI λ8

[0318] The more features are added, the more significant the error reduction becomes. The feature groups can be ordered by importance, and in one example, they can be ordered as: (1) MSI, (2) RI, (3) PPG. Some implementations of the classifier are reusable, meaning that the classifier can be trained with a first subject and then used to classify the impairment of a second subject.

[0319] Overview of outlier detection and removal

[0320] The disclosed techniques can be used for outlier detection and removal in data used to train machine learning classification models. Outlier detection and removal is an important area in statistics and pattern recognition, and has been widely used in various fields, such as sensitive events of concern, medical diagnosis, and cybersecurity. Outlier detection can also be called anomaly detection, oddity detection, etc. Most outlier detection is model-based and proximity-based. For model-based classifiers, the disclosed techniques can use statistical tests to estimate the parameters of the sample distribution, for example, it can be viewed as a Gaussian distribution based on the central limit theorem (CLT). For a Gaussian distribution, two parameters can be considered: mean and standard deviation. The disclosed techniques can obtain these parameters from maximum likelihood estimation or maximum a posteriori estimation. In model-based methods, outliers are points with low probability of occurrence, which can be estimated by calculating the Z-score (standard score). Empirically, observations can be considered outliers if their probability is greater than 0.95 or less than 0.05. This is based on univariate analysis. If it is a multivariate normal distribution:

[0321]

[0322] ∑ is the mean of all points, and ∑ is the covariance matrix based on the mean. The disclosed technique can calculate the points. arrive The Mahalanobis distance. The Mahalanobis distance follows the χ² value. 2 Distribution, where d is the degrees of freedom (d is the data dimension). Ultimately, for all points... If the Mahalanobis distance is greater than χ 2 If the value is (0.975), then the point can be considered an outlier. The methodology of statistical methods works in most cases; however, when estimating parameters (e.g., mean and variance), these parameters can be sensitive to outliers. Furthermore, for Mahalanobis distance, the minimum distance between an observation and the mean is a function of dimensionality, which increases with increasing dimensionality. Depth-based methods for identifying outliers at the boundaries of the data space and bias-based methods for minimizing the difference when outliers are removed are discussed.

[0323] In proximity-based outlier detection, the concept of nearest neighbors can be used to generate samples that include or exclude approximations. First, the concept of distance is crucial. Given N samples, M variables, and a matrix of size N*M, the disclosed technique, for example, using Euclidean distance, can calculate the distances between sample spaces using a defined distance.

[0324] Clustering methods are a common approach that employs this distance concept. In clustering, the disclosed technique defines a radius ω for any group of points with a known center (molecular center). Points smaller than or equal to this radius are considered good points, and the molecular center is adjusted based on the inclusions of that new data point. For the K-nearest neighbor method, this is the sum of distances from a point to its k nearest neighbors. However, this method is unreliable in high-dimensional datasets.

[0325] Other methods are based on definitions of central tendency. For example, the Local Outlier Factor (LOF) is based on density. Density can be estimated from clusters of points. If the density of some clusters or groups of points is less than the density of their neighboring points, then points in this cluster can be potential outliers. Furthermore, these techniques do not work if the dataset is high-dimensional. Angle-based Outlier Detection (ABOD) and grid-based subspace outlier detection have been proposed to handle high-dimensional datasets.

[0326] Ultimately, outlier detection and removal significantly improve the accuracy of MSI applications used for skin tissue classification. Outlier removal successfully reduces the variability in the sample space for each tissue class. By limiting variability in this way, overlap of spectral properties can be reduced accordingly. By reducing overlap, classification accuracy is significantly improved, thereby improving the training of the classification model. This model has the potential to assist physicians in making decisions regarding the treatment of amputees using quantitative data.

[0327] Overview of other example alternatives

[0328] Another clinical application of the device described in this article is the classification of pressure ulcers, also known as pressure sores or bedsores. These wounds develop because the pressure acting on the tissue obstructs blood flow to that tissue. Due to the obstruction, tissue necrosis and tissue damage occur. In many cases, this causes a noticeable change in tissue color in the later stages. Pressure ulcers can be classified into grades one through four, depending on the amount of tissue damage that has occurred.

[0329] Part of the difficulty in identifying pressure ulcers lies in the fact that early obstruction causes tissue changes that are not easily observed on the tissue surface. The device described herein is effective in identifying pressure ulcers in their early stages of development, which facilitates early treatment and preventative care. The device, as described herein, classifies pressure ulcers by reading light reflectance at different times and in different frequency bands, which can detect differences in tissue composition and blood flow to the tissue.

[0330] In contrast to pressure ulcers where blood flow to the tissue is obstructed, tissues also suffer from excessive bleeding. In congestion, which may manifest as erythema, increased blood flow to the tissue leads to swelling, discoloration, and necrosis. This is also accompanied by engorgement of capillaries and veins, excessive hemosiderin in the tissue, and fibrosis. Alternative embodiments of the present invention can be effective in identifying and assessing early tissue congestion. Furthermore, the ability to detect changes in the original appearance and quality of the tissue, combined with the detection of blood flow to the tissue, allows these alternative embodiments to easily identify and assess the severity of congested tissue.

[0331] The alternative devices described in this article have many other applications in medical fields requiring the classification and assessment of tissues. Similar to necrotic tissue, hypoperfused tissue, and hyperemia, there are other types of wounds that these alternatives can classify and assess, including abrasions, lacerations, bleeding, lacerations, punctures, penetrating wounds, chronic wounds, or any type of wound that alters the original appearance and quality of the tissue along with changes in blood flow to the tissue. The alternatives described in this article provide practitioners with physiological information related to tissue activity in the form of simple images. Information such as blood perfusion and oxidation at the wound site is an important indicator of wound healing. By imaging these hemodynamic features hidden beneath the skin, physicians can better understand the progress of wound healing and make more informed and timely patient care decisions. Simultaneously, some of the devices described in this article can provide information about skin composition, a characterization of skin condition.

[0332] Furthermore, the use of some of the devices described herein is not limited to applications involving damaged tissue. In fact, some alternatives can also detect healthy tissue and distinguish it from necrotic or soon-to-be-necrotic tissue.

[0333] By comparing with wound or skin conditions, healthy tissue in a normal location can be classified and assessed. For example, along with low-perfusion tissue, there will be areas of healthy tissue associated with or adjacent to the low-perfusion tissue. This can aid in amputation identification levels and amputation site treatment, enabling the identification of the boundaries between healthy tissue and necrotic or soon-to-be-necrotic tissue. Healthy tissue can be identified by imaging the skin at different times and frequency bands to assess skin composition, blood perfusion, and oxidation at tissue sites.

[0334] The alternative methods described herein can also classify tissues based on the likelihood of successful implantation of flaps, transplanted tissues, or regenerating cells at the amputation site. This classification can take into account the quality and original appearance of the recipient tissue, as well as its ability to receive new blood supply. Alternatively, receiving tissues can be classified based on the likelihood that the tissue can form a new blood supply for flap, transplant, or regenerating cell implantation, and the general health of the skin. In classifying flap tissue, transplanted tissue, or receiving tissue, some of the devices described herein can analyze multiple images corresponding to different times and frequency bands.

[0335] In addition to simply classifying tissue health, the alternative methods described herein can also measure various aspects of the tissue; for example, the thickness of skin areas and granulation tissue can also be assessed. In another example, the device described herein can be used to monitor and assess tissue health near sutured wounds and the healing of sutured wounds.

[0336] Another application of some of the devices described herein is monitoring tissue healing. These devices can also acquire multiple images at multiple points in time to monitor how wounds change or how healthy tissue forms. In some cases, therapeutic agents such as steroids, hepatocyte growth factor (HGF), fibroblast growth factor (FGF), antibiotics, including isolated or aggregated cell populations of stem cells and / or endothelial cells, or tissue transplants, can be used to treat wounds or other ailments, and this treatment can also be monitored using the devices described herein. Some alternatives involve monitoring the effectiveness of the therapeutic agent by estimating tissue healing before, during, or after the application of a specific treatment. Some alternatives involve monitoring the effectiveness of the therapeutic agent by acquiring multiple images at different times and in different frequency bands. Based on these images, light reflected from the skin can be used to assess the original appearance and quality of the tissue, as well as blood flow to the tissue. As a result, the devices described herein can provide valuable information about how tissue heals and the potency and speed with which the therapeutic agent promotes the healing process.

[0337] Several options are available for monitoring the introduction of a left ventricular assist device (LVAD) and its healing after implantation. As LVAD flow increases, diastolic blood pressure increases, systolic blood pressure remains constant, and pulse pressure decreases. Pulse pressure, the difference between systolic and diastolic blood pressure, is influenced by left ventricular contractility, intravascular volume, pre- and post-pressure loads, and pump rate. Therefore, assessment of arterial blood pressure values ​​and waveforms provides valuable information about the physiological interactions between the LVAD and the cardiovascular system. For example, poor left ventricular function is associated with arterial waveforms that do not show pulsation. The options described in this article can be used to monitor pulsatile flow return in patients after LVAD implantation and provide a powerful tool for monitoring and assisting patient recovery.

[0338] Certain alternatives can also be used for the intraoperative management of orthopedic tissue transfers and reconstructive surgeries. For example, in the case of breast cancer patients, treatment may involve total mastectomy followed by breast reconstruction. Complications in breast reconstruction have been reported in rates as high as 50%. The device described in this article can aid in the assessment of tissue prepared for transplantation and the transplanted tissue itself. The assessment of these alternatives uses the methods described above to examine tissue health and quality, blood perfusion, and oxygenation.

[0339] Certain alternatives can also be used to facilitate the analysis of treatments for chronic wounds. Patients with chronic wounds often receive expensive, advanced treatments without having their effectiveness measured. The alternatives described herein can use the aforementioned imaging techniques to image chronic wounds and provide quantitative data on their condition, including wound size, wound depth, presence of injured tissue, and presence of healthy tissue.

[0340] Some of the alternative approaches described herein can also be used to identify limb degeneration. In these applications, image recognition identifies peripheral perfusion of the limb. This can be used to monitor the health of normal limbs and to detect areas of insufficient peripheral blood flow to limbs that may require specialized treatment (e.g., areas of limb ischemia or peripheral vascular disease), such as the introduction of growth factors (FGF, HGF, or VEGF) and / or regenerative cells, including but not limited to stem cells, endothelial precursor cells, endothelial progenitor cells, or cell populations comprising aggregates or isolated cells of these types. In some cases, this allows for early intervention that can save a limb from amputation. In other, more severe cases, it can provide healthcare professionals with the data needed to make a rational decision regarding the necessity of amputation.

[0341] Another application of the device described in this article relates to the treatment of Raynaud's phenomenon, which occurs when a patient experiences a short-term episode of vasospasm (i.e., vascular stenosis). Vasospasm typically occurs in the digital arteries that supply blood to the fingers, but can also occur in the feet, nose, ears, and lips. Some optional devices can accurately and precisely identify when a patient is experiencing Raynaud's phenomenon, which can aid in diagnosis at any stage.

[0342] Some alternative devices can also be used to identify, classify, or assess the presence, proliferation, metastasis, tumor burden, or stage of cancer cells, as well as the reduction of cancer cells, proliferation, metastasis, tumor burden, or stage of cancer after treatment. These alternatives measure light reflected from tissue to determine the composition of the skin, which can reflect abnormal components associated with cancer cells. Alternatives can also measure blood flow to cancer cells by evaluating images at different times. Blood flow can reflect abnormal blood flow to tissue associated with the presence, proliferation, metastasis, tumor burden, or stage of cancer cells. After removal of cancer cells, alternatives of the invention can also be used to monitor healing, including the growth of healthy tissue and any recurrence of cancer cells.

[0343] All aspects of the aforementioned alternatives have been successfully tested in laboratory and clinical settings. For example, in experiments using optical tissue prostheses that mechanically simulate tissue dynamics caused by pulsating blood flow, the device described herein exhibits better optical penetration than laser Doppler imaging and accurately detects pulsating flow beneath the tissue prosthesis material. The pulsating flow was tested in the range of 40–200 bpm (0.67 Hz–3.33 Hz) to measure the full range of human heart rate from rest to high speeds during exercise or activity.

[0344] Another aspect of some of the alternatives described herein is that the device can be combined with a dynamic library comprising one or more tissue condition reference values. In some cases, the dynamic library may include an origin image containing information about healthy skin tissue. The dynamic library may also include various images of wounds or skin conditions as comparison points to determine the progression and / or healing of the wound or skin condition. The dynamic library may also include samples of relevant signals, such as normal heart rate, abnormal heart rate, noise signals, signals corresponding to healthy tissue, and samples of signals corresponding to unhealthy tissue. In some embodiments, the dynamic library may be stored in a centralized database, such as a server-based database, and may receive information from many of the devices described herein via one or more networks. The dynamic library can be used to provide training data for the disclosed machine learning classifier and can provide the device with updated classifiers in the field to enhance tissue classification and amputation location recommendations.

[0345] In some alternatives, the images in the dynamic library are other images or data acquired by the apparatus described herein. In some alternatives, the dynamic library includes images and / or data acquired by apparatus not related to this invention. These images may be used to evaluate or otherwise treat subjects.

[0346] Summarize

[0347] The above disclosure demonstrates the feasibility of using the disclosed instrument with PPG and MSI capabilities to identify tissue deficiencies in blood flow and oxygenation in wound models and patient case studies. As shown above, the combined use of PPG and MSI in the disclosed technique allows for a more accurate investigation of the pathology resulting from reduced epidermal microvessels and blood perfusion. The disclosed technique can predict healing potential at a given LOA; the addition of important patient health indicators affecting wound healing outcomes can further improve the accuracy of the automated amputation site analysis system.

[0348] Other alternative solutions

[0349] The following provides several preferred alternatives to the invention described herein.

[0350] 1. An organizational classification system, comprising:

[0351] At least one light emitter is configured to emit light of each of a plurality of wavelengths to illuminate a tissue region, and each of the at least one light emitter is configured to emit light that is spatially uniform.

[0352] At least one light detection element is configured to collect light emitted from the at least one light emitter and reflected from the tissue region;

[0353] One or more processors, which communicate with the at least one emitter and the at least one photodetector, and are configured to:

[0354] Identify at least one patient health indicator value corresponding to a patient with the said tissue region.

[0355] A classifier is selected from a plurality of classifiers using the at least one patient health indicator value, and each of the plurality of classifiers is trained from different subsets of the training dataset, wherein the classifier is selected based on a classifier that has been trained using a subset of the training dataset, the subset including data from other patients having the at least one patient health indicator value.

[0356] Control the at least one light emitter to sequentially emit light of each wavelength among a plurality of wavelengths;

[0357] Receive multiple signals from the at least one photodetector element, a first subset of the multiple signals representing light emitted at the multiple wavelengths and reflected from the tissue region;

[0358] Based on at least a portion of the plurality of signals, an image having a plurality of pixels depicting the tissue region is generated;

[0359] For each of the multiple pixels depicting the tissue region:

[0360] Based on a first subset of the plurality of signals, determine the reflection intensity value of the pixel at each of the plurality of wavelengths, and

[0361] The classification of a pixel is determined by inputting the reflectance intensity value into the classifier, which associates the pixel with one of a plurality of tissue categories; and

[0362] Based on the classification of each pixel, a mapping of the multiple tissue categories is generated on the multiple pixels depicting the tissue region.

[0363] 2. The system as described in Option 1, wherein a second subset of the plurality of signals represents light of the same wavelength reflected from the tissue region at multiple different times, and wherein for each of the plurality of pixels depicting the tissue region, the one or more processors are configured to:

[0364] Based on a second subset of the plurality of signals, the PPG amplitude value of the pixel is determined; and

[0365] Furthermore, the classification of the pixel is determined by inputting the PPG amplitude value into the classifier.

[0366] 3. The system as described in alternative 1 or 2, wherein the one or more processors are configured to output a visual representation of the mapping for display to a user.

[0367] 4. The system of Option 3, wherein the visual representation includes the image having pixels displayed with a specific visual representation based on a classification selection of the pixels, wherein pixels associated with each of the plurality of tissue categories are displayed with a different visual representation.

[0368] 5. The system of any one of options 1 to 4, wherein the plurality of tissue categories includes a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

[0369] 6. The system of alternative embodiment 5, wherein generating the map includes identifying regions of the plurality of pixels, the regions depicting tissue associated with at least one of a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

[0370] 7. The system of alternative embodiment 6, wherein the one or more processors are configured to output an image for display, wherein multiple pixels depicting the tissue region are displayed in different colors according to the associated tissue category.

[0371] 8. The system as described in Option 7, wherein the one or more processors are configured to determine a recommended location for amputation based on the identified region.

[0372] 9. The system of any one of options 1 to 8, wherein the one or more processors are configured to determine the melanin index of the tissue region based on the plurality of signals.

[0373] 10. The system of alternative embodiment 9, wherein the one or more processors are configured to select the classifier based at least on the melanin index.

[0374] 11. An organizational classification method, comprising:

[0375] A classifier is selected from multiple classifiers based on at least one patient health indicator value, and each of the multiple classifiers is trained from different subsets of the training dataset, wherein the classifier is selected based on a classifier that has been trained using a subset of the training dataset, the subset including data from other patients having the at least one patient health indicator value.

[0376] Receive multiple signals from at least one photodetector element, the at least one photodetector element being positioned to receive light reflected from a tissue region, a first subset of the multiple signals representing light emitted at the multiple wavelengths and reflected from the tissue region;

[0377] Based on at least a portion of the plurality of signals, an image having a plurality of pixels depicting the tissue region is generated;

[0378] For each of the multiple pixels depicting the tissue region:

[0379] Based on a first subset of the plurality of signals, determine the reflection intensity value of the pixel at each of the plurality of wavelengths, and

[0380] The classification of a pixel is determined by inputting the reflectance intensity value into a selected classifier, which associates the pixel with one of a plurality of tissue categories; and

[0381] Based on the classification of each pixel, a mapping of the multiple tissue categories is generated on the multiple pixels depicting the tissue region.

[0382] 12. The method of alternative embodiment 11, wherein the second subset of the plurality of signals represents light of the same wavelength reflected from the tissue region at a plurality of different times, and for each of the plurality of pixels depicting the tissue region, the method further comprises:

[0383] Based on a second subset of the plurality of signals, the PPG amplitude value of the pixel is determined; and

[0384] Furthermore, the classification of the pixel is determined by inputting the PPG amplitude value into the classifier.

[0385] 13. The method as described in alternatives 11 or 12 further includes determining the melanin index of the tissue region based on the plurality of signals.

[0386] 14. The method of alternative embodiment 13 further includes selecting the classifier based at least on the melanin index.

[0387] 15. The method of any one of alternatives 11 to 14, wherein generating the map includes identifying regions of the plurality of pixels, the regions depicting tissue associated with at least one of a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

[0388] 16. The method of alternative embodiment 15 further includes outputting an image for display, wherein multiple pixels depicting the tissue region are displayed in different colors according to the associated tissue category.

[0389] 17. A method for identifying a recommended location for amputation, the method comprising:

[0390] Select patients with tissue areas requiring amputation;

[0391] An imaging system is programmed to control via one or more hardware processors to capture data representing multiple images of the tissue region, the data representing the multiple images including a first subset, each of the multiple images being captured using light of a different wavelength among multiple different wavelengths reflected from the tissue region;

[0392] Based on at least one of the multiple images, generate an image having multiple pixels depicting the tissue region;

[0393] For each of the multiple pixels depicting the tissue region:

[0394] Based on a first subset of data representing multiple images, determine the reflection intensity value of the pixel at each of the multiple wavelengths, and

[0395] The classification of the pixel is determined at least by inputting the reflectance intensity value into a classifier, which associates the pixel with one of a plurality of tissue categories; and

[0396] Based on the classification of each pixel, a recommended location for amputation is identified within the tissue region.

[0397] 18. The method of alternative embodiment 17, wherein the plurality of images comprises a second subset captured sequentially over a plurality of time periods, and for each pixel of the plurality of pixels depicting the tissue region, the method further comprises:

[0398] Based on a second subset of data representing multiple images, the PPG amplitude value of the pixel is determined; and

[0399] Furthermore, the classification of the pixel is determined by inputting the PPG amplitude value into the classifier.

[0400] 19. The method as described in alternatives 17 or 18 further includes identifying at least one patient health indicator value of the patient.

[0401] 20. The method of alternative embodiment 19 further includes inputting the at least one patient health indicator value into the classifier to determine the classification of each pixel.

[0402] 21. The method of alternative embodiment 19 or 20, further comprising: selecting a classifier from a plurality of classifiers based on at least one patient health indicator value; training each of the plurality of classifiers from different subsets of a training dataset, wherein the classifier is selected based on a classifier that has been trained using a subset of the training dataset, the subset including data from other patients having the at least one patient health indicator value.

[0403] 22. The method of any one of options 17 to 21 further includes generating a mapping of the plurality of tissue categories on the plurality of pixels depicting the tissue region based on the classification of each pixel.

[0404] 23. The method of alternative embodiment 22 further includes outputting a visual representation of the mapping to a user, wherein the visual representation includes visual recognition of the recommended location of the amputation.

[0405] 24. The method of any one of options 17 to 23, wherein identifying the recommended location of the amputation is performed in a programmed manner by the one or more hardware processors.

[0406] 25. A method for training a convolutional neural network to classify tissue regions at amputation sites, the method comprising:

[0407] The system receives training data representing multiple images of an amputation site, the data representing multiple images including a first subset, each of the multiple images being captured using light of a different wavelength among multiple different wavelengths reflected from the amputation site;

[0408] The training data is fed as a three-dimensional volume into the input layer of the convolutional neural network. The height and width of the three-dimensional volume correspond to the number of pixels in the height and width of each of the multiple images, and the depth corresponds to the number of images in the multiple images.

[0409] Multiple convolutions are performed at multiple encoder convolution stages and multiple decoder convolution stages of the convolutional neural network, wherein the first encoder convolution stage of the multiple encoder convolution stages includes an input layer as a first convolutional layer;

[0410] The output of the last decoder convolutional stage in the plurality of decoder convolutional stages is fed into the normalized exponential function layer of the convolutional neural network;

[0411] Based on the output of the normalized exponential function layer, classification values ​​for each pixel are generated across the height and width of the multiple images;

[0412] The classification value of each pixel is compared with the ground truth classification of the pixel in the ground truth image, wherein the ground truth classification is based on physician analysis of the amputation site;

[0413] Based on the comparison results, identify any errors in the classification values ​​of the pixels; and

[0414] At least one weight of the plurality of convolutions is adjusted, at least in part, based on the error backpropagated through the convolutional neural network.

[0415] 26. The method of alternative embodiment 25, wherein the plurality of images comprises a second subset captured sequentially at different times at the same wavelength as each other.

[0416] 27. The method as described in alternatives 25 or 26, wherein generating the classification includes classifying each pixel as one of background, healthy tissue, diseased tissue, or necrotic tissue.

[0417] 28. The method of any one of alternatives 25 to 27, wherein the first subset of the plurality of images comprises eight images captured using light of different wavelengths of eight different wavelengths, and wherein the second subset of the plurality of images comprises hundreds of images captured sequentially at the same wavelength at a rate of 30 frames per second.

[0418] 29. The method of any one of options 25 to 28, wherein each of the plurality of encoder convolutional stages and each of the plurality of decoder convolutional stages comprises at least two convolutional layers, each convolutional layer being followed by a rectified linear unit layer.

[0419] 30. The method of any one of alternatives 25 to 29, wherein performing multiple convolutions includes:

[0420] At each of the plurality of encoder convolution stages:

[0421] Perform at least the first encoder convolution.

[0422] The output of the first encoder convolution is fed into the rectified linear unit layer, and

[0423] The output of the rectified linear unit layer is downsampled using a max-pooling layer; and

[0424] At each of the plurality of decoder convolution stages:

[0425] Receive the pooling mask from the max-pooling layer of the corresponding one of the plurality of encoder convolutional stages, and

[0426] At least a first decoder convolution is performed based at least in part on the pool mask.

[0427] 31. A method for classifying tissue regions of potential amputation sites using a convolutional neural network, the method comprising:

[0428] Receive data representing multiple images of a potential amputation site, the data representing multiple images including a first subset, each of the multiple images being captured using light of a different wavelength among multiple different wavelengths reflected from the potential amputation site;

[0429] The data is fed as a three-dimensional volume into the input layer of the convolutional neural network. The height and width of the three-dimensional volume correspond to the number of pixels in the height and width of each of the multiple images, and the depth corresponds to the number of images in the multiple images.

[0430] Perform at least one convolution on the three-dimensional volume;

[0431] The output of the at least one convolution is fed into the normalized exponential function layer of the convolutional neural network;

[0432] Based on the output of the normalized exponential function layer, classification values ​​for each pixel are generated across the height and width of the multiple images; and

[0433] Based on the classification values ​​for each pixel, a mapping of multiple tissue classifications of the tissue at the potential amputation site is generated.

[0434] 32. The method of alternative embodiment 31, wherein the plurality of images further comprises a second subset captured sequentially at different times at the same wavelength as each other.

[0435] 33. The method as described in alternatives 31 or 32, wherein generating the classification includes classifying each pixel as one of background, healthy tissue, diseased tissue, or necrotic tissue.

[0436] 34. The method of any one of alternatives 31 to 33, wherein the first subset of the plurality of images comprises eight images captured using light of different wavelengths of eight different wavelengths, and wherein the second subset of the plurality of images comprises hundreds of images captured sequentially at the same wavelength at a rate of 30 frames per second.

[0437] 35. The method of any one of alternatives 31 to 34, wherein performing at least one convolution includes performing multiple convolutions at multiple encoder convolution stages and multiple decoder convolution stages of the convolutional neural network, wherein a first encoder convolution stage of the multiple encoder convolution stages includes an input layer as a first convolutional layer.

[0438] 36. The method as described in alternative embodiment 35, wherein performing multiple convolutions includes:

[0439] At each of the plurality of encoder convolution stages:

[0440] Perform at least the first encoder convolution.

[0441] The output of the first encoder convolution is fed into the rectified linear unit layer, and

[0442] The output of the rectified linear unit layer is downsampled using a max-pooling layer; and

[0443] At each of the plurality of decoder convolution stages:

[0444] Receive the pooling mask from the max-pooling layer of the corresponding one of the plurality of encoder convolutional stages, and

[0445] At least a first decoder convolution is performed based at least in part on the pool mask.

[0446] 37. The method as described in alternatives 35 or 36, wherein each of the plurality of encoder convolutional stages and each of the plurality of decoder convolutional stages comprises at least two convolutional layers, each convolutional layer being followed by a rectified linear unit layer.

[0447] 38. The method of any one of options 31 to 37, wherein the method of option 25 is used to train the convolutional neural network.

[0448] Implementation systems and terminology

[0449] The embodiments disclosed herein provide systems, methods, and apparatus for identifying, assessing, and / or classifying subject tissues. Those skilled in the art will recognize that these alternatives can be implemented using hardware, software, firmware, or any combination thereof.

[0450] It should be understood that, in all the foregoing tests, features, materials, characteristics, or groups of elements described in conjunction with a particular aspect, alternative, or example are applicable to any other aspect, alternative, or example described herein, unless both are not simultaneously applicable. All features disclosed in this specification (including any appended claims, abstract, and drawings), and / or all steps of any method or process disclosed herein, may be combined in any combination, except for at least some mutually exclusive combinations of these features and / or steps. The scope of protection is not limited to the details of any of the foregoing alternatives. The scope of protection extends to any new feature, any new combination of features, or any new step or combination of steps disclosed in this specification (including any appended claims, abstract, and drawings).

[0451] Although certain alternatives have been disclosed, these alternatives are presented by way of example only and are not intended to limit the scope of the invention. In fact, the novel methods and systems described herein can be implemented in many other forms. Furthermore, various omissions, substitutions, and changes can be made to the form of the methods and systems described herein. Those skilled in the art will recognize that in some alternatives, the actual steps taken in the processes shown and / or disclosed may differ from those shown in the figures. Depending on the alternative, some of the above-described steps may be omitted, and others may be added. Moreover, the features and properties of the specific alternatives disclosed above can be combined in different ways to form other alternatives, all of which are within the scope of this disclosure.

[0452] It should be understood that the use of designations such as "first," "second," etc., to refer to any element herein is generally not intended to limit the number or order of these elements. Rather, these designations may be used herein as a convenient way to distinguish two or more elements or instances of elements. Therefore, referring to the first element and the second element does not imply that only two elements may be used there or that the first element must precede the second element in some form. Furthermore, unless otherwise specified, a group of elements may include one or more elements.

[0453] Those skilled in the art will understand that information and signals can be represented using any variety of terms and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and slices that may be referenced in the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0454] Those skilled in the art will also recognize that any of the various examples, modules, processors, devices, circuits, and algorithm steps described in conjunction with the aspects disclosed herein can be implemented as electronic hardware (e.g., a digital implementation, an analog implementation, or a combination thereof designed using source code or some other technology), various forms of program or design code incorporating instructions (which may be referred to herein as "software" or "software module"), or a combination thereof. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above according to their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in different ways for various specific applications, but such a decision on implementation should not be construed as departing from the scope of this disclosure.

[0455] The various example logics, components, modules, and circuits disclosed herein and illustrated in conjunction with the accompanying drawings may be implemented or performed within or by an integrated circuit (IC), access terminal, or access point. An IC may include a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, electronic components, optical components, mechanical components, or any combination thereof, designed to perform the functions described herein, and may execute code or instructions residing within, outside, or both within and outside the IC. Logic blocks, modules, and circuits may include antennas and / or transceivers communicating with various components within a network or device. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. The functionality of a module may also be implemented in some other ways taught herein. The functions described herein (e.g., relating to one or more of the accompanying drawings) may in some respects correspond to functions similarly written as "for a means of..." in the appended claims.

[0456] If implemented in software, the functionality can be stored or transferred as one or more instructions or code to a computer-readable medium. The steps of the methods or algorithms disclosed herein can be implemented in a processor-executable software module that may reside in a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium capable of transferring a computer program from one place to another. Storage media can be any available medium accessible to a computer. By way of example, and not limitation, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium accessible to a computer for storing desired program code in the form of instructions or data structures. Furthermore, any connector may also be appropriately referred to as a computer-readable medium. Optical discs and disks, as used herein, include compact discs (CDs), laser discs, optical discs, digital optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically reproduce data, while optical discs optically reproduce data by laser. The above combinations should also be included within the scope of computer-readable media. Furthermore, the execution of a method or algorithm may reside as one of code and instructions, any combination of code and instructions, or a set of code and instructions in a mechanically readable and computer-readable medium, which may be incorporated into a computer program product.

[0457] It should be understood that any specific order or hierarchy of steps in any disclosed process is an example of a sample method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged, which remains within the scope of this disclosure. The appended method claims present elements of the steps based on the sample order, but this does not imply limitation to the proposed specific order or hierarchy.

[0458] Various modifications to the embodiments described herein will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but rather to be given the widest scope consistent with the claims, principles, and novel features disclosed herein.

[0459] Some features described in this specification in the context of individual implementations can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented separately in multiple implementations or in any suitable sub-combination. Furthermore, although features are described above as being able to function in certain combinations, and were even originally proposed in this way, in some cases one or more features in the claimed combination can be removed from the combination, and the claimed combination can refer to a sub-combination or a variation of a sub-combination.

[0460] Similarly, although operations are illustrated in a specific order in the accompanying drawings, this should not be construed as requiring the operations to be performed in the specific order or sequence shown, or requiring all of the shown operations to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the above embodiments should not be construed as requiring such separation in all embodiments, but rather as meaning that the program components and system can generally be integrated into a single software product or packaged into multiple software products. Additionally, other embodiments are also within the scope of the appended claims. In some cases, the steps recited in the claims can be performed in a different order and still achieve the desired effect.

[0461] While this disclosure includes certain alternatives, examples, and applications, those skilled in the art will understand that this disclosure extends beyond the specifically disclosed alternatives to other alternatives and / or applications, obvious variations, and equivalents thereof, including alternatives that do not provide all the features and advantages described herein. Therefore, the scope of this disclosure is not intended to be limited to the specific content of the preferred alternatives herein, but may be limited by the claims set forth herein or to be proposed in the future. For example, the following alternatives, in addition to those set forth herein, are also intended to be included within the scope of this disclosure.

[0462] In the foregoing description, specific details are provided to offer a complete understanding of the examples. However, those skilled in the art will understand that these examples can also be implemented without these specific details. For example, electronic components / devices may be shown in blocks to avoid obscuring the examples with unnecessary detail. In other cases, these components, other structures, and techniques may be shown in detail to further explain the examples.

[0463] The above description of the disclosed embodiments is provided to enable those skilled in the art to implement or use the invention. Various variations of these embodiments will be apparent to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not intended to be limited to the embodiments shown herein, but is to be given the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An organizational classification system, comprising: At least one light emitter is configured to emit light of each of a plurality of wavelengths to illuminate a tissue region, and each of the at least one light emitter is configured to emit light that is spatially uniform. At least one light detection element is configured to collect light emitted from the at least one light emitter and reflected from the tissue region; One or more processors, which communicate with the at least one emitter and the at least one photodetector, and are configured to: Identify at least one patient health indicator value corresponding to a patient with the said tissue region. A classifier is selected from a plurality of classifiers using the at least one patient health indicator value, and each of the plurality of classifiers is trained from different subsets of the training dataset, wherein the classifier is selected based on a classifier that has been trained using a subset of the training dataset, the subset including data from other patients having the at least one patient health indicator value. Control the at least one light emitter to sequentially emit light of each wavelength among a plurality of wavelengths; Receive multiple signals from the at least one photodetector element, a first subset of the multiple signals representing light emitted at the multiple wavelengths and reflected from the tissue region; Based on at least a portion of the plurality of signals, an image having a plurality of pixels depicting the tissue region is generated; For each of the plurality of pixels depicting the tissue region: Based on a first subset of the plurality of signals, determine the reflection intensity value of the pixel at each of the plurality of wavelengths, and The healing classification score of the pixel associated with tissue healing potential is determined by inputting the reflection intensity value into the classifier; and Based on the healing classification score of each pixel, a total healing score associated with the tissue healing potential is generated for the plurality of pixels depicting the tissue region.

2. The system of claim 1, wherein a second subset of the plurality of signals represents light of the same wavelength reflected from the tissue region at a plurality of different times, and wherein for each of the plurality of pixels depicting the tissue region, the one or more processors are configured to: Based on a second subset of the plurality of signals, the PPG amplitude value of the pixel is determined; and Furthermore, the classification of the pixel is determined by inputting the PPG amplitude value into the classifier.

3. The system of claim 1, wherein the one or more processors are configured to output a visual representation of the image for display to a user.

4. The system of claim 3, wherein the visual representation includes the image having pixels displayed with a specific visual representation selected based on the tissue classification of the pixels, wherein pixels associated with each of a plurality of tissue categories are displayed with a different visual representation.

5. The system of claim 4, wherein the plurality of tissue categories includes a living tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

6. The system of claim 5, wherein generating the image includes identifying regions of the plurality of pixels, the regions depicting tissue associated with at least one of a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

7. The system of claim 6, wherein the one or more processors are configured to output an image for display, wherein a plurality of pixels depicting the tissue region are displayed in different colors according to the associated tissue category.

8. The system of claim 7, wherein the one or more processors are configured to determine a recommended location for amputation based on the identified region.

9. The system of claim 1, wherein the one or more processors are configured to determine the melanin index of the tissue region based on the plurality of signals.

10. The system of claim 9, wherein the one or more processors are configured to select the classifier based at least on the melanin index.

11. The system of claim 1, wherein the one or more processors are further configured to divide training data into groups according to the level of a classifier variable, confirm the classifier performance corresponding to each of the levels, and generate a separate classifier at each of the levels based on the confirmation.

12. An organizational classification method, comprising: A classifier is selected from multiple classifiers based on at least one patient health indicator value, and each of the multiple classifiers is trained from different subsets of the training dataset, wherein the classifier is selected based on a classifier that has been trained using a subset of the training dataset, the subset including data from other patients having the at least one patient health indicator value. Receive multiple signals from at least one photodetector element, the at least one photodetector element being positioned to receive light reflected from a tissue region, a first subset of the multiple signals representing light emitted at multiple wavelengths and reflected from the tissue region; Based on at least a portion of the plurality of signals, an image having a plurality of pixels depicting the tissue region is generated; For each of the plurality of pixels depicting the tissue region: Based on a first subset of the plurality of signals, determine the reflection intensity value of the pixel at each of the plurality of wavelengths, and The healing classification score of the pixel associated with tissue healing potential is determined by inputting the reflection intensity value into a selected classifier; and Based on the healing classification score of each pixel, a total healing score associated with the tissue healing potential is generated for the plurality of pixels depicting the tissue region.

13. The method of claim 12, wherein a second subset of the plurality of signals represents light of the same wavelength reflected from the tissue region at a plurality of different times, and for each of the plurality of pixels depicting the tissue region, the method further comprises: Based on a second subset of the plurality of signals, the PPG amplitude value of the pixel is determined; and Furthermore, the classification of the pixel is determined by inputting the PPG amplitude value into the classifier.

14. The method of claim 12, further comprising determining the melanin index of the tissue region based on the plurality of signals.

15. The method of claim 14, further comprising selecting the classifier based at least on the melanin index.

16. The method of claim 12, wherein generating the image includes identifying regions of the plurality of pixels, the regions depicting tissue associated with at least one of a live tissue category, a necrotic tissue category, and a tissue category with small vessel disease.

17. The method of claim 16, further comprising outputting an image for display, wherein a plurality of pixels depicting the tissue region are displayed in different colors according to an associated tissue category.

18. The method of claim 12, wherein, The method further includes dividing the training data into groups according to the level of the classifier variable, confirming the classifier performance corresponding to each of the levels, and generating a separate classifier for each of the levels based on the confirmation.

Images