Skill learning for dynamic treatment rules

Abstract

A method and system for training a medical treatment machine learning model includes segmenting a patient's trajectory, which includes a sequence of patient conditions and treatment actions (304). A machine learning model is trained based on the patient's trajectory segments (308), including a prototype layer that learns prototype vectors representing each class of the trajectory segment, and an imitation learning layer that learns a policy for selecting treatment actions based on input states and skill embeddings.

Description

[Technical Field]

[0001] Related Application Information This application claims priority to U.S. Patent Application No. 63 / 434,133, filed December 21, 2022, and U.S. Patent Application No. 18 / 539,506, filed December 14, 2023, each of which is incorporated by reference in its entirety. [Background technology]

[0002] The present invention relates to machine learning systems, and more particularly to systems that learn skills from historical records of medical treatments. 2. Description of Related Art

[0003] A dynamic treatment rule is a set of sequential treatment decision rules that can be used to provide recommendations for treating a patient. While it is possible to construct dynamic treatment rules from historical records using machine learning systems, the rationale underlying those past actions may not be interpretable, creating challenges in real-world clinical scenarios. Furthermore, a single policy may not be sufficient to address changing patient needs. Summary of the Invention

[0004] A method for training a healthcare treatment machine learning model includes segmenting a patient's trajectory, which includes a sequence of patient conditions and treatment actions. The machine learning model is trained based on the patient's trajectory segments, including a prototype layer that learns prototype vectors representing each class of the trajectory segment, and an imitation learning layer that learns a policy for selecting treatment actions based on input states and skill embeddings.

[0005] A system for training a healthcare treatment machine learning model includes a hardware processor and a memory that stores a computer program that, when executed by the hardware processor, causes the hardware processor to segment a patient's trajectory, which includes a sequence of patient conditions and treatment actions, and train a machine learning model based on the patient's trajectory segments, the machine learning model including a prototype layer that learns prototype vectors that represent each class of the trajectory segments, and an imitation learning layer that learns a policy for selecting a treatment action based on an input state and a skill embedding.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0007] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0008] [Figure 1] 1 is a block diagram of a healthcare facility where skill imitation is used to guide treatment, according to one embodiment of the present invention.

[0009] [Figure 2] 1 is a diagram of a patient with an automated treatment system, according to one embodiment of the present invention.

[0010] [Figure 3] FIG. 1 is a block diagram of a skill imitation model operable on a segment of a patient's trajectory, according to an embodiment of the present invention.

[0011] [Figure 4] FIG. 1 is a block / flow diagram of a method for training and using a skill imitation model for patient treatment, according to one embodiment of the present invention.

[0012] [Figure 5]FIG. 1 is a block / flow diagram of a method for monitoring and treating a patient using a skill imitation model, according to one embodiment of the present invention.

[0013] [Figure 6] FIG. 1 is a block diagram of a computer system capable of training and using a skill imitation model to treat patients, according to one embodiment of the present invention.

[0014] [Figure 7] FIG. 1 is a diagram of a neural network architecture that can be used as part of a skill imitation model, according to one embodiment of the present invention.

[0015] [Figure 8] FIG. 1 is a diagram of a deep neural network architecture that can be used as part of a skill imitation model, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Imitation learning can be used to learn mappings between states and actions when devising patient treatments in healthcare settings. Imitation learning aims to reproduce expert behaviors, such as diagnostic and therapeutic actions performed by physicians, based on demonstrations from a series of recordings. To do so, an interpretable sequence modeling framework can be used to identify expert trajectories based on sequence data with temporal characteristics. Learning and inference are performed at the segment level to capture temporal variations in states and identify transferable skills between different trajectories. This allows trajectories to be expressed in multiple segments, providing flexibility compared to trajectory-level formulations. By aggregating the prototypes learned at each segment, the resulting skills serve as conditional information that unconditionally guides the policy network to distinguish patterns and provide accurate treatment recommendations. These recommended treatments can then be further interpreted at the segment level by following exemplar segments.

[0017] Thus, leveraging segment-level expert demonstrations and results in representative and transferable skills across different trajectories, an interpretable skill learning model is provided for learning treatment policies. The model learns to capture exemplar segments and constructs faithful skill embeddings for imitation learning tasks.

[0018] 1, an illustration of skill learning is shown in the context of a healthcare facility 100, which may be used to assist in the monitoring and treatment of multiple patients, for example, to accommodate changes in the patients' specific healthcare needs. The healthcare facility may include one or more medical professionals 102 that provide information related to events and measurements of system status to a skill mimicry system 108. The treatment system 104 may further be designed to monitor the patient's condition to generate a medical record 106 and automatically manage and adjust treatment as needed.

[0019] The skill mimicry system 108 learns the skills applied by the medical professional 102 in response to a patient's clinical condition based on information drawn from at least the medical professional 102, the treatment system 104, and the medical record 106. For example, the medical record 106 may include the patient's past health conditions (e.g., biometric information and symptom descriptions) and actions taken by the medical professional 102 in response to those conditions.

[0020] The different elements of the healthcare facility 100 can communicate with each other via the network 110, for example, using any suitable wired or wireless communication protocol and medium. Thus, the skill mimicking system 108 can access the remotely stored medical records 106, communicate with the treatment system 104, receive instructions for the medical professionals 102, and send reports. In particular, the skill mimicking system 108 can automatically cause changes in the patient's treatment in response to new information obtained from the medical records 106 by sending instructions to the treatment system 104.

[0021] In some cases, the skill mimicry system 108 may generate a specific treatment plan for the patient, including a prescription plan including medications to help treat the patient, a meal plan to address the patient's dietary needs, a rehabilitation plan to provide physical therapy and other activities necessary for the patient to recover, and a discharge plan indicating whether the patient can return home, should remain, or should be transferred to another health care facility. Thus, the output of the skill mimicry system 108 may include one or a combination of the above automatic treatment and plan outputs.

[0022] Referring now to FIG. 2, a patient 202 is shown in the context of a health care system. For example, the patient 202 may be undergoing a hemodialysis (also simply referred to as "dialysis") session. During dialysis, a dialyzer 204 automatically draws the patient's blood, processes and cleanses the blood, and reintroduces the cleansed blood into the patient's body. Dialysis may take four hours and may occur every three days, although other durations and periods are contemplated. While dialysis is specifically intended, it should be understood that any suitable medical treatment or monitoring may be used instead.

[0023] Before, during, and after a dialysis session, the patient 202 may experience treatment-related health events. Such health events, which may be dangerous to the patient 202, can be predicted based on knowledge of previous health events and the patient's current health metrics. The recommendation 208 can further include information related to the type of predicted event and measurements of the patient's condition. It is specifically contemplated that the recommendation be made before the dialysis session begins so that treatment can be adjusted.

[0024] Recommendations can be made based on various input information. Some of the information includes a patient's static profile, including information such as age, gender, when dialysis was initiated, and previous health events. The information also includes dialysis measurement records, which may be taken for each dialysis session, as well as dynamic data such as blood pressure, weight, venous pressure, blood test measurements, and cardiothoracic ratio (CTR). Blood test measurements are taken periodically, such as twice a month, and can measure factors such as albumin, glucose, and platelet count. CTR can also be measured periodically, such as monthly. Dynamic information can also be recorded during a dialysis session, using sensors on the dialysis machine 204, for example. The dynamic information can be modeled as a time series over each frequency.

[0025] Additionally, the system itself may be monitored within a healthcare environment. For example, the operating parameters of a dialysis machine 204 or other system within a hospital or other healthcare facility are often monitored to predict events, as described below, along with the history of past events in the system.

[0026] During treatment, the condition of the patient 202 may be continuously monitored, such as by tracking the patient's heart rate and other vital signs. If the patient's vital signs indicate an impending or ongoing adverse health event, treatment may be modified accordingly. For example, the treatment system may automatically administer medication or stop treatment in response to a negative health event.

[0027] To train the skill imitation system 108, the medical record 106 can include a set of physician demonstration trajectories, each of which consists of a set of state-action pairs (s t ,a t ), where s t denotes the state of a given patient at time t, and a tindicates the therapeutic action taken by the medical professional 102. The skill imitation system 108 learns a policy that can replicate the therapeutic action taken by the medical professional 102. The imitation learning can be built on step-level demonstrations without considering the evolving patient symptoms and corresponding treatments.

[0028] The imitation learning task may instead be formulated at the level of a sequence of states to exploit the continuous nature of the demonstrations. Thus, each trajectory may be divided into segments, and imitation learning may be performed on each segment. Skills in each segment may be representative and transferable across different trajectories. Because each state within a segment is accompanied by previous state segments from the same trajectory, the dynamics between patient state and treatment demonstrations can also be exploited at the segment level.

[0029] A set of segments split from the original trajectory

number

[0030] 3, an imitation learning model is shown. The architecture of this model may include a segment embedding layer 304, a prototype layer 306, and an imitation learning layer 308. For simplicity, the subscript j, which indicates the order of instances, is omitted herein.

[0031] The input segment 302 relating to the patient's condition at step t is expressed as follows:

number

number

number

number

number

number

number

number

number

[0032] The segment embedding layer can also be implemented in other ways, for example, using long-short term memory (LSTM) or gated recurrent unit (GRU) architectures, or using a transformer architecture. However, in dynamic therapeutic domains, embedding long segments is rare, so a 1D convolutional layer may be more efficient and effective for extracting embeddings related to short segments.

[0033] The prototype layer 306 includes k prototype vectors

number

number

[0034] Each prototype vector represents a class of exemplary segments that reflects the patient's condition at a certain stage. t The similarity score between and each prototype vector is determined as follows:

number

number

number

[0035] The similarity scores for all prototype-embedding pairs are scaled between [0,1], so that the resulting scaled score for the ith prototype-embedding pair is:

number

number

number

number

number

number

number

number

number

number

[0036] The imitation learning layer 308 is a flat policy network π parameterized by θ. θ (a t |s t ) and s t From a t Mapping to skills embedding

number

number

number

number

[0037] 4, a method for training and using a skill imitation model is shown. Block 402 trains the model using a set of historical training examples comprising sequences of patient states and actions taken by medical professionals, which constitute trajectories. Once the model is trained, it is deployed to the skill imitation system 108 of the healthcare facility 100, where it can be used to monitor and treat 406 the patient, coordinating actions by the treatment system 104 and the medical professional 102.

[0038] During training 402, trajectories are divided into fixed-size segments from past data. Imitation learning is performed to train a policy to minimize the objective function, as described above. L=L 1M +λ1L cluster +λ2L evidence +λ3L diversity where λ1, λ2, and λ3 are weighting coefficients ranging from 0 to 1. The objective function includes an imitation learning term L 1M Given a batch of segments of size n, the situational policy aims to mimic the doctor's demonstration at the segment level in a supervised manner.

number

[0039] To improve the interpretability of the skill learning model, three regularization factors can be used in learning the prototype vectors, including the clustering structure of the segment embeddings, the evidence between the segments and the prototypes, and a term related to the diversity of the prototypes. The clustering structure regularization is based on regularizing the segment embeddings to get as close as possible to the closest prototypes.

number

number

[0040] Prototype-segment evidence regularization imposes a dual optimization objective on the segment embeddings and the prototype vectors: regularization encourages each prototype vector to be as similar as possible to its segment embedding.

number

[0041] The similarity between each pair of prototype vectors may be penalized, as indistinguishable prototype vectors representing similar patients may be redundant. On the other hand, encouraging diversity in prototypes improves generalization when encountering new segments and trajectories. A diversity regularization term can be imposed as follows:

number

[0042] Training 402 continues until the training loss L converges, and the prototype vectors are optimized to be close to the segment embeddings from the training data. However, at this stage, the prototype vectors cannot be interpreted because there is no correspondence between the prototype vectors and the actual segments. To associate the prototypes with the segments in the training data, each prototype

number

number

[0043] 5, further details regarding monitoring and treatment 406 are shown. Block 502 measures updated patient status using, for example, treatment system 104, and adds the updated patient status information to medical record 106. This information may include health indicators such as heart rate, blood pressure, and lab test results, as well as the patient's medication and treatment history. The collected data is used to monitor the patient's status. t Configure.

[0044] At block 504, a new patient condition is input into the trained model, which generates a corresponding skill output. The trained model converts the condition into an embedding and uses the embedding to identify the most appropriate prototype. Imitation learning is activated in the model, which generates an action plan based on both the input condition and the identified prototype. The action plan may include a recommendation or treatment instruction, such as specifying a specific medication dosage. The output skill indicates the treatment to be performed, and block 506 automatically performs the treatment, for example, by sending an instruction to the treatment system 502. The treatment may then be provided to the medical professional 102. This treatment information assists the medical professional 102 in making decisions to manage the patient.

[0045] 6, an exemplary computing device 600 is shown, in accordance with an embodiment of the present invention. The computing device 600 is configured to perform skill recognition.

[0046] Computing device 600 may be embodied as any type of computational or computing device capable of performing the functions described herein, such as, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a user electronic device. Additionally or alternatively, computing device 600 may be embodied as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically disaggregated computing device.

[0047] 6, computing device 600 illustratively includes a processor 610, an input / output subsystem 620, a memory 630, a data storage device 640, and a communications subsystem 650, and / or other components and devices typically found in a server or similar computing device. In other embodiments, computing device 600 may include other or additional components (e.g., various input / output devices) such as those typically found in a server computer. Furthermore, in some embodiments, one or more of the illustrative components may be incorporated into or otherwise form part of another component. For example, memory 630, or portions thereof, may be incorporated into processor 610 in some embodiments.

[0048] The processor 610 may be embodied as any type of processor capable of performing the functions described herein, and may be embodied as a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0049] Memory 630 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, memory 630 may store various data and software used during operation of computing device 600, such as an operating system, applications, programs, libraries, and drivers. Memory 630 is communicatively coupled to processor 610 via I / O subsystem 720 and may be embodied as circuits and / or components for facilitating input / output operations with processor 610, memory 630, and other components of computing device 600. For example, I / O subsystem 620 may be embodied as or otherwise include a memory controller hub, an input / output control hub, a platform controller hub, integrated control circuitry, firmware devices, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems for facilitating input / output operations. In some embodiments, the I / O subsystem 620 may form part of a system-on-chip (SOC) and be integrated with the processor 610, memory 630, and other components of the computing device 600 on a single integrated circuit chip.

[0050] The data storage device 640 may be embodied as any type of device or device configured for short-term or long-term storage of data, such as, for example, a memory device and circuitry, a memory card, a hard disk drive, a solid-state drive, or other data storage device. The data storage device 640 may store program code 640A for training a model, program code 640B for predicting events, and / or program code 640C for executing corrective actions in response to predicted events. The communications subsystem 650 of the computing device 600 may be embodied as any network interface controller or other communications circuitry, device, or collection thereof that can enable communications between the computing device 600 and other remote devices over a network. The communications subsystem 650 may be configured to achieve such communications using any one or more communications technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.).

[0051] As shown, computing device 600 may also include one or more peripheral devices 660. Peripheral devices 660 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 660 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0052] Of course, computing device 600 may include other elements (not shown) or omit certain elements, as would readily occur to one skilled in the art. For example, various other sensors, input devices, and / or output devices may be included in computing device 600, depending on the particular implementation of the same, as would be readily understood by one skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, additional processors, controllers, memory, etc. may be utilized in a variety of configurations. These and other variations of processing system 600 will be readily contemplated by one skilled in the art in view of the teachings of the present invention provided herein.

[0053] 7 and 8, exemplary neural network architectures are shown that can be used to implement portions of the present model, such as the segment embedding layer 304. Neural networks are generalized systems whose functionality and accuracy improve with exposure to additional empirical data. Neural networks learn by being exposed to empirical data. During training, the neural network stores and adjusts weights that are applied to input empirical data. Applying the adjusted weights to the data can identify that data belongs to a specific predefined class from a set of classes and output the probability that the input data belongs to each class.

[0054] Empirical data (also called training data) from a set of examples is formatted as a string of values ​​and fed to the input of a neural network. Each example is associated with a known outcome or output. Each column is represented as an (x,y) pair, where x represents the input data and y represents the known output. The input data can be of various data types and contain multiple different values. The network can have one input node for each value that makes up the example input data, and a separate weight can be applied to each input value. The input data can be formatted, for example, as a vector, array, or string, depending on the architecture of the neural network being built and trained.

[0055] Neural networks "learn" by comparing neural network outputs generated from input data with known values ​​from examples and adjusting stored weights to minimize the difference between the output value and the known value. Adjustments can be made to the stored weights through backpropagation, where the weight's influence on the output value is determined by calculating a mathematical gradient and adjusting the weight in a way that shifts the output toward the minimum difference. This optimization, called gradient descent, is a non-limiting example of how training occurs. A subset of examples with known values ​​that were not used in training can be used to test and validate the accuracy of the neural network.

[0056] During operation, a trained neural network can be used on new data not previously used for training or validation through generalization. The adjusted neural network weights can be applied to the new data, and the weights estimate the function developed from the training examples. The parameters of the estimated function, captured by the weights, are based on statistical inference.

[0057] In a layered neural network, nodes are arranged in layers. An exemplary simple neural network has an input layer 720 of source nodes 722 and a single computational layer 730 with one or more computational nodes 732 that also function as output nodes, with a single computational node 732 for each possible category into which an input example may be classified. The input layer 720 may have a number of source nodes 722 equal to the number of data values ​​712 in the input data 710. The data values ​​712 in the input data 710 may be represented as a column vector. Each computational node 732 in the computational layer 730 generates a weighted linear combination of values ​​from the input data 710 provided to the input node 720 and applies a differentiable nonlinear activation function to the sum. An exemplary simple neural network can perform classification on linearly separable examples (e.g., patterns).

[0058] A deep neural network, such as a multilayer perceptron, may have an input layer 720 of source nodes 722, one or more computational layers 730 with one or more computational nodes 732, and an output layer 740 with one output node 742 for each possible category into which an input example may fall. The input layer 720 may have a number of source nodes 722 equal to the number of data values ​​712 in the input data 710. The computational nodes 732 in the computational layer 730 are also referred to as hidden layers because they are located between the source nodes 722 and the output nodes 742 and are not directly observed. Each node 732, 742 in the computational layer generates a weighted linear combination of values ​​from the values ​​output by the nodes in the previous layer and applies a nonlinear activation function that is differentiable over the range of the linear combination. The weights applied to the values ​​from each previous node may be, for example, w1, w2, ... w n-i ,w n The output layer provides the network's overall response to the input data. Deep neural networks can be fully connected, where each node in a computational layer is connected to every node in the previous layer, or the connections between layers can have other configurations. If links between nodes are missing, the network is said to be partially connected.

[0059] Training a deep neural network involves two phases: a forward phase, in which the weights of each node are fixed and inputs are propagated through the network, and a backward phase, in which error values ​​are propagated back through the network and the weight values ​​are updated.

[0060] Computational nodes 732 in one or more computational (hidden) layers 730 perform nonlinear transformations on the input data 712 to generate a feature space, where classes or categories may be more easily separated in the feature space than in the original data space.

[0061] The embodiments described herein may be entirely hardware, entirely software, or contain both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0062] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. A computer-usable or computer-readable medium may include any apparatus that stores, communicates, propagates, or transports a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or propagation medium. The medium may include computer-readable storage media such as semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks.

[0063] Each computer program can be tangibly stored on a machine-readable storage medium or device (e.g., program memory or magnetic disk) readable by a general-purpose or special-purpose programmable computer to configure and control the operation of the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system of the present invention can also be considered to be embodied in a computer-readable storage medium configured with a computer program, where the configured storage medium causes the computer to operate in a particular, predetermined manner to perform the functions described herein.

[0064] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory employed during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I / O controllers.

[0065] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0066] As employed herein, the terms “hardware processor subsystem” or “hardware processor” can refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The data processing elements may be included in a central processing unit, a graphics processing unit, and / or a separate processor- or computing element-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more on-board memories (e.g., cache, dedicated memory array, read-only memory, etc.). In some embodiments, a hardware processor subsystem may include one or more memories (e.g., ROM, RAM, basic input / output system (BIOS), etc.) that may be on-board or off-board or dedicated for use by the hardware processor subsystem.

[0067] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code for achieving a specified result.

[0068] In other embodiments, the hardware processor subsystem may include dedicated, specialized circuitry that performs one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0069] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0070] References herein to "one embodiment" or "one embodiment" of the present invention, as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in one embodiment," as well as any other variations thereof, in various places throughout this specification do not necessarily all refer to the same embodiment. However, it should be understood that features of one or more embodiments may be combined given the teachings of the present invention provided herein.

[0071] For example, in the case of "A / B," the use of any of the following " / ," "and / or," "at least one," such as "A and / or B" or "at least one of A and B" will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are listed items.

[0072] The foregoing is understood in all respects to be illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are interpreted in accordance with the full breadth permitted by the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Various other feature combinations could be implemented by those skilled in the art without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is set forth in the appended claims.

Claims

1. 1. A computer-implemented method for training a healthcare treatment machine learning model, comprising: Segmenting 304 a patient trajectory comprising a sequence of patient states and treatment actions; training (308) a machine learning model based on the segments of the patient's trajectory, the machine learning model including a prototype layer that learns prototype vectors representing each class of trajectory segments, and an imitation learning layer that learns a policy for selecting a therapeutic action based on input states and skill embeddings.

2. 10. The method of claim 1, The method further comprising embedding the segmented patient trajectories using a segment embedding layer of the machine learning model.

3. 3. The method of claim 2, The segment embedding layer includes a multi-layer perceptron and a one-dimensional convolutional layer.

4. 10. The method of claim 1, Measuring patient status information; selecting a therapeutic action based on the measured state information and the skill predicted by the trained model; and informing a medical professional of said treatment action to assist said medical professional in making patient management decisions.

5. 10. The method of claim 1, The method, wherein the skill embedding comprises a weighted combination of the prototype vectors based on how similar the prototype vectors are to the segmented patient trajectories.

6. 10. The method of claim 1, The method, wherein training the machine learning model includes minimizing a loss function including an imitation learning term, a clustering structure regularization term, a prototype segment evidence regularization term, and a diversity regularization term.

7. 7. The method of claim 6, The imitation learning term is [Equation 1] where m is the length of the segment, n is the number of segments, and π Ｅは It is an expert policy, [Equation 2] is the action performed at step t of segment j, [Equation 3] is the patient's state at step t of segment j, and π θ is the learned policy, [Equation 4] is the skill embedding at step t for segment j.

8. 7. The method of claim 6, The clustering structure regularization term is [Equation 5] and the prototype segment evidence regularization term is [Equation 6] and the diversity regularization term is [Equation 7] where n is the number of segments and k is the number of prototype vectors, [Equation 8] is the i-th prototype vector, [Equation 9] is the segment embedding of segment j at step t, and d min is the proximity threshold.

9. 10. The method of claim 1, The method, wherein the treatment includes at least one of a prescription plan, a dietary plan, a rehabilitation plan, and a discharge plan.

10. 10. The method of claim 1, The method, wherein the therapeutic action includes instructions to a therapeutic system to automatically administer the therapy to the patient.

11. 1. A system for training a healthcare treatment machine learning model, comprising: a hardware processor (610); and a memory (640) for storing a computer program, the computer program, when executed by the hardware processor, causing the hardware processor to: Segmenting 304 a patient trajectory comprising a sequence of patient states and treatment actions; training (308) a machine learning model based on the patient's trajectory segments, the machine learning model including a prototype layer that learns prototype vectors representing each class of trajectory segments, and an imitation learning layer that learns a policy for selecting a therapeutic action based on input states and skill embeddings.

12. 12. The system of claim 11, The computer program further causes the hardware processor to embed the segmented patient trajectories using a segment embedding layer of the machine learning model.

13. 13. The system of claim 12, The segment embedding layer is a system including a multi-layer perceptron and a one-dimensional convolution layer.

14. 12. The system of claim 11, The prototype layer is a system that determines the similarity between the segments and the prototype vectors.

15. 12. The system of claim 11, The system, wherein the skill embedding comprises a weighted combination of the prototype vectors based on how similar the prototype vectors are to the segmented patient trajectory.

16. 12. The system of claim 11, The computer program further causes the hardware processor to minimize a loss function including an imitation learning term, a clustering structure regularization term, a prototype segment evidence regularization term, and a diversity regularization term.

17. 17. The system of claim 16, The imitation learning term is [Equation 10] where m is the length of the segment, n is the number of segments, and π Ｅ is the expert policy, [0011] is the action performed at step t of segment j, [0012] is the patient's state at step t of segment j, and π θ is the learned policy, [0013] is the skill embedding at step t of segment j in the system.

18. 17. The system of claim 16, The clustering structure regularization term is [0014] and the prototype segment evidence regularization term is [Equation 15] and the diversity regularization term is [0016] where n is the number of segments and k is the number of prototype vectors, [Equation 17] is the i-th prototype vector, [Equation 18] is the segment embedding of segment j at step t, and d min is the proximity threshold of the system.

19. 12. The system of claim 11, The system, wherein the treatment includes at least one of a prescription plan, a meal plan, a rehabilitation plan, and a discharge plan.

20. 12. The system of claim 11, The treatment action includes instructions to a treatment system to automatically administer treatment to a patient.

Images