Skills learning for dynamic treatment rules

Abstract

A method and system for training a medical treatment machine learning model includes segmenting a patient's trajectory, which includes a sequence of patient conditions and treatment actions (304). A machine learning model is trained based on the patient's trajectory segments (308), including a prototype layer that learns prototype vectors representing each class of the trajectory segment, and an imitation learning layer that learns a policy for selecting treatment actions based on input states and skill embeddings.

Description

[Technical Field]

[0001] Related application information This application claims priority to U.S. Patent Application No. 63 / 434,133, filed on 21 December 2022, and U.S. Patent Application No. 18 / 539,506, filed on 14 December 2023, both of which are incorporated herein by reference in their entirety. [Background technology]

[0002] This invention relates to a machine learning system, and more particularly to a system for learning skills from medical treatment history records. Description of related technologies

[0003] Dynamic treatment rules are a set of sequential treatment decision rules that can be used to provide suggestions for treating a patient. While it is possible to construct dynamic treatment rules from historical records using machine learning systems, the rationale behind those past actions may not be interpretable, leading to difficulties in real-world clinical scenarios. Furthermore, a single policy may be insufficient to address changing patient needs. [Overview of the Initiative]

[0004] A method for training a healthcare treatment machine learning model involves segmenting a patient's trajectory, which includes sequences of patient states and treatment actions. The machine learning model is trained on segments of the patient's trajectory, and includes a prototype layer that learns prototype vectors representing each class of the trajectory segment, and an imitation learning layer that learns policies for selecting treatment actions based on input states and skill embeddings.

[0005] A system for training a healthcare treatment machine learning model includes a hardware processor and memory for storing a computer program. When the computer program is executed by the hardware processor, it causes the hardware processor to segment the patient's trajectory, which includes sequences of patient states and treatment actions, and trains a machine learning model based on these segments of the patient's trajectory. This model includes a prototype layer that learns prototype vectors representing each class of the trajectory segment, and an imitation learning layer that learns policies for selecting treatment actions based on input states and skill embeddings.

[0006] These and other features and advantages will become apparent from the following detailed description of the exemplary embodiment, which will be read in conjunction with the attached drawings. [Brief explanation of the drawing]

[0007] This disclosure provides further details in the following description of preferred embodiments with reference to the following figures.

[0008] [Figure 1] This is a block diagram of a healthcare facility in which skill imitation is used to guide treatment, according to one embodiment of the present invention.

[0009] [Figure 2] This is a diagram of a patient with an automated treatment system according to one embodiment of the present invention.

[0010] [Figure 3] This is a block diagram of a skill imitation model that can operate on segments of a patient's trajectory, according to one embodiment of the present invention.

[0011] [Figure 4] This is a block / flow diagram of how to train and use a skill imitation model for patient treatment, according to one embodiment of the present invention.

[0012] [Figure 5]A block / flow diagram of a method for monitoring and treating a patient using a skill imitation model according to an embodiment of the present invention.

[0013] [Figure 6] A block diagram of a computer system that can train and use a skill imitation model for treating a patient according to an embodiment of the present invention.

[0014] [Figure 7] A diagram of a neural network architecture that can be used as part of a skill imitation model according to an embodiment of the present invention.

[0015] [Figure 8] A diagram of a deep neural network architecture that can be used as part of a skill imitation model according to an embodiment of the present invention.

MODE FOR CARRYING OUT THE INVENTION

[0016] Imitation learning can be used to learn the mapping between states and actions when devising patient treatment in the context of health management. Imitation learning aims to reproduce expert actions such as diagnoses and treatment actions performed by doctors based on demonstrations from a series of records. For this purpose, an interpretable sequence modeling framework can be used to identify the trajectories of experts based on sequence data with temporal characteristics. Learning and inference are performed at the segment level, capturing the temporal variations of states and identifying skills that can be transferred between different trajectories. As a result, trajectories can be represented by multiple segments, providing flexibility compared to trajectory-level formulations. By aggregating the prototypes learned in each segment, the resulting skills function as conditional information that unconditionally guides a policy network to distinguish patterns and provide accurate treatment recommendations. These recommended treatments can further be interpreted at the segment level by following exemplary segments.

[0017] Therefore, by leveraging expert demonstrations at the segment level and results in representative and transferable skills across different trajectories, an interpretable skills learning model is provided for learning treatment policies. This model learns to capture exemplary segments and builds faithful skill embeddings for imitation learning tasks.

[0018] Referring next to Figure 1, a diagram of skill learning in the context of a healthcare facility 100 is shown. This can be used to support the monitoring and treatment of multiple patients, for example, to respond to changes in the specific healthcare needs of patients. The healthcare facility may include one or more healthcare professionals 102 who provide information related to events and system status measurement to the skill imitation system 108. The treatment system 104 may also be designed to monitor the patient's condition to create a medical record 106 and to automatically manage and adjust treatment as needed.

[0019] The skill imitation system 108 learns the skills applied by the medical professional 102 in response to the patient's condition, based on information extracted from at least the medical professional 102, the treatment system 104, and the medical record 106. For example, the medical record 106 may include the patient's past health status (e.g., biometric information and symptom descriptions) and the actions taken by the medical professional 102 in response to those conditions.

[0020] Different elements of the healthcare facility 100 can communicate with each other via the network 110, for example, using any suitable wired or wireless communication protocol and medium. Thus, the skill imitation system 108 can access remotely stored medical records 106, communicate with the treatment system 104, receive instructions for medical professionals 102, and send reports. In particular, the skill imitation system 108 can automatically trigger changes in patient treatment in response to new information obtained from medical records 106 by sending instructions to the treatment system 104.

[0021] In some cases, the skill imitation system 108 can generate a specific treatment plan for a patient, including a prescription plan containing medications that would be helpful in treating the patient, a meal plan to address the patient's dietary needs, a rehabilitation plan to provide the physical therapy and other activities necessary for the patient's recovery, and a discharge plan indicating whether the patient can return home, should stay, or should be transferred to another healthcare facility. Thus, the output of the skill imitation system 108 may include one or a combination of the automated treatment and planning outputs described above.

[0022] Referring to Figure 2, patient 202 is shown in the background of a health management system. For example, patient 202 may be in a hemodialysis (also simply called "dialysis") session. During dialysis, the dialyzer 204 automatically draws the patient's blood, processes and purifies it, and reintroduces the purified blood into the patient's body. Dialysis may take up to four hours and may be performed every three days, but other durations and periods are also possible. While dialysis is specifically intended, it should be understood that appropriate medical treatment and monitoring may be used instead.

[0023] Before, during, and after dialysis sessions, patient 202 may experience treatment-related health events. While such events may be dangerous for patient 202, they can be predicted based on knowledge of previous health events and the patient's current health metrics. Recommendation 208 may further include information related to the type of event expected and measures of the patient's condition. This recommendation is particularly intended to be made before the start of a dialysis session so that treatment can be adjusted accordingly.

[0024] Recommendations can be made based on various input information. Some of this information includes a patient's static profile, such as age, sex, dialysis initiation date, and previous health events. Dynamic data, such as dialysis measurement records, blood pressure, weight, venous pressure, blood test measurements, and cardiothoracic ratio (CTR), may also be collected for each dialysis session. Blood test measurements can be performed regularly, for example, twice a month, and may measure factors such as albumin, glucose, and platelet count. CTR can also be measured regularly, for example, once a month. Dynamic information can also be recorded during dialysis sessions, for example, using sensors on the dialysis machine 204. Dynamic information may also be modeled as a time series over each frequency.

[0025] Furthermore, the system itself may be monitored within the healthcare environment. For example, the operating parameters of the dialysis machine 204 or other systems within the hospital or other healthcare facility are often monitored, along with a history of past events in the system, to predict events such as those described below.

[0026] During treatment, the patient's condition may be continuously monitored, for example, by tracking the patient's heart rate and other vital signs. If the patient's vital signs indicate an imminent or ongoing adverse health event, treatment may be modified accordingly. For example, the treatment system may automatically administer medication or discontinue treatment in response to a negative health event.

[0027] To train the skill imitation system 108, the medical record 106 may include a set of physician demonstration trajectories, each of which is a state-action pair (s t ,a t ) has a sequence, where, s t This indicates the state of a given patient at time t, and a tThis shows the treatment actions taken by medical professional 102. The skill imitation system 108 learns policies that allow it to replicate the treatment actions taken by medical professional 102. Imitation learning can be built on step-level demonstrations without considering evolving patient symptoms and corresponding treatments.

[0028] Since the imitation learning task leverages the continuous nature of demonstrations, it may instead be formulated at the level of a continuous sequence of states. Thus, each trajectory can be divided into segments, and imitation learning can be performed within each segment. The skills in each segment may be representative and transferable across different trajectories. Each state within a segment is accompanied by past state segments from the same trajectory, allowing the dynamics between patient states and treatment demonstrations to be utilized at the segment level as well.

[0029] A set of segments separated from the original trajectory.

number

[0030] Next, referring to Figure 3, the imitation learning model is shown. The architecture of this model may include a segment embedding layer 304, a prototype layer 306, and an imitation learning layer 308. For simplicity, the subscript j representing the order of instances is omitted in this specification.

[0031] The input segment 302 related to the patient's condition in step t is represented as follows:

number

number

number

number

number

number

number

number

number

[0032] Segment embedding layers can also be implemented in other ways, such as using LSTM (long-short term memory) or GRU (gated recurrent unit) architectures, or using a transformer architecture. However, since embedding long segments is rare in dynamic therapeutic domains, a one-dimensional convolutional layer may be more efficient and effective for extracting embeddings associated with short segments.

[0033] The prototype layer 306 contains k prototype vectors.

number

number

[0034] Each prototype vector represents a class of exemplary segments that reflect the patient's condition at a given stage. Segment embedding z t The similarity score between each prototype vector and the given vector is determined as follows:

number

number

number

[0035] The similarity scores for all prototype-embedding pairs are scaled between [0,1], and as a result, the scaled score for the i-th prototype-embedding pair is as follows:

number

number

number

number

number

number

number

number

number

number

[0036] The imitation learning layer 308 is a flat policy network π parameterized by θ θ (a t |s t ) is corrected, and the mapping from s t to a t is learned by incorporating the embedding of the skill

Number

Number

Number

Number

[0037] Next, referring to Figure 4, a method for training and using a skill imitation model is shown. Block 402 trains the model using a set of past training examples that include sequences of patient states and treatment actions taken by healthcare professionals, which constitute a trajectory. Once the model is trained, it is deployed to the skill imitation system 108 in the healthcare facility 100. There it can be used for patient monitoring and treatment 406, while coordinating treatment actions by the treatment system 104 and healthcare professionals 102.

[0038] During training 402, the trajectory is divided into segments of a fixed size from past data. Imitation learning is performed to train the policy to minimize the objective function, as described above. L=L 1M +λ1L cluster +λ2L evidence +λ3L diversity Here, λ1, λ2, and λ3 are weighting coefficients ranging from 0 to 1. The objective function includes the imitation learning term L. 1M It includes multiple terms, including the following. Given a batch of segments of size n, the situation policy aims to mimic a physician's demonstration at the segment level in a supervised manner.

number

[0039] To improve the interpretability of skill learning models, three regularization elements can be used for learning prototype vectors, including terms related to the clustering structure of segment embeddings, evidence between segments and prototypes, and the diversity of prototypes. The regularization of the clustering structure involves segment embeddings being positioned as close as possible to the nearest prototype.

number

number

[0040] Prototype-segment evidence regularization imposes a dual optimization objective on segment embeddings and prototype vectors. Regularization encourages each prototype vector to be as similar as possible to a segment embedding.

number

[0041] Similarity between each pair of prototype vectors may be penalized because it could lead to redundancy, with indistinguishable prototype vectors representing similar patients. On the other hand, encouraging prototype diversity improves generalization when encountering new segments and trajectories. A diversity regularization term can be imposed as follows:

number

[0042] Training continues until the training loss L converges, and the prototype vectors are optimized to closely resemble segment embeddings from the training data. However, at this stage, the prototype vectors cannot be interpreted because there is no correspondence between them and the actual segments. In order to associate prototypes with segments in the training data, each prototype

number

number

[0043] Next, referring to Figure 5, further details regarding monitoring and treatment 406 are shown. Block 502 measures the updated patient status using, for example, the treatment system 104 and adds the updated patient status information to the medical record 106. This information may include health indicators such as heart rate, blood pressure, and clinical test results, as well as the patient's medication and treatment history. The collected data is used to measure the patient's status s t It constitutes.

[0044] In block 504, the new patient state is input to the trained model, which generates a corresponding skill output. The trained model converts the state into an embedding and uses this embedding to identify the most appropriate prototype. The model's imitation learning is activated, and an action plan is generated based on both the input state and the identified prototype. The action plan may include recommendations or treatment instructions, such as specifying a particular dosage. The output skill indicates the treatment to be performed, and block 506 automatically executes the treatment, for example, by sending a command to the treatment system 502. The treatment may also be provided to the medical professional 102. This treatment information helps the medical professional 102 make decisions regarding the management of the patient.

[0045] Next, referring to Figure 6, an exemplary computing device 600 according to an embodiment of the present invention is shown. The computing device 600 is configured to perform skill recognition.

[0046] The arithmetic unit 600 can be embodied as any type of computing or computer device capable of performing the functions described herein, including but not limited to computers, servers, rack-based servers, blade servers, workstations, desktop computers, laptop computers, notebook computers, tablet computers, mobile computing devices, wearable computing devices, network devices, web devices, distributed computing systems, processor-based systems, and / or user electronic devices. Furthermore or alternatively, the arithmetic unit 600 may be embodied as one or more compute threads, memory threads, or other 28 racks, threads, computing chassis, or other components of a physically disassembled computing device.

[0047] As shown in Figure 6, the arithmetic unit 600 exemplary includes a processor 610, an input / output subsystem 620, memory 630, a data storage device 640, and a communication subsystem 650, and / or other components and devices commonly found in a server or similar arithmetic unit. In other embodiments, the arithmetic unit 600 may include other or additional components (e.g., various input / output devices) commonly found in a server computer. Furthermore, in some embodiments, one or more exemplary components may be incorporated into another component or form part of another component. For example, memory 630, or part thereof, may be incorporated into the processor 610 in some embodiments.

[0048] The processor 610 can be embodied as any type of processor capable of performing the functions described herein. The processor 610 may be embodied as a single processor, a multiprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a single or multicore processor, a digital signal processor, a microcontroller, or other processor or processing / control circuit.

[0049] Memory 630 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, memory 630 may store various data and software used during the operation of the arithmetic unit 600, such as operating systems, applications, programs, libraries, and drivers. Memory 630 may be communicatively coupled to the processor 610 via the I / O subsystem 720 and may be embodied as circuits and / or components to facilitate input / output operations with the processor 610, memory 630, and other components of the arithmetic unit 600. For example, the I / O subsystem 620 may be embodied as, or otherwise include, a memory controller hub, an input / output control hub, a platform controller hub, an integrated control circuit, a firmware device, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and / or other components and subsystems to facilitate input / output operations. In some embodiments, the I / O subsystem 620 may form part of a system-on-a-chip (SOC) and be integrated into a single integrated circuit chip together with other components of the processor 610, memory 630, and arithmetic unit 600.

[0050] The data storage device 640 can be embodied as any type of device or apparatus configured for short-term or long-term storage of data, such as a memory device and circuit, a memory card, a hard disk drive, a solid-state drive, or other data storage device. The data storage device 640 can store program code 640A for training a model, program code 640B for predicting events, and / or program code 640C for performing corrective actions in response to predicted events. The communication subsystem 650 of the arithmetic unit 600 can be embodied as any network interface controller or other communication circuit, apparatus, or assembly thereof that can enable communication between the arithmetic unit 600 and other remote devices over a network. The communication subsystem 650 can be configured to achieve such communication using any one or more communication technologies (e.g., wired or wireless) and associated protocols (e.g., Ethernet, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX®, etc.).

[0051] As shown in the figure, the arithmetic unit 600 may also include one or more peripheral devices 660. Peripheral devices 660 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 660 may include a display, a touchscreen, a graphics circuit, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0052] Of course, the arithmetic unit 600 may include other elements (not shown) as readily conceivable to those skilled in the art, and certain elements may be omitted. For example, various other sensors, input devices, and / or output devices may be included in the arithmetic unit 600, depending on specific implementations of the same, as readily understood to those skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, processors, controllers, memory, etc., may be added to enable various configurations. These and other variations of the processing system 600 are readily conceivable to those skilled in the art, given the teachings of the present invention provided herein.

[0053] Next, refer to Figures 7 and 8. Figures 7 and 8 show exemplary neural network architectures, including the segment embedding layer 304, which can be used to implement parts of this model. A neural network is a generalized system whose functionality and accuracy improve with exposure to additional empirical data. Neural networks are learned by exposure to empirical data. During training, the neural network remembers and adjusts multiple weights applied to the input empirical data. By applying the adjusted weights to the data, it can identify whether the data belongs to a specific predefined class from a set of classes, or output the probability that the input data belongs to each class.

[0054] The empirical data obtained from a series of examples (also called training data) is formatted as a string of values ​​and fed into the neural network. Each example is associated with a known result or output. Each column is represented as a pair (x,y), where x is the input data and y is the known output. The input data can be of various data types and may contain multiple different values. The network can have one input node for each value that makes up the example's input data, and each input value can be assigned a separate weight. The input data can be formatted as a vector, array, or string, for example, depending on the architecture of the neural network being built and trained.

[0055] A neural network "learns" by comparing the neural network output generated from input data with known values ​​from examples, and adjusting the stored weights to minimize the difference between the output and the known values. This adjustment can be performed on the stored weights through backpropagation, and the effect of the weights on the output is determined by calculating a mathematical gradient and adjusting the weights in a way that shifts the output to the minimum difference. This optimization, called gradient descent, is a non-restrictive example of how training takes place. A subset of examples with known values ​​not used in training can be used to test and validate the accuracy of the neural network.

[0056] During operation, the trained neural network can be used on new data that has not been previously used for training or validation through generalization. The weights of the tuned neural network can be applied to the new data, and the weights estimate the function developed from the training examples. The parameters of the estimated function captured by the weights are based on statistical inference.

[0057] In a layered neural network, nodes are arranged in layers. An exemplary simple neural network has an input layer 720 with source nodes 722 and a single computation layer 730 with one or more computation nodes 732 that also function as output nodes, with one computation node 732 for each possible category into which the input example can be classified. The input layer 720 can have a number of source nodes 722 equal to the number of data values ​​712 of the input data 710. The data values ​​712 of the input data 710 can be represented as a column vector. Each computation node 732 in the computation layer 730 generates a linear combination of weighted values ​​from the input data 710 supplied to the input nodes 720 and applies a differentiable nonlinear activation function to the sum. The exemplary simple neural network can perform classification on linearly separable examples (e.g., patterns).

[0058] A deep neural network, such as a multilayer perceptron, may have an input layer 720 with source nodes 722, one or more computation layers 730 with one or more computation nodes 732, and an output layer 740 with one output node 742 for each category into which the input example may be classified. The input layer 720 may have a number of source nodes 722 equal to the number of data values ​​712 of the input data 710. The computation nodes 732 of the computation layer 730 are located between the source nodes 722 and the output nodes 742 and are not directly observed, and are therefore also called hidden layers. Each node 732,742 of the computation layer generates a linear combination of weighted values ​​from the output values ​​of the nodes of the previous layer and applies a differentiable nonlinear activation function over the range of the linear combination. The weights applied to the values ​​from each previous node are, for example, w1, w2, ... w n-i ,w n It can be represented as follows: The output layer provides the network's overall response to the input data. Deep neural networks can be fully connected, where each node in the computational layer is connected to all nodes in the previous layer, or the connections between layers can be in other configurations. If there are missing links between nodes, the network is said to be partially connected.

[0059] Training a deep neural network involves two phases: a forward phase in which the weights of each node are fixed and the input is propagated through the network, and a backward phase in which error values ​​are propagated back through the network and the weight values ​​are updated.

[0060] One or more computational (hidden) layers 730 compute nodes 732 perform a nonlinear transformation on the input data 712 that generates the feature space. Classes and categories may be easier to separate in the feature space than in the original data space.

[0061] The embodiments described herein may be entirely hardware, entirely software, or may include both hardware and software elements. In preferred embodiments, the present invention is implemented in software including, but not limited to, firmware, resident software, and microcode.

[0062] Embodiments may include computer program products accessible from computer-enabled or computer-readable media that provide program code for use by or in connection with a computer or any instruction execution system. Computer-enabled or computer-readable media may include any device that stores, communicates, propagates, or transports programs for use by or in connection with an instruction execution system, apparatus, or device. The medium may be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor systems (or apparatus or devices), or propagation media. The medium may include computer-readable storage media such as semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random-access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks.

[0063] Each computer program can be substantially stored in a machine-readable storage medium or device (e.g., program memory or magnetic disk) that is readable by a general-purpose or special-purpose programmable computer, in order to configure and control the operation of the computer when the storage medium or device is read by the computer in order to perform the procedures described herein. The system of the present invention can also be considered to be implemented on a computer-readable storage medium configured with a computer program, in which case the configured storage medium causes the computer to operate in a specific predetermined manner to perform the functions described herein.

[0064] A data processing system suitable for storing and / or executing program code may include at least one processor directly or indirectly coupled to a memory element via a system bus. The memory element may include local memory, bulk storage, and cache memory that provides at least some temporary storage for the program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including, but not limited to, keyboards, displays, pointing devices, etc.) may be coupled to the system directly or via an intermediary I / O controller.

[0065] Network adapters can also be integrated into a system to enable a data processing system to connect to other data processing systems or remote printers or storage devices via an intervening private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0066] As used herein, the terms “hardware processor subsystem” or “hardware processor” may refer to a processor, memory, software, or combination thereof that works together to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). One or more data processing elements may be included in a central processing unit, a graphics processing unit, and / or a separate processor- or arithmetic element-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more onboard memories (e.g., caches, dedicated memory arrays, read-only memory, etc.). In some embodiments, a hardware processor subsystem may include one or more memories (e.g., ROM, RAM, Basic Input / Output System (BIOS), etc.) that may be onboard or offboard, or that may be dedicated for use by the hardware processor subsystem.

[0067] In some embodiments, a hardware processor subsystem may include and execute one or more software elements. These software elements may include an operating system and / or one or more applications and / or specific code to achieve a specified result.

[0068] In other embodiments, the hardware processor subsystem may include dedicated circuits that perform one or more electronic processing functions to achieve a specified result. Such circuits may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0069] These and other variations of the hardware processor subsystem are also intended in accordance with embodiments of the present invention.

[0070] In this specification, any reference to “one embodiment” or “one embodiment” of the present invention, and to other modifications, means that certain features, structures, properties, etc., described in relation to the embodiments are included in at least one embodiment of the present invention. Therefore, expressions such as “in one embodiment” or “in one embodiment” appearing elsewhere in this specification, and any other modifications, do not necessarily all refer to the same embodiment. However, it should be understood that, considering the teachings of the present invention provided herein, features of one or more embodiments can be combined.

[0071] For example, in the case of "A / B," the use of any of the following " / ," "and / or," or "at least one," such as "A and / or B" or "at least one of A and B," will be understood as intended to include the selection of only the first listed option (A), only the second listed option (B), or both options (A and B). As further examples, in the case of "A, B, and / or C" and "at least one of A, B, and C," such expressions are intended to include the selection of only the first listed option (A), only the second listed option (B), only the third listed option (C), only the first and second listed options (A and B), only the first and third listed options (A and C), only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended as many times as there are listed items.

[0072] The foregoing is to be understood in all respects to be illustrative and not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description but from the claims as interpreted in accordance with the full width permitted by patent law. The embodiments shown and described herein are merely illustrative of the invention, and those skilled in the art should understand that various modifications can be implemented without departing from the scope and spirit of the invention. Those skilled in the art can implement various other combinations of features without departing from the scope and spirit of the invention. Thus, while aspects of the invention have been described with the detail and specificity required by patent law, what is claimed and intended to be protected by the patent is as stated in the appended claims.

Claims

1. A computer-implemented method for training a health management and treatment machine learning model, Segmenting the patient's trajectory, including the sequence of the patient's condition and treatment procedures (304), A method comprising training a machine learning model based on segments of a patient's trajectory (308), the model comprising a prototype layer that learns prototype vectors representing each class of trajectory segments, and an imitation learning layer that learns policies for selecting therapeutic actions based on input states and skill embeddings.

2. In the method according to claim 1, A method further comprising embedding the segmented patient trajectory using the segment embedding layer of the machine learning model.

3. In the method of claim 2, The segment embedding layer comprises a multilayer perceptron and a one-dimensional convolutional layer.

4. In the method according to claim 1, Measuring information about the patient's condition, Based on the measured state information, and based on the skills predicted by the trained model, a treatment action is selected. A method further comprising notifying a medical professional of the treatment procedure in order to assist the medical professional in making decisions for patient management.

5. In the method according to claim 1, The skill embedding method includes a weighted combination of the prototype vectors based on how similar the prototype vectors are to the segmented patient trajectories.

6. In the method according to claim 1, Training the aforementioned machine learning model is a method that involves minimizing a loss function that includes an imitation learning term, a clustering structure regularization term, a prototype segment evidence regularization term, and a diversity regularization term.

7. In the method according to claim 6, The aforementioned imitation learning term [Math 1] This is shown as follows, where m is the length of the segment, n is the number of segments, and π Ｅは This is an expert policy, [Math 2] This is an action performed in step t of segment j, [Math 3] π represents the patient's condition at step t of segment j, θ This is a learned policy, [Math 4] This is a method of embedding skills in step t of segment j.

8. In the method according to claim 6, The clustering structure regularization term is [Math 5] As shown, the prototype segment evidence regularization term is [Math 6] As shown, the diversity regularization term is [Number 7] This is shown as follows, where n is the number of segments and k is the number of prototype vectors. [Number 8] This is the i-th prototype vector, [Number 9] This is the segment embedding of segment j at step t, and d min This is a method that sets the proximity threshold.

9. In the method according to claim 1, The aforementioned treatment method includes at least one of a prescription plan, a meal provision plan, a rehabilitation plan, and a discharge destination plan.

10. In the method according to claim 1, A method in which the aforementioned treatment includes instructing a treatment system to automatically perform treatment on a patient.

11. A system for training machine learning models for health management and treatment. Hardware processor (610), It has a memory (640) for storing a computer program, and when the computer program is executed by the hardware processor, the hardware processor has Segmenting the patient's trajectory, including the sequence of the patient's condition and treatment procedures (304), A system for performing the following: training a machine learning model based on segments of a patient's trajectory (308), which includes a prototype layer that learns prototype vectors representing each class of trajectory segments, and an imitation learning layer that learns policies for selecting therapeutic actions based on input states and skill embeddings.

12. In the system according to claim 11, The computer program is a system that causes the hardware processor to further embed the segmented patient trajectory using a segment embedding layer of the machine learning model.

13. In the system according to claim 12, The segment embedding layer is a system comprising a multilayer perceptron and a one-dimensional convolutional layer.

14. In the system according to claim 11, The prototype layer is a system that determines the similarity between the segment and the prototype vector.

15. In the system according to claim 11, The skill embedding system includes a weighted combination of the prototype vectors based on how similar the prototype vectors are to the segmented patient trajectories.

16. In the system according to claim 11, The computer program further provides a system for the hardware processor to minimize a loss function that includes an imitation learning term, a clustering structure regularization term, a prototype segment evidence regularization term, and a diversity regularization term.

17. In the system described in claim 16, The aforementioned imitation learning term [Number 10] This is shown as follows, where m is the length of the segment, n is the number of segments, and π Ｅ This is an expert policy, [Math 11] This is an action performed in step t of segment j, [Math 12] π represents the patient's condition at step t of segment j, θ This is a learned policy, [Number 13] This is a system where skills are embedded in step t of segment j.

18. In the system described in claim 16, The clustering structure regularization term is [Number 14] As shown, the prototype segment evidence regularization term is [Number 15] As shown, the diversity regularization term is [Number 16] This is shown as follows, where n is the number of segments and k is the number of prototype vectors. [Number 17] This is the i-th prototype vector, [Number 18] This is the segment embedding of segment j at step t, and d min This is a system where proximity is the threshold.

19. In the system according to claim 11, The aforementioned treatment procedure is a system that includes at least one of a prescription plan, a meal provision plan, a rehabilitation plan, and a discharge destination plan.

20. In the system according to claim 11, The aforementioned treatment procedure is a system that includes instructions to a treatment system for automatically performing treatment on a patient.

Images