Apparatus and method for tracking patient symptoms through dialog system using language model

Abstract

Provided is a method for tracking patient symptoms through a dialog system using a language model. The method comprises the steps of: extracting, through a router chain of a dialog system, at least one side effect list on the basis of a query message and a response message input by a patient with respect to the query message; routing to an inspection chain of the dialog system corresponding to the side effect list; and providing, through the inspection chain, a side effect grade on the basis of an additional response message input by the patient with respect to an additional query message based on the side effect list.

Description

Device and method for tracking patient symptoms through a conversation system using a language model

[0001] The present disclosure relates to a device and method for tracking patient symptoms through a dialogue system using a language model, and more specifically, to a device and method for tracking patient symptoms through a dialogue system using a language model, which outputs a list of side effects and provides a grade of side effects using a language model that processes the patient's natural language.

[0002] In order to evaluate clinical side effects after clinical trials on patients, a standardized list of side effects and a side effect grading table are required. Normally, to evaluate a patient's side effects, medical staff must directly survey the patient to obtain information, and the decision is made based on the medical staff's artificial judgment.

[0003] In addition, the side effect grade determined by the artificial judgment of medical staff requires an additional process of computerization through user input, but the scale of patient side effect tracking is limited due to the limited number of specialized medical staff.

[0004] In addition, since the list of side effects and the grade of side effects are determined by the artificial judgment of medical staff, there are limitations in securing statistical significance based on the list of side effects and the grade of side effects of multiple patients.

[0005] Accordingly, the Common Terminology Criteria for Adverse Events (CTCAE) was developed by the National Cancer Institute (NCI) as an example of a classification system that defines a standardized list of adverse events and their grades. CTCAE is a standardized classification system for assessing adverse events occurring in patients during clinical trials. This classification system is used to systematically evaluate adverse events caused by drugs or treatments.

[0006] Specifically, the main purpose of CTCAE is to systematically collect, analyze, and report adverse events occurring in clinical trials, and to assign adverse event grades to a list of adverse events related to various systems of the body, such as neurological, gastrointestinal, skin, cardiovascular, and hematological adverse events occurring in clinical trials and research, to provide a level of severity of adverse events.

[0007] Accordingly, while medical professionals can utilize the standardized classification system, CTCAE, to evaluate and record adverse events experienced by clinical trial participants, there are limitations in that it is difficult to ensure consistency in evaluating adverse events due to the involvement of arbitrary judgment.

[0008] The technical problem of the present disclosure is to provide a device and method for tracking patient symptoms through a dialogue system using a language model, which outputs a list of side effects and provides a grade of side effects using a language model that processes the patient's natural language.

[0009] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0010] According to one aspect of the present disclosure, a method for tracking patient symptoms through a dialogue system utilizing a language model is provided. The method may include: extracting at least one list of side effects based on a query message and a response message input by a patient to the query message through a router chain of the dialogue system; routing the list of side effects to a test chain of the dialogue system corresponding to the list of side effects; and providing a side effect grade based on an additional response message input by the patient to an additional query message based on the list of side effects through the test chain.

[0011] According to another aspect of the present disclosure, a device for tracking patient symptoms through a dialogue system using a language model is provided. The device includes a memory storing at least one instruction and a processor executing the at least one instruction stored in the memory based on data obtained from the memory, wherein the processor extracts at least one list of side effects based on a query message and a response message input by a patient in response to the query message through a router chain of the dialogue system, routes the list of side effects to a test chain of the dialogue system corresponding to the list of side effects, and provides a side effect grade based on an additional response message input by the patient in response to an additional query message based on the list of side effects through the test chain.

[0012] According to one aspect of the present disclosure, the step of extracting the side effect list may be extracting the side effect list that matches the candidate symptoms among the response messages input by the patient.

[0013] According to one aspect of the present disclosure, the step of extracting the side effect list may be extracting the side effect list by referring to a router prompt template that instructs to refer to a prompt document that has classified the candidate symptoms.

[0014] According to one aspect of the present disclosure, the additional query message may be composed of a plurality of messages corresponding to the extracted side effect list.

[0015] According to one aspect of the present disclosure, the step of providing the side effect grade may be to determine and provide the side effect grade by referring to an inspection prompt template that instructs to refer to a prompt document that has previously classified the side effect grade.

[0016] According to one aspect of the present disclosure, the step of providing the side effect grade may include providing the side effect grade and, in addition, providing information on the content of the additional response message that serves as a basis for determining the side effect grade.

[0017] According to one aspect of the present disclosure, the step of providing the side effect grade may further include the step of re-determining the side effect grade when an input of disagreement is received with respect to the provided side effect grade.

[0018] According to one aspect of the present disclosure, after the step of providing the side effect level, the method may further include a step of storing conversation data including the query message, the response message, the additional query message, and the additional response message, and a summary of the conversation data according to a setting.

[0019] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0020] According to the present disclosure, a device and method for tracking patient symptoms can be provided through a conversation system using a language model model that outputs a list of side effects and provides a side effect grade using a language model model that processes the patient's natural language.

[0021] According to the present disclosure, a conversational system utilizing a language model enables consistent evaluation and reporting of side effects, thereby providing reliability as comparative data for analysis of side effects available between different clinical trials or studies.

[0022] According to the present disclosure, it is possible to ensure the safety of various medical interventions by providing reliable side effect lists and analysis data on side effect grades.

[0023] According to the present disclosure, a basis for designing a system that automatically tracks patient symptoms can be provided through an approach using a systematic and standardized dialogue system.

[0024] According to the present disclosure, the limitation on the scale of tracking side effects of patients due to the limited number of specialized medical staff can be overcome by using a conversational system operated in the form of a chatbot.

[0025] According to the present disclosure, a uniform and consistent list of side effects and corresponding side effect grades can be secured through a conversation system operated in the form of a chatbot, eliminating the bias of medical staff.

[0026] According to the present disclosure, a chatbot can be provided that compensates for the lack of flexibility and conversation context awareness by readjusting the side effect rating to reflect the response of a patient who disagrees with the side effect rating provided by the conversation system.

[0027] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0028] FIG. 1 is a diagram illustrating a device for tracking patient symptoms according to the present disclosure communicating with each other to transmit and receive data.

[0029] FIG. 2 is a schematic diagram illustrating a module included in a device for tracking patient symptoms in one embodiment of the present disclosure.

[0030] Figure 3 is a diagram illustrating an example of a module that implements a method for tracking patient symptoms.

[0031] Figure 4 is a schematic diagram illustrating the structure of a conversation system implemented through a language model for tracking patient symptoms.

[0032] Figure 5 is a diagram illustrating the configuration of a dialogue chain that constitutes a dialogue system using a language model.

[0033] Figure 6 is a flowchart illustrating the process of tracking patient symptoms using a conversation system based on a language model.

[0034] Figure 7 is a flowchart illustrating the process of extracting a list of side effects from a patient through conversation data.

[0035] Figure 8 is a diagram illustrating an example of a prompt document that has pre-classified candidate symptoms or side effect grades to output a list of side effects or side effect grades for a patient.

[0036] Figure 9 is a diagram illustrating a process of extracting a list of side effects from a patient through a chatbot by referring to a routing prompt template.

[0037] Figure 10 is a flowchart illustrating a process for providing a patient's side effect rating through conversation data.

[0038] Figure 11 is a diagram illustrating a process of providing a patient's side effect rating through a chatbot, referring to the inspection prompt template.

[0039] Figure 12 is a diagram illustrating a process in which a side effect list is extracted through a chatbot according to the operation of a router chain, and the chatbot operates according to an inspection chain corresponding to the extracted side effect list.

[0040] Figure 13 is a diagram illustrating a process of providing a side effect rating through a chatbot according to the operation of the inspection chain.

[0041] Figure 14 is a diagram illustrating the process of re-determining the side effect grade through a chatbot according to the operation of the inspection chain.

[0042] Figure 15 is a diagram illustrating a summary of conversation data according to conversation data and settings.

[0043] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0044] In describing embodiments of the present disclosure, detailed descriptions of known configurations or functions will be omitted if they are deemed to obscure the gist of the present disclosure. Furthermore, portions unrelated to the description of the present disclosure in the drawings have been omitted, and similar portions have been designated with similar reference numerals.

[0045] In the present disclosure, when a component is said to be "connected," "coupled," or "connected" to another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is said to "include" or "have" another component, unless otherwise specifically stated, this does not exclude the other component, but rather implies that the other component may be included.

[0046] In this disclosure, terms such as first, second, etc. are used only for the purpose of distinguishing one component from another, and do not limit the order or importance of components, unless specifically stated otherwise. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0047] In this disclosure, distinct components are used to clearly illustrate their respective characteristics, and do not necessarily imply that the components are separated. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed into multiple hardware or software units. Therefore, even if not specifically mentioned, such integrated or distributed embodiments are also included within the scope of this disclosure.

[0048] In the present disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments comprising a subset of the components described in one embodiment are also within the scope of the present disclosure. Furthermore, embodiments including other components in addition to the components described in various embodiments are also within the scope of the present disclosure.

[0049] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below, along with the accompanying drawings. However, the present invention is not limited to the embodiments presented below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the invention of the scope of the invention.

[0050] FIG. 1 is a diagram illustrating a device for tracking patient symptoms according to the present disclosure communicating with each other to transmit and receive data.

[0051] The user device (100) is a communication device, and may refer to a mobile computing device or a fixed computing device equipped with hardware and software capable of executing a user interface for transmitting and receiving data used in the present disclosure. For example, the user device (100) may include a mobile phone, a smartphone, or a wearable device. In addition, although FIG. 1 illustrates a smartphone as an example of the user device (100), the present invention is not limited thereto, and any electronic device capable of network communication with the main server (200) via wired or wireless communication, such as a notebook, laptop, or tablet PC, may be included.

[0052] A user device (100) according to an embodiment of the present disclosure can execute a user interface that implements a chatbot-based conversation system for outputting query messages and additional query messages to track patient symptoms, receiving response messages and additional response messages accordingly, extracting a list of side effects, and providing a side effect grade. The process of extracting a list of side effects through a chatbot and providing a side effect grade based on the extracted list of side effects is described with reference to FIG. 6.

[0053] The conversation system according to the present disclosure may be provided in the form of a chatbot based on a language model implemented using natural language processing artificial intelligence. The language model may be pre-trained on a server (200).

[0054] The user device (100) and the main server (200) can communicate with each other through a network. The network can be implemented with any form of digital data communication and can include various wired and wireless communications such as, for example, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a campus area network (CAN), 3G, 5G, WiFi, and LTE.

[0055] The main server (200) may be implemented as at least one computer device that communicates with the user device (100) through a network to provide commands, codes, files, software, etc.

[0056] According to the present disclosure, the main server (200) can transmit various information, software and applications used in the conversation system in response to requests and data transmitted from the user device (100) to support the conversation system and various services for tracking patient symptoms.

[0057] For example, the main server (200) may transmit a side effect list and a side effect grade thereof to the user device (100) based on conversation data including a query message, an additional query message, a response message, and an additional response message received through a conversation system implemented in the user device (100). Through this, the side effect list and the side effect grade thereof may be uploaded to a web page, or the patient may be allowed to check the side effect list and the side effect grade thereof through a user interface that can visually present the conversation system to the patient.

[0058] As another example, software or an application that implements a conversation system may be transmitted to the user device (100), allowing the patient to receive results through processing within the user device (100). Accordingly, the main server (200) may transmit software or an application that executes a user interface for implementing a conversation system for tracking patient symptoms to the user device (100).

[0059] FIG. 2 is a schematic diagram illustrating a module included in a device for tracking patient symptoms in one embodiment of the present disclosure.

[0060] The user device (100) and main server (200) described in FIG. 1 can be implemented by the modules illustrated in FIG. 2. For convenience of description, the description will focus on the main server (200), but the user device (100) can also be implemented substantially in the same manner.

[0061] The main server (200) may include a processor (205), a communication unit (210), and a memory (215). Each component included in the main server (200) is not an essential component, and may have additional components or be omitted. One component may be included in or combined with another component, so that a single component may perform multiple functions. For example, in the case of the user device (100), a user interface may be additionally provided, and the user interface may include an input / output interface.

[0062] For example, the input / output interface may be a means for interfacing with an input / output device for interaction with a user of the user device (100), and the input / output device may include a microphone, a keyboard, a mouse, a display, a speaker, a touch screen, etc.

[0063] The communication unit (210) of the main server (200) may provide a function for communicating with the user device (100). Specifically, the communication unit (210) may transmit requests, commands, data, files, etc. generated by the processor (205) of the main server (200) based on program codes stored in a recording device such as a memory (210) to the user device (100). In addition, requests, commands, data, files, etc. generated in the user device (100) may be received by the communication unit (210) of the main server (200) via a network.

[0064] The communication unit (210) according to the present disclosure can transmit and receive required information for tracking patient symptoms. For example, the communication unit (210) can transmit and receive conversation data, a list of side effects, a grade of side effects, or software or applications for implementing a conversation system.

[0065] The communication unit (210) may utilize a network implemented with any form of digital data communication to transmit and receive the above-described information. For example, various wired and wireless communications such as a local area network (LAN), a wide area network (WAN), 3G, 5G, WiFi, and LTE may be utilized.

[0066] The memory (215) of the main server (200) stores applications and various data for controlling the main server (200), and can load applications or read and write data at the request of the processor (205).

[0067] According to the present disclosure, the memory (215) may store an application and at least one instruction that extracts at least one side effect list based on a query message and a response message input by a patient for the query message through a router chain of a dialogue system, routes the side effect list to a test chain of the dialogue system corresponding to the side effect list, and provides a side effect grade based on an additional response message input by the patient for an additional query message based on the side effect list through the test chain.

[0068] The processor (205) can perform overall control by executing applications and instructions stored in the memory (215) of the main server (200). Through this, the processor (205) can practically implement a method for tracking patient symptoms according to the present disclosure.

[0069] Below, an example of a module operating in a user device (100) or a main server (200) to implement a method for tracking patient symptoms is described through FIG. 3.

[0070] FIG. 3 is a diagram illustrating an example of a module implementing a method for tracking patient symptoms according to the present disclosure.

[0071] The module illustrated in FIG. 3 can be implemented substantially identically to the processing for implementing the method of tracking patient symptoms of the processor (205), communication unit (210), and memory (215) described in FIG. 2. For convenience of description below, the description will focus on the main server (200), but the same can be implemented substantially identically in the user device (100).

[0072] The main server (200) may include a router chain unit (305), a test chain unit (310), and a prompt document unit (315) to track patient symptoms. The router chain unit (305) may include a language model unit (305a), a router prompt template unit (305b), and a dialogue storage unit (305c). The test chain unit (310) may include a language model unit (310a), a test prompt template unit (310b), and a dialogue storage unit (310c). Each of the above-described components is not an essential component, and additional components may be provided or omitted, and one component may be included in or combined with another component so that a single component may perform multiple functions. For example, although FIG. 3 illustrates two test chain units (310), the test chain unit (310) may be provided according to the number of side effect lists since it is a component that routes according to the side effect list extracted from the router chain unit (305). In addition, Fig. 3 only illustrates a module for tracking patient symptoms in multiple configurations for the convenience of understanding, and is not limited to being hardware-wise divided, but can be interpreted as processing of software or applications that can be implemented in a single configuration.

[0073] The router chain unit (305) extracts a list of side effects through a patient response message obtained from a conversation system implemented as a chatbot, and routes the extracted list of side effects to the inspection chain unit (310) corresponding to the extracted list of side effects. This will be described later.

[0074] The inspection chain (310) routed by the router chain (305) can determine and provide a side effect rating through an additional response message from the patient obtained from a conversation system implemented as a chatbot. This will be described later.

[0075] The language model unit (305a, 310a) can process the patient's input response message through a language model implemented with natural language processing artificial intelligence. The language model unit (305a, 310a) can utilize a conversation chain structure to organically generate conversations in a chatbot using the language model. The conversation chain is a structure that uses a prompt template, conversation data, and input message as input values ​​for the language model.

[0076] The language model used in this disclosure can be a Large Language Model (LLM). A large language model is a deep learning model capable of processing input natural language, and can utilize models from the Generative Pre-trained Transformer (GPT) family, HyperClova, or Bert.

[0077] The router prompt template unit (305b) and the examination prompt template unit (310b) are written using prompt engineering, and can provide specific instructions to the language model so that the language model can operate in a direction specialized for a specific task. The router prompt template unit (305b) according to the present disclosure can serve to provide a series of instructions, rules, and additional information so that the language model unit (305a) can efficiently proceed with the conversation process with the patient through the chatbot. For example, the router prompt template unit (305b) can instruct the language model unit (305a) to compare the response message entered by the patient with the candidate symptoms recorded in the prompt document unit (315) and extract a list of side effects corresponding to the candidate symptoms. Similarly, the examination prompt template unit (310b) according to the present disclosure can serve to provide a series of instructions, rules, and additional information so that the language model unit (310a) can efficiently proceed with the conversation process with the patient through the chatbot. For example, the inspection prompt template section (310b) may output an additional query message based on the extracted side effect list and may instruct the patient to refer to the side effect grade recorded in the prompt document section (315) when providing a side effect grade based on an additional response message entered by the patient.

[0078] The conversation storage unit (305c, 310c) can be designed with a buffer structure to store conversation data, enabling the language model to be implemented as a chatbot. This allows the language model to remember conversation data and generate more consistent and robust response messages. Furthermore, as the conversation progresses, the language model references previous interactions (i.e., conversations) to respond more appropriately to the context.

[0079] The conversation storage unit (305c, 310c) according to the present disclosure can store conversation data generated by a conversation with a chatbot implemented by a language model model, such as a response message input by a patient, an additional response message, and a response message, a query message, and an additional query message output by the language model model unit (305a). The conversation data stored by the conversation storage unit (305c, 310c) can be re-input to the language model model unit (305a) to form an organic flow of conversation with the chatbot.

[0080] Hereinafter, the configuration of a conversation system implemented as a chatbot will be described with reference to FIG. 4. As described in FIG. 2, the conversation system can be implemented by the processor (205) of a user device (100) or a main server (200) executing an application stored in a memory (215) and at least one instruction. Similarly, the conversation system can be implemented through modules (305, 310, 315) included in the user device (100) or the main server (200), as described in FIG. 3.

[0081] Figure 4 is a schematic diagram illustrating the structure of a conversation system implemented through a language model for tracking patient symptoms.

[0082] The language model that can be used in the present disclosure can be used as a backbone for creating a domain-specific chatbot corresponding to extracting a list of side effects or providing a grade of side effects.

[0083] Specifically, the conversation system may consist of a router chain and a test chain. The router chain may extract a list of side effects based on the conversation with the patient and route the test chain corresponding to the extracted list of side effects.

[0084] More specifically, the router chain may include a language model, a router prompt template, and a dialogue memory. The test chain may also include a language model, a test prompt template, and a dialogue memory. The language model models included in the router chain and the test chain may vary depending on system or user settings.

[0085] A router chain can output a query message formed with a question structure that leads to extracting a list of side effects without omission, and receive a response message accordingly.

[0086] Specifically, the language model, implemented using natural language processing AI, can process the patient's input response message and form a dialogue chain structure. For ease of understanding, the dialogue chain structure is described with reference to Figure 5.

[0087] Figure 5 is a diagram illustrating the configuration of a dialogue chain that constitutes a dialogue system using a language model.

[0088] Looking at Figure 5, the conversation chain refers to a structure that uses a prompt template, conversation data from a conversation memory, and an input message as input to a language model, outputs an output message based on the input information, and then saves the output message and input message as conversation data and then re-inputs them into the language model.

[0089] This allows the conversation to flow organically, remember previous conversation data, and derive more consistent and robust response messages. As the conversation progresses, the language model refers to previous interactions, i.e., conversations, and responds more appropriately to the context.

[0090] Returning to Figure 4, prompt templates are created using prompt engineering to provide specific instructions to the language model, allowing the language model to operate in a direction specialized for a specific task.

[0091] The router prompt template according to the present disclosure can instruct a language model to compare a response message entered by a patient with candidate symptoms recorded in a prompt document and extract a list of side effects corresponding to the candidate symptoms.

[0092] The test prompt template according to the present disclosure can output an additional query message based on the extracted side effect list and instruct the language model to refer to the side effects recorded in the prompt document when providing a side effect rating based on an additional response message entered by the patient.

[0093] For example, the test chain according to the present disclosure can determine and provide adverse event grades based on a prompt document created based on the Common Terminology Criteria for Adverse Events (CTCAE), a standardized classification system developed by the National Cancer Institute of the United States for evaluating adverse events occurring during clinical trials. The prompt document can be created according to a user-defined format or a format required by the dialog system according to the present disclosure, and the referenced prompt document can be provided differently for each router chain or test chain. In the present disclosure, a single prompt document is illustrated, but it is understood that multiple prompt documents can be configured.

[0094] The conversation memory can be designed as a buffer structure to store conversation data, so that the language model can be implemented as a chatbot.

[0095] Finally, the conversation system can store and output a summary of the conversation data, including the adverse event ratings provided by the test chain and the list of adverse events used as input to the test chain. This summary can be used as analytical data for available adverse events across different clinical trials or studies, and can be stored in a JSON file format.

[0096] Figure 6 is a flowchart illustrating the process of tracking patient symptoms using a conversation system based on a language model.

[0097] For the convenience of the description below, the process of tracking patient symptoms using a conversation system will be described with a focus on implementation in the main server (200).

[0098] The main server (200) extracts at least one list of side effects through a router chain (S405). The executable conversation system on the main server (200) is implemented as a chatbot, enabling conversations with patients.

[0099] Specifically, the router chain of the conversation system can perform a conversation to output a query message through a chatbot and extract a list of side effects based on the patient's response message.

[0100] For example, a chatbot could output a query message to extract a list of side effects from a patient, such as, "Please tell us about the symptoms you have experienced since your last treatment, and we will help you diagnose."

[0101] The main server (200) can output an additional query message upon receiving the patient's response message to the query message through the chatbot in order to extract a list of side effects.

[0102] For example, a chatbot could output a query message to extract additional side effects, such as "Do you have any other symptoms?"

[0103] Finally, the main server (200) can analyze the response message received through the conversation with the chatbot via the router chain to extract a list of side effects and output the extracted list of side effects to the patient via the chatbot. The router chain can reference a prompt document to extract the list of side effects, which is described in detail in FIG. 7.

[0104] Additionally, the query message can be structured as a question that prompts the extraction of a complete list of side effects. Specifically, the main server (200) can output a message to the chatbot to indicate the absence of any further side effects to be extracted.

[0105] For example, the chatbot can check whether there are additional side effects in addition to the extracted side effects list, such as, "Based on the symptoms you have so far, it seems like you have a headache and dizziness. If you have these symptoms, it would be a good idea to proceed with the CTCAE test. Please press the proceed button to proceed." In addition, if the missing side effects list does not exist, it can output a message to confirm whether or not the side effect grade is determined through the test chain.

[0106] Next, the main server (200) routes the extracted side effect list to a corresponding inspection chain using a router chain (S410). The inspection chain may be provided according to the number of side effect lists, and may include a default inspection chain if the extracted side effect list does not exist.

[0107] Next, the main server (200) provides a side effect grade for the side effect list through the inspection chain (S415).

[0108] Specifically, the chatbot can output additional inquiry messages through the inspection chain of the conversation system, and determine and provide a side effect grade based on the patient's additional response message.

[0109] Additional query messages may include queries to specifically determine the severity of each symptom in the extracted side effect list. For example, if the extracted side effect is headache, the chatbot may output additional query messages to determine the severity of headache symptoms, such as "How often have you had headaches in the past week?"

[0110] The inspection chain can refer to the prompt document when outputting additional query messages, which is described in detail in Figure 10.

[0111] The main server (200) may output an additional additional query message upon receiving an additional response message from the patient to the additional query message through the chatbot in order to extract the side effect grade.

[0112] For example, the chatbot could output additional questions to more specifically determine the severity of your headache symptoms, such as, "Can I ask if your headache has caused you any difficulties in your daily life? For example, have you had any difficulties performing household or personal tasks?"

[0113] Next, the main server (200) can analyze additional response messages received through the inspection chain and conversation with the chatbot to determine the adverse effect grade and provide the result to the patient via the chatbot. Furthermore, the chatbot can provide the adverse effect grade and, in addition, provide information on the basis of the additional response messages received to determine the adverse effect grade.

[0114] For example, a chatbot could determine and provide the side effect grade and the basis for determining the side effect grade, such as, "Your symptom is grade 1, which means mild pain. This refers to pain that does not significantly interfere with daily life. This decision was made based on your statement that there was no significant inconvenience in daily life."

[0115] The inspection chain can be performed based on the CTCAE criteria to determine adverse event grades. The determination of adverse event grades based on the CTCAE criteria can be referenced in the prompt document. This is described in detail in Figure 10.

[0116] Figure 7 is a flowchart illustrating the process of extracting a list of side effects from a patient through conversation data.

[0117] Referring to Figure 7, the router chain outputs a query message based on a router prompt template (S505). The router prompt template can instruct the language model to generate at least one query message to identify the patient's side effect list. Accordingly, the chatbot can output at least one query message to conduct a conversation with the patient.

[0118] Next, the router chain receives a response message to the query message output by the chatbot (S510) and extracts a list of side effects matching the candidate symptoms by referencing the router prompt template (S515). The router prompt template can instruct the language model to extract a list of side effects by referencing a prompt document that has already classified the candidate symptoms.

[0119] Specifically, the router chain stores conversation data, consisting of the chatbot's query message and the patient's response message, in the conversation memory. It then repeatedly re-inputs the conversation data and the response message for the next query message into the language model to extract a list of side effects. In other words, by memorizing the existing conversation data, the router chain can analyze all response messages entered by the patient and extract a list of side effects that match the candidate symptoms without omission.

[0120] A prompt document according to the present disclosure may include a list of candidate symptoms for extracting a list of side effects, and may include information on side effect grades and additional query messages. This is described in detail with reference to FIG. 8.

[0121] Figure 8 is a diagram illustrating an example of a prompt document that has pre-classified candidate symptoms or side effect grades to output a list of side effects or side effect grades for a patient.

[0122] Candidate symptoms (805) may include dry mouth, difficulty swallowing (Dysphagia), mucositis oral, cracked or split lip edges (Cheilitis), etc., and the candidate symptoms (805) are not limited to the candidates exemplified in FIG. 8 and may be written in the prompt document based on the classification system of CTCAE.

[0123] Information (810) regarding additional query messages may be written in the prompt document corresponding to each candidate symptom (805), and multiple instances may be written for each candidate symptom (805) (not shown). For example, in the case of dry mouth, multiple examples of additional query messages may be written, such as "Over the past week, how severe was your dry mouth at its worst?" and "How long did your dry mouth last?" In addition, the information (810) may include specific examples of additional query messages. For example, it may include specific examples of additional query messages, such as "How severe was your dry mouth for in minutes, hours, or all day?"

[0124] The side effect grade (815) can be written in the prompt document for each side effect list step by step based on the classification system of CTCAE. For example, in the case of dry mouth, the side effect grade (815) can be written in the prompt document by classifying it into mild (grade 1, symptoms of sticky saliva or dry mouth but no decrease in appetite), moderate (grade 2, almost no saliva is produced so only soft and moist food is possible, reducing appetite), and severe (grade 3, sufficient oral intake is impossible so conventional diet or intravenous nutrition is required). The side effect grade (815) written in the prompt document can include information that serves as the basis for determining the grade.

[0125] Returning to Figure 7 again, in step S515, the router chain analyzes the response message input through the chatbot and extracts a list of side effects matching the candidate symptoms by referring to the prompt document.

[0126] For example, if a patient types a response message to the chatbot such as "I think I have a headache," the router chain can extract "headache" from the patient's list of side effects by referencing "headache" written in the prompt document.

[0127] For the convenience of understanding, the conversation process of extracting a list of side effects through a chatbot is explained with reference to the instructions of the routing prompt template through Figure 9.

[0128] Figure 9 is a diagram illustrating a process of extracting a list of side effects from a patient through a chatbot by referring to a routing prompt template.

[0129] A routing prompt template can include instructions for the language model to reference a prompt document. For example, given a patient response message, the model can be instructed to analyze the patient response message and select the most similar candidate symptom from a list of side effects based on the candidate symptom and description, or to format the extracted side effect list and generate a message to determine the side effect grade through a test chain based on the extracted side effect list.

[0130] Additionally, the routing prompt template may include instructions for the route to route to the corresponding inspection chain based on the extracted side effect list.

[0131] Since the routing prompt template includes an instruction to extract at least one side effect list, the chatbot can output query messages to extract the side effect list without omission, such as “Are you experiencing any additional discomfort?”, “Are you experiencing any other discomfort?”, and “Are there any other symptoms other than the ones you mentioned?”

[0132] Figure 10 is a flowchart illustrating a process for providing a patient's side effect rating through conversation data.

[0133] Referring to Figure 10, the inspection chain routed to correspond to the extracted side effect list outputs an additional query message by referencing the inspection prompt template (S605). The inspection prompt template may instruct the language model to generate at least one additional query message to determine the side effect grade for the side effect list. Specifically, the inspection prompt template may instruct the language model to refer to information (810) regarding additional query messages written in the prompt document to generate the additional query message.

[0134] Accordingly, the chatbot can perform a conversation with the patient by outputting an additional query message with reference to information (810) about the additional query message written in the prompt document.

[0135] Next, the inspection chain receives an additional response message for the additional query message output by the chatbot (S610) and determines the patient's adverse event grade by referencing the inspection prompt template (S615). The inspection prompt template can input instructions to the language model to determine the adverse event grade by referencing a prompt document that has previously classified the adverse event grade.

[0136] Specifically, the test chain stores conversation data consisting of additional query messages from the chatbot and additional response messages from the patient in the conversation memory, and repeatedly re-inputs the conversation data and additional response messages for the next additional query message into the language model to determine the adverse event rating. In other words, by remembering the existing conversation data, the test chain can analyze all additional response messages entered by the patient to determine a robust adverse event rating.

[0137] Referring to the prompt document illustrated in Figure 8, for example, if a patient inputs an additional response message to the chatbot, such as “Eating the meat was not uncomfortable, but I felt a little uncomfortable after eating,” the test chain can determine the side effect grade corresponding to grade 1 by referring to “Mild (grade 1, symptoms of sticky saliva or dry mouth, but no decrease in food intake)” written in the prompt document.

[0138] In addition, the inspection chain provides information on the grade of side effects determined through the chatbot and the basis for determining the grade (S620).

[0139] For example, the chatbot could output something like, "Okay. Now, based on the information we have so far, we'll determine the CTCAE grade. Considering that there was no vomiting, but there was discomfort after eating, but not enough to require hospitalization, the current situation is grade 1. This means that no special treatment or intervention is required."

[0140] For ease of understanding, the conversation process of providing a side effect rating through a chatbot is described with reference to the instructions of the inspection prompt template through Figure 11.

[0141] The test prompt template may include instructions for the language model to reference a prompt document. For example, the test prompt template may instruct the language model to output an additional query message by referencing information (810) regarding additional query messages in the prompt document (Rule 1 in the drawing). Additionally, the test prompt template may instruct the model to output an additional query message to more specifically determine the severity of the patient's symptoms (Rule 2 in the drawing).

[0142] Accordingly, if the side effect list is "vomiting," the chatbot will output additional inquiry messages such as "How often have you vomited in the past week?", "I see. So, did you have difficulty eating whenever you felt that way?", "So, you had discomfort after eating. So, did you ever have to go to the hospital or get help from a medical professional because of that discomfort?"

[0143] The test prompt template may instruct the user to determine the adverse effect grade by referencing the adverse effect grades already classified in the prompt document when sufficient data is available to determine the adverse effect grade for the patient's adverse effect list (Rule 3 in the drawing). The availability of sufficient data may mean that all additional response messages have been received, as information (810) regarding additional query messages written in the prompt document is output as additional query messages via the chatbot.

[0144] In addition, the inspection prompt template may instruct the investigator to determine and provide information on the basis of which the adverse event grade was determined when the adverse event grade was determined (Rule 4 of the drawing).

[0145] Accordingly, the chatbot could output something like, "I see. Then, based on the information provided so far, we will determine the CTCAE grade. Considering that there was no vomiting, but there was discomfort after eating, but not enough to require hospital treatment, the current situation is 'Grade 1,' which means that no special treatment or intervention is required."

[0146] Returning to Figure 10, if the inspection chain receives disagreement input regarding the side effect grade output through the chatbot or the information on which the side effect grade was determined, the side effect grade can be re-determined (S625).

[0147] A negative input may include any response message that is interpreted as negative to a message containing information about the side effect grade output by the chatbot or the basis for determining the side effect grade.

[0148] For example, if the inspection chain receives a disagreement with the side effect rating provided via the chatbot, such as "It does not limit some aspects of daily life," the inspection chain can redetermine and provide the side effect rating. A specific example of this is described in Figure 14.

[0149]

[0150] Figures 12 to 14 illustrate an embodiment of a conversation system according to the present disclosure implemented as a chatbot in the form of a web UI.

[0151] Figure 12 is a diagram illustrating a process in which a side effect list is extracted through a chatbot according to the operation of a router chain, and the chatbot operates according to an inspection chain corresponding to the extracted side effect list.

[0152] The chatbot can output at least one query message to the patient to extract a list of side effects. Additionally, the chatbot can output a query message to the patient to extract all side effects without missing any.

[0153] For example, as shown in the drawing, a query message can be output to extract a list of side effects without omission, such as "I see. Based on the symptoms you have had so far, it seems you have a headache and dizziness. If you have these symptoms, it would be a good idea to proceed with a CTCAE test. To proceed, please press the proceed button." The query message output by the chatbot is not limited to the embodiment illustrated in this drawing.

[0154] Figure 13 is a diagram illustrating a process of providing a side effect rating through a chatbot according to the operation of the inspection chain.

[0155] Specifically, FIG. 13 is an example of a conversation performed by a chatbot that determines and provides a side effect grade as a list of side effects corresponding to “vomiting” is extracted.

[0156] The chatbot can output at least one additional query message to determine the side effect grade of the extracted side effect list. For example, the chatbot can output at least one additional query message such as, "How often have you vomited in the past week?", "I see. So, did you have difficulty eating whenever you felt that way?", or "You experienced discomfort after eating. So, did you ever need to go to the hospital or seek medical help for that discomfort?"

[0157] The chatbot can determine and provide a side effect rating when it has received all additional response messages corresponding to the additional query messages that can be output by referencing the prompt document. Additionally, the chatbot can provide information regarding the basis for determining the side effect rating.

[0158] For example, if the chatbot references a prompt document containing the CTCAE to assess the side effect grade, it can output something like, "Okay. Then, based on the information provided so far, I will determine the CTCAE grade. Considering that there was no vomiting, but there was discomfort after eating, but not enough to require hospital treatment, the current situation is 'grade 1'. This means that no special treatment or intervention is required." The method of outputting additional query messages output by the chatbot, information about the side effect grade, and the basis for determining the side effect grade is not limited to the embodiment illustrated in this drawing.

[0159] Figure 14 is a diagram illustrating the process of re-determining the side effect grade through a chatbot according to the operation of the inspection chain.

[0160] Specifically, FIG. 14 is an example of a conversation performed by a chatbot that redetermines a side effect grade when receiving disagreement input regarding information about the side effect grade and the basis for determining the side effect grade.

[0161] When a chatbot receives an input that is not in agreement, it can re-determine the side effect level and output it.

[0162] For example, if a statement like "Your symptom is 'Grade 2,' which means 'moderate pain,' refers to pain that limits some aspects of your daily life. This was determined based on your statement that you were able to continue your daily life despite the headache," and a disagreement such as "It does not limit some aspects of your daily life," is received, the side effect grade is re-determined and output through steps S615 and S620. This can complement the chatbot's flexibility and limitations in conversational context recognition.

[0163] Figure 15 is a diagram illustrating a summary of conversation data according to conversation data and settings.

[0164] The conversation system according to the present disclosure can store and output a summary of conversation data that includes all conversation details, such as the adverse effect rating provided by the inspection chain and the list of adverse effects used as input to the inspection chain. The format of the summary may vary depending on system settings and user settings.

[0165] While the exemplary methods of this disclosure are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and individual steps may be performed simultaneously or in different orders, if desired. To implement a method according to this disclosure, additional steps may be included in addition to the steps illustrated, some steps may be excluded and the remaining steps included, or some steps may be excluded and additional steps included.

[0166] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0167] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0168] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0169] The present invention can be used in a device for tracking patient symptoms through a conversation system using a language model that outputs a list of side effects and provides a side effect grade using a language model that processes the patient's natural language.

Claims

1. A method for tracking patient symptoms through a conversation system using a language model. A step of extracting at least one list of side effects based on a query message and a response message entered by a patient for the query message through a router chain of a dialogue system; A step of routing to the inspection chain of the dialogue system corresponding to the above side effect list; and A method for tracking patient symptoms, comprising the step of providing a side effect grade based on an additional response message entered by the patient in response to an additional query message based on the side effect list through the above inspection chain.

2. In paragraph 1, The step of extracting the above side effect list is: A method for tracking patient symptoms, comprising extracting a list of side effects that match candidate symptoms from the response message entered by the patient.

3. In paragraph 2, The step of extracting the above side effect list is: A method for tracking patient symptoms, wherein the list of side effects is extracted by referring to a router prompt template that instructs to refer to a prompt document that has classified the above candidate symptoms.

4. In paragraph 1, The above additional query message is, A method for tracking patient symptoms corresponding to the extracted side effect list and consisting of multiple symptoms.

5. In paragraph 1, The step of providing the above side effect grade is: A method for tracking patient symptoms, wherein the side effect grade is determined and provided by referring to an examination prompt template that instructs the user to refer to a prompt document that has classified the side effect grade.

6. In paragraph 5, The step of providing the above side effect grade is: A method for tracking patient symptoms, which provides the side effect grade and, in addition, provides information on the basis of which the side effect grade was determined among the additional response messages.

7. In paragraph 1, The step of providing the above side effect grade is: A method for tracking patient symptoms, further comprising the step of re-determining the side effect grade if a disagreement is received with respect to the provided side effect grade.

8. In paragraph 1, After the step of providing the above side effect grade, A method for tracking patient symptoms, further comprising the step of storing conversation data including the query message, the response message, the additional query message, and the additional response message, and a summary of the conversation data according to a setting.

9. In a device for tracking patient symptoms through a conversation system using a language model, memory that stores at least one instruction; and A processor that executes at least one instruction stored in the memory based on data obtained from the memory, The above processor, Through the router chain of the dialogue system, extract at least one list of side effects based on a query message and a response message entered by the patient for the query message, Route to the inspection chain of the above conversation system corresponding to the above side effect list, A device for tracking patient symptoms, which provides a side effect grade based on an additional response message entered by the patient in response to an additional inquiry message based on the side effect list through the above inspection chain.

10. In paragraph 9, Extracting the above list of side effects is: A device for tracking patient symptoms, which extracts a list of side effects that match candidate symptoms from the response message entered by the patient.

11. In paragraph 10, Extracting the above list of side effects is: A device for tracking patient symptoms, wherein the list of side effects is extracted by referring to a router prompt template that instructs to refer to a prompt document that has classified the above candidate symptoms.

12. In paragraph 9, The above additional query message is, A device for tracking patient symptoms, which corresponds to the extracted side effect list and is composed of multiple items.

13. In paragraph 9, Providing the above side effect ratings, A device for tracking patient symptoms, wherein the adverse effect grade is determined and provided by referring to an examination prompt template that instructs the user to refer to a prompt document that has classified the adverse effect grade.

14. In paragraph 13, Providing the above side effect ratings, A device for tracking patient symptoms, which provides the above side effect grade and also provides information on the basis for determining the side effect grade among the above additional response messages.

15. In paragraph 9, Providing the above side effect ratings, A device for tracking patient symptoms, further comprising re-determining the side effect grade if a disagreement input is received regarding the side effect grade provided.

16. In paragraph 9, After providing the above side effect ratings, A device for tracking patient symptoms, further comprising storing conversation data including the query message, the response message, the additional query message, and the additional response message, and a summary of the conversation data according to a setting.

Images