Device for implementing LLM model for predicting condition of patient on basis of consultation data and control method therefor

Abstract

The present disclosure relates to a device and a method for implementing an LLM model for predicting a condition of a patient on the basis of consultation data, and the device may comprise: a memory for storing at least one piece of consultation data and at least one process for predicting a condition of a patient on the basis of the consultation data; and a processor for performing an operation according to the process, wherein the processor controls an LLM model to be trained on the basis of result data constructed by matching international classification of diseases (hereinafter, ICD) and the consultation data, inputs the consultation data to the trained LLM model to perform inference, and generates condition prediction information of the patient on the basis of an inference result.

Description

Device for implementing an LLM model for predicting patient status based on consultation data and its control method

[0001] The present disclosure relates to a device for implementing a patient condition prediction LLM model. More specifically, the present disclosure relates to a device for implementing a patient condition prediction LLM model based on consultation data and a control method thereof.

[0002] In the field of psychiatry, the classification system is based on the International Classification of Diseases and the Diagnostic and Statistical Manual of Mental Disorders. It is recommended that diagnoses based on these classification systems be made by a skilled professional in the field of psychiatry through a psychiatric interview, medical history taking, mental status examination, and a comprehensive review of various records and tests.

[0003] While technologies utilizing various assessment methods applied to psychiatric diagnosis are being developed, the complexity of the language content and the need for empathy in psychiatric interviews and medical history taking currently hinder the utilization of interview-based diagnostic technologies. Nevertheless, the importance of language in psychiatric diagnosis is so significant that the need for technologies that utilize it is critical.

[0004] However, in the case of conventional technologies, there were limitations in that they could not universally respond to general patient inquiries and could only be used in limited ways. In addition, existing technologies that use deep learning had the disadvantage that some parts of artificial intelligence's inference could not be explained, which caused inconvenience to users.

[0005] The embodiment disclosed in this disclosure aims to provide a device for implementing a patient condition prediction LLM model based on consultation data that interprets the contents of a patient's consultation and helps infer explainable patient conditions based on the International Classification of Diseases and expert knowledge on mental health.

[0006] The purpose of the embodiment disclosed in the present disclosure is to provide a device for implementing a patient condition prediction LLM model based on counseling data that constructs mental health-related expert knowledge data and preprocesses it for artificial intelligence model learning.

[0007] The embodiment disclosed in the present disclosure aims to provide a device for implementing a large-scale language model (LLM) model for predicting patient status based on consultation data, utilizing techniques such as in-context learning and chain-of-thought, to fully utilize the excellent performance of the LLM.

[0008] The purpose of the embodiment disclosed in the present disclosure is to provide a device for implementing a consultation data-based patient condition prediction LLM model that sets up a training / inference process to derive results by citing consultation text so that the inference results of a large-scale language model can be logically explained.

[0009] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] In order to achieve the above-described technical task, a device for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure comprises: a memory storing at least one consultation data and at least one process for predicting a patient condition based on the consultation data; and a processor performing an operation according to the process, wherein the processor controls learning of an LLM model based on result data constructed by matching the International Classification of Diseases (hereinafter, ICD) and the consultation data, inputs the consultation data into the learned LLM model to make an inference, and generates patient condition prediction information based on an inference result.

[0011] At this time, according to the present disclosure, the processor can preprocess the consultation data by performing summary labeling for at least one of the experience and symptom.

[0012] Additionally, according to the present disclosure, the processor can preprocess the counseling data by performing labeling to segment out symptoms that are medically significant in terms of mental health among the vocabulary of the counseling data.

[0013] In addition, according to the present disclosure, the processor may perform preprocessing by processing at least one of the Diagnostic and Statistical Manual of Mental Disorders (hereinafter, DSM) and the International Classification of Disease (hereinafter, ICD) into a question-and-answer format to learn mental health knowledge.

[0014] Additionally, according to the present disclosure, when the processor modifies the LLM model, the LLM model can be controlled to learn mental health-related domain knowledge using question-and-answer for the DSM and the ICD.

[0015] In addition, according to the present disclosure, when the processor modifies the LLM model, the LLM model can be controlled to learn to identify psychiatrically significant symptoms based on the consultation data, and to present a basis for judgment by citing a section suggesting the significant symptoms.

[0016] In addition, according to the present disclosure, when the LLM model is used as is, the processor can control the patient to learn by inputting the interview content between the patient and the evaluator included in the consultation data as text, or when the LLM model is used as is, the patient's symptoms can be inferred by using a label indicating a section suggesting a mental health-related symptom, or when the LLM model is used as is, the patient's symptoms can be inferred by using a label indicating a section suggesting a mental health-related symptom generated in a data preprocessing process based on in-context learning.

[0017] In addition, a method for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure may include a step of acquiring consultation data of a patient by the processor; a step of controlling, by the processor, to learn an LLM model based on result data constructed by matching the International Classification of Diseases (hereinafter, ICD) and the consultation data; a step of inputting, by the processor, the consultation data into the learned LLM model to infer; and a step of generating, by the processor, prediction information of the patient's condition based on the inferred result.

[0018] At this time, the method for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure may further include a step of preprocessing by performing summary labeling for at least one of experience and symptom based on the consultation data.

[0019] In addition, the method for implementing a patient condition prediction LLM model based on counseling data according to the present disclosure may further include a preprocessing step of performing labeling to segment symptoms that are medically significant in terms of mental health among the vocabulary of the counseling data.

[0020] In addition, the method for implementing a patient condition prediction LLM model based on counseling data according to the present disclosure may further include a step of performing preprocessing by processing at least one of the Diagnostic and Statistical Manual of Mental Disorders (hereinafter, DSM) and the International Classification of Disease (hereinafter, ICD) into a question-and-answer format in order to learn mental health knowledge.

[0021] In addition, the method for implementing a patient condition prediction LLM model based on counseling data according to the present disclosure may further include a step of controlling the LLM model to learn mental health-related domain knowledge using question-and-answer for the DSM and the ICD when modifying the LLM model.

[0022] In addition, the method for implementing a patient condition prediction LLM model based on counseling data according to the present disclosure may further include, when modifying the LLM model, a step of identifying psychiatrically significant symptoms based on the counseling data by the LLM model; and a step of controlling the LLM model to learn to present a basis for judgment by citing a section suggesting the significant symptoms.

[0023] In addition, the method for implementing a patient condition prediction LLM model based on counseling data according to the present disclosure may further include at least one of the steps of: when the LLM model is used as is, controlling the content of an interview between the patient and the evaluator included in the counseling data to be input as text and learned; when the LLM model is used as is, inferring the symptoms of the patient using a label indicating a section suggesting a mentally health-related symptom; and when the LLM model is used as is, inferring the symptoms of the patient using a label indicating a section suggesting a mentally health-related symptom generated in a data preprocessing process based on in-context learning.

[0024] According to the aforementioned problem solving means of the present disclosure, user convenience can be improved by providing a result report that interprets the patient's consultation content and helps infer the patient's condition in an explainable manner based on the International Classification of Diseases and expert knowledge on mental health.

[0025] According to the aforementioned problem solving means of the present disclosure, mental health-related expert knowledge data can be constructed and preprocessed for artificial intelligence model learning to predict the patient's condition, thereby having the advantage of being able to predict the patient's mental health condition more precisely.

[0026] According to the aforementioned problem solving means of the present disclosure, the training / inference process can be set up to derive results by citing counseling text so that the inference results of a large-scale language model can be logically explained, thereby providing an advantage in more accurately predicting the mental health status of a patient.

[0027] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0028] Figure 1 is a configuration diagram of a device for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure.

[0029] FIG. 2 is a diagram illustrating a flowchart of a method for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure.

[0030] FIG. 3 is a diagram illustrating training an LLM model using preprocessed data according to the present disclosure.

[0031] FIG. 4 is a diagram illustrating a process of assisting diagnosis of a new patient by utilizing a learning model according to the present disclosure.

[0032] FIG. 5 is a diagram illustrating a process of adding a specialist label to existing patient consultation data according to the present disclosure.

[0033] Figure 6 is a diagram illustrating a table describing psychiatrically significant symptoms according to the present disclosure.

[0034] FIG. 7 is a diagram illustrating an example of generating data for learning mental health-related domain knowledge in a question-and-answer format according to the present disclosure.

[0035] FIG. 8 is a diagram illustrating an embodiment in which an LLM model according to the present disclosure learns questions and answers.

[0036] FIG. 9 is a diagram illustrating an embodiment in which an LLM model according to the present disclosure learns to determine whether there are significant symptoms and to provide a basis for judgment by citing symptoms and compartments.

[0037] FIG. 10 is a diagram illustrating an example of inputting the content of an interview between a patient and an evaluator as text into the input prompt of the LLM model according to the present disclosure.

[0038] FIG. 11 and FIG. 12 are diagrams illustrating an example of inferring a patient's symptoms using a chain-of-thought when the LLM model according to the present disclosure is used as is.

[0039] FIG. 13 is a diagram illustrating an example of inferring a patient's symptoms using in-context learning when the LLM model according to the present disclosure is used as is.

[0040] FIG. 14 is a diagram illustrating an example of generating a user's answer using Prompt Engineering when the LLM model according to the present disclosure is used as is.

[0041] FIG. 15 is a diagram illustrating a configuration of a device for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure.

[0042] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0043] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0044] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0045] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0046] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0047] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0048] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0049] The operating principle and embodiments of the present invention will be described with reference to the attached drawings below.

[0050] The present invention can be implemented not only in a server system but also in various devices capable of performing computational processing and providing results to a user. For example, the present invention according to the present disclosure may include a computer, a server device, and a mobile terminal, or may be implemented in any one of the following forms.

[0051] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0052] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0053] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0054] The related functions according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0055] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0056] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0057] The processor can create a neural network, train (or learn) a neural network, perform computations based on received input data, and generate information signals based on the results of the computations, or retrain the neural network.

[0058] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0059] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0060] Figure 1 is a configuration diagram of a device for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure.

[0061] Referring to FIG. 1, a device (100) for implementing a patient condition prediction LLM model based on consultation data includes an input unit (110), a sensor unit (120), a control unit (130), a display (140), a memory (150), a communication unit (160), and a camera (170).

[0062] The input unit (110) receives consultation data from the patient.

[0063] The sensor unit (120) senses consultation data by attaching a sensor.

[0064] The control unit (130) obtains the consultation data through the input unit (110), stores the consultation data in the memory (120), learns an LLM model based on result data constructed by matching the International Classification of Diseases (ICD) and the consultation data, inputs the consultation data into the learned LLM model to make inferences, generates prediction information on the patient's condition based on the inference results, and displays the generated prediction information on the patient's condition.

[0065] The display (140) displays a graphic image according to a control command from the control unit (130).

[0066] The memory (150) stores the above consultation data.

[0067] The communication unit (160) transmits and receives data with an external device (200).

[0068] Here, the external device (200) includes an external device such as a smartphone, PC, laptop, tablet PC, etc.

[0069] The camera (170) photographs a subject in front according to a control command from the control unit (130).

[0070] The control unit (130) preprocesses the acquired consultation data by performing summary labeling for at least one of the experiences and symptoms.

[0071] The control unit (130) preprocesses the acquired counseling data by performing labeling to segment out symptoms that are meaningful in terms of psychiatry among the vocabulary.

[0072] The control unit (130) performs preprocessing by processing data for learning mental health knowledge.

[0073] The control unit (130) performs preprocessing by processing at least one of the Diagnostic and Statistical Manual of Mental Disorders (hereinafter, DSM) and the International Classification of Disease (hereinafter, ICD) into a question-and-answer format in order to learn mental health knowledge.

[0074] When modifying the LLM model, the control unit (130) controls the LLM model to learn mental health-related domain knowledge using question-and-answer sessions about the DSM and ICD. A detailed description of this will be provided later in FIG. 7.

[0075] When modifying the LLM model, the control unit (130) controls the LLM model to learn to identify psychiatrically significant symptoms based on the consultation data and to provide a basis for judgment by citing sections suggesting the identified significant symptoms. A detailed description of this will be provided later in Fig. 9.

[0076] When the LLM model is used as is, the control unit (130) controls the learning by inputting the interview content between the patient and the evaluator included in the consultation data as text. A detailed description of this is provided later in Fig. 10.

[0077] When the control unit (130) utilizes the LLM model as is, it infers the patient's symptoms using labels that indicate sections that indicate psychiatrically significant symptoms. A detailed description of this is provided later in FIGS. 11 and 12.

[0078] When the control unit (130) utilizes the LLM model as is, it infers the patient's symptoms by utilizing labels indicating psychiatrically significant symptoms generated during the data preprocessing process based on in-context learning. A detailed description of this is provided later in Fig. 13.

[0079] Describes categorizing a patient's mental health status.

[0080] The control unit (130) categorizes the patient's mental state.

[0081] For example, a patient's mental state can be classified as high, medium, or low. Similarly, a patient's mental state can be categorized as severe, weak, normal, or good.

[0082] Additionally, if a patient has depression, the depression status can be divided into severe, moderate, mild, and none.

[0083] The application part of the present invention is described.

[0084] The present invention can be used as diagnostic aid data for doctors and as self-diagnosis data for patients during psychiatric treatment.

[0085] The control unit (130) can transmit patient condition prediction information to an external device (200) through the communication unit (160). Here, the external device (200) can include at least one of a hospital server, a psychiatrist's laptop, a PC, and a tablet PC.

[0086] According to the present invention, in the case of mental illness, there is a problem of poor patient accessibility to hospitals, but there is an advantage in that a psychiatrist can easily understand the patient's condition and take immediate action by using consultation data and patient condition prediction information before seeing the patient.

[0087] However, the components illustrated in FIG. 1 are not essential for implementing the patient condition prediction LLM model implementation device (100) based on consultation data according to the present disclosure, and thus the present invention described in this specification may have more or fewer components than the components listed above.

[0088] The communication unit (160) may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0089] The input unit (110) is for inputting video information (or signal), audio information (or signal), data, or information input from a user, and may include at least one camera, at least one microphone, and at least one user input unit. Voice data or image data collected by the input unit (110) may be analyzed and processed into a user control command.

[0090] The display (140) displays (outputs) information processed in the consultation data-based patient condition prediction LLM model implementation device (100). For example, the consultation data-based patient condition prediction LLM model implementation device (100) can display execution screen information of a running application program (e.g., an application), or UI (User Interface) or GUI (Graphical User Interface) information according to such execution screen information.

[0091] The memory (150) can store data supporting various functions of the patient condition prediction LLM model implementation device (100) based on consultation data, a program for the operation of the control unit (130), input / output data (e.g., music files, still images, videos, etc.), at least one process for predicting the patient condition based on consultation data can be stored, and a plurality of application programs (or applications) run by the patient condition prediction LLM model implementation device (100) based on consultation data, data for the operation of the device, and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0092] The memory (150) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (150) may be a database connected by wire or wirelessly, although separate from the present invention, and may be implemented as a database system.

[0093] The control unit (130) may be implemented as at least one core, at least one process for predicting a patient's condition based on consultation data, a memory for storing data for an algorithm for controlling the operation of components within the consultation data-based patient condition prediction LLM model implementation device (100) or a program reproducing the algorithm, and at least one processor (not shown) for performing the aforementioned operation using the process and / or data stored in the memory. In this case, the memory and the processor may be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

[0094] In addition, the control unit (130) may control any one or a combination of the components discussed above in order to implement various embodiments of the present disclosure described in FIGS. 2 to 15 below in accordance with the present invention. In addition, the control unit (130) may also be in the form of a processor (1603) of FIG. 15 below.

[0095] At least one component may be added or deleted to correspond to the performance of the components illustrated in Figure 1. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the system.

[0096] Meanwhile, each component illustrated in FIG. 1 refers to software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0097] FIG. 2 is a flowchart illustrating a method for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure. The present invention is performed under the control of a consultation data-based patient condition prediction LLM model implementation device (100) or a control unit (130).

[0098] Referring to Fig. 2, the control unit (130) obtains the patient's consultation data through the input unit (110) (S210).

[0099] The control unit (130) learns the LLM model based on the result data constructed by matching the International Classification of Diseases (ICD) and the consultation data according to at least one process stored in the memory (150) (S220).

[0100] The control unit (130) inputs the above-mentioned counseling data into the learned LLM model and makes an inference (S230).

[0101] The control unit (130) generates prediction information on the patient's condition based on the inferred result (S240).

[0102] The control unit (130) displays the patient's condition prediction information generated above (S250).

[0103] FIG. 3 is a diagram illustrating training an LLM model using preprocessed data according to the present disclosure.

[0104] Referring to Figure 3, implementing the present invention requires a process of training a large-scale language model. Specifically, after loading a pre-trained large-scale language model (LLM) for a general language, the LLM must be fine-tuned for specific training data to perform the desired task.

[0105] The overall learning process (300) is diagrammed in Figure 3 and is largely divided into two detailed processes: a preprocessing process for preparing data to be used for learning, and a learning process for performing fine-tuning using the prepared data.

[0106] Given a fine-tuned large-scale language model derived through the process in Fig. 3, the present invention performs a process of assisting in the diagnosis of a new patient by utilizing the model.

[0107] Although this specification is based on GPT, Lamma2 can also be used.

[0108] Further explanation of natural language processing.

[0109] Natural language processing (NLP) is an applied field that utilizes machine learning models to perform various tasks related to natural language. Specifically, machine learning algorithms have demonstrated outstanding performance in diverse applications such as document translation, summarization, and sentiment analysis.

[0110] In particular, among natural language processing models, models that generate language by utilizing large parameters are called large language models (LLMs), and their potential has been in the spotlight recently as the performance of LLMs has increased explosively. In particular, the generative pretrained transformer (GPT) is a large-scale language model based on the transformer structure, and GPT 3.5 (aka ChatGPT), which was released at the end of 2022, and GPT 4, which was released in the spring of 2023, are known to be capable of language generation at a level similar to that of humans.

[0111] Generally, the input of LLM is called a prompt, and the output is called completion. In the present invention, the input-output relationship of LLM is expressed in the form completion = LLM(prompt).

[0112] There are two main ways to utilize LLMs such as GPT for new tasks.

[0113] First, there is the method of directly utilizing pre-trained LLM (zero-shot inference).

[0114] Second, fine-tuning LLM for new tasks.

[0115] The first method is to ask about the desired task in the LLM input prompt and use the LLM output as the execution result of the task.

[0116] For example, if you set the input prompt to “What is the capital of Korea?” and think of LLM (“What is the capital of Korea”) as the answer to that question, you can measure LLM’s performance on the Q&A task.

[0117] The second way is to fine-tune the parameters of LLM while giving us the correct answer for the task we want, rather than using LLM as is.

[0118] For example, after fine-tuning LLM to satisfy LLM(x=1, y=2. Then, f(x,y))= “5”, LLM(x=2, y=3. Then, f(x,y))= “13”, and LLM(x=3, y=4. Then, f(x,y))= “25”, we can measure the regression performance corresponding to f(x,y) = x^2 + y^2 by asking for the values ​​of f(x,y) for new x,y.

[0119] Large-scale language models possess various properties. Specifically, a technique called in-context learning [1] can achieve higher performance than zero-shot inference without fine-tuning the LLM. Specifically, in-context learning involves presenting specific task-related examples within the LLM's input prompt and obtaining answers for new examples.

[0120] For example, when configuring the input prompt in the form of Prompt = “Question 1: What is the capital of Korea? Answer 1: Seoul, / , Question 2: What is the capital of Japan? Answer 2: Tokyo, / , Question 3: What is the capital of the United States? Answer 3: ”, recent models often produce the desired answer in the form of LLM(prompt) = “Washington”.

[0121] Among the methods to improve the performance of LLM, the chain-of-thought technique [2] is useful for tasks that require logical thinking.

[0122] Specifically, by explicitly explaining the logical thinking process for deriving answers when constructing LLM's input prompts, we facilitate LLM's ability to logically derive answers to new problems.

[0123] Prompt = “Question 1: Roger has 5 tennis balls and he bought 2 more cans of tennis. There are 3 tennis balls in each can. How many tennis balls does Roger have now? Answer 1: Roger started with 5 tennis balls. There are 6 tennis balls in 2 cans of 3 tennis balls each. Since 5 + 6 = 11, Roger now has 11 balls in total. Question 2: There are 16 balls in a box. Half of them are golf balls, and half of the golf balls are blue. How many blue golf balls are in the box? Answer 2:” It has been recently discovered in large-scale models that when an input prompt is constructed in the form of “Question 1: Roger has 5 tennis balls and he bought 2 more cans of tennis balls. There are 6 tennis balls in 2 cans of 3 tennis balls each. Since 5 + 6 = 11, Roger now has 11 balls in total. Question 2: There are 16 balls in a box. Half of them are golf balls, and half of the golf balls are blue. How many blue golf balls are in the box? Answer 2:”, it has been found that LLM can derive the correct answer while explaining the derivation process of the formula.

[0124] The most representative diagnostic system in medicine is the International Classification of Diseases (ICD) established by the World Health Organization (WHO). In the field of psychiatry, diagnosis is based on both the ICD and the Diagnostic and Statistical Manual of Mental Disorders (DSM). Diagnosis according to these classifications is recommended to be made by a skilled psychiatrist, incorporating a psychiatric interview, medical history, mental status examination, and a comprehensive review of various records and tests.

[0125] Accordingly, technologies utilizing various assessment methods applied to diagnosis in psychiatry are being developed. However, the complexity of the language content and the need for empathy in psychiatric interviews and medical history taking have hindered the utilization of interview-based diagnostic techniques. However, given the profound importance of language in psychiatric diagnosis, the need for technologies that utilize this approach is critical.

[0126] Large-scale language models, whose utility has recently increased, have demonstrated the ability to perform natural language processing, including summarizing conversational content, making inferences based on linguistic evidence, and understanding context. Therefore, utilizing large-scale language models to assist in diagnosis in psychiatry is expected to enable advanced diagnostic assistance based on the psychiatric interview, a crucial linguistic element in diagnosis.

[0127] In order for this diagnostic classification to be used in clinical practice, it is important to first consider whether it is designed to be consistent with the ICD and DSM classification criteria, and second, whether it is possible to explain which part of the psychiatric interview content the large-scale language model based its judgment on.

[0128] FIG. 4 is a diagram illustrating a process of assisting diagnosis of a new patient by utilizing a learning model according to the present disclosure.

[0129] Referring to Figure 4, when new patient consultation data is input, the fine-tuned LLM model is used to infer citation-based explainable experiences and symptoms.

[0130] The LLM model utilization process (400) is diagrammed in Figure 4. In the present invention, the technical elements utilized in Figures 3 and 4 are described in the following table of contents order.

[0131] 1. Data preprocessing process

[0132] a) The process of adding a specialist's label to existing patient consultation data.

[0133] b) The process of creating data to learn knowledge related to mental health

[0134] 2. Model learning and utilization process

[0135] a) How to learn and use

[0136] i) When modifying the LLM model

[0137] ii) If the LLM model is used as is

[0138] b) Technical elements

[0139] i) chain-of-thought

[0140] ii) In-context Learning

[0141] iii) Prompt engineering

[0142] FIG. 5 is a diagram illustrating a process of adding a specialist label to existing patient consultation data according to the present disclosure.

[0143] Describes the data preprocessing process.

[0144] This is a process (500) of adding a specialist's label to existing patient consultation data.

[0145] The raw input data, which is not preprocessed, is a textual record of the patient and evaluator interviews without any lexical changes, but the speaker may be identified.

[0146] The initial input data is evaluated by a psychiatric expert to generate two types of labels. Specifically, labels are generated to summarize the patient's experiences and symptoms, and labels identify sections of the interview vocabulary that suggest psychiatrically significant symptoms.

[0147] The summary label for the patient's experiences and symptoms is a summary of information from the patient and evaluator's interview that may be helpful in making a diagnosis, as determined by a mental health professional. This summary should not suggest a final diagnosis, but should instead include key experiences and symptoms relevant to the patient's current condition.

[0148] The label that indicates a section of the interview vocabulary that suggests a specific symptom is a section that a psychiatrist judges to indicate as suggesting a psychiatrically significant symptom in the entire interview between the patient and the evaluator.

[0149] Referring to Figure 5, for example, a section can be specified for a part suggesting 'suicidal thoughts'.

[0150] In the above example, the part suggesting the symptom (I sometimes feel like I wish tomorrow would never come. I keep having thoughts related to suicide.) is delimited by square brackets ([, ]), and the symptom (suii) it suggests is specified by brackets (<, >).

[0151] The names of the symptoms included in the brackets are determined in advance to clarify their types. For example, this means establishing a list of symptoms, such as the table (600) in Fig. 6.

[0152] Figure 6 is a diagram illustrating a table describing psychiatrically significant symptoms according to the present disclosure.

[0153] Referring to Table 6 (600) of FIG. 6, it can be divided into symptoms, labels, and related diagnoses.

[0154] In terms of symptoms, they can be categorized into re-experiencing, avoidance, suicidal ideation, and depression, and in terms of the labels for each symptom, they can be categorized into reexperiencing, avoidance, suicidal ideation, and depression. In terms of the related diagnosis for each symptom, they can be categorized into post-traumatic stress disorder and major depressive disorder. In addition, suicidal ideation <suii>can be specified as

[0155] FIG. 7 is a diagram illustrating an example of generating data for learning mental health-related domain knowledge in a question-and-answer format according to the present disclosure.

[0156] This paper describes the process of creating data for learning domain knowledge related to mental health.

[0157] It is possible to process mental health-related domain knowledge data to train large-scale language models to acquire specific domain knowledge using text data containing domain knowledge. Specifically, the contents of the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Diseases (ICD), which serve as the standard handbooks for psychiatric diagnosis, can be processed into a question-and-answer format, as in the example sentence (700) in Figure 7, and used for training large-scale language models.

[0158] In the case of example (700) of Fig. 7, the content of the question and answer requires review by a psychiatric expert, and is not used for learning other than the content specified in the ICD and DSM.

[0159] FIG. 8 is a diagram illustrating an embodiment in which an LLM model according to the present disclosure learns questions and answers.

[0160] Explains how to learn and use it.

[0161] First, we will explain the case of transforming the model.

[0162] The process of fine-tuning a pre-trained LLM is a two-step process.

[0163] The first phase enables LLM students to acquire domain knowledge related to mental health. This phase involves Q&A on the contents of the DSM and ICD obtained through data preprocessing.<Q, A> It uses pairs. When you input Q in the prompt, it learns to answer with the corresponding A, so LLM understands the correlation between the question and the answer, and fine-tunes it to understand the content based on this.

[0164] That is, the control unit (130) creates a prompt and completion (800) in the form of Fig. 8, and then performs fine-tuning to learn it.

[0165] FIG. 9 is a diagram illustrating an embodiment in which an LLM model according to the present disclosure learns to determine whether there are significant symptoms and to provide a basis for judgment by citing symptoms and compartments.

[0166] The second step involves learning to identify psychiatrically significant symptoms from patient consultation data and to cite segments suggesting symptoms to provide a basis for judgment. During the data preprocessing process, existing patient consultation data with symptom labeling added by a specialist is used. The raw, unprocessed data is segmented and entered into the prompt. If a labeled portion of the segment is found, the symptom and the segment suggesting it are entered as completion.

[0167] That is, the control unit (130) creates a prompt and completion (900) in the form of Fig. 9, and then performs fine-tuning to learn it.

[0168] With this type of fine-tuning, when a new patient's initial consultation data is segmented and entered, the task of informing the patient of the mentally significant symptoms present in the segment and the compartments suggesting them can be performed.

[0169] FIG. 10 is a diagram illustrating an example of inputting the content of an interview between a patient and an evaluator as text into the input prompt of the LLM model according to the present disclosure.

[0170] In the case of Fig. 10, the model is used as is.

[0171] Pre-trained LLMs can be used to infer psychiatrically relevant experiences and symptoms. The LLM input prompts textually record patient-evaluator interviews, and questions are then asked about the patient's experiences and symptoms as they appear in the interview.

[0172] That is, when the control unit (130) configures the input prompt (1000) in the form of FIG. 10, it can infer the patient's experience in the form of LLM (prompt)="Violent incident: anxiety and fear due to the violent act of the sixth cousin."

[0173] Next, we will explain Expandable AI.

[0174] Additionally, the control unit (130) converts audio into text and highlights and displays an area of ​​the text that serves as the basis for diagnosis.

[0175] For example, in the text of Figure 10, the area that serves as the basis for the diagnosis could be "He followed me around like a stalker, tried to skip ship me, and hit me in the ear while I was out." This part is highlighted and displayed.

[0176] FIG. 11 and FIG. 12 are diagrams illustrating an example of inferring a patient's symptoms using a chain-of-thought when the LLM model according to the present disclosure is used as is.

[0177] Explains Chain-of-thoght.

[0178] The control unit (130) can utilize a chain-of-thought to infer patient symptoms when utilizing a pre-trained LLM. This utilizes labels generated during the data preprocessing process, which indicate segments suggesting psychiatrically significant symptoms. Specifically, when configuring the LLM input prompt, a description of the logical thought process that elicits the patient's symptoms is explicitly provided, allowing the LLM to logically derive the patient's symptoms based on new interview content.

[0179] That is, when the control unit (130) configures the input prompt (1100, 1200) in the respective forms of FIGS. 11 and 12, it is possible to infer psychiatrically significant symptoms for the new interview content in the form of LLM (prompt) = "Re-experiencing means that dreams related to a traumatic event repeatedly appear, or that intense images and memories that seem to be happening again here and now. In the interview, the respondent said that he or she has a memory of being arrested by the police in the past and now gets goosebumps just seeing the police on the street. This shows an absorption in past memories and a strong physical sensation, so it can be said that he or she is showing re-experiencing symptoms."

[0180] FIG. 13 is a diagram illustrating an example of inferring a patient's symptoms using in-context learning when the LLM model according to the present disclosure is used as is.

[0181] Explains In-context Learning.

[0182] The control unit (130) can infer patient symptoms by utilizing in-context learning when utilizing the pre-trained LLM. This utilizes labels generated during the data preprocessing process, which indicate segments suggesting psychiatrically significant symptoms. Specifically, when configuring the LLM input prompt, only the aforementioned symptom labels are specified, allowing the LLM to derive patient symptom labels for new interview content.

[0183] That is, when the control unit (130) configures the input prompt (1300) in the form of FIG. 13, it can infer a symptom that is medically significant in terms of mental health in the form of LLM(prompt)="anxiety".

[0184] FIG. 14 is a diagram illustrating an example of generating a user's answer using Prompt Engineering when the LLM model according to the present disclosure is used as is.

[0185] Prompt engineering is a technique for effectively designing input prompts that generate user-preferred responses without requiring separate training of a pre-trained LLM. For example, the GPT-4 model, among LLMs, allows input of text into both "user" and "system" fields when configuring input prompts using the API. Enter only the task you wish to solve in "user", and enter specific instructions for completing the task entered in "system" in "user."

[0186] That is, when the control unit (130) configures the input prompt (1400) in the form of Fig. 14, it can infer psychiatrically significant symptoms from the interview content along with the quotation in the form of LLM(prompt)="Avoidance: {"Quote": "Then it seems like he is avoiding it on purpose. It feels somewhat similar, a man in his 40s."}".

[0187] FIG. 15 is a diagram illustrating a configuration of a device for implementing a patient condition prediction LLM model based on consultation data according to the present disclosure.

[0188] Referring to FIG. 15, a device for implementing a patient condition prediction LLM model based on consultation data includes a device (1500). The device (1500) may include a memory (1502), a processor (1503), a transceiver (1504), and a peripheral device (1501). In addition, as an example, the device (1500) may further include other configurations and is not limited to the above-described embodiment.

[0189] More specifically, the device (1500) of FIG. 15 may be an exemplary hardware / software architecture, such as an NDN device, an NDN server, a content router, etc. In this case, as an example, the memory (1602) may be a non-removable memory or a removable memory. In addition, as an example, the peripheral device (1601) may include a display, GPS, or other peripheral devices, and is not limited to the above-described embodiment.

[0190] In addition, as an example, the above-described device (1500) may include a communication circuit such as the transceiver (1504), and may perform communication with an external device based thereon.

[0191] Additionally, as an example, the processor (1503) may be at least one of a general-purpose processor, a digital signal processor (DSP), a DSP core, a controller, a microcontroller, ASICs (Application Specific Integrated Circuits), FPGA (Field Programmable Gate Array) circuits, any other type of integrated circuit (IC), and one or more microprocessors associated with a state machine. In other words, it may be a hardware / software configuration that performs a control role for controlling the above-described device (1500).

[0192] At this time, the processor (1503) can perform the same operation as the control unit (130) and execute computer-executable instructions stored in the memory (1502) to perform various essential functions of the patient condition prediction LLM model implementation device based on consultation data. For example, the processor (1503) can control at least one of signal coding, data processing, power control, input / output processing, and communication operations. In addition, the processor (1503) can control the physical layer, the MAC layer, and the application layers. In addition, for example, the processor (1503) can perform authentication and security procedures in the access layer and / or the application layer, and is not limited to the above-described embodiment.

[0193] For example, the processor (1503) can communicate with other devices via the transceiver (1504). For example, the processor (1503) can control a node to communicate with other nodes via a network through the execution of computer-executable instructions. That is, the communication performed in the present invention can be controlled. For example, the other nodes can be an NDN server, a content router, and other devices. For example, the transceiver (1504) can transmit an RF signal via an antenna and transmit the signal based on various communication networks.

[0194] In addition, as an example, MIMO technology, beamforming, etc. can be applied as antenna technology, and the present invention is not limited to the above-described embodiment. In addition, the signal transmitted and received through the transceiver (1504) can be modulated and demodulated and controlled by the processor (1503), and the present invention is not limited to the above-described embodiment.

[0195] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.< / suii>

Claims

1. A memory storing at least one consultation data and at least one process for predicting the patient's condition based on the consultation data; and A processor comprising: a processor for performing operations according to the above process; The above processor, Control to learn the LLM model based on the result data constructed by matching the International Classification of Diseases (ICD) and the above consultation data. Input the above counseling data into the learned LLM model and infer it. A device for implementing a patient condition prediction LLM model based on consultation data, which generates patient condition prediction information based on the inference results.

2. In the first paragraph, the processor, A device for implementing a patient condition prediction LLM model based on consultation data, which preprocesses the consultation data by performing summary labeling for at least one of the experiences and symptoms.

3. In the first paragraph, the processor, A device for implementing a patient status prediction LLM model based on counseling data, which preprocesses and labels the vocabulary of the above counseling data to segment out symptoms that are medically significant for mental health.

4. In the first paragraph, the processor, A device for implementing a patient status prediction LLM model based on counseling data, which performs preprocessing by processing at least one of the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Disease (ICD) into a question-and-answer format to learn mental health knowledge.

5. In the first paragraph, the processor, A device for implementing a patient condition prediction LLM model based on consultation data, which controls the LLM model to learn mental health-related domain knowledge by using question-answering about the DSM and the ICD when modifying the above LLM model.

6. In the first paragraph, the processor, When modifying the above LLM model, the LLM model identifies symptoms that are psychiatrically significant based on the above counseling data, A device for implementing a patient status prediction LLM model based on consultation data, which controls the LLM model to learn to provide a basis for judgment by citing a section suggesting the significant symptom.

7. In the first paragraph, the processor, If the above LLM model is used as is, the interview content between the patient and the evaluator included in the above consultation data is input as text and controlled for learning, or If the above LLM model is used as is, the patient's symptoms can be inferred by using labels that indicate compartments that suggest symptoms that are medically significant for mental health, or A device for implementing a patient status prediction LLM model based on consultation data, which infers the symptoms of the patient by utilizing labels indicating symptoms that are medically significant for mental health generated in the data preprocessing process based on in-context learning when the above LLM model is used as is.

8. A method for implementing a patient condition prediction LLM model based on consultation data, performed by a processor of a device, A step of obtaining patient consultation data by the above processor; A step of controlling the LLM model to be learned based on the result data constructed by matching the International Classification of Diseases (hereinafter, ICD) and the consultation data by the above processor; A step of inputting the counseling data into the learned LLM model and making an inference by the above processor; and A method for implementing a patient condition prediction LLM model based on consultation data, comprising a step of generating patient condition prediction information based on the inferred result by the processor.

9. In paragraph 8, A method for implementing a patient condition prediction LLM model based on consultation data, further comprising a step of preprocessing by performing summary labeling for at least one of experiences and symptoms based on the consultation data.

10. In paragraph 8, A method for implementing a patient status prediction LLM model based on counseling data, further comprising a step of performing labeling to segment symptoms that are medically significant in terms of mental health among the vocabulary of the above counseling data and performing preprocessing.

11. In paragraph 8, A method for implementing a patient status prediction LLM model based on counseling data, further comprising a step of performing preprocessing by processing at least one of the Diagnostic and Statistical Manual of Mental Disorders (hereinafter, DSM) and the International Classification of Disease (hereinafter, ICD) into a question-and-answer format to learn mental health knowledge.

12. In paragraph 8, A method for implementing a patient condition prediction LLM model based on consultation data, wherein when modifying the above LLM model, the method further includes a step of controlling the LLM model to learn domain knowledge related to mental health by using question-answering about the DSM and the ICD.

13. In paragraph 8, In the case of modifying the above LLM model, the step of identifying symptoms that are psychiatrically significant based on the above counseling data is performed by the LLM model; and A method for implementing a patient status prediction LLM model based on consultation data, further comprising a step of controlling the LLM model to learn to provide a basis for judgment by citing a section suggesting the significant symptom.

14. In paragraph 8, In case of using the above LLM model as it is, a step of controlling the content of the interview between the patient and the evaluator included in the consultation data to be input as text and learned; When the above LLM model is used as is, a step of inferring the patient's symptoms by using labels that indicate compartments that suggest symptoms that are medically significant for mental health; A method for implementing an LLM model for predicting patient status based on consultation data, which further includes at least one of the steps of: inferring the symptoms of the patient by utilizing a label indicating a symptom that is medically significant for mental health generated in a data preprocessing process based on in-context learning when the above LLM model is used as is; 15. A computer-readable recording medium having recorded thereon a computer program for implementing a patient condition prediction LLM model based on consultation data of Article 8.

Images