Method for providing explanation for patient condition prediction and electronic device therefor

Abstract

Disclosed are a method for providing explanation for patient condition prediction and an electronic device therefor. The method for providing explanation for patient condition prediction, according to the present disclosure, may comprise the steps of: acquiring time-series data of a patient; acquiring, from an artificial intelligence-based prediction model, a prediction result output by the prediction model on the basis of the time-series data; acquiring, from an explainable artificial intelligence-based explanation model, explanation data output by the explanation model on the basis of the time-series data and the prediction result; inputting, into an artificial intelligence-based language model, input data including the time-series data, the prediction result, and the explanation data; and acquiring, from the language model, natural language text data that is output by the language model on the basis of the input data and includes reasoning grounds of the prediction model for the prediction result.

Description

Method for providing an explanation of patient condition prediction and electronic device for the same

[0001] The present disclosure relates to a technology that provides an explanation of a patient condition prediction using an electronic device, and more specifically, to a technology that explains the basis of inference of a prediction model regarding the predicted result of a patient condition.

[0002] With the use of artificial intelligence models to predict patients' conditions, the speed and efficiency of diagnosis have increased significantly. This has helped supplement the shortage of medical personnel and substantially reduced the workload of saturated medical institutions, making it possible to provide a more pleasant medical environment for both medical staff and patients.

[0003] Specifically, AI models currently utilized in the medical field for the prediction and measurement of risks such as the Major Adverse Events Score (MAES), Sepsis Score, and Mortality Score can encompass a wide range of approaches, from traditional statistical models to the latest deep learning-based techniques. These risk prediction models primarily use inputs such as patients' Electronic Health Records (EHRs), clinical test data, vital signs, prescription records, imaging, and biometric data extracted from wearable devices to calculate the probability or score of a patient experiencing a critical condition (Major Adverse Event), sepsis, or mortality at a specific point in time or during hospitalization, providing this information to medical personnel to alleviate their workload. However, these AI models are fundamentally "black boxes," making it difficult for humans to interpret their decision-making processes. Consequently, medical staff and patients cannot determine the basis for the model's predictions, and patients lose trust in the model's diagnosis, leading to hesitation in complying even when the model diagnoses the need for medication or surgery.

[0004] Furthermore, while the medical field often requires meeting various regulatory requirements based on a clear understanding and evidence of the diagnosis, AI models sometimes fail to receive approval for clinical use due to regulations, as their diagnostic basis cannot be identified.

[0005] Furthermore, artificial intelligence models are at risk of bias during the learning process, and if the basis for the model's diagnoses cannot be explained, it can be very difficult to accurately identify such biases and eliminate the risk.

[0006] The present disclosure is proposed to solve the aforementioned problems and aims to gain patient trust in the model's diagnosis by providing the reasoning basis of the artificial intelligence model as natural language text.

[0007] Furthermore, the present disclosure aims to enhance the reliability of the reasoning grounds provided by an artificial intelligence model by utilizing various additional information that helps infer the reasoning grounds of the model.

[0008] The technical problems to be solved by the present disclosure are not limited to those described above, and other technical problems can be inferred from the following embodiments.

[0009] A method for providing an explanation regarding a patient condition prediction according to one embodiment through an electronic device may include: a step of acquiring time-series data of a patient; a step of acquiring a prediction result output by an artificial intelligence-based prediction model based on the time-series data from an artificial intelligence-based prediction model; a step of acquiring explanation data output by an explanation model based on the time-series data and the prediction result from an explainable artificial intelligence-based explanation model; a step of inputting input data including the time-series data, the prediction result, and the explanation data into an artificial intelligence-based language model; and a step of acquiring natural language text data from the language model that includes the reasoning basis of the prediction model for the prediction result, which is output by the language model based on the input data.

[0010] In one embodiment, the time series data may include information regarding the time at which the patient's vital sign was measured, the value of the vital sign, and the type of the vital sign, and the prediction result may include at least some of the patient's sepsis risk score, major adverse events risk score, and mortality risk score, and the explanatory data may include the input attribution score of the time series data to the prediction result.

[0011] In one embodiment, the input step may include the step of concatenating the time series data, the prediction result, and the explanatory data to generate input data corresponding to the input format of the language model.

[0012] In one embodiment, the natural language text data may include a biosignal specified by the explanatory data among a plurality of biosignals constituting the time series data as the basis for inference of the prediction model.

[0013] In one embodiment, the method for providing an explanation regarding the prediction of the patient's condition may further include the step of obtaining anomaly data in which the anomaly degree is evaluated based on rules for at least some of the biosignals among the plurality of biosignals constituting the time series data. In this case, the input data further includes the anomaly data, and the natural language text data may include, among the plurality of biosignals constituting the time series data, at least some of the biosignals identified by the explanation data or biosignals determined to be abnormal according to the anomaly data as the basis for inference of the prediction model.

[0014] In this regard, the abnormality degree data may include whether at least some of the biological signals among the plurality of biological signals constituting the time series data are normal or abnormal, a normal range value for the at least some of the biological signals, and a class value classifying the abnormality degree of the biological signal determined to be abnormal.

[0015] Meanwhile, the above natural language text data may include a biosignal identified by the above explanatory data and simultaneously judged to be abnormal according to the degree of abnormality data as the first-rank inference basis of the prediction model, a biosignal identified by the above explanatory data but not judged to be abnormal according to the degree of abnormality data as the second-rank inference basis of the prediction model, and a biosignal judged to be abnormal according to the degree of abnormality data but not identified by the above explanatory data as the third-rank inference basis of the prediction model.

[0016] In one embodiment, the method for providing an explanation regarding the prediction of the patient's condition may further include the step of acquiring Electronic Health Records (EHR) data including the patient's past medical history and treatment history. In this case, the input data further includes the EHR data, and the natural language text data may include, among a plurality of biosignals constituting the time series data, at least some of the biosignals specified by the explanation data or the biosignals specified by the EHR data as the basis for inference of the prediction model.

[0017] In this regard, if the biosignal specified by the EHR data corresponds to the basis for inference of the prediction model, the natural language text data may additionally include text information that explains the patient's past medical history and treatment history as the basis for judgment regarding the patient's symptoms, and explains the diagnostic content corresponding to the biosignal specified by the EHR data as the result of judgment regarding the patient's symptoms.

[0018] In one embodiment, the method for providing an explanation regarding the prediction of the patient's condition may further include the step of obtaining an expert's analysis result regarding the time series data and the prediction result. In this case, the input data further includes the expert's analysis result, and the natural language text data may include at least some of the biosignals specified by the explanation data or the biosignals specified by the expert's analysis result among a plurality of biosignals constituting the time series data as the basis for inference of the prediction model.

[0019] Meanwhile, according to one embodiment, an electronic device that provides an explanation regarding a patient condition prediction includes a transceiver configured to communicate with the outside; a memory for storing instructions; and a processor. The processor controls the transceiver and the memory to acquire time-series data of the patient, acquires a prediction result output by an artificial intelligence-based prediction model based on the time-series data, acquires explanation data output by an explanation model based on explainable artificial intelligence based on the time-series data and the prediction result, inputs input data including the time-series data, the prediction result, and the explanation data into an artificial intelligence-based language model, and acquires natural language text data including the reasoning basis of the prediction model for the prediction result output by the language model based on the input data.

[0020] Specific details of other embodiments are included in the detailed description and drawings.

[0021] According to the present disclosure, by providing the reasoning basis of an artificial intelligence model as natural language text, the trust of medical staff and patients in the model's diagnosis can be enhanced, and the explainability of the diagnosis results can be improved.

[0022] In addition, according to the present disclosure, the degree of abnormality, the patient's past medical history, treatment history, and expert opinions are comprehensively reflected in the model's prediction process to provide patient-tailored medical services and facilitate communication between the patient and medical staff.

[0023] The effects of the invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the claims.

[0024] FIG. 1 is a schematic diagram illustrating a system that provides an explanation of patient condition prediction according to the present disclosure.

[0025] FIG. 2 is a flowchart illustrating a method for providing an explanation of a patient condition prediction according to one embodiment.

[0026] Figures 3 to 6 are exemplary diagrams in which input data is input into a language model and natural language text data is output.

[0027] FIG. 7 is a block diagram illustrating an electronic device according to various embodiments.

[0028] Specific embodiments will be described below with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, this is merely illustrative and the disclosed embodiments are not limited thereto.

[0029] In describing the embodiments, if it is determined that a detailed description of the related prior art could unnecessarily obscure the essence of the disclosed embodiments, such detailed description will be omitted. Furthermore, the terms described below are defined in consideration of their functions in the disclosed embodiments, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments and should not be limiting. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0030] The terms used in the embodiments have been selected to be as widely used as possible, taking into account their functions in the present disclosure; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section. Therefore, terms used in the present disclosure should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0031] When a part throughout the specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "...module" as used in the specification refer to a unit that processes at least one function or operation; this unit may be implemented in hardware or software, or as a combination of hardware and software, and may not be clearly distinguishable in terms of specific operation, unlike the illustrated examples.

[0032] The expression "at least one of a, b, and c" described throughout the specification may include 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c', or 'a, b, and c all'.

[0033] The "terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all types of handheld-based wireless communication devices such as communication-based terminals like IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), smartphones, tablet PCs, etc.

[0034] In the following description, terms such as "transmission," "communication," "sending," "receiving," and other terms with similar meanings regarding signals or information include not only the direct transmission of signals or information from one component to another but also transmission through other components.

[0035] In particular, "transmitting" or "transmitting" a signal or information as a component indicates the final destination of the signal or information, not the direct destination. The same applies to the "reception" of the signal or information. Furthermore, in this specification, two or more data or information are "related" means that if one data (or information) is obtained, at least a portion of another data (or information) can be obtained based thereon.

[0036] Additionally, terms such as first, second, etc., may be used to describe various components, but said components should not be limited by said terms. said terms may be used for the purpose of distinguishing one component from another.

[0037] For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0038] In describing the embodiments, technical details that are well known in the technical field to which the present invention belongs and are not directly related to the present invention are omitted. This is intended to convey the essence of the present invention more clearly without obscuring it by omitting unnecessary explanations.

[0039] For the same reason, some components in the attached drawings have been exaggerated, omitted, or schematically depicted. Additionally, the size of each component does not entirely reflect its actual dimensions. Identical or corresponding components in each drawing have been assigned the same reference numbers.

[0040] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0041] It will be understood that each block of the process flow diagrams and combinations of the flow diagrams can be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means to perform the functions described in the flow diagram block(s). Since these computer program instructions can also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory can also produce a manufactured item containing the means of instruction to perform the function described in the flow diagram block(s). Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0042] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to their corresponding functions.

[0043] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0044] FIG. 1 is a schematic diagram illustrating a system that provides an explanation of a patient condition prediction according to the present disclosure. It will be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components shown in FIG. 1.

[0045] Referring to FIG. 1, the system includes an electronic device (110), a database (120), a user terminal (130), and a network (140). The electronic device (110), the database (120), and the user terminal (130) can communicate with each other through a network, and the network includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof, and is a data communication network in a comprehensive sense that enables each system component shown in FIG. 1 to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks. Wireless communication may include, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra-wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), but is not limited to these.

[0046] The electronic device (110) is a device that explains why the model predicted such a result regarding the patient's condition predicted by the artificial intelligence model. To this end, the electronic device (110) can obtain information from various sources. For example, the electronic device (110) can obtain data that the artificial intelligence model uses for prediction (inference) from a database of a medical institution, and the prediction result of the artificial intelligence model can be obtained from a cloud server that provides the model's service. In addition, since data explaining the model's decision-making process can be generated through a separate entity that analyzes the model's decision-making logic, the electronic device (110) can also obtain such data from that entity. Although FIG. 1 is illustrated as if the electronic device (110) receives information from a single database (120) through a network (140), this is for convenience of explanation, and depending on the embodiment, the number and type of sources from which the electronic device (110) obtains information may vary.

[0047] Meanwhile, the electronic device (110) can generate information that explains the basis of the model's prediction and store the generated information within the electronic device (110) or in a database or cloud server linked to the electronic device (110). At this time, the electronic device (110) can create a database that is easy to query later based on the type of the prediction result, the medical institution where the model is used, or the user account that requested the model's prediction by tagging the generated information with an identifier corresponding to the type of the prediction result, the medical institution where the model is used, or the user account that requested the prediction.

[0048] Additionally, the electronic device (110) may generate information that can explain the basis for the model's prediction and provide it to a user terminal (130) via a network (140). The user terminal (130) is a terminal of a user account that uses the service provided by the electronic device (110) or the service provided by the model, and may be, for example, a computer of a medical staff member working in a medical institution or a smartphone of a patient hospitalized in a medical institution, but is not limited thereto. The electronic device (110) may generate information that can explain the basis for the model's prediction in response to a request from the user terminal (130) and transmit the generated information to the user terminal (130) that sent the request.

[0049] According to an embodiment, the user terminal (130) transmitting the information generated by the electronic device (110) may be a separate terminal from the device that transmitted the request. For example, the electronic device (110) may generate information that can explain the basis for the model's prediction at the request of a doctor (administrator account) and transmit the generated information to a user terminal (130) logged in with a patient account. In this regard, if the terminal requesting the generation of information and the terminal to transmit the information are different, or if the terminal to transmit the information is not a pre-registered terminal (e.g., a terminal of a medical institution), the electronic device (110) may encrypt the generated information with the private key of the electronic device (110) and transmit the public key of the electronic device (110) along with the generated information to the terminal receiving the information. This is to prevent the information from being leaked to an unspecified third party, as the information generated by the electronic device (110) may be sensitive information containing the result of predicting the patient's condition and the basis thereof.

[0050] Meanwhile, the electronic device (110) can generate information that explains the basis for the model's prediction, process the information in response to a request from the user terminal (130), and then transmit it to the user terminal (130). For example, in response to a request from the user terminal (130), the electronic device (110) may apply graphic effects (e.g., highlighting) or font effects (e.g., bolding or changing the font size) to parts of the generated information that correspond to the basis for the model's prediction. As another example, for users who are not skilled personnel such as medical staff, the electronic device (110) may translate words written in English in the generated information into Korean or convert technical terms into general terms in response to a request from the user terminal (130). As yet another example, in response to a request from the user terminal (130), the electronic device (110) may summarize the generated information into a number of characters or less. This is to prevent an excessive amount of information from being provided to medical staff who must treat many patients within a limited time.

[0051] In relation to the above, a more detailed explanation will be provided through the drawings below.

[0052] FIG. 2 is a flowchart illustrating a method for providing an explanation of a patient condition prediction according to one embodiment. Although the method illustrated in FIG. 2 is described as being performed by the electronic device (110) described above, this is exemplary and the method illustrated in FIG. 2 may be performed by other devices or by a combination of the electronic device (110) and separate devices.

[0053] In step S210, the electronic device (110) can acquire the patient's time series data.

[0054] In one embodiment, the patient's time series data may include information about the time at which the patient's vital sign was measured, the value of the vital sign, and the type of the vital sign.

[0055] In an embodiment, the electronic device (110) may acquire all of the patient's time-series data from a single device, server, or database, and depending on the type of time-series data, some may be acquired from a digital device that measures data by contacting the patient's body or a device that assists said digital device (hereinafter collectively referred to as the 'first device'), and the remainder may be acquired from a device that stores medical information recorded, measured, or analyzed for the patient (hereinafter collectively referred to as the 'second device'). The electronic device (110) may also acquire data from the first device by accessing an IoT (Internet of Things) application, and the network used by the electronic device (110) and the first device to access the IoT application may be the same or different from the network used by the electronic device (110) to communicate with the second device.

[0056] According to one embodiment, the time series data may include information regarding the time at which the patient's biosignal was measured, the value of the biosignal, and the type of the biosignal, and the electronic device (110) may generate the time series data by acquiring the time at which the patient's biosignal was measured, the value of the biosignal, and the type of the biosignal, and then inputting them into a template corresponding to the time series data.

[0057] In step S220, the electronic device (110) can obtain a prediction result output by the prediction model based on time series data from an artificial intelligence-based prediction model.

[0058] According to one embodiment, the prediction model may include a Large Language Model (LLM) based on a Transformer architecture. For example, the prediction model may analyze and tokenize text-based input, and a self-attention mechanism may be applied.

[0059] In addition, the prediction result output by the prediction model may include information regarding the probability of an abnormality occurring in the patient's predicted target state within the time interval from the current time to the prediction time. For example, the prediction result output by the prediction model may include at least some of the patient's risk of developing sepsis, risk of developing an acute severe event, and risk of developing acute death.

[0060] In step S230, the electronic device (110) can obtain explanatory data output by an explanatory model based on time series data and prediction results from an explainable artificial intelligence (XAI)-based explanatory model.

[0061] In one embodiment, the explanatory data output by the explanatory model may include the input attribution score of the time series data for the prediction result output by the prediction model. Specifically, the XAI-based explanatory model may represent the extent to which a feature among the time series data contributed to the prediction result output by the prediction model as the input attribution score for each feature included in the time series data.

[0062] For example, an explanatory model based on the SHAP (SHapley Additive exPlanations) technique utilizes Shapley values ​​from cooperative game theory. The input contribution of each feature to the prediction can be calculated by considering all possible subsets of features that include or exclude that feature. Assuming that the prediction result is produced by adding the baseline value (referring to the output of the prediction model for neutral inputs, such as the average prediction for all inputs) and the sum of the input contributions of each feature, as shown in Equation 1 below, the SHAP-based explanatory model can evaluate how the prediction result changes when each feature is included or excluded from the prediction model's input and calculate the difference between each prediction result. In Equation 1, is the prediction result, is the reference value, represents the input contribution of feature i.

[0063] [Mathematical Formula 1]

[0064]

[0065] These identical data can be converted into a single sentence and input into input format A. In this case, the language model can easily identify the progression of a specific biosignal over time. As another example, the electronic device (110) can perform interpolation based on the measured biosignal values ​​as a preprocessing step for biosignal data types (electrocardiogram, heart rate variability, blood pressure, pulse, respiration, blood sugar, temperature, etc.) for which the time-series change pattern of the biosignal data value needs to be considered in order to output a prediction result using a prediction model among the types of biosignal data.

[0066] Meanwhile, the electronic device (110) can input the prediction results output by the prediction model into input format A along with time series data. An example of the prediction result illustrated in FIG. 3 is MAES, which indicates the risk of an acute severe event occurring to a patient within a certain period of time as a result of the prediction model analyzing the time series data.

[0067] Meanwhile, the electronic device (110) can input the explanatory data output by the explanatory model into input format A along with time series data and prediction results. In FIG. 3, the explanatory model can output the contribution (value) of each pulse (Pulse), respiration (RESP), systolic blood pressure (SBP), diastolic blood pressure (DBP), and temperature (TEMP) to the prediction result over time in the form of time series data, and the electronic device (110) can input this as explanatory data to the language model as part of the input data. In the embodiment, the electronic device (110) may use the explanatory data output by the explanatory model directly as input data, or if the explanatory data is written in a format other than a pre-set format, it may convert it into a pre-set format (e.g., data composed of numbers and text) and use it as input data. For example, the electronic device (110) can convert the explanatory data in the form of a graph of FIG. 3 into structured data (Tabular Data) in which the contribution (value) for each biosignal is matched over time.

[0068] Meanwhile, in Figure 3, the decrease in PULSE from 72 to 48 at 0.0 hours prior (i.e., immediately before the generation of time series data) was identified as the basis for the prediction model's inference, because it was determined that the contribution (value) of PULSE at 0.0 hours to the prediction of MAES as 80 points in the explanatory data was the greatest.

[0069] According to one embodiment, regarding the prediction of MAES as 80 points, there may be changes in the same biosignal at multiple points in time, and various natural language text data may be generated based on this. Additionally, there may be changes in multiple biosignals at the same point in time, and various natural language text data may be generated based on this. Thus, in response to a single prediction result, multiple natural language text data may exist for each point in time or biosignal, and the electronic device (110) may select specific natural language text data among them and provide it to the user terminal (130). The criteria for the electronic device (110) to select natural language text data may be set in various ways; for example, the electronic device (110) may select the most recent natural language text data among multiple natural language text data distinguished by time. As another example, the electronic device (110) may select natural language text data containing the most biosignals with a relatively high input contribution based on explanatory data among multiple natural language text data distinguished by biosignals. As another example, the electronic device (110) can select the natural language text data containing the most biosignals with a relatively high input contribution based on the explanatory data as the first priority and the most recent natural language text data as the second priority among multiple natural language text data distinguished by time and biosignals, and provide both to the user terminal (130).

[0070] Next, Figure 4 describes an example in which input data including abnormality degree data, time series data, prediction results, and explanatory data is input into a language model and natural language text data is output.

[0071] In one embodiment, the abnormality degree data may include whether at least some of the biological signals among the plurality of biological signals constituting the time series data are normal or abnormal, a normal range value for at least some of the biological signals, and a class value classifying the abnormality degree of the biological signal determined to be abnormal.

[0072] In one embodiment, the electronic device (110) can obtain anomaly data in which the anomaly degree is evaluated based on rules for at least some of the biosignals among a plurality of biosignals constituting time series data. Subsequently, the electronic device (110) can input input data including anomaly data, time series data, prediction results, and explanatory data into a language model.

[0073] Natural language text data output from a language model may include, among a plurality of biosignals constituting time series data, at least some of the biosignals identified by explanatory data or biosignals determined to be abnormal according to abnormality degree data as the basis for inference of a prediction model. For example, the natural language text data may include a biosignal identified by explanatory data and simultaneously determined to be abnormal according to abnormality degree data as the first-rank basis for inference of the prediction model, a biosignal identified by explanatory data but not determined to be abnormal according to abnormality degree data as the second-rank basis for inference of the prediction model, and a biosignal determined to be abnormal according to abnormality degree data but not identified by explanatory data as the third-rank basis for inference of the prediction model.

[0074] As another example, natural language text data may include biological signals corresponding to a class with a high degree of abnormality among biological signals judged as abnormal based on abnormality degree data as the basis for inference of a prediction model. Furthermore, if there are multiple biological signals corresponding to a class with a high degree of abnormality, natural language text data may include the biological signal that deviates the most from the normal range value among such multiple biological signals as the basis for inference of a prediction model. In this case, the degree to which the biological signal deviates from the normal range value may be calculated as an absolute value or as a relative value of the deviation from the normal range. For example, in Figure 4, the biological signals judged to have a 'high' degree of abnormality are CRP (C-Reactive Protein), PLATELET, SBP, and DBP, and natural language text data may include CRP, which deviates the most relatively from the normal range among these, as the basis for inference of a prediction model.

[0075] Meanwhile, in Figure 4, HEMATOCRIT was identified as the basis for inference of the prediction model based on rule-based abnormality judgment. Although HEMATOCRIT belongs to a class with a low degree of abnormality, it can be identified as the basis for inference of the prediction model because it has a significant impact on the prediction of MAES according to predefined rules.

[0076] Next, Figure 5 describes an example in which input data including electronic health record data, time series data, prediction results, and explanatory data is input into a language model and natural language text data is output.

[0077] In one embodiment, the electronic device (110) can acquire electronic health record data including the patient's past medical history and treatment history. Subsequently, the electronic device (110) can input input data including electronic health record data, time series data, prediction results, and explanatory data into a language model.

[0078] Natural language text data output from a language model may include at least some of the biosignals specified by explanatory data or biosignals specified by EHR data among a plurality of biosignals constituting time series data as the basis for inference of a prediction model. For example, the natural language text data may include a biosignal specified by explanatory data and simultaneously specified by EHR data as the first-rank basis for inference of the prediction model, a biosignal specified by explanatory data but not specified by EHR data as the second-rank basis for inference of the prediction model, and a biosignal specified by EHR data but not specified by explanatory data as the third-rank basis for inference of the prediction model.

[0079] Meanwhile, in the case where there are multiple biosignals identified according to EHR data, natural language text data may include the biosignal identified most frequently according to EHR data as the basis for inference in the prediction model, or the biosignal identified most recently according to EHR data as the basis for inference in the prediction model.

[0080] Meanwhile, if a biosignal identified by EHR data corresponds to the basis for inference of the prediction model, the natural language text data may additionally include text information that explains the patient's past medical history and treatment history as the basis for judgment regarding the patient's symptoms, and explains the diagnosis corresponding to the biosignal identified by the EHR data as the result of judgment regarding the patient's symptoms. In Figure 5, by referring to the patient's past medical history and treatment history as EHR data, it is explained in the natural language text data as the 'basis for judgment' that the patient had chronic renal failure in their past history and that there is a record in the nurse's treatment records of administering dopamine to maintain blood pressure. Furthermore, according to this basis for judgment, the diagnosis that the decrease in HEMATOCRIT to 27 suggests exacerbation of metabolic acidosis due to renal failure is explained as the 'result of judgment'. According to an embodiment related thereto, when the patient's past medical history is identified in the EHR, the corresponding treatment history can also be identified, and a diagnosis can be generated based on the identified medical history and treatment history. The generation of the diagnosis can be rule-based, but a pre-trained language model can also generate the diagnosis using the medical history and treatment history as input.

[0081] Next, Figure 6 describes an example in which input data including expert analysis results, time series data, prediction results, and explanatory data is input into a language model and natural language text data is output.

[0082] In one embodiment, the electronic device (110) may obtain an expert's analysis results regarding time series data and prediction results. Subsequently, the electronic device (110) may input input data including the expert's analysis results, time series data, prediction results, and explanatory data into a language model. According to one embodiment, the format of the input data including the expert's analysis results, time series data, prediction results, and explanatory data may have various forms, such as text or images. According to one embodiment, the electronic device (110) may obtain input data including the expert's analysis results, time series data, prediction results, and explanatory data from various electronic devices, including a user terminal (130).

[0083] Natural language text data output from a language model may include, among a plurality of biosignals constituting time series data, at least some of the biosignals specified by explanatory data or biosignals specified by expert analysis results as the basis for inference of a prediction model. For example, the natural language text data may include a biosignal specified by explanatory data and simultaneously specified by expert analysis results as the first-ranked basis for inference of the prediction model, a biosignal specified by explanatory data but not specified by expert analysis results as the second-ranked basis for inference of the prediction model, and a biosignal specified by expert analysis results but not specified by explanatory data as the third-ranked basis for inference of the prediction model.

[0084] In one embodiment, the language model can be trained in advance using few-shot analysis results including expert opinions. That is, while FIG. 6 illustrates that analysis results are input along with time-series data, prediction results, and explanatory data during the inference of the language model, depending on the embodiment, the analysis results may be data input only during the training process of the language model. Since labeling expert opinions for a few cases where time-series data and prediction results are matched requires significant cost and time, the language model can generate natural language text data that accurately contains the basis for the prediction model's inference for various input data by using analysis results that include a small number of expert opinions through a few-shot learning technique.

[0085] In FIG. 6, HEMATOCRIT decreased from 28 to 27 at 0.0 hours. According to expert opinion (medical domain knowledge), "HEMATOCRIT was measured at 27 0.50 hours ago, indicating a very serious patient condition, and MAES is predicted to be high at 90 points," this was identified as the basis for inference of the prediction model. However, when the language model identifies the basis for inference of the prediction model, it does not consider only the analysis results for a single case; depending on the embodiment, it may consider one or more cases of biosignals that are identical to or highly correlated with the biosignal identified by the explanatory data. For example, if the biosignal identified by the explanatory data is CRP, the language model may refer to expert opinion regarding CRP and one biosignal most highly correlated with CRP. At this time, the correlation between biosignals may be defined in advance in the form of a table and stored in a storage medium linked to a database (120) or an electronic device (110).

[0086] In this way, through FIGS. 4 to 6, embodiments in which abnormality degree data, electronic health record data, and information on expert opinions are added as input data to a language model and utilized as a basis for inference regarding the prediction result of a prediction model have been individually described. In addition to the embodiments of FIGS. 4 to 6, the electronic device (110) may be implemented such that the language model includes information on the basis for inference regarding the prediction result of a prediction model in the output data by inputting at least two of the abnormality degree data, electronic health record data, and information on expert opinions into the language model as input data.

[0087] According to the embodiments, among the abnormality degree data, electronic health record data, and expert opinion information, it can be assumed that two or more are added as input data to input format A.

[0088] For example, the electronic device (110) may prioritize abnormality data among the three elements described above, prioritize electronic health record data as the second priority, and prioritize information on expert opinions as the third priority. Accordingly, if there is a biosignal that is duplicated by the abnormality data among the biosignals specified by the explanatory data, the electronic device (110) may include it as the basis for inference of the prediction model. However, if there are multiple biosignals that are duplicated by the explanatory data and abnormality data, the electronic device (110) may include the biosignal that is duplicated by the electronic health record data among them as the basis for inference of the prediction model, and if there are still multiple biosignals specified, it may include the biosignal that is duplicated by information on expert opinions as the basis for inference of the prediction model. If there are no biosignals duplicated by the first and second priority elements, or if there are no biosignals duplicated by the first to third priority elements, all biosignals specified by the element with the highest priority (which may include cases where there are multiple biosignals) may be included as the basis for inference of the prediction model.

[0089] Meanwhile, as another example, the electronic device (110) may input a prompt separately from the input data requesting that, when considering two or more of the three elements described above, it determine which element is more important to the language model and prioritize it.

[0090] FIG. 7 is a block diagram illustrating an electronic device (110) according to various embodiments.

[0091] In an embodiment, the electronic device (110) may include a memory (111), a processor (113), and a transceiver (115). The electronic device (110) may exchange data with the outside through a transceiver (115) configured to communicate with a server or client.

[0092] The processor (113) may perform at least one method described above through the drawings or support an external device in performing the method. The memory (111) may store information for performing at least one method described above through the drawings. The memory (111) may be volatile memory or non-volatile memory.

[0093] The processor (113) can control the electronic device (110) to execute a program and provide information. The code of the program executed by the processor (113) can be stored in memory (111).

[0094] In one embodiment, the processor (113) is connected to the memory (111) and the transceiver (115) to acquire time series data of a patient, acquire a prediction result output by the prediction model based on the time series data from an artificial intelligence-based prediction model, acquire explanation data output by the explanation model based on the time series data and the prediction result from an explainable artificial intelligence-based explanation model, input data including the time series data, the prediction result and the explanation data into an artificial intelligence-based language model, and acquire natural language text data including the reasoning basis of the prediction model for the prediction result output by the language model based on the input data from the language model.

[0095] The electronic device (110) illustrated in FIG. 7 is illustrated only with components related to the present embodiment. Therefore, it can be understood by those skilled in the art related to the present embodiment that other general-purpose components may be included in addition to the components illustrated in FIG. 7.

[0096] The device according to the embodiments described above may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, and user interface devices such as a touch panel, a key, a button, etc. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium is computer-readable, stored in memory, and can be executed on a processor.

[0097] The present embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions under the control of one or more microprocessors or other control devices. Similar to how components may be implemented as software programming or software elements, the present embodiment may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present embodiment may employ prior art for electronic configuration, signal processing, message processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

[0098] The aforementioned embodiments can be implemented as artificial intelligence (AI) through the processor and memory of an electronic device. The processor may consist of one or more processors, and the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (digital signal processors), graphics-dedicated processors such as GPUs and VPUs (vision processing units), or AI-dedicated processors such as NPUs. The one or more processors may be controlled to process input data according to predefined operation rules or AI models stored in memory. Alternatively, if the one or more processors are AI-dedicated processors, the AI-dedicated processors may be designed with a hardware structure specialized for processing a specific AI model.

[0099] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform a desired characteristic (or objective) are created by a basic artificial intelligence model being trained using a number of learning data by a learning algorithm. Such learning may be performed on the electronic device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0100] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and can perform neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include, but is not limited to, deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), bidirectional recurrent deep neural networks (BRDNN), or deep Q-networks.

[0101] The aforementioned embodiments are merely examples, and other embodiments may be implemented within the scope of the claims set forth below.

Claims

1. In a method for providing an explanation of patient condition prediction through an electronic device, Step of acquiring patient time series data; A step of obtaining a prediction result output by an artificial intelligence-based prediction model based on the time series data from the prediction model; A step of obtaining explanatory data output by an explanatory model based on an explainable artificial intelligence, based on the time series data and the prediction result; A step of inputting input data including the above time series data, the above prediction result, and the above explanatory data into an artificial intelligence-based language model; and A step comprising obtaining natural language text data from the above language model, the natural language text data including the reasoning basis of the prediction model for the prediction result output by the language model based on the input data. Method for providing an explanation of patient condition prediction.

2. In Paragraph 1, The above time series data is, Information including the time at which the patient's vital sign was measured, the value of the vital sign, and the type of the vital sign, Method for providing an explanation of patient condition prediction.

3. In Paragraph 1, The above prediction results are, Including at least some of the sepsis risk score, major adverse events risk score, and mortality risk score of the above patient Method for providing an explanation of patient condition prediction.

4. In Paragraph 1, The above description data is, including the input attribution score of the time series data for the above prediction result, Method for providing an explanation of patient condition prediction.

5. In Paragraph 1, The above input step is, The method includes the step of concatenating the time series data, the prediction result, and the explanatory data to generate input data corresponding to the input format of the language model. Method for providing an explanation of patient condition prediction.

6. In Paragraph 1, The above natural language text data is, Among the plurality of biosignals constituting the time series data, the biosignal specified by the explanatory data is included as the basis for inference of the prediction model. Method for providing an explanation of patient condition prediction.

7. In Paragraph 1, The method further includes the step of obtaining anomaly degree data in which the anomaly degree is evaluated based on rules for at least some of the multiple biosignals constituting the time series data. The above input data is, In addition to the above abnormal degree data, The above natural language text data is, Among a plurality of biosignals constituting the time series data, at least some of the biosignals identified by the explanatory data or the biosignals determined to be abnormal according to the degree of abnormality data are included as the basis for inference of the prediction model. Method for providing an explanation of patient condition prediction.

8. In Paragraph 7, The above abnormality degree data is, A method comprising whether at least some of the biological signals among the plurality of biological signals constituting the time series data are normal or abnormal, a normal range value for the at least some of the biological signals, and a class value classifying the degree of abnormality of the biological signal determined to be abnormal. Method for providing an explanation of patient condition prediction.

9. In Paragraph 7, The above natural language text data is, A biosignal identified by the above-described data and simultaneously judged as abnormal according to the above degree of abnormality data as the first-priority inference basis of the above prediction model, A biosignal identified by the above-described data but not judged as abnormal according to the above degree of abnormality data as the second-ranked inference basis of the above prediction model, Including a biosignal judged as abnormal based on the above abnormality degree data but not specified by the above explanatory data as the third-rank inference basis of the above prediction model Method for providing an explanation of patient condition prediction.

10. In Paragraph 1, The method further includes the step of acquiring Electronic Health Records (EHR) data including the patient's past medical history and treatment history. The above input data is, In addition to the above EHR data, The above natural language text data is, Among the plurality of biosignals constituting the time series data, at least some of the biosignals specified by the explanatory data or the biosignals specified by the EHR data are included as the basis for inference of the prediction model. Method for providing an explanation of patient condition prediction.

11. In Paragraph 10, If the biosignal identified by the above EHR data corresponds to the basis of inference of the above prediction model, The above natural language text data is, Text information further comprising explaining the patient's past medical history and treatment history as the basis for judgment regarding the patient's symptoms, and explaining the diagnostic content corresponding to the biosignals identified by the EHR data as the result of judgment regarding the patient's symptoms. Method for providing an explanation of patient condition prediction.

12. In Paragraph 1, The method further includes the step of obtaining expert analysis results regarding the above time series data and the above prediction results, The above input data is, In addition to including the analysis results of the aforementioned expert, The above natural language text data is, Among the plurality of biosignals constituting the time series data, at least some of the biosignals specified by the explanatory data or the biosignals specified by the analysis results of the expert are included as the basis for inference of the prediction model. Method for providing an explanation of patient condition prediction.

13. A computer-readable, non-transient recording medium having a program for executing the method of claim 1 on a computer.

14. As an electronic device that provides an explanation of the prediction of a patient's condition, A transceiver configured to communicate with the outside; It includes memory and a processor for storing instructions, The above processor controls the transceiver and the memory, Acquire patient time series data, and From an artificial intelligence-based prediction model, a prediction result output by the prediction model based on the time series data is obtained, and From an explainable artificial intelligence-based explanation model, the explanation data output by the explanation model based on the time series data and the prediction result is obtained, and Input data including the above time series data, the above prediction result, and the above explanatory data is input into an artificial intelligence-based language model, and An electronic device for obtaining natural language text data from the above language model, the natural language text data including the reasoning basis of the prediction model for the prediction result output by the language model based on the input data.

Images