Method for predicting condition of patient and electronic apparatus therefor

Abstract

Disclosed are a method for predicting a condition of a patient and an electronic apparatus therefor. The method for predicting a condition of a patient according to the present disclosure may comprise: a step for receiving, from a server, bio-signal data measured for a patient and unstructured medical information related to the patient; a step for converting the received bio-signal data to first text data; a step for converting the received unstructured medical information to second text data; a step for inputting the first text data and the second text data into an AI-based analysis model to generate analysis information about the patient; and a step for inputting the analysis information into an AI-based prediction model to predict information about the condition of the patient.

Description

Method for predicting patient condition and electronic device for the same

[0001] The present disclosure relates to a technology for predicting a patient's condition using an electronic device, and more specifically, to a technology for predicting information regarding a patient's condition by analyzing data collected from a patient through an artificial intelligence-based model.

[0002] In intensive care units or emergency rooms, a patient's condition can change rapidly, so it is crucial to accurately determine the timing of medical staff and equipment deployment. Additionally, it is necessary to quickly establish a treatment plan considering the patient's progress; to achieve this, smooth communication between the patient and medical staff must be facilitated through a patient-centered approach.

[0003] However, conventionally, data collected from patients could not be utilized in diverse ways, and in particular, there was a lack of a methodology to analyze patients by integrally considering both dynamic data that changes over time and static data that does not.

[0004] Furthermore, although patients' conditions were analyzed and treatment plans were established based on the professional judgment of skilled medical staff, limitations in accuracy and speed were encountered due to limited personnel and time; this acted as a factor that lowered patients' trust and satisfaction with the medical institution.

[0005] The present disclosure is proposed to solve the aforementioned problems and aims to improve the accuracy of analysis of a patient's condition and accumulate in-depth treatment cases by integrally utilizing various types of data collected from patients.

[0006] Furthermore, the present disclosure aims to overcome the shortage of medical personnel and improve the quality of medical services by analyzing patient data through an artificial intelligence-based model and further predicting the patient's future condition.

[0007] The technical problems to be solved by the present disclosure are not limited to those described above, and other technical problems can be inferred from the following embodiments.

[0008] A method for predicting a patient's condition according to one embodiment through an electronic device may include: receiving biosignal data measured for a patient and unstructured medical information related to the patient from a server; converting the received biosignal data into first text data; converting the received unstructured medical information into second text data; inputting the first text data and the second text data into an artificial intelligence-based analysis model to generate analysis information about the patient; and inputting the analysis information into an artificial intelligence-based prediction model to predict the patient's condition information.

[0009] In one embodiment, the biosignal data includes information regarding the time at which the patient's biosignal was measured, the value of the biosignal, and the type of the biosignal, and the step of converting into the first text data may include the step of inputting the measured time, the value of the biosignal, and the type of the biosignal into a template corresponding to the first text data.

[0010] In one embodiment, the unstructured medical information may include at least some of the patient's profile data, the patient's initial examination record data, the patient's medical history data regarding the patient's current disease, and the patient's initial biosignal data.

[0011] In one embodiment, the step of converting into the second text data may include the step of generating the second text data in the form of a prompt in which the table of contents is separated by items constituting the unstructured medical information.

[0012] In one embodiment, the step of generating the analysis information may include: generating input data for the analysis model, the input data comprising the first text data, the second text data, and instruction data for the analysis model; and inputting the input data into the analysis model to generate the analysis information. In this regard, the instruction data may include data regarding a predicted target state and a prediction time point that the prediction model considers when generating the state information.

[0013] In one embodiment, the analysis information may include information analyzed for at least some of non-time series data, time series data, a predicted target state, and recommendations regarding the patient.

[0014] In one embodiment, the state information may include information regarding the probability of an anomaly occurring in the predicted state within the time interval from the current time to the predicted time.

[0015] In one embodiment, the analysis model may be few-shot trained based on training text data and expert analysis results regarding the training text data.

[0016] In one embodiment, the prediction model may be supervised learning based on learning analysis information and whether an abnormality occurs in the predicted target state when predicting based on the learning analysis information.

[0017] In one embodiment, the analysis model and the prediction model may be trained based on refined analysis information obtained by processing some of the analysis information output by the analysis model based on the first text data and the second text data. In this regard, the partial analysis information may include a predetermined number of analysis information among the plurality of analysis information that has a low loss in the first training epoch of the prediction model using the plurality of analysis information. Additionally, the refined analysis information may be generated by deleting some sentences from each of the partial analysis information in which the attention score of the prediction model is lower than a reference value.

[0018] Meanwhile, according to one embodiment, an electronic device for predicting a patient's condition includes a transceiver configured to communicate with a server, a memory for storing instructions, and a processor. The processor controls the transceiver and the memory to receive biosignal data measured for the patient and unstructured medical information related to the patient from the server, converts the received biosignal data into first text data, converts the received unstructured medical information into second text data, inputs the first text data and the second text data into an artificial intelligence-based analysis model to generate analysis information about the patient, and inputs the analysis information into an artificial intelligence-based prediction model to predict the patient's condition information.

[0019] Specific details of other embodiments are included in the detailed description and drawings.

[0020] According to the present disclosure, by converting biosignal data collected from patients and unstructured medical information and utilizing them as input to an artificial intelligence-based analysis model, data having various characteristics can be considered in an integrated manner.

[0021] In addition, according to the present disclosure, by analyzing patient data through artificial intelligence-based analysis models and prediction models and further predicting the patient's future condition, the accuracy of medical services can be improved and patient trust and satisfaction can be enhanced.

[0022] The effects of the invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the claims.

[0023] FIG. 1 is a schematic diagram illustrating a system for predicting a patient's condition according to the present disclosure.

[0024] FIG. 2 is a diagram relating to a method for predicting a patient's condition according to one embodiment.

[0025] Figure 3 is an exemplary diagram of converting biosignal data into first text data.

[0026] Figure 4 is an exemplary diagram of converting unstructured medical information into second text data.

[0027] Figure 5 is an exemplary diagram showing the generation of analysis information from first and second text data through an analysis model.

[0028] Figure 6 is an exemplary diagram showing how to predict patient condition information from analysis information through a prediction model.

[0029] FIG. 7 is a diagram relating to a method for predicting a patient's condition according to an additional embodiment.

[0030] FIG. 8 is an exemplary diagram showing the generation of multiple analysis information from first and second text data through an analysis model.

[0031] FIG. 9 is an exemplary diagram showing the selection of some analysis information among multiple analysis information.

[0032] Figure 10 is an exemplary diagram showing the generation of refined analysis information through processing.

[0033] FIG. 11 is a block diagram illustrating an electronic device according to various embodiments.

[0034] Hereinafter, specific embodiments will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, this is merely illustrative and the disclosed embodiments are not limited thereto.

[0035] In describing the embodiments, if it is determined that a detailed description of the related prior art could unnecessarily obscure the essence of the disclosed embodiments, such detailed description will be omitted. Furthermore, the terms described below are defined in consideration of their functions in the disclosed embodiments, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments and should not be limiting. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "compose" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0036] The terms used in the embodiments have been selected to be as widely used as possible, taking into account their functions in the present disclosure; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section. Therefore, terms used in the present disclosure should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0037] When a part throughout the specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "...module" as used in the specification refer to a unit that processes at least one function or operation; this unit may be implemented in hardware or software, or as a combination of hardware and software, and may not be clearly distinguishable in terms of specific operation, unlike the illustrated examples.

[0038] The expression "at least one of a, b, and c" described throughout the specification may include 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c', or 'a, b, and c all'.

[0039] The "terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all types of handheld-based wireless communication devices such as communication-based terminals like IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), smartphones, tablet PCs, etc.

[0040] In the following description, terms such as "transmission," "communication," "sending," "receiving," and other terms with similar meanings regarding signals or information include not only the direct transmission of signals or information from one component to another but also transmission through other components.

[0041] In particular, "transmitting" or "transmitting" a signal or information as a component indicates the final destination of the signal or information, not the direct destination. The same applies to the "reception" of the signal or information. Furthermore, in this specification, two or more data or information are "related" means that if one data (or information) is obtained, at least a portion of another data (or information) can be obtained based thereon.

[0042] Additionally, terms such as first, second, etc., may be used to describe various components, but said components should not be limited by said terms. said terms may be used for the purpose of distinguishing one component from another.

[0043] For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0044] In describing the embodiments, technical details that are well known in the technical field to which the present invention belongs and are not directly related to the present invention are omitted. This is intended to convey the essence of the present invention more clearly without obscuring it by omitting unnecessary explanations.

[0045] For the same reason, some components in the attached drawings have been exaggerated, omitted, or schematically depicted. Additionally, the size of each component does not entirely reflect its actual dimensions. Identical or corresponding components in each drawing have been assigned the same reference numbers.

[0046] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0047] It will be understood that each block of the process flow diagrams and combinations of the flow diagrams can be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means to perform the functions described in the flow diagram block(s). Since these computer program instructions can also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory can also produce a manufactured item containing the means of instruction to perform the function described in the flow diagram block(s). Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0048] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to their corresponding functions.

[0049] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0050] FIG. 1 is a schematic diagram illustrating a system for predicting a patient's condition according to the present disclosure. It will be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components shown in FIG. 1.

[0051] Referring to FIG. 1, the system includes a first device (10), a second device (20), and an electronic device (110). The first device (10), the second device (20), and the electronic device (110) can communicate with each other within a network, and the network includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof, and is a data communication network in a comprehensive sense that enables each system component shown in FIG. 1 to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks. Wireless communication may include, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), but is not limited thereto.

[0052] The first device (10) is a device that measures a patient's biosignal data and transmits it to an electronic device (110), or receives biosignal data measured from an external source and mediates it to the electronic device (110). As an embodiment of the first device (10) that measures a patient's biosignal data and transmits it to the electronic device (110), a smart watch that measures biosignal data by contacting the patient's body and transmits it to the electronic device (110) either as is or after processing may be included in the first device (10). As an embodiment of the first device (10) that receives biosignal data measured from an external source and mediates it to the electronic device (110), a tablet PC, a cloud server, etc. that receives biosignal data from a contact device such as a smart watch may be included in the first device (10), and this may transmit the biosignal data to the electronic device (110) either as is or after processing.

[0053] In other words, the first device (10) may be a digital device for measuring biosignal data, or a separate device that assists the said digital device. The first device (10) is capable of precise measurement compared to biosignal measurement by a person and can periodically measure biosignal data having predetermined standardized attributes (e.g., consistent format and / or type of acquired data). Additionally, the first device (10) may measure data such as an electrocardiogram (ECG) having an electrical waveform.

[0054] In one embodiment, the first device (10) may include a smart device connected to the electronic device (110) via an IoT (Internet of Things) application. Accordingly, when the smart device corresponding to the first device (10) uploads the patient's biosignal data to a database that services the IoT application, the electronic device (110) can obtain the patient's biosignal data simply by accessing the IoT application. At this time, the network used by the first device (10) and the electronic device (110) to access the IoT application may be the same or different from the network used by the second device (20) to communicate with the first device (10) and the electronic device (110). For example, the network used by the electronic device (110) to receive the patient's biosignal data from the first device (10) via a dedicated URL (Uniform Resource Locator) may be different from the network used by the second device (20) to communicate with the electronic device (110).

[0055] The second device (20) is a device that stores medical information recorded, measured, or analyzed for a patient and transmits it to an electronic device (110). The medical information stored in the second device (20) may be data measured by a person (medical personnel) (e.g., chart data, electronic health records, etc.), but it is not necessary to interpret it as being limited thereto. That is, the second device (20) may be an external device capable of recording and storing various medical information other than a wearable device such as a smart watch that measures biosignal data by contacting the patient's body like the first device (10), and may be a medical device that records analysis data on the patient's samples (e.g., blood, urine, etc.). The second device (20) may store medical information within its own storage space and then transmit it to the electronic device (110), but it may also store medical information in a separate storage medium linked to the second device (20).

[0056] Biosignal data transmitted from the first device (10) to the electronic device (110) and medical information transmitted from the second device (20) to the electronic device (110) can be distinguished by various criteria. For example, biosignal data may be data directly measured by the first device (10), and medical information may be data extracted through separate equipment or laboratory processing after sample extraction and stored by the second device (20). For another example, biosignal data may be data that is available immediately after measurement or within a very short period of time and is capable of continuous and real-time monitoring, whereas medical information may be data that requires a relatively long time for sample preparation, processing, analysis, etc., and results are not provided in real time. For yet another example, biosignal data may be data collected by the first device (10) designed to directly interface with the patient, whereas medical information may be data collected by laboratory equipment that does not directly interface with the patient. As another example, biosignal data may be data requiring only partial preprocessing, such as noise removal or filtering, whereas medical information may be data requiring more comprehensive preprocessing for normalization, quantification, or interpretation itself. As another example, biosignal data may be data collected by a first device (10) for home use or remote medical use, whereas medical information may be data collected by professional equipment in a controlled environment and provided to an electronic device (110) by a second device (20). As yet another example, biosignal data may be time-series data acquired periodically, while medical information may be non-time-series data acquired non-periodically. However, it should be noted that the criteria for distinguishing between biosignal data and medical information, and between the first device (10) and the second device (20), may be defined in various ways other than those mentioned above.

[0057] The electronic device (110) is a server that provides a prediction service for a patient's condition. It can obtain analysis information by processing data received from the first device (10) or the second device (20), converting it into first text data and second text data, respectively, and inputting them into an analysis model. The electronic device (110) can obtain patient condition information based on the output information by inputting the analysis information into a prediction model distinct from the analysis model. In this way, the electronic device (110) can process data received from the first device (10) or the second device (20) and transmit the processed data to a client using the service. The client can access the service through an application or a web browser to receive the processed data.

[0058] In relation to the above, a more detailed explanation will be provided through the drawings below.

[0059] FIG. 2 is a drawing relating to a method for predicting a patient's condition according to one embodiment. Hereinafter, the method illustrated in FIG. 2 is described as being performed by the electronic device (110) described above, but this is exemplary and the method illustrated in FIG. 2 may be performed by other devices or by a combination of the electronic device (110) and separate devices. The same applies to the method illustrated in FIG. 7 described later.

[0060] In step S210, the electronic device (110) may receive biosignal data measured for the patient and unstructured medical information related to the patient from a server. Here, although it is described that the electronic device (110) receives biosignal data and unstructured medical information from a 'server', the biosignal data may be transmitted from the first device (10) of FIG. 1 through a network to the electronic device (110), and the medical information may be transmitted from the second device (20) of FIG. 1 through a network to the electronic device (110). Accordingly, in an embodiment where the first device (10) and the second device (20) can directly transmit biosignal data and medical information to the electronic device (110) (for example, an embodiment where the first device (10) and the second device (20) communicate with the electronic device (110) via a communication protocol such as Bluetooth, NFC, Zigbee, and / or Wi-Fi Direct), the data may be transmitted directly to the electronic device (110) without passing through a separate server. As another example, biosignal data may be transmitted to the electronic device (110) from a database storing Electronic Health Records (EHR) of a medical institution, and medical information may be transmitted to the electronic device (110) from a different database separate from the above database.

[0061] In step S220, the electronic device (110) can convert the received biosignal data into first text data.

[0062] According to one embodiment, the biosignal data may include information regarding the time at which the patient's biosignal was measured, the value of the biosignal, and the type of the biosignal, and the electronic device (110) may generate the first text data by inputting the time at which the patient's biosignal was measured, the value of the biosignal, and the type of the biosignal into a template corresponding to the first text data. This will be explained in more detail with reference to FIG. 3.

[0063] Figure 3 is an exemplary diagram of converting biosignal data into first text data.

[0064] The tensor of train_x illustrated on the left side of Fig. 3 is a two-dimensional array representing numerical data, where the rows represent each measured biosignal and the columns represent different variables. Specifically, each column can represent, in order, the time at which the biosignal was measured, the value of the biosignal, and the type of the biosignal. For example, [-16.0833, 2.800, 16.0000] in the first row and [-13.8667, 357.0000, 15.0000] in the second row represent separately measured biosignals, where -16.0833 in the first row represents a past time relative to the current time, 2.800 represents the value of the biosignal, and 16.0000 represents the type of the corresponding biosignal.

[0065] The first text data illustrated on the right side of FIG. 3 is the result of converting the tensor of train_x on the left into natural language based on a predefined template, for example, the electronic device (110) can identify that 2.8000 is a CRP (C-Reactive Protein) value through 16.0000, which indicates the type of biosignal in the first line [-16.0833, 2.800, 16.0000], and can identify that the value was measured 16.0833 hours ago from the present. As a result, the first line of the first text data is converted to "CRP measured at 2.8000 16.08 hours ago" (time or value may be rounded up, rounded down, or truncated at a specific decimal place, which may be set differently depending on the embodiment).

[0066] In one embodiment, the electronic device (110) may sort the order of the received biosignal data before converting it into first text data. For example, the electronic device (110) may sort each row of the biosignal data in ascending or descending order based on the time at which the biosignal was measured. This is to ensure that the first text data generated as a result of conversion is sorted by time, thereby enabling the user of the electronic device (110) (e.g., professional medical personnel, medical institution servers) to facilitate the storage, processing, and application of the first text data. As another example, the electronic device (110) may sort each row of the biosignal data in ascending or descending order based on the type of the biosignal. By doing so, biosignals of the same or similar types are sorted together, allowing the user of the electronic device (110) to interpret the first text data more easily and to facilitate storage, processing, and application.

[0067] In one embodiment, the electronic device (110) can convert biosignal data, in which the time at which the biosignals were measured is the same, into a single sentence in the first text data. For example, in the left train_x tensor of FIG. 3, rows 9 through 12 are biosignals measured 0.73 hours prior to the present. The electronic device (110) can convert them into a single sentence, "LACTATE was measured at 0.7000, PH at 7.4250, HEMATOCRIT at 28.0000, and HCO3 at 21.3000 0.73 hours prior," without separating them into multiple sentences in the first text data. Considering that specific biosignals are collected through a single test or measurement, the electronic device (110) can thereby prevent the first text data from becoming unnecessarily verbose and make the capacity and volume of the first text data more efficient. In addition, if the task of sorting biosignal data in the order of the time the biosignals were measured is performed first, this sentence integration task can be carried out more easily.

[0068] Meanwhile, in one embodiment, if there are two or more biosignals of the same type among the received biosignal data, the electronic device (110) may perform a preprocessing operation on the biosignals of the same type before converting the biosignal data into the first text data. For example, the electronic device (110) may delete the biosignals of the same type, excluding the one most recently measured. As another example, the electronic device (110) may convert the biosignals of the same type into a single sentence in the first text data. In this case, the user can easily understand the progression of a specific biosignal over time.

[0069] Meanwhile, referring again to FIG. 2, in step S230, the electronic device (110) can convert the received unstructured medical information into second text data. In the present disclosure, 'unstructured medical information' may collectively refer to medical information to which the format of biosignal data has not been applied.

[0070] According to one embodiment, the unstructured medical information may include at least some of the patient's profile data, the patient's initial examination record data, the patient's medical history data regarding the patient's current disease, and the patient's initial vital sign data. The electronic device (110) may generate second text data in the form of a prompt with a table of contents separated by items constituting the unstructured medical information. This will be explained in more detail with reference to FIG. 4.

[0071] Figure 4 is an exemplary diagram of converting unstructured medical information into second text data.

[0072] The left side of FIG. 4 illustrates, exemplarily, unstructured medical information received from the second device (20) or extracted from the received information. The unstructured medical information is data received by the electronic device (110) from the second device (20) and may include various types of information, such as text types, handwritten record information, and image information. The types of unstructured medical information received and extracted by the electronic device (110) from the second device (20) are very diverse and are not limited to those shown in FIG. 4, but may include, for example, patient profile data, initial examination record data, medical history data regarding the current disease, and initial biosignal data. Among these, the term "initial biosignal data" may refer to the biosignal data (or the average of biosignal data measured over a predetermined number of times) that is first measured for each biosignal type during the entire period of observing the progress of the patient's biosignal. According to one embodiment, by inputting such initial biosignal data together with the patient's profile data, the initial biosignal data may be assigned characteristics of data serving as a reference point for identifying the patient's health status at the time of admission, evaluating subsequent changes based on this, determining whether there is an emergency situation, and estimating the prognosis.

[0073] For example, patient profile data may include at least some of the patient's age, gender, and type of ward, and initial examination record data may include information on the symptoms the patient primarily complained of at the time of the initial examination. Additionally, patient history data regarding the patient's current disease may include at least some of the time of symptom onset, the medical professional's opinion based on diagnosis, and information on treatment performed by the medical institution. Furthermore, patient initial vital sign data may include at least some of the time the vital sign was initially measured, and the type and value of the initially measured vital sign. Specifically, patient initial vital sign data may include at least some of the mental status, systolic blood pressure (SBP), diastolic blood pressure (DBP), pulse rate (PR), respiratory rate (RR), body temperature (BT), and oxygen saturation (SPO2) measured at the time of the initial examination of the patient.

[0074] The right side of FIG. 4 exemplarily shows second text data converted by an electronic device (110). The patient's age / gender ("81 years old / female"), major symptoms ("bloody sputum, cough"), current medical history ("occurred 3 months ago"), and initial vital sign values ​​("20.5 hours ago") each correspond to items 1 to 4 on the left, and the electronic device (110) generated second text data in the form of prompts divided into table of contents 1) to 4) for each item of medical information.

[0075] Meanwhile, referring again to FIG. 2, in step S240, the electronic device (110) can input the first text data and the second text data into an artificial intelligence-based analysis model to generate analysis information about the patient.

[0076] According to one embodiment, the analysis model may include a Large Language Model (LLM) based on a Transformer architecture. Specifically, the analysis model may include multiple layers, including encoder and decoder structures, through which text-form input is tokenized, converted into embedding vectors containing semantic information about the tokens, and a text-form output is generated by sequentially predicting the probability of the token to be located next to each token. Furthermore, a self-attention mechanism may be applied to the analysis model.

[0077] According to one embodiment, the electronic device (110) may generate input data for an analysis model, comprising first text data, second text data, and instruction data for the analysis model, and then input the input data into the analysis model to generate analysis information for a patient. Specifically, the instruction data may include data regarding a predicted target state and a prediction time point that the prediction model considers when generating patient state information. Additionally, the analysis information generated by the electronic device (110) may include information analyzed regarding at least some of non-time series data, time series data, a predicted target state, and recommendations for the patient. This will be explained in more detail with reference to FIG. 5.

[0078] Figure 5 is an exemplary diagram showing the generation of analysis information from first and second text data through an analysis model.

[0079] Referring to the left side of FIG. 5, input data to be input into the analysis model may include instruction data, first text data, and second text data, and each data may be distinguished by an identifier or tag indicating the type of data. Furthermore, the arrangement order of each data within the input data may be determined by an identifier or tag distinguishing each data, or by a predefined prompt.

[0080] In one embodiment, the target state and the time of prediction included in the instruction data may be hyperparameters input by the user of the electronic device (110) for each prediction, but in some cases, they may be parameters set on the patient state prediction service provided by the electronic device (110). For example, individual settings may be made to predict the probability of an abnormal state X occurring in a patient after N hours for each account logged into the service. Furthermore, the instruction data may include text requesting the generation of analysis information necessary for predicting the target state during the time interval from the present to the time of prediction. The text may include text specifying the type of data that the analysis model will refer to during the prediction (in FIG. 5, 'recent Vital-Sign and Lab-Test results'). In this case, the electronic device (110) may generate the instruction data such that the text of the instruction data specifying the 'type of data that the analysis model will refer to' corresponds to at least a part of the first text data or at least a part of the second text data. That is, before inputting data into the analysis model, the electronic device (110) checks whether the text of the instruction data specifying the 'type of data the analysis model will refer to' corresponds to at least a part of the first text data or at least a part of the second text data, and if there is no correspondence at all, it may regenerate the text of the instruction data specifying the 'type of data the analysis model will refer to'.

[0081] According to one embodiment, the instruction data may include data that is entered by a user through the input interface of the electronic device (110) or data that has been modified from default settings. According to another embodiment, the instruction data may be information that the electronic device (110) has obtained by processing default settings based on the first text data and / or the second text data. For example, among the information included in the instruction data, data indicating the predicted target state and the prediction time point considered by the prediction model when generating patient state information may include default values, and as shown on the left side of FIG. 5, information regarding "Vital-Sign and Lab-Test results" among the information included in the instruction data may be included as information to represent data extracted by analyzing the first text data and the second text data.

[0082] Referring to the right side of FIG. 5, an example of analysis information generated by an analysis model processing and analyzing first text data and second text data based on instruction data is shown. Among the analysis information, non-time series data and time series data may be generated by classifying at least a portion of the first text data and second text data based on whether they are static data or dynamic data. Additionally, the predicted state may include predicted content regarding the patient's major condition up to the prediction point in time based on the first text data and second text data. Furthermore, the recommendations may include content regarding necessary measures after the current point in time based on the non-time series data, time series data, and predicted state.

[0083] According to one embodiment, the analysis model may be trained to output information indicating the predicted target state and the prediction time point that the prediction model considers when generating patient state information.

[0084] Meanwhile, referring again to FIG. 2, in step S250, the electronic device (110) can predict the patient's condition information by inputting the analysis information into an artificial intelligence-based prediction model.

[0085] According to one embodiment, the prediction model may include an LLM based on a transformer architecture. Similar to the analysis model, the prediction model can analyze text-based input to generate text-based output, and in some cases, a self-attention mechanism may be applied. However, the prediction model does not have the same structure as the analysis model, and the detailed architecture may differ depending on the embodiment.

[0086] According to one embodiment, the patient condition information predicted by the electronic device (110) may include information regarding the probability of an abnormality occurring in the predicted condition within the time interval from the current time to the predicted time. This will be explained in more detail with reference to FIG. 6.

[0087] Figure 6 is an exemplary diagram showing how to predict patient condition information from analysis information through a prediction model.

[0088] The left side of FIG. 6 shows analysis information generated by the electronic device (110) through an analysis model. The electronic device (110) can generate state information including information on the probability of an abnormality occurring in the predicted target state ('death status: Positive / Negative') within a time interval ('within x hours from the present') up to the prediction point in time, as shown on the right side of FIG. 6, through a prediction model. As state information, the probability that the patient will die (Positive) and the probability that the patient will not die (Negative) within x hours can be output, respectively.

[0089] To this end, since the prediction model must predict the probability of each class for the state to be predicted, it may additionally include a classifier structure in addition to the LLM structure. However, the number of classes predicted by the prediction model can be defined in various ways and is not limited to the two shown in FIG. 6.

[0090] Figures 2 through 6 described above relate to an inference process for predicting a patient's condition using an analysis model and a prediction model. On the other hand, regarding the training process of the model, the analysis model and the prediction model are not limited to artificial intelligence models of any specific structure, so they can be trained based on various learning techniques.

[0091] For example, the analysis model can be trained using few-shot learning based on training text data and expert analysis results regarding the training text data. Specifically, analysis information regarding a patient's condition is provided as training text data to experts (e.g., medical professionals) in a medical institution or region where a patient condition prediction service is to be provided, and the expert's judgment regarding the analysis information can be collected as an analysis result. The electronic device (110) can train the analysis model by applying a few-shot learning technique so that the analysis model can learn the expert's judgment using only scarce training text data and analysis results, in order to minimize the cost incurred from the expert's services.

[0092] In addition, for example, a prediction model can be supervised based on training analysis information and whether an anomaly occurs in the target state when making a prediction based on the training analysis information.

[0093] However, the training techniques for analysis and prediction models can be configured in various ways, taking into account the amount of data from the medical environment itself where the patient condition prediction service will be applied, or from environments similar to that environment. For example, if there is sufficient high-quality data available for training, analysis and prediction models can be pre-trained using unsupervised learning with unlabeled data and an auto-aggressive approach that utilizes the values ​​predicted by each model for subsequent predictions. Alternatively, the performance of each model can be improved by fine-tuning these pre-trained models through supervised learning using partially labeled data. Furthermore, in-context learning techniques can be applied during unsupervised learning to train each model by extracting a series of rules from the data. As another example, considering the amount of analysis and condition information to be used as ground truth, it is possible to train one of the analysis or prediction models using unsupervised learning techniques while training the other using supervised learning techniques.

[0094] FIG. 7 is a diagram relating to a method for predicting a patient's condition according to an additional embodiment. In order to further improve the performance of the analysis model and the prediction model, it is necessary to train each model with training data that has a relatively small loss and is mainly considered during the analysis and prediction process. FIG. 7 relates to a process for extracting such training data.

[0095] In step S710, the electronic device (110) can input the first text data and the second text data into an analysis model to generate multiple analysis information. This will be explained in more detail with reference to FIG. 8.

[0096] FIG. 8 is an exemplary diagram showing the generation of multiple analysis information from first and second text data through an analysis model.

[0097] The left side of FIG. 8 represents input data for an analysis model, which may include instruction data, first text data, and second text data. The form of the input data can be defined in various ways, for example, as templated tabular data. For a single input data, the analysis model may perform multiple analyses to generate multiple analysis information. The electronic device (110) may adopt a method for the analysis model to generate multiple analysis information based on the same input data. For example, the electronic device (110) may train the analysis model through various techniques such as prompt engineering, few-shot learning, and conditional generation to generate multiple forms of output so that the analysis model can perform various contextual interpretations of the input data. The number of analysis information generated may be a hyperparameter set by the user of the electronic device (110), or, in some cases, a parameter set on the patient condition prediction service provided by the electronic device (110).

[0098] Meanwhile, referring again to FIG. 7, in step S720, the electronic device (110) may select some of the generated analysis information. For example, the electronic device (110) may select a preset number of analysis information from among the analysis information, wherein the loss in the first training epoch of the prediction model using the analysis information is low. This will be explained in more detail with reference to FIG. 9.

[0099] FIG. 9 is an exemplary diagram showing the selection of some analysis information among multiple analysis information.

[0100] The electronic device (110) may perform the process of FIG. 9 to select analysis information with a relatively low loss for prediction among a plurality of analysis information. Specifically, the electronic device (110) may input a plurality of analysis information generated by an analysis model into a prediction model to obtain a result of prediction regarding the patient's condition information, calculate the loss between each result and the correct answer, and store it in a storage space within the electronic device (110) or in a medium linked to the electronic device (110).

[0101] In this case, it is desirable that the loss on the prediction result be the loss from the first training epoch of the prediction model. This is because as the analysis model is trained, the loss tends to decrease due to memorization of the training data, so using the loss from the early stages of training allows for a more accurate estimation of the quality of the actual analysis information.

[0102] The electronic device (110) can select a preset number of analysis information from a plurality of analysis information in order of lowest loss for the prediction result. This is based on the assumption that the lower the loss, the more helpful the analysis information is for training a prediction model for predicting the patient's condition (clinical event). In the embodiment, the number of selected analysis information may be hyperparameters input by the user of the electronic device (110), but in some cases, it may be parameters set on the patient condition prediction service provided by the electronic device (110).

[0103] Meanwhile, referring again to FIG. 7, in step S730, the electronic device (110) can process selected partial analysis information to generate refined analysis information. For example, the electronic device (110) can generate refined analysis information by deleting some sentences in each of the selected partial analysis information among a plurality of analysis information where the attention score of the prediction model is lower than a reference value. This will be explained in more detail with reference to FIG. 10.

[0104] Figure 10 is an exemplary diagram showing the generation of refined analysis information through processing.

[0105]

[0106] The left side of FIG. 10 represents a single piece of analysis information before processing. The electronic device (110) can tokenize the analysis information into units of sentences, paragraphs, or items ('non-time series data', 'time series data', 'prediction status', 'recommendation') and calculate an attention score for each token. The attention score is a numerical value that quantifies how much weight a prediction model gives to other tokens when processing a specific token. The attention score of token i related to token j represents the importance of token j for processing token i and can be derived through a dot-product attention mechanism.

[0107] Specifically, an input sequence consisting of N tokens can be represented by a matrix X of size N*d (where d is the embedding dimension). Each token can be transformed into three vectors: query (Q), key (K), and value (V), and each vector can be produced by multiplying the input embedding matrix by a learned weight matrix (Q=XW Q , K=XW K , V=XW V ). Subsequently, for token i, the attention score associated with all tokens j, including itself, is the query vector Q of token i. i and the key vector K of token j j It can be calculated by taking the inner product, and for gradient stabilization, each attention score is of key dimension d kIt can be scaled by dividing by the square root of . Subsequently, the scaled attention scores can be normalized using the softmax function, thereby ensuring that the sum of the attention scores for all tokens becomes 1. This series of processes can be expressed mathematically as Equation 1 below. Here, W Q , W K , W V are weight matrices for the query, key, and value, respectively, and is the attention score of token i associated with token j, and is a scaled attention score, and is the key dimension, and is a normalized scaled attention score, and represents the final attention score of token i.

[0108] [Mathematical Formula 1]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114] The right side of FIG. 10 shows refined analysis information resulting from processing the analysis information on the left side. The electronic device (110) can calculate an attention score for each token according to the mechanism described above and delete tokens with an attention score lower than a reference value from the analysis information. According to an embodiment, the electronic device (110) can compare the attention scores for each token and delete a preset number of tokens from the analysis information in order of lowest attention score. In FIG. 10, the token "Required action: Early treatment when metabolic acidosis or anemia progresses," which is the last sentence, was deleted. As a result, the part with the lowest importance is deleted when the prediction model processes the analysis information, thereby allowing for the efficient use of the capacity and volume of analysis information while securing high-quality training data.

[0115] Meanwhile, referring again to FIG. 7, in step S740, the electronic device (110) can train an analysis model and a prediction model using refined analysis information.

[0116] The electronic device (110) can generate multiple pieces of analysis information by inputting the first text data and the second text data into the analysis model multiple times to secure a sufficient amount of training data, while also refining the analysis information to generate high-quality training data that contains only information useful for prediction and where the prediction result is relatively close to the correct answer from the perspective of the prediction model.

[0117] FIG. 11 is a block diagram illustrating an electronic device according to various embodiments.

[0118] In an embodiment, the electronic device (110) may include a memory (111), a processor (113), and a transceiver (115). The electronic device (110) may exchange data with the outside through a transceiver (115) configured to communicate with a server or client.

[0119] The processor (113) may perform at least one method described above through the drawings or support an external device in performing the method. The memory (111) may store information for performing at least one method described above through the drawings. The memory (111) may be volatile memory or non-volatile memory.

[0120] The processor (113) can control the electronic device (110) to execute a program and provide information. The code of the program executed by the processor (113) can be stored in memory (111).

[0121] In one embodiment, the processor (113) is connected to the memory (111) and the transceiver (115) to receive biosignal data measured for the patient and unstructured medical information related to the patient from the server, convert the received biosignal data into first text data, convert the received unstructured medical information into second text data, input the first text data and the second text data into an artificial intelligence-based analysis model to generate analysis information about the patient, and input the analysis information into an artificial intelligence-based prediction model to predict the patient's condition information.

[0122] The electronic device (110) illustrated in FIG. 11 is illustrated only with components related to the present embodiment. Therefore, it can be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components illustrated in FIG. 11.

[0123] The device according to the embodiments described above may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, and user interface devices such as a touch panel, a key, a button, etc. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium is computer-readable, stored in memory, and can be executed on a processor.

[0124] The present embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions under the control of one or more microprocessors or other control devices. Similar to how components may be implemented as software programming or software elements, the present embodiment may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present embodiment may employ prior art for electronic configuration, signal processing, message processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

[0125] The aforementioned embodiments can be implemented as artificial intelligence (AI) through the processor and memory of an electronic device. The processor may consist of one or more processors, and the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (digital signal processors), graphics-dedicated processors such as GPUs and VPUs (vision processing units), or AI-dedicated processors such as NPUs. The one or more processors may be controlled to process input data according to predefined operation rules or AI models stored in memory. Alternatively, if the one or more processors are AI-dedicated processors, the AI-dedicated processors may be designed with a hardware structure specialized for processing a specific AI model.

[0126] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform a desired characteristic (or objective) are created by a basic artificial intelligence model being trained using a number of learning data by a learning algorithm. Such learning may be performed on the electronic device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0127] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and can perform neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include, but is not limited to, deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), bidirectional recurrent deep neural networks (BRDNN), or deep Q-networks.

[0128] The aforementioned embodiments are merely examples, and other embodiments may be implemented within the scope of the claims set forth below.

Claims

1. In a method for predicting a patient's condition through an electronic device, A step of receiving biosignal data measured for a patient and unstructured medical information related to said patient from a server; A step of converting the received biosignal data into first text data; A step of converting the received unstructured medical information into second text data; A step of generating analysis information for the patient by inputting the first text data and the second text data into an artificial intelligence-based analysis model; and A step comprising inputting the above analysis information into an artificial intelligence-based prediction model to predict the patient's condition information, Method for predicting patient condition.

2. In Paragraph 1, The above biosignal data is, It includes information on the time at which the patient's vital sign was measured, the value of the vital sign, and the type of the vital sign, and The step of converting into the first text data above is, A method comprising the step of inputting the measured time, the value of the biosignal, and the type of the biosignal into a template corresponding to the first text data. Method for predicting patient condition.

3. In Paragraph 1, The above unstructured medical information is, including at least some of the patient’s profile data, the patient’s initial examination record data, the patient’s medical history data regarding the patient’s current disease, and the patient’s initial biosignal data, Method for predicting patient condition.

4. In Paragraph 1, The step of converting into the above second text data is, A step comprising generating the second text data in the form of a prompt with a table of contents separated by items constituting the unstructured medical information, Method for predicting patient condition.

5. In Paragraph 1, The step of generating the above analysis information is, A step of generating input data for the analysis model, comprising the first text data, the second text data, and instruction data for the analysis model; and A step comprising inputting the above input data into the above analysis model to generate the above analysis information, Method for predicting patient condition.

6. In Paragraph 5, The above instruction data is, including data regarding the predicted target state and the prediction time point considered when the above prediction model generates the above state information, Method for predicting patient condition.

7. In Paragraph 1, The above analysis information is, Information including analyzed non-time series data, time series data, predicted target status, and recommendations for the above-mentioned patient, Method for predicting patient condition.

8. In Paragraph 1, The above status information is, Includes information on the probability of an anomaly occurring in the predicted target state within the time interval from the current point in time to the predicted point in time, Method for predicting patient condition.

9. In Paragraph 1, The above analysis model is, Few-shot learning based on training text data and expert analysis results regarding the training text data, Method for predicting patient condition.

10. In Paragraph 1, The above prediction model is, Supervised learning based on learning analysis information and whether an anomaly occurs in the predicted target state when predicting based on the said learning analysis information, Method for predicting patient condition.

11. In Paragraph 1, The above analysis model and the above prediction model are, Among the plurality of analysis information output by the analysis model based on the first text data and the second text data, learning is performed based on refined analysis information obtained by processing some of the analysis information. Method for predicting patient condition.

12. In Paragraph 11, The above partial analysis information is, Among the plurality of analysis information above, including a preset number of analysis information having a low loss in the first training epoch of the prediction model using the plurality of analysis information, Method for predicting patient condition.

13. In Paragraph 11, The above refined analysis information is, In each of the above partial analysis information, generated by deleting some sentences in which the attention score of the prediction model is lower than the threshold value, Method for predicting patient condition.

14. A computer-readable, non-transient recording medium having a program for executing the method of claim 1 on a computer.

15. As an electronic device for predicting a patient's condition, A transceiver configured to communicate with a server; It includes memory and a processor for storing instructions, The above processor controls the transceiver and the memory, Receiving biosignal data measured for a patient and unstructured medical information related to said patient from said server, and Converting the above-mentioned received biosignal data into first text data, and Converting the received unstructured medical information into second text data, and The above first text data and the above second text data are input into an artificial intelligence-based analysis model to generate analysis information about the patient, and An electronic device that predicts the condition information of the patient by inputting the above analysis information into an artificial intelligence-based prediction model.

Images