Information processing device, information processing method, and storage medium

Abstract

An information processing device 1X mainly comprises a first acquisition means 15X, a second acquisition means 16X, and an integration means 17X. The first acquisition means 15X acquires first time-series information representing a medical event in a time series generated from structured data of a medical record of a patient. The second acquisition means 16X acquires second time-series information representing a medical event in a time series generated from unstructured data of the medical record of the patient. The integration means 17X generates third time-series information in which the first time-series information and the second time-series information are integrated.

Description

Information processing device, processing method, and storage medium

[0001] The present disclosure relates to the technical field of an information processing device, a processing method, and a storage medium for processing medical information.

[0002] Based on electronic medical records, timelines of medical events such as examinations, medication, and other medical procedures are being generated. For example, Patent Literature 1 discloses a technology for extracting clinical medical expressions and time expressions from clinical texts created in clinical settings, and converting the extracted data into time-series data for display.

[0003] Japanese Patent Application Laid-Open No. 2023-055570

[0004] In addition to structured data structured as tables, electronic medical records also contain unstructured data, which is unstructured text information. To generate a detailed patient timeline, it is preferable to use both types of data. However, Patent Document 1 does not disclose anything about how to handle structured data.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide an information processing device, a processing method, and a storage medium that are capable of suitably generating time-series information of medical events.

[0006] One aspect of the information processing device is an information processing device having: a first acquisition means for acquiring first time series information representing medical events in time series generated from structured data of a patient's medical record; a second acquisition means for acquiring second time series information representing medical events in time series generated from unstructured data of the medical record; and an integration means for generating third time series information by integrating the first time series information and the second time series information.

[0007] One aspect of the processing method is a processing method in which a computer acquires first time series information representing medical events in time series generated from structured data of a patient's medical record, acquires second time series information representing medical events in time series generated from unstructured data of the medical record, and generates third time series information by integrating the first time series information and the second time series information.

[0008] One aspect of the storage medium is a storage medium that stores a program that causes a computer to execute the following processes: acquire first time series information representing medical events in time series generated from structured data of a patient's medical record; acquire second time series information representing medical events in time series generated from unstructured data of the medical record; and generate third time series information that integrates the first time series information and the second time series information.

[0009] According to the present disclosure, time-series information of medical events can be suitably generated.

[0010] 1 shows a schematic configuration of a medical information processing system; FIG. 2 shows an example of a data structure of medical records in a medical record storage unit; FIG. 3 shows a hardware configuration of a medical information processing device; FIG. 4 shows an example of a functional block of a processor of a medical information processing device; FIG. 5 shows an overview of a matching process between an entity of a first timeline and an entity of a second timeline; (A) shows a first example of a method for matching entity A of the first timeline with entity a of the second timeline; (B) shows a second example of a method for matching entity A of the first timeline with entity a of the second timeline; FIG. 6 shows specific examples of a first timeline and a second timeline, which clearly show the matching results, and an integrated timeline; FIG. 7 shows an overview of a process for generating a presentation timeline from the integrated timeline; (A) shows a first aspect of a machine learning model; (B) shows a second aspect of a machine learning model; (C) shows a third aspect of a machine learning model; (D) shows a fourth aspect of a machine learning model; 1 is a block diagram of an information processing apparatus;

[0011] Hereinafter, embodiments of an information processing device, a processing method, and a storage medium will be described with reference to the drawings.

[0012] <First embodiment> (1) System configuration Fig. 1 shows a schematic configuration of a medical information processing system 100. The medical information processing system 100 executes processing related to the generation and display of a timeline (time-series information) of medical events of a patient specified by a user.

[0013] Hereinafter, "medical records" refers to the various records created in the course of medical treatment regarding a patient's condition (physical condition, medical condition, treatment applied, etc.), and includes medical records (charts), prescriptions, surgery records, nursing records, test findings records, referral letters, summaries of the medical progress of discharged patients during their hospital stay, and other medical documents. Hereinafter, unless otherwise specified, medical records will refer to electronic data (including electronic medical records) in which the above-mentioned records are stored electronically.

[0014] The medical information processing system 100 mainly includes a medical information processing device 1, an input device 2, an output device 3, and a storage device 4.

[0015] The medical information processing device 1 generates a timeline of medical events for a specified patient and controls the display of the timeline according to the user's viewing purpose, which includes outpatient consultations for patients and the creation of medical documents (such as referral letters, medical certificates, and discharge summaries).

[0016] The medical information processing device 1 communicates data with the input device 2, output device 3, and storage device 4 via a communication network or by direct wireless or wired communication. The medical information processing device 1 accepts user designation of a patient (also referred to as a "target patient") for which a timeline of medical events is to be generated, and the purpose of viewing the timeline, based on, for example, an input signal supplied from the input device 2 operated by the user. The medical information processing device 1 also supplies an output signal to the output device 3, which includes display information for the timeline generated in accordance with the designated target patient and viewing purpose. The output signal may further include audio information.

[0017] The input device 2 is an interface that accepts manual input (external input) of information by a user. The input device 2 may be, for example, various user input interfaces such as a touch panel, buttons, a keyboard, a mouse, or a voice input device. The input device 2 supplies the generated input signal to the medical information processing device 1. The output device 3 outputs predetermined information based on the output signal supplied from the medical information processing device 1. Examples of the output device 3 include a display, a projector, a speaker, a printer, etc.

[0018] The storage device 4 is a memory that stores various information necessary for processing executed by the medical information processing device 1. The storage device 4 may be an external storage device such as a hard disk connected to or built into the medical information processing device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the medical information processing device 1. The storage device 4 may also be composed of multiple devices. At least a portion of the information stored in the storage device 4 may be stored by the medical information processing device 1.

[0019] Functionally, the storage device 4 mainly includes a medical record storage unit 41 that stores the medical records of patients.

[0020] FIG. 2 shows an example of the data structure of medical records in the medical record storage unit 41. The medical record storage unit 41 stores, for example, medical records created for each medical treatment, linked to patient identification information (also referred to as "patient ID") for identifying the patient who received the treatment. Here, the medical records include structured data structured as a table and unstructured data, which is unstructured text information. In the example of FIG. 2, the medical record storage unit 41 stores structured data "CD1" and unstructured data "NCD1" linked to the patient ID "PID1" as the medical record of a patient identified by the patient ID "PID1," and stores structured data "CD2" and unstructured data "NCD2" linked to the patient ID "PID2" as the medical record of a patient identified by the patient ID "PID2." For example, the structured data (CD1, CD2, ...) includes basic patient information, medication and prescription orders, test orders, etc., while the unstructured data (NCD1, NCD2, ...) includes medical examination notes (including radiology reports), etc. Note that the structured data does not need to be separated by patient ID, and may be configured as one or more tables common to all patients.

[0021] The configuration of the medical information processing system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, the input device 2 and the output device 3 may be configured as an integrated unit. In this case, the input device 2 and the output device 3 may be configured as a tablet terminal (including a smartphone) that is integrated with or separate from the medical information processing device 1. The medical information processing device 1 may also be configured as a plurality of devices. In this case, the plurality of devices that make up the medical information processing device 1 exchange information necessary to execute pre-assigned processing between these plurality of devices. In this case, the medical information processing device 1 functions as a system.

[0022] (2) Hardware Configuration Fig. 3 shows the hardware configuration of the medical information processing device 1. The medical information processing device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.

[0023] The processor 11 executes programs stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the medical information processing device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0024] The memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 12 also stores programs for executing processes performed by the medical information processing device 1. Note that part of the information stored in the memory 12 may be stored in one or more external storage devices capable of communicating with the medical information processing device 1, or may be stored in a storage medium that is detachable from the medical information processing device 1.

[0025] The interface 13 is an interface for electrically connecting the medical information processing device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0026] The hardware configuration of the medical information processing device 1 is not limited to the configuration shown in Fig. 3. For example, the medical information processing device 1 may include at least one of the input device 2 and the output device 3.

[0027] (3) Processing Overview Next, an overview of the processing related to displaying a timeline of medical events of a target patient executed by the medical information processing device 1 will be described. In general, the medical information processing device 1 integrates a timeline based on structured data and a timeline based on unstructured data of the target patient by matching based on natural language processing, filters the integrated timeline according to the viewing purpose, and displays it on the output device 3. In this way, the medical information processing device 1 displays a timeline based on both structured data and unstructured data with the necessary and sufficient amount of information according to the user's viewing purpose.

[0028] 4 shows an example of functional blocks of the processor 11 of the medical information processing device 1. Functionally, the processor 11 of the medical information processing device 1 has a medical record acquisition unit 14, a first timeline generation unit 15, a second timeline generation unit 16, a timeline integration unit 17, a filtering unit 18, and an output control unit 19. Note that, although the blocks where data is exchanged are connected by solid lines here, the combination of blocks where data is exchanged is not limited to this. The same applies to the other functional block diagrams described below.

[0029] The medical record acquisition unit 14 acquires the medical record of the target patient from the medical record storage unit 41 via the interface 13. In this case, for example, the medical record acquisition unit 14 receives a user input specifying the target patient from the input device 2, and identifies the target patient based on an input signal generated by the input device 2. Then, the medical record acquisition unit 14 extracts the medical record of the target patient in structured data and unstructured data linked to the patient ID of the target patient from the medical record storage unit 41. Then, the medical record acquisition unit 14 supplies the structured data to the first timeline generation unit 15 and supplies the unstructured data to the second timeline generation unit 16.

[0030] Here, the structured data supplied to the first timeline generator 15 is composed of entities generated as records of medical records for each medical event of the target patient. Each entity includes, for example, information on the medical event that occurred and the date and time of the occurrence of the medical event (which may be the date and time when the medical event was recorded). A medical event is an event (including a medical procedure) related to medical care for the target patient, such as an examination, test, surgery, or medication.

[0031] The first timeline generation unit 15 generates a timeline (also referred to as a "first timeline") in which entities of the structured data of the medical records of the target patient supplied from the medical record acquisition unit 14 are arranged in chronological order. The first timeline is a group of structured data entities in which temporal relationships are determined based on the order in which medical events occur. The first timeline generation unit 15 supplies the generated first timeline to the timeline integration unit 17.

[0032] The second timeline generation unit 16 generates entities for each medical event based on the unstructured data of the target patient's medical record supplied from the medical record acquisition unit 14, and generates a timeline (also referred to as the "second timeline") in which the entities are arranged in chronological order. The second timeline is a group of unstructured data entities in which temporal relationships are determined based on the order in which the medical events occur. In this case, for example, the second timeline generation unit 16 generates entities using a machine learning model or the like that extracts medical named entities and their temporal relationships from text, which is unstructured data. The entities include entity data representing medical events (clinical medical expressions) and relationship data representing temporal relationships. The second timeline generation unit 16 then generates a second timeline in which the entities are arranged based on the relationship data. Note that an example of a method for generating a timeline of medical events from unstructured data in this manner is disclosed in Patent Document 1. Note that the second timeline generation unit 16 may generate chronological entities from text information, which is unstructured data, using any method, not limited to the method described in Patent Document 1. The second timeline generation unit 16 supplies the generated second timeline to the timeline integration unit 17 .

[0033] The timeline integration unit 17 generates a timeline (also referred to as an "integrated timeline") by integrating the first timeline supplied from the first timeline generation unit 15 and the second timeline supplied from the second timeline generation unit 16. In this case, the timeline integration unit 17 estimates equivalence between entities on the first timeline and entities on the second timeline, and aligns the time axes of the first timeline and the second timeline based on the equivalent entities. As a result, the timeline integration unit 17 integrates the entities on the first timeline and the second timeline and identifies their temporal relationships, thereby generating a detailed integrated timeline based on both structured data and unstructured data. Details of the processing performed by the timeline integration unit 17 will be described later. The timeline integration unit 17 supplies the integrated timeline to the filtering unit 18.

[0034] The filtering unit 18 filters entities on the integrated timeline in accordance with the viewing purpose specified by the user, and generates a coarse-grained timeline from the integrated timeline. In this case, the filtering unit 18 receives user input specifying the viewing purpose via the input device 2, and identifies the viewing purpose based on the input signal generated by the input device 2. The timeline generated by the filtering unit 18 through filtering is the timeline presented to the user, and will hereinafter also be referred to as the "presented timeline." The filtering unit 18 supplies the presented timeline to the output control unit 19.

[0035] The output control unit 19 generates an output signal based on the presentation timeline supplied from the filtering unit 18 and supplies the generated output signal to the output device 3. As a result, the output control unit 19 causes the output device 3 to output information related to the presentation timeline. By checking the presentation timeline, medical professionals can, for example, consider the next course of action for the patient or appropriately create medical documents related to the patient. In this way, the output control unit 19 can support the decision-making of medical professionals. Examples of displays that the output control unit 19 displays on the output device 3 will be described later.

[0036] The components of the medical record acquisition unit 14, the first timeline generation unit 15, the second timeline generation unit 16, the timeline integration unit 17, the filtering unit 18, and the output control unit 19 described in FIG. 4 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to realize each component. At least some of these components may not necessarily be realized by software programs, but may be realized by any combination of hardware, firmware, and software. At least some of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above components. At least some of the components may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be realized by various hardware. The same applies to other embodiments described below. Furthermore, these components may be realized by the cooperation of multiple computers using, for example, cloud computing technology, etc. The same applies to other embodiments described below.

[0037] (4) Generation of Integrated Timeline Next, a specific description will be given of the processing for generating an integrated timeline by the timeline integration unit 17. First, a description will be given of the processing for matching entities of the first timeline with entities of the second timeline.

[0038] 5 is a diagram showing an overview of the matching process between entities on a first timeline and entities on a second timeline. Fig. 5 clearly shows a first timeline, which is information in which structured data entities (including entities A to D) are arranged in chronological order, and a second timeline, which is information in which unstructured data entities (including entities A to D) are arranged in chronological order. Entities A to D are examples of first entities, and entities A to D are examples of second entities.

[0039] The timeline integration unit 17 performs matching (in other words, a determination of match / mismatch) regarding medical events between entities on the first timeline and entities on the second timeline. As a result, the timeline integration unit 17 identifies a pair of matching entities on the first timeline and entities on the second timeline. When matching between entities on the first timeline and entities on the second timeline, for example, the timeline integration unit 17 uses an arbitrary language model that vectorizes character strings for each token, embeds information representing medical events included in each entity (vectorization for each token), and performs the above-described determination based on the distance between the vectors in the converted vector space.

[0040] 6A shows a first example of a method for determining whether an entity A on the first timeline matches or mismatches an entity a on the second timeline. In the example shown in FIG. 6A, each entity on the first timeline includes a code such as a number for identifying a medical event (also referred to as an "event code"). Event code dictionary data (i.e., a master table showing the correspondence between each code used in the structured data and the corresponding text) is stored in the medical record storage unit 41 or the like. Each entity on the second timeline includes a character string related to the medical event (also referred to as an "event character string").

[0041] In the first example shown in FIG. 6A , the timeline integration unit 17 converts the event code CA of entity A into a string based on dictionary data to generate an event string TA. Then, the timeline integration unit 17 matches the event string TA with the event string Ta included in entity a using an arbitrary language model. In this case, the timeline integration unit 17 embeds the event string TA and the event string Ta for each token using the language model to generate vectors, and determines whether the characters match or mismatch based on the distance between the vectors and a threshold. This allows the timeline integration unit 17 to accurately determine whether the characters match or mismatch while absorbing variations in notation, etc. Examples of the above-mentioned language model include a language model that takes context into account, such as BERT (Bidirectional Encoder Representations from Transformers). The above-mentioned language model is an example of a machine learning model that converts tokens related to medical events into vectors.

[0042] 6B illustrates a second example of a method for determining whether an entity A on the first timeline matches an entity a on the second timeline. In the second example illustrated in FIG. 6B , the timeline integration unit 17 matches a character string representing a medical event included in the dictionary data with an event character string Ta included in entity a using an arbitrary language model, and extracts an event code Ca corresponding to the event character string Ta from the dictionary data. In this case, the timeline integration unit 17 first embeds the character string in the dictionary data and the event character string Ta using the language model to generate a vector. Then, based on the generated vector, the timeline integration unit 17 identifies a character string in the dictionary data that substantially matches the event character string Ta and identifies an event code Ca corresponding to the identified character string. If the event code CA and the event code Ca are identical, the timeline integration unit 17 determines that the entity A and the entity a substantially match. If the event code CA and the event code Ca are not identical, the timeline integration unit 17 determines that the entity A and the entity a do not substantially match. The second example shown in FIG. 6B also enables the timeline integration unit 17 to accurately determine whether there is a substantial match or mismatch between the entities of the first timeline and the entities of the second timeline.

[0043] Note that the method of matching the entities on the first timeline with the entities on the second timeline is not limited to the examples shown in Figures 6(A) and 6(B) . For example, the timeline integration unit 17 may determine whether the entities on the first timeline and the entities on the second timeline match or mismatch by directly comparing character strings (tokens) with each other without using a language model.

[0044] Next, the process of integrating the first timeline and the second timeline based on the matching result will be described.

[0045] The timeline integration unit 17 integrates (incorporates) the second timeline into the first timeline based on the matching result. At this time, the timeline integration unit 17 integrates entities of the second timeline that match with entities of the first timeline with the entities of the first timeline. On the other hand, the timeline integration unit 17 adds entities of the second timeline that do not match any entities of the first timeline to the first timeline so that their temporal relationships with the integrated entities of the second timeline are consistent.

[0046] 7 is a diagram showing a specific example of a first timeline and a second timeline indicating the matching results, and an integrated timeline. In the example of FIG. 7, the matching results of the first timeline and the second timeline shown in FIG. 5 are indicated, and an integrated timeline based on the matching results is shown. Specifically, the timeline integration unit 17 determines that entity B and entity a match (substantially coincide), and that entity C and entity c, and entity C and entity d match, respectively.

[0047] In this case, the timeline integration unit 17 integrates, among the entities in the second timeline, any entity that matches any of the entities in the first timeline into the matching entity in the first timeline. For example, the timeline integration unit 17 generates entity "B(a)" by integrating entity a into entity B, and generates entity "C(c, d)" by integrating entity c and entity d into entity C. In this case, the timeline integration unit 17 may integrate the entities based on any data combination process. Note that the timeline integration unit 17 may consider the entity in the first timeline to be more accurate than the entity in the second timeline, and use the entity in the first timeline as the integrated entity.

[0048] Furthermore, the timeline integration unit 17 adds, to the first timeline, entities in the second timeline that do not match any of the entities in the first timeline so that their temporal relationships with the matched entities are consistent. For example, entity b in the second timeline that does not match any of the entities in the first timeline indicates a medical event that occurs after entity a and before entity c. Therefore, the timeline integration unit 17 inserts entity b between entity B (a) into which entity a is integrated and entity C (c, d) into which entity c is integrated. Note that in this case, if there are multiple candidate positions for insertion, the timeline integration unit 17 may insert the entity in the second timeline into a candidate position randomly selected from the multiple candidate positions. Alternatively, if the entity includes date and time information of the occurrence of a medical event, the position may be determined based on the date and time information of the occurrence.

[0049] In this way, the timeline integration unit 17 generates an integrated timeline by integrating the second timeline into the first timeline based on the matching result between the entities of the first timeline and the second timeline. The integrated timeline generated in this way is a detailed timeline that takes into account both structured data and unstructured data.

[0050] (5) Filtering Next, we will explain the filtering of the integrated timeline by the filtering unit 18 according to the viewing purpose. Generally, the information required by the viewing user differs depending on the viewing purpose (outpatient consultation, creation of medical documents, etc.). Taking the above into consideration, the filtering unit 18 filters the integrated timeline according to the viewing purpose, removes entities that are less important for the specified viewing purpose, and generates a presented timeline that lists only entities that are necessary and sufficient for the viewing user.

[0051] For example, the filtering unit 18 calculates a score (also referred to as an "aptitude score") for each entity on the integrated timeline that indicates the suitability of display according to the viewing purpose (i.e., suitability as an entity on the presented timeline). In this case, for example, the filtering unit 18 calculates an suitability score for each entity from each entity on the integrated timeline using a machine learning model that calculates suitability scores according to the viewing purpose. The suitability score is an example of "information that indicates the suitability of display according to the purpose."

[0052] FIG. 8 is a diagram showing an outline of a process for generating a presentation timeline from the integrated timeline shown in FIG.

[0053] The filtering unit 18, for example, selects a machine learning model according to the viewing purpose specified by the user, and calculates an aptitude score corresponding to each entity on the integrated timeline using the selected machine learning model. Here, based on the machine learning model, the filtering unit 18 calculates an aptitude score of "0.7" for entity A, an aptitude score of "0.3" for entity B (a), an aptitude score of "0.9" for entity b, an aptitude score of "0.9" for entity C (c, d), and an aptitude score of "0.8" for entity D. Here, as an example, the aptitude score has a value range of 0 to 1, and the higher the suitability of the display according to the viewing purpose, the higher the value.

[0054] The filtering unit 18 then considers entities whose aptitude scores are equal to or greater than a predetermined threshold to be information necessary for the specified browsing purpose and uses them as entities in the presentation timeline. On the other hand, the filtering unit 18 considers entities whose aptitude scores are less than the predetermined threshold to be entities that are not important for the specified browsing purpose and does not use them as entities in the presentation timeline. The above-mentioned threshold is, for example, set to a different value for each browsing purpose or a common value regardless of the browsing purpose and is stored in advance in the memory 12 or the storage device 4.

[0055] In the example of Figure 8, the filtering unit 18 uses entity A, entity b, entity C (c, d), and entity D, whose suitability scores are equal to or greater than the predetermined threshold "0.6," as entities of the presentation timeline. On the other hand, the filtering unit 18 does not use entity B (a), whose suitability score is less than the predetermined threshold "0.6," as an entity of the presentation timeline. This allows the filtering unit 18 to generate a presentation timeline made up of entities that provide necessary and sufficient information for the specified viewing purpose.

[0056] Here, the machine learning model used to calculate the aptitude score will be described in more detail. Figure 9(A) shows a first embodiment of the machine learning model, Figure 9(B) shows a second embodiment of the machine learning model, Figure 9(C) shows a third embodiment of the machine learning model, and Figure 9(D) shows a fourth embodiment of the machine learning model.

[0057] The machine learning model of the first aspect shown in FIG. 9A is a model that learns the relationship between event information included in an entity and an aptitude score, and is generated by machine learning for each browsing purpose. The event information may be the event string described above or an event code. The learned parameters of the machine learning model are associated with each browsing purpose and stored in the memory 12, the storage device 4, or the like. When the machine learning model is configured using a neural network, various parameters (including hyperparameters), such as the layer structure, the neuron structure of each layer, the number and filter size of filters in each layer, and the weight of each element of each filter, are stored in the memory 12, the storage device 4, or the like. The filtering unit 18 then selects a machine learning model associated with the browsing purpose and inputs the event information of the entity into the selected machine learning model. The filtering unit 18 then acquires the aptitude score output by the machine learning model based on the above input as the aptitude score of the entity.

[0058] The machine learning model undergoes machine learning using training data including a plurality of records each pairing an input sample (event information in this case) to the machine learning model and a correct answer (aptitude score in this case) that the machine learning model should output when the sample is input. In this case, the parameters of the machine learning model are determined so that the error (loss) between the correct answer and the inference result output by the machine learning model when the sample is input to the machine learning model is minimized. The algorithm for determining the parameters so as to minimize the loss may be any learning algorithm used in machine learning, such as gradient descent or backpropagation. Subsequent machine learning models are similarly trained, and trained parameters are obtained.

[0059] The second-mode machine learning model shown in FIG. 9B is a model that learns the relationship between a set of event information included in an entity and relative time information indicating the relative occurrence time of a medical event indicated by the event information, and an aptitude score, and is generated by machine learning for each browsing purpose. Here, the relative time information is, for example, relative time information based on the occurrence date of an important medical event (e.g., surgery, initial consultation, etc.). For example, if the date of surgery is used as the reference, the relative time information indicates the number of days before or after the surgery. The learned parameters of the machine learning model are associated with each browsing purpose and stored in the memory 12, the storage device 4, etc. Then, the filtering unit 18 selects a machine learning model according to the browsing purpose and inputs the event information of the entity and the relative time information based on the occurrence date and time information included in the entity into the selected machine learning model. Then, the filtering unit 18 obtains the aptitude score output by the machine learning model based on the above input as the aptitude score of the entity. The above-mentioned event information and relative time information are examples of "entity-based information."

[0060] The machine learning model of the third aspect shown in FIG. 9(C) is a model that learns the relationship between a set of event information included in an entity and information indicating a browsing purpose specified by a user, and an aptitude score, and is generated by machine learning. The information indicating a browsing purpose is, for example, identification information unique to each browsing purpose. The learned parameters of the machine learning model are stored in the memory 12, the storage device 4, or the like. The filtering unit 18 then inputs the event information of the entity and information indicating the specified browsing purpose into the machine learning model of the third aspect, and acquires the aptitude score output by the machine learning model as the aptitude score of the entity. Note that the machine learning model of the third aspect may be combined with the machine learning model of the second aspect. In this case, the machine learning model is a model to which event information, relative time information, and information indicating a browsing purpose are input. The information input into the machine learning model of the third aspect is an example of "information based on an entity and a purpose."

[0061] Note that instead of calculating the aptitude score using a machine learning model, the filtering unit 18 may determine the aptitude score based on a rule. In this case, for example, if table information indicating the correspondence between medical events and aptitude scores is stored for each viewing purpose in the memory 12 or the storage device 4, the filtering unit 18 selects table information according to the specified viewing purpose. Then, the filtering unit 18 determines the aptitude score associated in the table information with the medical event indicated by the event information of the entity as the aptitude score of the entity.

[0062] The machine learning model of the fourth aspect shown in FIG. 9(D) is a model that learns the relationship between information input to the machine learning model of any of the first to third aspects and the suitability of an entity, and is generated by machine learning. When entity-based information such as event information is input, the machine learning model of the fourth aspect outputs a classification result (e.g., a binary classification result of 0 or 1) indicating the suitability of the entity. In this case, the filtering unit 18 inputs entity-based information into the machine learning model and, based on the classification result output by the machine learning model, determines whether to leave the entity on the presentation timeline. Note that the machine learning model of the fourth aspect may be a machine learning model trained for each browsing purpose, as in the first or second aspect, or may be a common machine learning model that inputs the browsing purpose, as in the third aspect. The above-mentioned classification result is an example of "information indicating the suitability of display according to the purpose."

[0063] (6) Display Example Fig. 10 is an example of a display screen that the output control unit 19 causes the output device 3 to display. The output control unit 19 generates display information for displaying the above-mentioned display screen based on the presentation timeline generated by the filtering unit 18, and transmits the generated display information to the output device 3 via the interface 13, thereby causing the output device 3 to display the display screen. The output control unit 19 provides a patient information display area 51, a timeline display area 52, a viewing purpose selection field 53, and a target patient selection button 54 on the display screen.

[0064] The output control unit 19 displays basic information of the target patient in the patient information display area 51. Here, the output control unit 19 refers to the medical records of the target patient stored in the medical record storage unit 41, and displays the patient name, age, sex, disease name, treatment period, etc. in the patient information display area 51.

[0065] Furthermore, the output control unit 19 displays in the timeline display area 52 a table showing the presentation timeline generated by the filtering unit 18 in accordance with the viewing purpose specified in the viewing purpose selection field 53. Here, the above-mentioned table shows the entities of the presentation timeline in chronological order, and one record is generated based on one entity. Here, the output control unit 19 displays a table having columns for "Date and Time," "Event," "Symptoms," "Examination," "Diagnosis," "Treatment," and "Schedule." Note that the display format of the presentation timeline in the timeline display area 52 is not limited to the format (table) shown in FIG. 10 . For example, the presentation timeline may be displayed in the timeline display area 52 as a chart in any format.

[0066] The output control unit 19 accepts a user input specifying a viewing purpose in the viewing purpose selection field 53. Here, as an example, the viewing purpose selection field 53 is a selection field in the form of a pull-down menu, and "outpatient examination" is selected. The filtering unit 18 supplies the output control unit 19 with a presentation timeline generated by filtering the integrated timeline of the target patient according to the viewing purpose selected in the viewing purpose selection field 53, and the output control unit 19 determines the display content of the timeline display area 52 based on the presentation timeline. Furthermore, when the output control unit 19 detects that the target patient selection button 54 has been selected, it displays a display screen (not shown) for selecting a target patient and accepts a user input specifying the target patient.

[0067] 10, the output control unit 19 can suitably present to the user a timeline of the target patient that has been filtered according to the viewing purpose selected by the user. Such a timeline has a desired time scale according to the viewing purpose.

[0068] (7) Processing Flow FIG. 11 is an example of a flowchart executed by the medical information processing device 1.

[0069] First, the medical information processing device 1 determines whether a target patient has been designated (step S11). For example, when the medical information processing device 1 receives a user input from the input device 2 specifying a target patient, the medical information processing device 1 determines that a target patient has been designated.

[0070] Then, when a target patient is designated (step S11; Yes), the medical information processing device 1 acquires structured data and unstructured data corresponding to the designated target patient from the medical record storage unit 41 (step S12). On the other hand, when a target patient is not designated (step S11; No), the medical information processing device 1 continues to execute step S11. Note that the medical information processing device 1 may perform the processes from step S12 onwards only when a target patient is designated and a viewing purpose is specified.

[0071] Next, the medical information processing device 1 generates a first timeline based on the structured data acquired in step S12 (step S13). Furthermore, the medical information processing device 1 generates a second timeline based on the unstructured data acquired in step S12 (step S14). Note that steps S13 and S14 may be performed in any order and may be executed in parallel.

[0072] Next, the medical information processing apparatus 1 integrates the first timeline generated in step S13 and the second timeline generated in step S14 by matching (step S15), thereby generating an integrated timeline.

[0073] Next, the medical information processing device 1 determines whether a viewing purpose has been specified (step S16). For example, when the medical information processing device 1 receives a user input specifying a viewing purpose from the input device 2, the medical information processing device 1 determines that a viewing purpose has been specified. If it determines that a viewing purpose has been specified (step S16; Yes), the medical information processing device 1 filters the integrated timeline according to the specified viewing purpose (step S17). As a result, the medical information processing device 1 acquires a timeline obtained by deleting entities that are unnecessary information in consideration of the viewing purpose from the integrated timeline as a presented timeline. Then, the medical information processing device 1 presents the filtered presented timeline on the output device 3 (step S18).

[0074] <Second embodiment> Fig. 12 shows a schematic configuration of a medical information processing system 100A. The medical information processing system 100A according to the second embodiment is a server-client model system, in which a medical information processing device 1A functioning as a server device performs the processing of the medical information processing device 1 in the first embodiment. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0075] The medical information processing system 100A mainly includes a medical information processing device 1A that functions as a server, a storage device 4A that stores data similar to the storage device 4 according to the first embodiment, and a terminal device 8 that functions as a client. The medical information processing device 1A and the terminal device 8 communicate data via a network 7.

[0076] The terminal device 8 is a terminal having input, display, and communication functions, and functions as the input device 2 and the output device 3 shown in Fig. 1. The terminal device 8 may be, for example, a personal computer, a tablet terminal, a PDA (Personal Digital Assistant), etc. The terminal device 8 transmits input signals based on user input to the medical information processing device 1A.

[0077] The medical information processing device 1A has the same hardware configuration and functional block configuration as the medical information processing device 1 according to the first embodiment. The medical information processing device 1A receives input signals and the like from a terminal device 8 via a network 7, identifies the target patient and the purpose of viewing based on the received information, and generates a presentation timeline. The medical information processing device 1A also transmits output signals indicating information related to the results of generating the presentation timeline to the terminal device 8 via the network 7. In this way, the terminal device 8 functions as the input device 2 and output device 3 according to the first embodiment.

[0078] According to this configuration, the medical information processing system 100A can generate a presentation timeline corresponding to the target patient and viewing purpose designated by the user, and present it to the user.

[0079] Third Embodiment In a third embodiment, in the configuration shown in FIG. 12, the processing executed by the medical information processing apparatus 1 in the first embodiment is shared and executed by a medical information processing apparatus 1A and a terminal device 8.

[0080] For example, the medical information processing device 1A has functions corresponding to the medical record acquisition unit 14, first timeline generation unit 15, second timeline generation unit 16, and timeline integration unit 17 shown in Figure 4, and the terminal device 8 has functions corresponding to the filtering unit 18 and output control unit 19.

[0081] In this case, when information indicating a designated patient is supplied from the terminal device 8 or another device, the medical information processing device 1A generates an integrated timeline from the medical records of the designated patient and supplies information indicating the generated integrated timeline to the terminal device 8. When the terminal device 8 detects an input specifying the viewing purpose, it generates a presentation timeline based on the information indicating the integrated timeline received from the medical information processing device 1A and the viewing purpose, and performs output control based on the generated presentation timeline.

[0082] With this configuration, the medical information processing device 1A can generate an integrated timeline of a target patient using both structured data and unstructured data. Furthermore, the terminal device 8 can generate a presentation timeline corresponding to a viewing purpose specified by the user and present it to the user.

[0083] <Fourth Embodiment> Figure 13 shows the configuration of a medical information processing system 100B. The medical information processing system 100B in the fourth embodiment is a system that generates medical document data representing medical documents using an LLM (Large Language Model), and includes a medical information processing device 1B and a medical document generation device 8B that communicates with the medical information processing device 1B via a network 7B. Hereinafter, for convenience of explanation, the medical information processing device 1B is assumed to include the input device 2, output device 3, and storage device 4 described in the first embodiment, etc. However, as in the first embodiment, these may be configured as devices separate from the medical information processing device 1B. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0084] The medical information processing device 1B accepts input from the user selecting medical events to be included in the medical document from the presented timeline, and generates text information (also referred to as "input text information") that will serve as input data (input prompt) to the LLM based on the selection result. The input text information is, for example, text information that lists the selected medical events in itemized chronological order. The medical information processing device 1B then transmits the generated input text information to the medical document generation device 8B. The medical information processing device 1B then receives medical document data from the medical document generation device 8B in response to the input text information, and displays the medical document indicated by the received medical document data.

[0085] The medical document generation device 8B is a device capable of executing the trained LLM and is a device or server (including a cloud server) capable of data communication with the medical information processing device 1B via the network 7B. When the medical document generation device 8B receives input text information from the medical information processing device 1B, it inputs the input text information into an LLM configured by referencing the trained parameters of the LLM, thereby obtaining medical document data output by the LLM. In this case, the LLM is an arbitrary natural language processing model that has been machine-learned to generate medical document data indicating a medical document based on the input text information. The medical document generation device 8B then transmits the generated medical document data to the medical information processing device 1B.

[0086] 14 is an example of a display screen displayed by the medical information processing device 1B. The medical information processing device 1B displays the above-mentioned display screen based on the presentation timeline generated by the filtering unit 18. The medical information processing device 1B has a patient information display area 51, a timeline display area 52, a viewing purpose selection field 53, a target patient selection button 54, and a selected event list display area 55 on the display screen. The medical information processing device 1B displays the same content as the display example of FIG. 10 in the patient information display area 51, the timeline display area 52, and the viewing purpose selection field 53.

[0087] The medical information processing device 1B accepts an input from the user (medical professional) selecting a medical event to be recorded in the medical document in the "Event" column of the table showing the presented timeline displayed in the timeline display area 52. Here, the medical information processing device 1B highlights the medical events "Hospitalization," "Surgery," and "Discharge" selected by the user by clicking the pointer on the above-mentioned table by color coding or the like. Note that the highlighting method is not limited to this.

[0088] The medical information processing device 1B also generates itemized text for the medical event selected in the "Event" column of the table showing the presentation timeline, and displays the generated text in the selected event list display area 55. As a result, the itemized text is presented (displayed) to the user who is a medical professional. The user who is a medical professional may review and, if necessary, modify the itemized text displayed in the selected event list display area 55. In this case, the medical information processing device 1B accepts input of deleting and adding text in the selected event list display area 55.

[0089] When the medical information processing device 1B detects that the document creation button 56 provided in the selected event list display area 55 has been selected, it regards the text information indicating the itemized sentences displayed in the selected event list display area 55 as input text information (i.e., an input prompt for the LLM) and transmits it to the medical document generation device 8B. The medical document generation device 8B inputs the input text information received from the medical information processing device 1B into the LLM and transmits the medical document data obtained to the medical information processing device 1B. The medical information processing device 1B receives the medical document data from the medical document generation device 8B, displays the medical document indicated by the received medical document data, and presents it to the user, who is a medical professional. In this way, the medical information processing device 1B can preferably present to the user a medical document based on the medical event selected by the user.

[0090] Fifth Embodiment Fig. 15 is a block diagram of an information processing device 1X. The information processing device 1X mainly includes a first acquisition unit 15X, a second acquisition unit 16X, and an integration unit 17X. Note that the information processing device 1X may be composed of multiple devices. The information processing device 1X may be, for example, the medical information processing device 1 in the first embodiment, the medical information processing device 1A in the second embodiment, or the medical information processing device 1A in the third embodiment.

[0091] The first acquisition unit 15X acquires first time-series information representing medical events in chronological order, generated from structured data of a patient's medical records. The first time-series information may be, for example, the first timeline in the first to third embodiments. The first acquisition unit 15X may generate the first time-series information, or may receive the generated first time-series information from another device. In the former example, the first acquisition unit 15X may be the first timeline generation unit 15 in the first to third embodiments.

[0092] The second acquisition unit 16X acquires second time-series information representing medical events in chronological order, generated from unstructured data of a patient's medical records. The second time-series information may be, for example, the second timeline in the first to third embodiments. The second acquisition unit 16X may generate the second time-series information, or may receive the generated second time-series information from another device. In the former example, the second acquisition unit 16X may be the second timeline generation unit 16 in the first to third embodiments.

[0093] The integration unit 17X generates third time-series information by integrating the first time-series information and the second time-series information. The third time-series information can be, for example, the integrated timeline in the first to third embodiments. The integration unit 17X can be, for example, the timeline integration unit 17 in the first to third embodiments.

[0094] 16 is an example of a flowchart executed by the information processing device 1X in the fifth embodiment. The first acquisition unit 15X acquires first time-series information representing medical events in time series generated from structured data of the patient's medical record (step S21). The second acquisition unit 16X acquires second time-series information representing medical events in time series generated from unstructured data of the patient's medical record (step S22). The integration unit 17X generates third time-series information by integrating the first time-series information and the second time-series information (step S23). Note that steps S21 and S22 may be executed in any order and may be executed in parallel.

[0095] According to the fifth embodiment, the information processing device 1X can suitably generate time-series information of medical events based on both structured data and unstructured data.

[0096] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor, etc. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs). The program may also be supplied to a computer by various types of transient computer-readable media. Examples of transient computer-readable media include electric signals, optical signals, and electromagnetic waves. The transient computer-readable medium can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0097] In addition, part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0098] [Supplementary Note 1] An information processing device comprising: a first acquisition means for acquiring first time series information representing medical events in time series generated from structured data of a patient's medical record, a second acquisition means for acquiring second time series information representing medical events in time series generated from unstructured data of the medical record, and an integration means for generating third time series information by integrating the first time series information and the second time series information. [Supplementary Note 2] The information processing device according to Supplementary Note 1, wherein the first time series information includes a plurality of first entities each representing the medical events in time series, the second time series information includes a plurality of second entities each representing the medical events in time series, and the integration means generates the third time series information based on matching of the first entities with the second entities regarding the medical events. [Supplementary Note 3] The information processing device according to Supplementary Note 2, wherein the integrating means generates the third time series information by integrating the second entities that match any of the first entities into the first entity and adding the second entities that do not match any of the first entities to the first time series information so that their temporal relationships with the second entities integrated into the first entities are consistent. [Supplementary Note 4] The information processing device according to Supplementary Note 2, wherein the integrating means performs the matching based on a machine learning model that converts tokens related to the medical event into vectors. [Supplementary Note 5] The information processing device according to Supplementary Note 2, wherein the matching is performed based on dictionary data that indicates a correspondence between codes used in the structured data and text. [Supplementary Note 6] The information processing device according to Supplementary Note 1, wherein the first acquiring means, when the patient is designated, generates the first time series information from the structured data included in the medical record of the patient, and the second acquiring means, when the patient is designated, generates the second time series information from the unstructured data included in the medical record of the patient. [Supplementary Note 7] The information processing device according to Supplementary Note 1, further comprising a display control means for displaying information relating to the third time-series information on a display device.[Supplementary Note 8] The information processing device according to Supplementary Note 7, further comprising filtering means for filtering the third time-series information based on a specified purpose, wherein the display control means causes the display device to display the filtered third time-series information. [Supplementary Note 9] The display control device according to Supplementary Note 7, wherein the display control means causes the display device to display information related to the third time-series information in order to support decision-making by a medical professional. [Supplementary Note 10] The display control device according to Supplementary Note 7, further comprising generation means for generating text information to be input to an LLM, based on the third time-series information selected by a user from the third time-series information. [Supplementary Note 11] A processing method, wherein a computer: acquires first time-series information representing medical events in time series generated from structured data of a patient's medical record; acquires second time-series information representing medical events in time series generated from unstructured data of the medical record; and generates third time-series information by integrating the first time-series information and the second time-series information. [Supplementary Note 12] A storage medium storing a program that causes a computer to execute the following processes: acquire first time series information representing medical events in time series generated from structured data of a patient's medical record; acquire second time series information representing medical events in time series generated from unstructured data of the medical record; and generate third time series information by integrating the first time series information and the second time series information.

[0099] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and non-patent documents are incorporated herein by reference.

[0100] 1, 1A, 1B Medical information processing device 1X Information processing device 2 Input device 3 Output device 4 Storage device 8 Terminal device 100, 100A, 100B Medical information processing system

Claims

1. An information processing device having: a first acquisition means for acquiring first time series information representing medical events in a time series generated from structured data of a patient's medical records; a second acquisition means for acquiring second time series information representing medical events in a time series generated from unstructured data of the medical records; and an integration means for generating third time series information by integrating the first time series information and the second time series information.

2. An information processing device as described in claim 1, wherein the first time series information includes a plurality of first entities each representing a medical event in the time series, the second time series information includes a plurality of second entities each representing a medical event in the time series, and the integration means generates the third time series information based on matching of the medical events between the first entities and the second entities.

3. The information processing device according to claim 2, wherein the integration means generates the third time series information by integrating the second entities that match any of the first entities into the first entity, and adding the second entities that do not match any of the first entities to the first time series information so that their time relationships with the second entities integrated into the first entities are consistent.

4. The information processing device according to claim 2, wherein the integration means performs the matching based on a machine learning model that converts tokens related to the medical events into vectors.

5. The information processing device according to claim 2, wherein the matching is performed based on dictionary data indicating a correspondence between a code used in the structured data and text.

6. The information processing device according to claim 1, wherein the first acquisition means generates the first time series information from the structured data included in the medical record of the patient when the patient is specified, and the second acquisition means generates the second time series information from the unstructured data included in the medical record of the patient when the patient is specified.

7. The information processing device according to claim 1, further comprising display control means for causing a display device to display information relating to said third time-series information.

8. An information processing device as described in claim 7, further comprising a filtering means for filtering the third time series information based on a specified purpose, and the display control means causes the display device to display the filtered third time series information.

9. The display control device according to claim 7, wherein said display control means causes information relating to said third time-series information to be displayed on said display device in order to assist medical personnel in making decisions.

10. A display control device according to claim 7, further comprising a generating means for generating text information for input to an LLM based on the third time-series information selected by a user from among the third time-series information.

11. A processing method, comprising: a computer acquiring first time series information representing medical events in a time series generated from structured data of a patient's medical record; acquiring second time series information representing medical events in a time series generated from unstructured data of the medical record; and generating third time series information by integrating the first time series information and the second time series information.

12. A storage medium storing a program that causes a computer to execute a process of acquiring first time series information representing medical events in a time series generated from structured data of a patient's medical records, acquiring second time series information representing medical events in a time series generated from unstructured data of the medical records, and generating third time series information by integrating the first time series information and the second time series information.

Images