Healthcare device for displaying EHR data

Abstract

According to one aspect, there is provided a healthcare device comprising: a memory comprising instruction data representing an instruction set; and a processor configured to communicate with the memory and execute the instruction set. The instruction set, when executed by the processor, causes the processor to: i) receive a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command being indicated with respect to a first portion of a display, displaying first data from an electronic health record, EHR, in the first portion of the display; ii) retrieving second data that is semantically linked to the first data; and iii) in response to receiving the first user input, instructing the display to display the second data in a second portion of the display.

Description

Technical Field

[0001] Embodiments herein relate to healthcare devices, and in particular but not exclusively, embodiments herein relate to retrieving information from electronic health records (EHRs). Background Art

[0002] The disclosure herein relates to the field of clinical decision support systems (CDSs) and to the retrieval of medical information in electronic health records (EHRs). An EHR is a digital version of the paper records traditionally kept by hospitals related to patient care and treatment. EHRs can include a patient's medical history and treatment history. They can also include diagnoses, medications, treatment plans, immunization history, allergies, radiology images and laboratory test results, and any other information or notes made by the patient's clinician or caregiver.

[0003] Healthcare professionals (HCPs) need the right information at the right time to make informed decisions. This typically requires gathering a large amount of patient data. In addition, HCPs need different information depending on their role.

[0004] Typically, current tools for retrieving patient information from EHRs can be configured to retrieve specific data entries or specific types of data entries based on a user's authorization, meaning that only data relevant to each HCP role may be retrieved.

[0005] In addition, professionals and non-medical professionals (including the patients themselves) have different data needs. Professionals are trained to use medical terms and local jargon. Laypersons may need more explanations and jargon closer to their own level of education. Note that a specialized HCP can be a professional in one field and a layperson in another field.

[0006] It would be beneficial to have an improved data retrieval scheme such that medical data from EHRs can be retrieved in a predictable manner for different users. Summary of the Invention

[0007] Typically, a hierarchical model of progressive data disclosure is implemented in an EHR retrieval system to enable HCPs to explore patient information. The data is delivered in small manageable bits. Based on the initial data, the user can decide to retrieve more data related to the initial data. The aim of the embodiments herein is to improve the data retrieval by HCPs from EHRs.

[0008] According to a first aspect, there is provided a healthcare device comprising: a memory including instruction data representing an instruction set and a processor configured to communicate with the memory and execute the instruction set. The instruction set, when executed by the processor, causes the processor to: i) receive a first user input from a user input device, the first user input indicating a first zoom command that is indicated relative to a first portion of a display, in which first data from an electronic health record (EHR) is displayed; ii) retrieve second data semantically linked to the first data; and iii) in response to receiving the first user input, cause the display to display the second data in a second portion of the display.

[0009] According to a second aspect, there is provided a computer-implemented method performed by a healthcare device. The method comprises: i) receiving a first user input from a user input device, the first user input indicating a first zoom command that is indicated relative to a first portion of a display, in which first data from an electronic health record (EHR) is displayed; ii) retrieving second data semantically linked to the first data; and iii) in response to receiving the first user input, cause the display to display the second data in a second portion of the display.

[0010] According to a third aspect, there is provided a computer program product comprising a computer-readable medium having computer-readable code embodied therein, the computer-readable code being configured to, when executed by a suitable computer or processor, cause the computer or processor to perform the method of the second aspect.

[0011] Thus, the apparatus and method herein facilitate what may be considered specialized semantic zooming of medical information, whereby receiving a zoom command from a user initiates a retrieval in the area of the zoom command of further semantically linked information from an EHR or other data sources (related to the first data). In this way, a healthcare provider (HCP) can zoom in and out of medical records to receive further information related to the first data. The method can further improve the time taken to access relevant data and can ensure retrieval of more relevant data.

[0012] These and other aspects will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Example embodiments will now be described, by way of example only, with reference to the following drawings, in which:

[0014] Figure 1 An apparatus according to some embodiments is shown;

[0015] Figure 2 shows a method according to some embodiments;

[0016] Figure 3 shows an example display according to some embodiments herein; and

[0017] Figure 4 shows an example system according to some embodiments herein. Detailed Description

[0018] Turning now to Figure 1 , in some embodiments, there is a healthcare device 100 for a healthcare setting according to some embodiments herein. Generally, the device may form part of a computer device or system, such as a laptop computer, a desktop computer, or other computing device. In some embodiments, the device 100 may form part of a distributed computing arrangement or cloud.

[0019] The device includes: a memory 104 that includes instruction data representing a set of instructions; and a processor 102 (e.g., processing circuitry or logic) that is configured to communicate with the memory and execute the set of instructions. Generally, when executed by the processor, the set of instructions may cause the processor to perform any embodiment of method 200 as described below.

[0020] Embodiments of the healthcare device 100 may be used to retrieve information from an EHR. More specifically, when executed by the processor, the set of instructions causes the processor to: i) receive a first user input from a user input device, the first user input indicating a first zoom command that is indicated relative to a portion of a display, and in a first portion of the display, display first data from an electronic health record EHR; ii) retrieve second data that is semantically linked to the first data; and iii) in response to receiving the first user input, cause the display to display the second data in a second portion of the display.

[0021] The processor 102 may include one or more processors, processing units, multi-core processors, or modules, which are configured or programmed to control the device 100 in the manner described herein. In a particular embodiment, the processor 102 may include multiple software and / or hardware modules each configured to perform or for performing individual or multiple steps of the methods described herein. In some embodiments, for example, the processor 102 may include multiple (e.g., interoperable) processors, processing units, multi-core processors, and / or modules configured for distributed processing. Those skilled in the art will understand that such processors, processing units, multi-core processors, and / or modules may be located in different positions and may perform different steps of the methods described herein and / or different portions of a single step.

[0022] The memory 104 is configured to store program code executable by the processor 102 to perform the methods described herein. Alternatively or additionally, one or more memories 104 may be external to the device 100 (i.e., separate from or remote from the device 100). For example, one or more memories 104 may be part of another device. The memory 104 may be used to store an EHR or a portion thereof, a scaling command, a first portion (or its location) of a display, second data, and / or any other information or data received, computed, or determined by the processor 102 of the device 100 or from any interface, memory, or device external to the device 100. The processor 102 may be configured to control the memory 104 to store an EHR or a portion thereof, a scaling command, a first portion (or its location) of a display, second data, and / or any other information.

[0023] In some embodiments, the memory 104 may include multiple sub-memories, each sub-memory capable of storing one instruction data. For example, at least one sub-memory may store instruction data representing at least one instruction of the instruction set, while at least one other sub-memory may store instruction data representing at least one other instruction of the instruction set.

[0024] It will be understood that Figure 1 only the components necessary to illustrate this aspect of the disclosure are shown, and in an actual implementation, the device 100 may include additional components other than those shown. For example, the device 100 may include a battery or other power source for powering the device 100 or a device for connecting the device 100 to a mains power supply.

[0025] As another example, the apparatus 100 may further include a display. For example, the display mentioned in steps i) and iii) may form part of the apparatus 100. Generally, the display may include, for example, a computer screen, a screen on a mobile phone, a screen on a tablet, or a screen associated with a hospital monitor. In some examples, the display may be a three-dimensional display, a virtual reality display, or any other display mechanism capable of displaying data from an EHR.

[0026] However, it will be understood that the display need not be part of the apparatus 100. For example, the display may be separate from the apparatus 100. Thus, in some embodiments, the apparatus 100 may communicate with the display via a wired or wireless connection.

[0027] The apparatus 100 may also include a user input device. For example, the user input device mentioned in step i) may form part of the apparatus 100. For example, in some embodiments, the display is a touchscreen display, and the user input device may be the touch-sensitive aspect of the touchscreen display. In such an embodiment, a first zoom command may be indicated by the user using a known touch movement. For example, a "zoom in" command may be indicated by placing two fingers on the screen in a first portion of the display and spreading the fingers apart. A "zoom out" command may be indicated by the reverse movement, for example, by placing two fingers on the screen and bringing them closer together. As another example, a zoom command may be input by touching the screen more than a threshold number of times within a certain time interval (e.g., a double tap or triple tap may be used to input a zoom command), or by using a scroll bar or the like on the screen to input a zoom command. However, these are merely examples, and those skilled in the art will be familiar with touchscreen displays and the movements performed thereon to indicate a zoom command.

[0028] In other embodiments, the user input device is, for example, a keyboard, a mouse, or some other input device that enables a user to interact with the apparatus to, for example, provide a first zoom command. As an example, the wheel on a mouse may be used to indicate a "zoom in" command. For example, an upward movement may indicate a zoom in command, and a downward movement may indicate a "zoom out" command. Those skilled in the art will understand, however, that these are merely examples, and a zoom command may be indicated in other ways different from those described herein.

[0029] However, it will be understood that the user input device need not be part of the apparatus 100. For example, the user input device may be separate from the apparatus 100. Thus, in some embodiments, the apparatus 100 may communicate with the user input device via a wired or wireless connection.

[0030] Moving on to Figure 2, there exists a computer-implemented method 200. The computer-implemented method 200 can be executed by a healthcare device such as the above-mentioned healthcare device 100.

[0031] Briefly, in a first step 202, the method 200 includes: i) receiving 202 a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command being indicated relative to a first portion of a display, in which first data from an electronic health record (EHR) is displayed. In a second step 204, the method includes ii) retrieving 204 second data that is semantically linked to the first data. In a third step 206, the method 200 includes: iii) in response to receiving the first user input, instructing the display to display the second data in a second portion of the display.

[0032] More specifically, before step i), the method 200 can include instructing the display to display the first data from the EHR. In this sense, the first data can be a part (or portion) of all the information in the EHR. As mentioned above, an electronic health record (EHR) is the digital version of the paper records traditionally kept by hospitals related to patient care and treatment. The EHR can include a patient's medical history and treatment history. They can also include diagnoses, medications, treatment plans, immunization history, allergies, radiology images and laboratory test results, and any other information or notes made by the patient's clinician or caregiver.

[0033] Generally, the EHR can include other parameters different from those mentioned above, for example, monitoring data and signals, vital signs information, algorithm outputs, billing and financial information, scheduling, resource management. There can be different dashboards showing metrics such as operating status, clinical status, and / or any other key performance indicators.

[0034] The data in the EHR can be dynamic, for example, it can be updated regularly. For example, additional new data can be added, and / or existing data can be updated. As an example, if vital signs information is taken every 15 minutes, the new vital signs information can be stored in the EHR.

[0035] The data stored in the EHR can be stored in an automatic manner (e.g., via a device-to-device communication protocol), manually by a human, or in a hybrid manner, for example, where a human verifies the data before storing it in the EHR.

[0036] The first data can be displayed together with other data from the EHR or from other sources.

[0037] A user of device 100, such as a doctor, clinician, radiologist, nurse, other HCP, the patient themselves, or any other user, can view the first data displayed and provide a first user input. The first user input can be received from a user input device. The user input device has been described above with respect to device 100, and the details thereof will be understood to apply equally to method 200. The first user input can be received, for example, via a wired or wireless connection. In other embodiments where the user input device forms part of device 100, the first user input can be received directly from the user input device.

[0038] The first user input indicates a first zoom command. For example, the first user input can indicate the type of zoom command, such as a "zoom in" command or a "zoom out" command. The user can use the user input device as described above with respect to device 100 to indicate the first zoom command, and the details thereof will be understood to apply equally to method 200.

[0039] The zoom command is indicated with respect to a first portion of the display. In other words, the user provides an indication to zoom in or out on the first data displayed on the display.

[0040] Generally, the display can be divided into multiple portions. For example, there can be different portions associated with each piece of data displayed on the display, or with different groups of data on the display. The zoom command can be executed differently in each portion.

[0041] However, it will be understood that the display does not necessarily have to be divided into different portions. Thus, in some embodiments, the first portion of the display (as referred to herein) can refer to the entire display or the graphical user interface (GUI).

[0042] In response to receiving the first user input indicating a zoom command, method 200 moves to step ii) and retrieves second data that is semantically linked to the first data.

[0043] In this context, second data is semantically linked to first data if there is a meaningful or logical connection between the second data and the first data. In other words, the second data is related to the first data in some way. For example, if the second data relates to the same or a similar subject or topic as the first data, the second data can be semantically linked to the first data.

[0044] For example, if the first data indicates a medical condition, the second data can be semantically linked to the first data in the sense that it provides more detailed (or conversely more general) information about the medical condition.

[0045] As another example, if the first data indicates a course of action or treatment, the second data can be semantically linked to the first data in the sense that it provides more detailed (or conversely more general) information about the course of action or treatment.

[0046] As another example, the first data can relate to a medical condition mentioned in an EHR, and the second data can relate to the treatment given for that medical condition. In another example, the first data relates to a medical condition mentioned in an EHR, and the second data relates to further information related to the medical condition. In another example, the first data relates to the type of a drug mentioned in an EHR, and the second data relates to a specific drug within that type (or the brand of the same drug).

[0047] As another example, if the first data is a medical image (such as a CT scan, an x-ray scan, an MRI scan, etc.), the second data can be semantically linked to the first data in the sense that it provides more information about a specific structure, lesion, or other feature detected in the medical image.

[0048] Other examples of the first data and the second data include but are not limited to:

[0049] Vital sign measurements and further related vital signs,

[0050] Output of a prediction algorithm and details explaining the output, such as a list of the most important inputs contributing to the output,

[0051] Appointments in a calendar and further details, including location, staff, required resources, etc.,

[0052] Surgical planning and surgical documentation,

[0053] Clinical notes and related clinical notes,

[0054] Parts (text) of clinical notes and related clinical notes,

[0055] Clinical protocols (or procedures) and more information about the protocols,

[0056] Physical order entries and more information / details about the orders,

[0057] Billing and more detailed billing information,

[0058] Patient administrations and more detailed patient administrations,

[0059] Patient registrations and more detailed patient registrations,

[0060] Patient overviews and individual patient details,

[0061] Operation dashboards and more details about specific elements.

[0062] An example sequence of four types of data that can be displayed in a sequence of consecutive "zoom in" commands is: disease -> disease subtype -> disease mechanism -> genetic information.

[0063] Note that in the above example, the second data can relate to a specific patient. For example, the second data can be taken from an EHR. Alternatively or additionally, the second data can be taken from one or more other sources. For example, medical guidelines, medical logs, medical texts, medical resource databases, medical repositories, medical reference tools, clinical decision support systems, or any other source. References from government agencies can also be used, such as resources from the Food and Drug Administration (FDA).

[0064] It should also be noted that the first data and the second data can be of different types. For example, the first data can be in text format and the second data can be in image format (e.g., such as a medical image), and vice versa. Thus, the first data can be of a first data type and the second data can be of a second data type different from the first data type. Examples of data types include, but are not limited to: images, text strings, animations, data in tabular format, data presented in graphical format, hyperlinks, simulation data, audio data, 3D data or images, video data, or any other data type.

[0065] In some embodiments, if the first user input indicates a zoom in command, the second data is more detailed than the first data. In some embodiments, if the first user input includes a zoom out command, the second data is less detailed than the first data. Thus, in this way, the user can zoom in and out respectively to view more or less details. According to the methods herein, data is retrieved in a logical and intuitive manner in response to a zoom command.

[0066] The second data can be retrieved in different ways. For example, the second data can be logically connected to the first data as part of a hierarchical data structure in an EHR. An example hierarchical data structure is an XML data structure, whereby data can be stored in a sequence of hierarchical levels. Another example of a hierarchical data structure is a Flexible Image Transport System (FITS) header structure, which is often used to store radiological images.

[0067] Another example of a hierarchical structure is the Health Level Seven Fast Healthcare Interoperability Resources standard, HL7 FHIR. In such an example, the structure within the standard can determine how data is linked in a zoom command. The example can be summarized as follows: Food Allergy: Category (Food | Drug | Environment | Biological) -> Zoom In -> Type (Allergy | Intolerance - Deep Mechanism). For more information, see the HL7 FHIR documentation, section 9.1, titled: "Resource Allergy Intolerance".

[0068] An example of a hierarchical structure that represents the relationships between different medical terms and concepts is to use a coding mechanism that shows how different concepts are hierarchically linked to each other. For example, for drugs, a structure similar to the Anatomical Therapeutic Chemical Classification System (ATC drug codes) can be used.

[0069] Generally, in an embodiment where data is in a hierarchical data structure, the structure can be learned from the data or can be predetermined, for example, by referring to EMR architectures and data transfer, management, and interoperability standards.

[0070] In this way, the second data can be in an adjacent level of the hierarchical structure to the first data. In other words, the user may be able to zoom in to view data in lower levels of the hierarchical data structure and zoom out to view data in higher levels of the hierarchical data structure.

[0071] In other embodiments, the receipt of a zoom command can perform a computational function to retrieve second data that is semantically linked to the first data. As an example, the first data can be one of the inputs to such a computational function that is linked to a first portion of the display. In such an example, the zoom action can cause the computational function to be executed and a function output to be generated (it is possible that, for example, if the function is executed at a set frequency or based on other triggers, the function is executed and the output is pre-generated). In such an embodiment, the second data is then determined based on the function output.

[0072] Inputs to such functions include, but are not limited to, the following examples: EHR, user identity, information from the user profile, and data displayed in the first portion of the display.

[0073] The function that is executed can be predetermined, but different executions may result in different outputs because it is possible that:

[0074] - The first data changes;

[0075] - In addition to the first data, it utilizes other inputs, and if these inputs change, the function output will change. Examples of other inputs in addition to the first data are time, other EHR data, or user-related inputs (previous zoom actions, profiles);

[0076] - The function changes because it is random (and indeterministic), or because the function algorithm has been retrained / updated.

[0077] As described above, the EHR may change due to the potential dynamic nature of the information in it.

[0078] Therefore, the functional output can be considered the current version of the meaning / interpretation of the first data, and if this implies a change, the semantically linked data (second data) can also change.

[0079] As an example, the computational function described above can use natural language processing or machine learning to determine the meaning of the first data. Examples of ways to determine semantic meaning based on clinical data are described in the paper by Kersloot, M.G., van Putten, F.J.P., Abu-Hanna, A et al., titled "Natural language processing algorithms for mapping clinical text fragments onto ontology concepts: a systematic review and recommendations for future studies" J Biomed Semant 11, 14 (2020) https: / / doi.org / 10.1186 / s13326-020-00231-z. The concepts in this paper can be used to determine semantically linked words, which can be used as a search string for retrieving the second data or as an input parameter to a data query.

[0080] Some additional example methods that can be used to calculate semantic meaning and semantic links (e.g., semantic meaning and semantic links can be used to determine the second data semantically linked to the first data) are as follows:

[0081] Natural Language Processing (NLP) : NLP algorithms can be trained to analyze the structure and meaning of sentences, identify relationships between words and phrases, and extract relevant information from the text. For example, using medical literature on diseases for training, the relationships between different words and phrases can be learned. This relationship can be used later to determine the output of the scaling function. For example, it can be learned that certain drugs are mentioned in the text describing a specific disease. Based on this information, it can be assumed that the disease and the drug name are semantically linked.

[0082] Another example is to look at the links between different words within the same sentence or in the text that are within a set distance of each other. Then, it can be assumed that the words / phrases within a certain distance of each other are semantically linked.

[0083] Using natural language algorithms for semantic analysis is a well-developed area, and the skilled person will be familiar with the appropriate techniques available. Some additional references are: Chintalapudi N et al. (2021) “Text mining with sentiment analysis on seafarers’ medical documents”; Feldman R., Regev Y et al. (2004) “Mining the biomedical literature using semantic analysis and natural language processing techniques” and Velupillai S. et al. (2015) “Recent Advances in Clinical Natural Language Processing in Support of Semantic Analysis”; 10(1):183-93; doi:10.15265 / IY-2015-009.

[0084] Examples of algorithm types are as follows.

[0085] Word Embedding Process : Methods that represent words as vectors (word embedding algorithms), such as word2vec and GloVe, where words with similar meanings are located close to each other. Then, these distances can be used to identify semantic relationships between words.

[0086] Semantic Network : This is a graphical representation of the meaning of words or phrases in a given text. It is represented as a network of nodes and edges, where the nodes are concepts or entities, and the edges represent the relationships between them.

[0087] Semantic Role Labeling (SRL) : Here, the roles of different words or phrases are identified in an automated manner. For example, some words can be labeled as “agents”, some as “actions”, etc.

[0088] When using machine learning algorithms (such as NLP) to find semantic links in medical data, one approach is to use large datasets of medical data. Learning can be performed based on a supervised or unsupervised approach.

[0089] When performed in an unsupervised manner, the relationship can be learned automatically from the text. For example, this is possible because linked words will occur very close to each other. Using section divisions, chapter divisions, paper clustering / classification, etc. used in medical literature, the relationship between words and concepts can be learned. K-means clustering and hierarchical clustering are example techniques that can be used.

[0090] Another example is using annotated data to learn semantic links. In this example, typical uses of the EMR can be used to determine and learn relationships between different types of EMR data and fields. For example, usage data indicating the source of a click and its resulting destination and the content of that source and destination can be collected. Such information can be used as a label for connections between different EMR fields. In other words, by tracking how clinicians use the EMR, if they travel from data 1 field to data 2 field and the content of those data fields, and by using such information as ground truth during training, machine learning semantic links applicable to human users can be learned. For example, it can be assumed that data 1 and data 2 data or data fields are linked. Using this information, given input data or data fields, a trained machine learning algorithm can predict the linked data or data fields. Support Vector Machine (SVM) and decision trees are example methods that can be used.

[0091] Other techniques include but are not limited to: deep learning algorithms such as convolutional neural networks, recurrent neural networks, and long short-term memory (LSTM) networks.

[0092] Latent Dirichlet Allocation (LDA): A probabilistic generative model that can be used to identify topics in a data corpus. For example, semantic relationships such as synonyms and antonyms can be identified.

[0093] Latent Semantic Analysis (LSA): A mathematical technique that uses Singular Value Decomposition (SVD) to identify relationships between words and concepts.

[0094] Word2vec: A deep learning algorithm for vector representation of words. These vector representations can be semantic relationships such as similarity, relevance, etc. See: Mikolov, Tomas et al. (2013) “Efficient Estimation of Word Representations in Vector Space”

[0095] GloVe is another algorithm similar to Word2vec. See: Jeffrey Pennington, Richard Socher, and Christopher Manning, 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532 - 1543, Doha, Qatar, Association for Computational Linguistics.

[0096] BERT: This is a deep learning algorithm based on transformers. It is very powerful and has many variants that can be used to identify semantic relationships. See: "BERT: Pre-training of Deep Bidirectional Transformer for Language Understanding" by Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Touanova, arXiv:1810.04805

[0097] Now returning to method 200, after step ii), the method moves to step iii), where in response to receiving a first user input, method 200 includes instructing 206 the display to display second data in a second portion of the display.

[0098] The second portion of the display can be the same as the first portion of the display. In other words, step 206 can include instructing the display to replace the first data (e.g., instead of the first data or in the same area as the first data) with the second data. In some embodiments, the second portion of the display at least partially overlaps the first portion of the display. As a result, step 206 can include instructing the display to replace the first data with the second data.

[0099] In other embodiments, the zoom command causes a new window or graphical user interface (GUI) to appear. In such embodiments, the zoom command can cause a new window to pop up and display the second data. In other words, the first portion of the display can be in a first GUI (e.g., corresponding to the first GUI), and the second portion of the display can be in a second GUI (e.g., corresponding to the second GUI). The processor can cause the second GUI to be launched on the display in response to receiving the first user input.

[0100] Thus, in this way, a zoom command initiated with respect to first data displayed on the display can be used to retrieve and display second data that is semantically related to the first data from or related to the EHR.

[0101] Subsequently, a further zoom command can be received from the user and the further zoom command can be processed according to method 200. For example, if a second zoom command is received that indicates zooming in a direction opposite to the first zoom command (e.g., if the first zoom command is a zoom-in command and the second zoom command is a zoom-out command, or vice versa), the second zoom command can (at least partially) reverse the first zoom command.

[0102] In other words, method 200 can also include receiving a second user input from a user input device, the second user input indicating a second zoom command relative to a first portion of the display. The second zoom command can at least partially reverse the first zoom command. Accordingly, method 200 can also include instructing the display to redisplay the first data in the first portion of the display.

[0103] In an embodiment where the first data and the second data are part of a hierarchical structure, a zoom command in a first direction can cause data to be displayed from a higher level in the hierarchy, while a received zoom command in a second, opposite direction can cause data to be displayed from a lower level in the hierarchy.

[0104] In an embodiment where the zoom command is linked to a computing function, a second zoom command in the opposite direction can be interpreted as a computation that is the inverse of the computing function. As another example, the computing function can be a different function, where the second data is one of the inputs. In another example, it can be a function with both the first data and the second data as inputs. In other words, all previous links used to reach the current data field can be saved in the memory for the usage session of the EMR, and the zoom / unzoom history can be used as one of the inputs to the computing function. The use of the historical zoom actions (previous zooms) is also one of the reasons why zooming at the same area at different time points can result in different outputs.

[0105] In some embodiments, in response to receiving the second zoom command, method 200 includes retrieving third data that is semantically linked to the second data, and instructing the display to display the third data in a third portion of the display. In such a scenario, the third data may be different from the first data if, due to the processor performing step ii): there has been a change in the first data; there has been a change in the second data; or if the third data has become available (e.g., because the processor has performed step ii)), and as a result, the third data is semantically more highly linked to the second data compared to the first data.

[0106] In other words, in this embodiment, if the second zoom command is an inverse zoom command of the first zoom command (e.g., if the second zoom command is in the opposite direction of the first zoom command), this will not necessarily result in the re-display of the first data if information that is more semantically relevant to the second data (compared to the first data) is now available.

[0107] Now turning to Figure 3 , which shows an example according to some embodiments herein. In this example, in Figure 3 (a), a display 300 (or viewport) with medical concepts is shown, where different regions (301, 302) indicated by dashed lines and shaded areas are identified for each medical concept. In this example, each region is associated with a zoom function for that particular region. Zooming on the first region (301, not the mouse position) at position 310 using a mouse activates the first zoom function (also referred to herein as a calculation function), resulting in Figure 3 the display shown in b. For example, zooming on the second region (302) at the mouse position 320 activates the second zoom function, resulting in Figure 3 the display shown in c. Zooming on the background (340) activates the third zoom function, resulting in Figure 3 the display shown in d. Taking the zooming of drug data as an example. The first zoom function can provide the generic name for a professional, the second zoom function can provide the brand name for a layperson, and the third zoom function can affect the font size of the text.

[0108] Thus, Figure 3 a system for zooming semantic information in a viewport is shown, where the information includes multiple regions (e.g., regions 301, 302), and each region has a zoom function associated with that region.

[0109] Taking an anticoagulant as an example as shown in Figure 3 a. When zooming with a mouse cursor on the left side of this concept at 310, the first zoom function can display the generic name of the actual drug (e.g., bivalirudin as shown in Figure 3 b), and the region for the muscle relaxant remains unchanged. When zooming with a mouse cursor on the right side of this concept at 320, the second zoom function can display the brand name of the same drug (e.g., Angiomax as shown in Figure 3 c). Note that, as described above, these regions can be configured such that the zoom in and zoom out commands support inverse operations.

[0110] In addition, the system can be extended to support multiple zoom functions for a region, where a specific zoom function is associated with a semantic type of a deep medical concept. Take, for example, a zoom function for zooming anatomical data in addition to the aforementioned zoom function for drug data. The first zoom function can show the Latin name of the anatomical structure (e.g., hepaticus for liver), and the second zoom function can show the translated anatomical name for laypersons (e.g., liver). Thus, the proposed concept provides a single solution for exploring medical data, identifying the needs of different users, and allowing users to decide for themselves what level of information they need while minimizing the data shown in the viewport.

[0111] Note that the regions do not have to be predefined and do not have to be part of the source data. The regions can be obtained dynamically based on the deep meaning of the data. For example, by grouping the content shown according to meaning (e.g., using the NLP method described above) and accordingly dividing the display into different parts. The system can dynamically link the content to a specific zoom function. Thus, the configuration can be minimal, and the medical data can be scaled according to the database without information on the zoom regions.

[0112] Now turning to Figure 4 , which shows a plurality of computing modules 400 that can be used to implement method 200. The modules 400 can be implemented in a device such as device 100 described above.

[0113] In this embodiment, there is a display system 430 configured to display medical concepts such as EHR from a patient database 450 on a viewport unit 440.

[0114] The system further includes a tagging unit 420 configured to form a plurality of display regions of the medical concepts displayed in the viewport unit. First, it uses a semantic DB 480 to identify the type of the medical concept. For example, the type of "anticoagulant" is a drug.

[0115] Drug classes and types (including brands) are available in hospital systems and NLP, or any of the other techniques described above can be used to create the semantic DB 480, as described above. Also as described above, the FHIR standard or other data structures can be used to link the data in the semantic database. As an example, the output of the described NLP algorithm can be used to show the relationships between different words and concepts. For example, if words (or groups of words) are represented as numerical vectors, the semantic DB 480 can contain this information. In addition, the relationships between different words / concepts can be represented as decision trees or connected graphs.

[0116] As another example, the semantic database can follow a structure similar to the Unified Medical Language System (UMLS), see Reference Manual; Bethesda (MD): National Library of Medicine (US); 2009.. As an example, the semantic DB 480 can follow a structure similar to the UMLS, but can, for example, consider updated patient data, updated protocol and workflow execution data, clinician and organization profiles to learn and automatically update the relationships.

[0117] Second, the system receives region definitions for a specific type from the semantic region DB 470. The semantic region DB 470 can include rules / codes that determine aspects related to the user interface or user interaction. These are aspects linked to how to determine the active region (the region with the zoom function) and indicate the active region to the user. These can be based on predefined rules. For example, the rule can be: "Use the left 20% as the left region and the right 20% as the right region". Another example of a rule can be: "Use the outermost pixels of the word to determine the boundary of the word, and use the pixels within a 20-pixel distance from the boundary as the active region". However, it should be understood that these are merely examples, and the semantic region DB can include rules or combinations of rules different from those described herein.

[0118] In some embodiments, the regions can be automatically updated, learned, or determined, for example, based on the traced interactions of a human user with the EHR system. They can also learn from or copy from other systems showing similar types of data.

[0119] Depending on the type of information displayed on the EMR screen (e.g., billing, patient data, scheduling data, etc.), the definition of the active region can be different.

[0120] In some cases, the definition of the regions can be determined or influenced by the number or type of links assigned to a particular data entry. For example, an increase in the number of hierarchical links can result in an increase in the number of active regions (and / or a decrease in the size of the active regions).

[0121] Each region is identified with a "left" or "right" position indicator. Third, it applies the region definition to a specific medical concept (e.g., anticoagulant, see Figure 3 a).

[0122] The semantic display system 430 assigns the zoom functions from the zoom function database 460 to each of the constituent display regions.

[0123] Then it receives a zoom function (also referred to herein as a zoom command) for a specific medical concept type, such as a drug. Then, the zoom function is associated with a specific region in the semantic region database 470, where the zoom function is tagged with a location indicator (e.g., "left", "right").

[0124] The system includes an input unit 410 for selecting a region from a plurality of regions on the viewport 440. For example, the user selects the left region on an anticoagulant.

[0125] System 430 is configured to apply the selected zoom function to the selected medical concept. For example, the zoom function is defined to reveal all potential brand names of a drug. To achieve this, the semantic database 480 may contain various known brand names (e.g., Angomax). Additionally, the database may include the relationship from the brand name to the drug type. The list of filtered brand names is crossed with the data in the patient database. The intersection is displayed in the viewport unit 440 (see Figure 4 b).

[0126] The implementation can be enhanced in various ways.

[0127] - The region location can be extended with additional region indicators such as top, bottom, and middle positions.

[0128] - Not all data sources are structured and codified. The system can recognize semantics from the text on the display in an automated manner. This enables semantic zooming on sources that may not have been intended for semantic zooming traditionally. As another example, an external document can be used as a table of contents for patient information (such as EHR). In this way, it allows for the progressive disclosure of patient information manipulated by documents that are not part of the system.

[0129] Those skilled in the art will be able to apply this method to other medical phrases as well as isolated medical concepts.

[0130] - The word / concept / content of the document on which the mouse is pointed can be recognized, and the zoom function can be associated with displaying other words linked to the corresponding word. For example, for a given drug, other drugs with the same active ingredient can be displayed or hidden depending on the zoom-in and zoom-out actions respectively.

[0131] As another example, the zoom action can be associated with displaying or hiding linked content. The link can be, for example, a hyperlink to a web page or a QR code. In this case, zooming when hovering over the link will cause the content associated with the link to be shown with increased detail and quantity, while zooming out will reduce the amount of information.

[0132] - The system can be enhanced to recognize parts of words. For example, it can magnify parts of a drug name. Many treatments use monoclonal antibodies (MABs). When magnifying infliximab, it will result in all monoclonal antibodies or all MAB treatments related to the patient's disease.

[0133] - The system can be enhanced to recognize the way information is presented on the screen. For example, font styles, specific characters used, or specific pixel combinations of graphics, allowing multiple zoom functions based on the way information is presented on the display.

[0134] Thus, in this way, semantic zoom can be used to search for and retrieve information in the EHR in a more intuitive and comprehensive manner.

[0135] Now turning to other embodiments, in some embodiments, a computer program product including a computer-readable medium is provided, the computer-readable medium having computer-readable code embodied therein, the computer-readable code being configured to cause a computer or processor to perform one or more of the methods described herein when executed by a suitable computer or processor.

[0136] Therefore, it should be understood that the present disclosure also applies to computer programs, particularly to computer programs on or in a carrier, which are adapted to put the various embodiments into practice. The program can be in the form of source code, object code, intermediate source code, and object code in a partially compiled form, or any other form suitable for the implementation of the methods according to the various embodiments described herein.

[0137] It should also be understood that such a program can have many different architectural designs. For example, the program code implementing the functions of the method or system can be divided into one or more subroutines. Many different ways of allocating functions among these subroutines will be obvious to those skilled in the art. The subroutines can be stored together in an executable file to form a self - contained program. Such an executable file can include computer - executable instructions, such as processor instructions and / or interpreter instructions (e.g., Java interpreter instructions). Alternatively, one or more or all of the subroutines can be stored in at least one external library file and linked to the main program statically or dynamically, for example, at runtime. The main program contains at least one call to at least one subroutine. The subroutines can also include function calls to each other.

[0138] The carrier of a computer program can be any entity or device capable of carrying the program. For example, the carrier can include a data memory, such as a ROM (e.g., CD ROM or semiconductor ROM) or a magnetic recording medium (e.g., hard disk). In addition, the carrier can be a transmissible carrier, such as an electrical or optical signal, which can be transmitted via a cable or optical fiber or by radio or other means. When the program is implemented in such a signal, the carrier can consist of such a cable or other device or apparatus. Alternatively, the carrier can be an integrated circuit in which the program is embedded, and the integrated circuit is adapted to execute or for executing the relevant method.

[0139] From a study of the drawings, the present disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments in practicing the principles and techniques described herein. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. The fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but it may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A healthcare device, comprising: a memory including instruction data representing an instruction set; and a processor configured to communicate with the memory and execute the instruction set, wherein the instruction set, when executed by the processor, causes the processor to: i) receive a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command being indicated with respect to a first portion of a display, and in the first portion of the display, first data from an electronic health record, EHR, is displayed; ii) retrieve second data semantically linked to the first data; and iii) in response to receiving the first user input, instruct the display to display the second data in a second portion of the display.

2. The device according to claim 1, wherein Causing the processor to retrieve the second data includes causing the processor to: perform a computational function to retrieve the second data semantically linked to the first data.

3. The device according to claim 2, wherein The computational function uses natural language processing, NLP, or machine learning to retrieve the second data semantically linked to the first data.

4. The device according to claim 1, wherein, As part of a hierarchical data structure in the EHR, the second data is logically connected to the first data; and wherein the second data is in an adjacent level of the hierarchical structure to the first data.

5. The device according to any one of the preceding claims, wherein, If the first user input includes a zoom-in command, the second data is more detailed than the first data, and / or, if the first user input includes a zoom-out command, the second data is less detailed than the first data.

6. The device according to any one of the preceding claims, wherein, The second portion of the display at least partially overlaps the first portion of the display, and / or, wherein the processor instructs the display to display the second data instead of the first data.

7. The apparatus according to any one of the preceding claims, wherein, The first portion of the display is in a first graphical user interface, GUI, the second portion of the display is in a second GUI, and wherein the processor causes the second GUI to be launched on the display in response to receiving the first user input.

8. The device according to any one of the preceding claims, wherein, The instruction set, when executed by the processor, further causes the processor to: receive a second user input from the user input device, the second user input indicating a second zoom command with respect to the first portion of the display; and wherein the second zoom command at least partially reverses the first zoom command.

9. The apparatus according to claim 8, wherein, In response to receiving the second zoom command, the instruction set, when executed by the processor, further causes the processor to: instruct the display to redisplay the first data in the first portion of the display.

10. The apparatus according to claim 8, wherein, In response to receiving the second zoom command, the instruction set, when executed by the processor, further causes the processor to: retrieve third data semantically linked to the second data; and instruct the display to display the third data in a third portion of the display; and wherein, if as a result of the processor executing step ii): there have been changes in the first data; there have been changes in the second data; or The third data has become available; and as a result, the third data is semantically more highly linked to the second data than the first data, the third data being different from the first data.

11. The apparatus according to any one of the preceding claims, wherein: the first data relates to a medical condition mentioned in the EHR, and the second data relates to a treatment given for the medical condition; the first data relates to a medical condition mentioned in the EHR, and the second data relates to further information related to the medical condition; or the first data relates to a type of drug mentioned in the EHR, and the second data relates to a specific drug within the type.

12. The device according to any one of the preceding claims, wherein, The first data belongs to a first data type, and the second data belongs to a second data type different from the first data type.

13. A computer-implemented method, the method comprising: i) receiving a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command being indicated with respect to a first portion of a display, the first data from an electronic health record, EHR, being displayed in the first portion of the display; ii) retrieving second data that is semantically linked to the first data; and iii) in response to receiving the first user input, instructing the display to display the second data in a second portion of the display.

14. The method according to claim 13, wherein If the first user input includes a zoom-in command, the second data is more detailed than the first data, and / or, if the first user input includes a zoom-out command, the second data is less detailed than the first data.

15. A computer program product comprising a computer-readable medium having computer-readable code embodied therein, the computer-readable code being configured to cause a computer or processor to perform the method according to claim 13 or 14 when executed by a suitable computer or processor.

Images