Conversational interaction with databases

Abstract

A system includes one or more memory that stores instructions and one or more processors that executes the instructions. When executed by the one or more processors, the instructions cause the system to: interrogate a first database; extract, based on interrogating the first database, a first schema of the first database; and generate, based on the first schema of the first database, a first system prompt structure to be accessed by a large language model to generate a database query for input to search the first database. The instructions also cause the system to output results of a search of the first database based on input received via a user interface and a resultant selection of a first system prompt structure using the large language model to generate a query to search a first database.

Description

CONVERSATIONAL INTERACTION WITH DATABASESBACKGROUND

[0001] Software solutions deployed in hospitals provide users ways to view patient demographics, medical history, and data related to imaging and non-imaging studies. The software solutions are provided to support clinical decision-making. Clinical data and operational data for these software solutions are stored in specialized databases. An advanced analytics software solution may connect to these specialized databases to produce pre-configured reports with key performance indicators (KPIs) relating to, for example, procedure volumes, productivity, efficiency, and quality. KPIs can be classified into operational metrics or clinical metrics. Operational metrics include performance, distribution of procedure characteristics, staffing, reporting, interventionspecific metrics, and protocol and guideline adherence. Clinical metrics include quality and safety indicators, clinical outcomes, trends, and imaging findings and measurements.

[0002] Users of the software solutions may include clinicians, department heads, administrators, supporting staff, researchers, quality and safety officials, and executives within a healthcare organization. The users may need timely access to KPIs and pre- configured dashboards computed from such specialized databases. However, simply providing pre- configured dashboards may not be sufficient for users to receive answers to their specific questions. Moreover, the users are not always equipped with the knowledge to query the specialized databases directly using the correct tools such as structured query language (SQL) tools and access controls which enable users to arrive at the answers to their specific questions. Users often delegate tasks to a capable database support specialist or data science colleagues within their organization with variable, and often significant, turnaround times. Sometimes the users have to reach out to service teams of third- party vendors to provide a solution. These approaches slow down decision making in the hospital environment.SUMMARY

[0003] According to an aspect of the present disclosure, a system includes one or more memory and one or more processors. The one or more memory stores instructions. The one or moreprocessors executes the instructions. When executed by the one or more processors, the instructions cause the system to: interrogate a first database; extract, based on interrogating the first database, a first schema of the first database; and generate, based on the first schema of the first database, a first system prompt structure to be accessed by a large language model (LLM) to generate a database query for input to search the first database.

[0004] According to another aspect of the present disclosure, a system includes one or more memory and one or more processors. The one or more memory stores instructions. The one or more processors executes the instructions. When executed by the one or more processors, the instructions cause the system to: receive input via a user interface; select a first system prompt structure using a large language model that receives the input from the user interface; generate, by the large language model based on the first system prompt structure and the input, a query to search a first database; send the query to the first database to search the first database; receive results of the search of the first database; and generate output of the results of the search of the first database.

[0005] According to another aspect of the present disclosure, a computer-implemented method of querying clinical data repositories includes receiving a natural language query as input via a user interface; selecting a first system prompt structure using a large language model that receives the input from the user interface; generating, by the large language model based on the first system prompt structure and the input, a query to search a first database; sending the query to the first database to search the first database; receiving results of the search of the first database; and generating output of the results of the search of the first database, and receiving a response with results for the query from a language model.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order to describe the manner in which the advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments or examples thereof which are illustrated in the appended drawings. Understanding that these drawings depict only exemplary embodiments of the disclosure, and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0007] The example embodiments are best understood from the following detailed description when read with the accompanying drawing figures. It is emphasized that the various features are not necessarily drawn to scale. In fact, the dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and practical, like reference numerals refer to like elements.

[0008] FIG. 1 illustrates a system for conversational interaction with databases, in accordance with a representative embodiment.

[0009] FIG. 2 illustrates another system for conversational interaction with databases, in accordance with a representative embodiment.

[0010] FIG. 3 illustrates a hybrid system and method for conversational interaction with databases, in accordance with a representative embodiment.

[0011] FIG. 4 illustrates a method for conversational interaction with databases, in accordance with a representative embodiment.

[0012] FIG. 5 illustrates a user interface for conversational interaction with databases, in accordance with a representative embodiment.

[0013] FIG. 6 illustrates dashboards for conversational interaction with databases, in accordance with a representative embodiment.

[0014] FIG. 7 illustrates a system for conversational interaction with databases, in accordance with a representative embodiment.

[0015] FIG. 8 illustrates a supporting infrastructure for conversational interaction with databases, in accordance with a representative embodiment.

[0016] FIG. 9 illustrates a module group of a platform service for conversational interaction with databases, in accordance with a representative embodiment.

[0017] FIG. 10 illustrates an IO bridge module for conversational interaction with databases, in accordance with a representative embodiment.DETAILED DESCRIPTION

[0018] In the following detailed description, for the purposes of explanation and not limitation, representative embodiments disclosing specific details are set forth in order to provide a thorough understanding of embodiments according to the present teachings. However, other embodiments consistent with the present disclosure that depart from specific details disclosed herein remainwithin the scope of the appended claims. Descriptions of known systems, devices, materials, methods of operation and methods of manufacture may be omitted so as to avoid obscuring the description of the representative embodiments. Nonetheless, systems, devices, materials and methods that are within the purview of one of ordinary skill in the art are within the scope of the present teachings and may be used in accordance with the representative embodiments. It is to be understood that the terminology used herein is for purposes of describing particular embodiments only and is not intended to be limiting. Definitions and explanations for terms herein are in addition to the technical and scientific meanings of the terms as commonly understood and accepted in the technical field of the present teachings.

[0019] It will be understood that, although the terms first, second, third etc. may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another element or component. Thus, a first element or component discussed below could be termed a second element or component without departing from the teachings of the inventive concept.

[0020] As used in the specification and appended claims, the singular forms of terms ‘a,’ ‘an’ and ‘the’ are intended to include both singular and plural forms, unless the context clearly dictates otherwise. Additionally, the terms "comprises", and / or "comprising," and / or similar terms when used in this specification, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0021] Unless otherwise noted, when an element or component is said to be “connected to,” “coupled to,” or “adjacent to” another element or component, it will be understood that the element or component can be directly connected or coupled to the other element or component, or intervening elements or components may be present. That is, these and similar terms encompass cases where one or more intermediate elements or components may be employed to connect two elements or components. However, when an element or component is said to be “directly connected” to another element or component, this encompasses only cases where the two elements or components are connected to each other without any intermediate or intervening elements or components.

[0022] The present disclosure, through one or more of its various aspects, embodiments and / or specific features or sub-components, is thus intended to bring out one or more of the advantages as specifically noted below.

[0023] As described herein, conversational interaction with databases enables users to carry on a conversation with an embedded artificial intelligence (Al) agent that translates the user’s questions, expressed in natural language, into appropriate database queries such that resulting data may be transformed into appropriate text or visual elements that answer the user’s question. The users may include, for example, clinicians, department heads, administrators, supporting staff, researchers, quality and safety officials, and executives. Such users sometimes need timely access to metrics computed from such databases insofar as hospitals and similar institutions have such databases that store large amounts of healthcare information including contain patient records, clinical, operational, and financial data. Elements of the systems described herein therefore may provide an intuitive, natural language interface powered by a generative Al model, real-time query generation and execution capabilities, dynamic dashboards, session memory, and multilingual support. Together, some or all of these features solve the core problems of data accessibility, usability, and flexibility, enabling transformation of healthcare data interaction.

[0024] FIG. 1 illustrates a system for conversational interaction with databases, in accordance with a representative embodiment.

[0025] The system in FIG. 1 is a system on which a method for conversational interaction with databases may be implemented. FIG. 1 illustrates a system with an LLM that receives a user’s question, a system prompt and a history of previously asked questions and answers from the user’s session as inputs to the LLM. The LLM registers these inputs and then acts upon the inputs in a manner instructed by the system prompt. Visualizations generated by the system in FIG. 1 are referred to as dashboards.

[0026] Using the system of FIG. 1, significant limitations in the current methods used for healthcare data interaction may be addressed, particularly in cardiovascular analytics within hospital environments. Traditional approaches require multiple manual steps and heavy reliance on information technology (IT) or data science teams to access and analyze data, resulting in delays that slow decision-making, impact patient care, and add to the workload of support teams. Technically, data analysis is primarily handled through static dashboards, SQL queries, and support from data science teams, solutions that are functional but slow, and which requireexpertise not always available to clinicians. Clinically, healthcare providers rely on pre-set reports or wait for data requests to be fulfilled by technical staff, which limits their ability to respond to the latest information and hinders timely, data-driven decisions. While these dashboards display KPIs, they lack the adaptability needed to answer real-time, specific queries, limiting healthcare providers' ability to customize data views on demand. These systems also pose challenges for non-technical users, who are often unable to interact directly with databases due to a lack of technical expertise, further increasing dependence on support teams.

[0027] These challenges are well-recognized within healthcare data management. The need for efficient, real-time access to actionable insights has long been an issue and is resolved by the teachings herein. Although some advances have been made, existing analytics tools still fall short in terms of ease of use, flexibility, and conversational accessibility. The methods and systems disclosed herein overcome these limitations by introducing a generative Al-powered conversational interface that allows healthcare professionals to interact with data through natural language queries, both verbal and textual. Users can simply ask questions in plain language and receive immediate, data-driven answers without needing technical skills or intermediary support.

[0028] The system of FIG. 1 provides real-time, on-demand data access, which significantly speeds up the process of retrieving insights. The system of FIG. 1 enables users to generate and customize dashboards on the fly, allowing them to explore various metrics and views as needed. This flexibility empowers clinicians to independently track patient outcomes, administrators to monitor resource utilization, and department heads to make strategic, data-informed decisions. Additionally, by making data interaction accessible to non-technical users, the invention reduces bottlenecks caused by technical dependencies and fosters a data-driven culture across different roles within healthcare institutions.

[0029] FIG. 2 illustrates another system for conversational interaction with databases, in accordance with a representative embodiment.

[0030] The system 200 in FIG. 2 is a system on which a method for conversational interaction with databases may be implemented. The system 200 in FIG. 2 includes components that may be provided together or that may be distributed. The system 200 in FIG. 2 includes a user device 250, a first database 270, a second database 272, a display 280 connected to the user device 250, and a prompt structure generator system 290. Each of the user device 250 and the prompt structure generator system 290 may include a separate controller that includes at least a memory that storesinstructions and a processor that executes the instructions, though a controller may include more the a memory and a processor.

[0031] The user device 250 may generate and send a query to the first database 270 or the second database 272 based on a prompt structure generated by the prompt structure generator system 290 and particular to the first database 270 or the second database 272. Primary elements of the system 200 include a set of technical features designed to address the problems of inefficient, delayed, and non-intuitive data interaction in healthcare environments. These features collectively enable real-time, conversational data access, empowering healthcare professionals to retrieve and analyze data independently and efficiently. The primary elements and additional elements may include a conversational interface with generative Al capabilities, automated database query generation and execution, session history and context awareness, and multilingual support. The system 200 is thus enabled with multi-source data capabilities. The system 200 is configured to pull data from various sources, including relational and document-based databases. Multilingual support may be provided using language localization to support diverse healthcare teams. The multilingual capabilities may arise from the training of the LLM. Users can interact with the system in their preferred language, making it accessible in multicultural and international healthcare environments.

[0032] The display 280 may be local to the user device 250 or may be remotely connected to the user device 250. The display 280 may be connected to the user device 250 via a local wired interface such as an Ethernet cable or via a local wireless interface such as a Wi-Fi connection. The display 280 may be interfaced with other user input devices by which users can input instructions, including mouses, keyboards, thumbwheels and so on. The display 280 may be a monitor such as a computer monitor, a display on a mobile device, an augmented reality display, a television, an electronic whiteboard, or another screen configured to display electronic imagery. The display 280 may also include one or more input interface(s) that may connect to other elements or components, as well as an interactive touch screen configured to display prompts to users and collect touch input from users. The display 280 may be used to visualize the user interface 581 in FIG. 5 and / or the user interface 681 in FIG. 6.

[0033] The user device 250 and the prompt structure generator system 290 may perform some of the operations described herein directly and may implement other operations described herein indirectly. For example, the user device 250 may indirectly control operations such as bygenerating and transmitting content to be displayed on the display 280. The user device and the prompt structure generator system 290 may directly control other operations such as logical operations performed by a processor executing instructions from a memory based on input received from electronic elements and / or users via interfaces. Accordingly, the processes implemented by the user device 250 and the prompt structure generator system 290 when a processor executes instructions from a memory may include steps not directly performed by the user device 250 and the prompt structure generator system 290.

[0034] In an embodiment, the system 200 may provide a generative Al LLM for natural language processing and response generation; a speech-to-text and text-to-speech web browser or cloud provider functionalities; an automated query generation agent that translates natural language queries into database queries (e.g., SQL); a data aggregation agent to pull data from multiple sources (e.g., relational and document-based databases); a dynamic dashboard generation and visualization web-based interface to provide real-time, customizable data views; and a sessionbased memory and context management to track and utilize interaction history. The system 200 may be implemented partly or fully upon an existing healthcare informatics platforms, such as the infrastructure and platform described herein, and be integrated directly into a cardiovascular analytics software solution. By integrating this conversational Al-driven analytics tool into existing software solutions, the functionality of existing offerings may be amplified and provide clients with a cutting-edge data interaction experience.

[0035] FIG. 3 illustrates a hybrid system and method for conversational interaction with databases, in accordance with a representative embodiment.

[0036] At S302, an LLM operation is performed. For example, a user may provide instructions to an LLM via a user interface.

[0037] At S304, a database interrogation operation is performed. For example, the LLM may output instructions to retrieve data and / or metadata from a database, so that the LLM can generate a logical understanding such as a map or other arrangement of the data in the database and the logical arrangement of the data in the database. The logical arrangement may record where particular types of data are stored in the database. Locations may be stated as logical addresses and / or physical addresses.

[0038] At S306, a schema is obtained as a logical understanding of the data in the database and the logical arrangement of the data in the database.

[0039] At S308, a system prompt is generated so that the LLM from S302 or another LLM can be used to access the database in the future.

[0040] At S312, a determination is made as to whether the system prompt generated at S308 is to be filtered. Here, filtering may involve a re-interrogation of the database, a re-extraction of the date and metadata to obtain the schema, and / or a regeneration of the system prompt.

[0041] If the system prompt is to be filtered (S312 = No) the system prompt is set at S314. If the system prompt is to be filtered (S312 = Yes), the process returns to S302.

[0042] FIG. 3 shows that an LLM gets a schema and filters the results to only include meaningful relationships to use from the schema. This forward path in the figure corresponds to the interrogation, extraction, and generation aspects that result in a system prompt for a selected database. The feedback loop shown in FIG. 3 reflects the ability to re-interrogate, re-extract and regenerate the system prompt as described herein. The result of FIG. 3 is an ability to generate a system prompt for one or more databases.

[0043] FIG. 4 illustrates a method for conversational interaction with databases, in accordance with a representative embodiment.

[0044] The method of FIG. 4 may be performed by the system 200.

[0045] At S410, the method of FIG. 4 begins with interrogating a first database. The interrogation at S410 may be performed by a custom agent configured to interrogate databases including the first database.

[0046] At S420, a first schema of the first database is extracted based on interrogating the first database. The first schema may comprise or otherwise be based on a first logical arrangement of the first database, and may include at least one of table names, column attributes, data types, relationships between tables, relationships between columns, or relationships between datatypes, constraints, indexes and views. The schema acts as an architectural blueprint for the database but not the actual data.

[0047] At S425, the first database may be mapped based on the extracted first logical arrangement from S420. The mapping at S425 may be generated by an automated framework comprising a discovery module, interpretation module, and schema learning module. The discovery module may be a module that systematically studies each item of data and metadata in terms of labelling and location of data. The interpretation module may interpret the results of the discovery module according to a predetermined interpretation framework so that the data and metadata from thedatabase can be understood in context of the predetermined interpretation framework. The schema learning module may record the schema that states the metadata from the database including logical and / or physical locations of data characterized by the metadata and relationships between different data elements from the database.

[0048] At S430, the method of FIG. 4 includes generating, based on the first schema of the first database, a first system prompt structure to be accessed by a large language model to generate a database query for input to search the first database.

[0049] After S430, the method of FIG. 4 may return to S410 for several different reasons. In one set of embodiments, the system performing S410 to S430 may return to apply the custom agent to a second database at S410 again, then extract, by the custom agent, a second logical arrangement of the second database at S420, and generate, based on the second logical arrangement of the second database, a second system prompt structure to be accessed by the large language model to generate a database query for input to search the second database at S430. In these embodiments, the first system prompt structure may be customized for the first database and the second system prompt structure may be customized for the second database.

[0050] In another set of embodiments, the system performing S410 to S430 may return to S410 to reinterrogate the first database at S410, re-extract an updated first schema of the first database at S420 based on re-interrogating the first database at S410, and regenerate an updated first system prompt based on the updated first schema of the first database from S420. The interrogation and re-interrogation of the first database at S410 may be performed by a custom agent configured to interrogate and re-interrogate multiple databases of different types including the first database and a second database different from the first database. The updated first system prompt is to be accessed by the large language model to generate a database query for input to search the first database. The re-interrogation of the first database may be followed by the system re-extracting, based on re-interrogating the first database, an updated first schema of the first database; and re-generating, based on the updated first schema of the first database, an updated first system prompt structure to be accessed by the large language model to generate a database query for input to search the first database.

[0051] A semi-automatic prompt generation framework may be used to provide the LLM the necessary preliminary information to interact and comprehend the associated databases. This automatic prompt generation framework developed from S410 to S430 uses custom agents to enhance the efficiency and adaptability of LLMs in interacting with databases. These custom agents dynamically construct prompts by interpreting database structures and user intentions, enabling more accurate and contextually appropriate responses.

[0052] For automated database schema interpretation, custom agents analyze database schemas from S410 to S430 to extract table names, column attributes, data types, and relationships. This data allows the agents to auto-generate prompts that align with the database’s context, enabling the LLM to retrieve or process information accurately. For contextual prompt composition, the custom agents retrieve metadata and sample data to form a context-rich background, allowing the LLM to interpret the dataset’s nature and formatting needs. This contextualized prompt ensures the model’s responses are relevant and precise. For real-time prompt adaptation to database changes, the returns from S430 to S410 enable updating as databases evolve insofar as custom agents detect and reflect these changes in generated prompts, allowing the LLM to stay current without manual reconfiguration. An adaptive feedback loop enables agents to improve prompts based on user interaction and response effectiveness. This adaptive approach ensures continuous prompt refinement for optimal LLM performance. The custom agent framework significantly reduces manual intervention in prompt generation, allowing for scalable, consistent, and adaptive interaction between LLMs and databases.

[0053] At S440, another system such as a user device may receive input such as conversational input via a user interface. The user device may include a large language model in some embodiments. In other embodiments, the large language model may be implemented remotely such as in a cloud. For example, instructions executed by a user device may comprise an access program to access a remote instantiation of the large language model, and the access program may be configured to provide to the remote instantiation of the large language model the conversational input, the first system prompt structure, and a session history for a session in which the conversational input is received. The access program may be or include a system configuration such as a configuration file with a uniform resource locator (URL) and user-authentication mechanisms to indicate when the user is using an on-premise LLM, a local LLM or a cloud instance of the LLM. The configuration file or other system configuration may also indicate the type of database, the logical address of the database, the system prompt for the database, user session history etc.

[0054] For S440, a conversational interface such as keyboard or a microphone in a headset, handheld device, or monitor may be coupled to a generative artificial intelligence (Al) large language model (LLM) (GenAI Large Language Model). A user can “speak” to an application that interfaces with the LLM with questions expressed in natural language through text or speech. S440 initiates use of a conversational interface with generative Al capabilities to access a prompt generation framework for system prompt structures.

[0055] At or after S440 a session history may be created for a session with the LLM the first time input is received via the user interface in the session. A query that is generated later at S460 may be generated further based on the session history. The session history may be updated as the user asks questions. Session history may include the questions asked by a user, SQL queries or other types of queries output by the LLM for the database, and the final answers received by the system. The session history may also be maintained over multiple sessions for the user, and may be stored in data files for the user. In some embodiments, session history may be stored in subfiles for the user and a patient when the user uses the LLM to query the database for different patients. As explained below, the LLM may use the system prompt, the questions asked by the user, and the session history together to produce the database SQL queries or another type of database query as well as to produce the responses from the database to the questions by the user.

[0056] At S445, the other system may translate the conversational input into the query based on the first system prompt structure. For speech inputs, the conversational interface uses speech recognition to convert the user’s spoken question to text. Speech recognition is available with high-accuracy real-time transcription capabilities from a variety of sources including from multiple cloud providers such as Amazon, Microsoft, Google etc. For example, the other system may translate voice input as the conversational input into text input, and display the text input via a user interface for confirmation of the text input. The large language model may be configured to translate the conversational input into the query for the first database in accordance with formatting required for the first database. For natural language processing (NLP), the conversational interface powered by an LLM enables user interactions. This NLP-based interface allows users to input queries in natural language, either verbally or via text. The system 200 in FIG. 2 enables speech-to-text and text-to-speech integration. The system 200 may support both text and voice inputs, using speech recognition technology for real-time transcription of verbal queries. These features may leverage high-accuracy transcription from cloud providers and offers text-to-speech output for responses, allowing for a fully conversational experience.

[0057] At S447, the other system may display the test and the proposed text to the user for confirmation. The transcribed text may be displayed in a user interface for the application that interfaces with the LLM where the transcribed text and the proposed query may be reviewed and corrected by the user if needed.

[0058] The user may alternatively enter the question at S440 simply by typing the question instead of speaking the question, and the typed text may then be submitted to the Generative Al LLM. Additionally, in some embodiments S447 may be performed before S445, such as when only proposed text is presented to the user at S447 before the approved text is translated into the query based on the first system prompt structure.

[0059] At S450, a determination is made as to whether a session history exists for the session in which the conversational input is received at S440. The conversational Al component described herein includes session-based memory, allowing it to retain context from previous interactions. The session-based memory feature may create a seamless and context-aware interaction within the conversational Al. By recording and concatenating each question-and-answer exchange in a labeled sequence — such as “first question and answer,” “second question and answer” — the system 200 may build a coherent memory structure that the LLM can reference throughout the session. This approach allows the Al to retain important details from previous questions and responses, effectively “remembering” what has been discussed and using that knowledge to enrich future answers. For example, if a user asks a follow-up question, the Al can draw upon the specific context provided by earlier exchanges rather than requiring repeated clarification, making interactions more fluid and natural.

[0060] Additionally, the system 200 may provide context awareness. Users are able to ask followup questions and request text or visualization edits leveraging the complete conversation history in the current session. The session-based memory may also help the Al understand complex or multi-part inquiries by keeping a clear, step-by-step history of the conversation. This accumulated context ensures that the LLM maintains continuity, enhances response relevance, and can handle more sophisticated dialogues by building upon prior interactions in real-time.

[0061] At S453, if a session history exists (S450 = Yes), the other system may obtain a session history for a session in which the conversational input is received.

[0062] At S455, if there is no session history (S350 = Yes) or otherwise after S453, the other system may access, by a large language model based on the conversational input, a first system prompt structure. The first system prompt structure is based on a mapping of the first database at S425. S455 may involve selecting a first system prompt structure based on a large language model that receives the input from the user interface at S440.At S460, the other system may generate, by the large language model based on the first system prompt structure and the conversational input, a query to search the first database. The query may be generated further based on the session history. For automated database query generation and execution, natural language to SQL translation may be used. The LLM may be designed to interpret users’ questions and automatically translate the questions into structured database queries, such as SQL commands using customizable agents. This allows retrieval of relevant data from one or more healthcare databases without requiring users to understand SQL or other query languages. The LLM works by parsing the user’s question received at S440 and breaking the questions into sub-parts that can be answered from the database at S480. The LLM uses the system tools to, for example, list the database tables or their schema at S490, check if a SQL query is valid and run the SQL query at S460 to send the query to the database at S470. The LLM is instructed to apply these tools one or more times until a determination when the data provided by the queries are relevant to the user’s question. The LLM then terminates the iterative process and provides a final answer to the question in the form of text, speech, or data visualization. The first system prompt structure may be augmented to include instructions for rendering the graphical portion of the response. The LLM may generate code and the code may be executed to produce output such as a visualization. A method performed by the system with the LLM may include augmenting a system prompt structure with instructions for rendering aspects of the results in a graphical chart. A portion of the received response may be processed by a graph generating module to render contextual data visualizations corresponding to the augmentation.

[0063] At S470, the other system may send the query to the first database to search the first database.

[0064] At S480, the other system may receive results of the search of the first database. A portion of the received response may be processed by a graph generating module to render contextual data visualizations

[0065] At S490, the other system may generate output such as a display of the results of the search of the first database. For example, the other system may generate a dashboard for analytics of data stored in the first database for the display of the results of the search of the first database. S490 may involve more than producing an output. Results from the LLM may be parsed in S490 to a post-processing step that checks if the result contains markup text or code that can be renderedinto a visualization. Markup text may include, for example, paragraphs or bulleted lists. Code may include, for example, JavaScript or Python. At S490 the user will be presented with results such as text, images, video, or sound. Therefore, the output may be visual and / or audio, and may take any of a variety of possible forms.

[0066] FIG. 5 illustrates a user interface for conversational interaction with databases, in accordance with a representative embodiment.

[0067] The systems described herein may be used to generate dynamic, customizable visualizations and dashboards. Based on the conversational interactions enabled by user interfaces such as in FIG. 5, systems described herein may generate interactive data visualizations. Such systems may automatically generate relevant visualizations including charts, graphs and / or tables based on query results, making data interpretation more intuitive. Effectively, LLM outputs are transformed into a suitable and configurable format intuitive use. The data visualization features may transform query results into visually intuitive formats such as charts, graphs, and / or tables.

[0068] FIG. 6 illustrates dashboards for conversational interaction with databases, in accordance with a representative embodiment. The user interface in FIG. 6 shows a collection of visualizations or dashboards in the form of charts generated by the system. These charts are responses to some of the user’s queries over a session. These charts are saved using the user interface into this collection for future reference,

[0069] FIG. 7 illustrates a system for conversational interaction with databases, in accordance with a representative embodiment.

[0070] FIG. 7 depicts a block diagram illustrating a computer system 700 according to examples of the disclosure. Instances of the computer system 700 may be used to implement the user device 250 and / or the prompt structure generator system 290, for example. Instances of the computer system 700 may also be used to perform some or all aspects of the method of FIG. 3 and / or FIG. 4, for example. The computer system 700 includes a processor 701, an external bus 716, a display 780, and a variety of elements connected to the external bus 716. The computer system 700 may include a processor 701 for implementing one or more processors described herein. Processor 701 may be any suitable processor type including, but not limited to, a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable array (FPGA) where the FPGA has been programmed to form a processor, a graphical processing unit (GPU), anapplication specific circuit (ASIC) where the ASIC has been designed to form a processor, or a combination thereof.

[0071] Processor 701 may include one or more cores 702. Core 702 may include one or more ALU 704 (arithmetic logic unit(s)). In some examples, core 702 may include a FPLU 706 (floating point logic unit and / or a DSPU 708 (digital signal processing unit) in addition to, or instead of, ALU 704.

[0072] Processor 701 may include one or more registers 712 communicatively coupled to core 702. Registers 712 may be implemented using dedicated logic gate circuits (e.g., flip-flops) and / or any memory technology. In some embodiments, one or more register 712 may be implemented using static memory. The register may provide data, instructions and addresses to core 702.

[0073] In some examples, processor 701 may include one or more levels of cache memory 710 communicatively coupled to core 702. Cache memory 710 may provide computer-readable instructions to core 702 for execution. Cache memory 710 may provide data for processing by core 702. In some embodiments, the computer-readable instructions may have been provided to cache memory 710 by a local memory, for example, local memory attached to external bus 716. Cache memory 710 may be implemented with any suitable cache memory type, for example, metal-oxide semiconductor (MOS) memory such as static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0074] Processor 701 may include a controller 714, which may control input to processor 701 from other processors and / or components included in a system and / or outputs from processor 701 to other processors and / or components included in the system. Controller 714 may control the data paths in ALU 704, FPLU 706, and / or DSPU 708. Controller 714 may be implemented as one or more state machines, data paths, and / or dedicated control logic. The gates of controller 714 may be implemented as standalone gates, FPGA, ASIC or any other suitable technology.

[0075] Registers 712 and cache memory 710 may communicate with controller 714 and core 702 via internal connections 720A, 720B, 720C, and 720D. Internal connections may be implemented as a bus, multiplexor, crossbar switch, and / or any other suitable connection technology.

[0076] Inputs and outputs for processor 701 may be provided via an external bus 716, which may include one or more conductive lines. External bus 716 may be communicatively coupled to one or more components of processor 701, for example, controller 714, cache memory 710, and / or register 712. External bus 716 may be coupled to one or more components of the system.

[0077] External bus 716 may be coupled to one or more external memories. The external memories may include ROM 732 (read only memory). ROM 732 may be a masked ROM, Electronically Programmable Read Only Memory (EPROM), or any other suitable technology. The external memory may include Random Access Memory (RAM) 733. RAM X33 may be a static RAM, battery backed up static RAM, Dynamic RAM (DRAM), or any other suitable technology. The external memory may include Electrically Erasable Programmable Read Only Memory (EEPROM) 735. The external memory may include Flash memory 734. The External memory may include a magnetic storage device such as disc 736. In some examples, the external memories may be included in a system.

[0078] In some examples, external bus 716 may include a communications interface 738 by way of which the computer system 700 can connect to networks and receive data useful in executing the methods and system set out herein as well as transmitting information to other devices. The computer system 700 may further include an input / output (IO) interface 740 for communicatively accessing connected devices. The connected devices may include a video display unit such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a flat panel display, a solid-state display, or a cathode ray tube (CRT), and / or any other suitable technology. The connected devices may further include input devices such as a keyboard / virtual keyboard, touch- sensitive input screen, speech input with speech recognition, and / or a cursor control device such as a mouse or touch-sensitive input screen or pad, and / or other suitable technologies. The connected devices may also optionally include a disk drive unit, a signal generation device such as a speaker or remote control, an external network interface device, and / or other suitable technologies.

[0079] The display 780 may be local to the processor 701 or may be remotely connected to the processor 701. The display 780 may be connected to the processor 701 via the external bus 716 and a local wired interface such as an Ethernet cable or via a local wireless interface such as a Wi-Fi connection. The display 780 may be interfaced with other user input devices by which users can input instructions, including mouses, keyboards, thumbwheels and so on. The display 780 may be a monitor such as a computer monitor, a display on a mobile device, an augmented reality display, a television, an electronic whiteboard, or another screen configured to display electronic imagery. The display 780 may also include one or more input interface(s) that may connect to other elements or components, as well as an interactive touch screen configured to displayprompts to users and collect touch input from users. The display 780 may be used to visualize the user interface 581 in FIG. 5 and / or the user interface 681 in FIG. 6.

[0080] The processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform one or more method steps. The structure for a variety of these systems is discussed in the description below. In addition, any programming language that is sufficient for achieving the techniques and implementations of the present disclosure may be used. In addition, the language used in the specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the disclosed subject matter. Accordingly, the present disclosure is intended to be illustrative, and not limiting, of the scope of the concepts discussed herein.

[0081] In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes software programs. Further, in exemplary implementations can include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing may implement one or more of the methods or functionalities as described herein, and a processor described herein may be used to support a virtual processing environment.

[0082] Some portions of the description are presented in terms of symbolic representations of operations on non-transient signals stored within a computer memory. These descriptions and representations are used by those skilled in the art to convey the substance of their work most effectively to others . Such operations typically require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0083] All of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similarelectronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage components. Portions of the present disclosure include processes and instructions that may be embodied in software, firmware, or hardware, and when embodied in software, may be downloaded to reside on and be operated from different platforms used by a variety of operating systems.

[0084] In a networked deployment, the computer system may operate in the capacity of a server, or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer or distributed network environment. The computer system can also be implemented as or incorporated into various devices, such as a server or another type of computer such as a workstation that includes a controller, a stationary computer, a mobile computer, a personal computer (PC), a laptop computer, a tablet computer, or any other machine capable of executing a set of software instructions sequentially or non-sequentially that specify actions to be taken by that machine. The computer system can be incorporated as an integrated system part of a larger system that includes additional devices. In an example, the computer system can be implemented using one or more electronic devices that provide voice, video, or data communication possibilities. Further, while the computer system is illustrated in the singular, the term “system” shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set or multiple sets, of software instructions to perform one or more computer functions.

[0085] The computer system may also include one or more processors. The processor executes instructions to implement some, or all aspects of methods and processes described herein. The processor is tangible and non-transitory. As used herein, the term “non-transitory” is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term “non-transitory” specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time. The processor is an article of manufacture and / or a machine component.

[0086] The processor is configured to execute software instructions to perform functions as described in the various examples herein. The processor may be a general-purpose processor or may be part of an application specific integrated circuit (ASIC). The processor may also be a microprocessor, a microcomputer, a processor chip, a controller, a microcontroller, a digital signalprocessor (DSP), a state machine, or a programmable logic device, a logical circuit, including a programmable gate array (PGA), such as a field programmable gate array (FPGA), or another type of circuit that includes discrete gate and / or transistor logic. The processor may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in, or coupled to, a single device or multiple devices. The processor can include one or more internal levels of cache, and a bus controller or bus interface unit to direct interaction with a bus. The term “processor” as used herein encompasses an electronic component able to execute a program or machine executable instruction. References to a computing device comprising “a processor” should be interpreted to include more than one processor or processing core, as in a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed among multiple computer systems. The term computing device should also be interpreted to include a collection, or network, of computing devices each including a processor or processors. Programs have software instructions that can be performed by one or multiple processors that may be within the same computing device or which may be distributed across multiple computing devices. Further, the software instructions, when executed by the processor, perform one or more steps of the methods and processes as described herein.

[0087] FIG. 8 illustrates a supporting infrastructure for conversational interaction with databases, in accordance with a representative embodiment.

[0088] FIG. 8 depicts an example of a supporting infrastructure 800 in diagrammatic form. Supporting infrastructure 800 is structured in multiple layers, each designed to support different aspects of providing a medical software product deployed at least in part over the internet. Here, the multiple layers include CSL 804 (clinical storage layer), PL 802 (platform layer), FL 806 (foundation layer), and CPL 808 (cloud provision layer). The PL 802 may be used to implement the user device 250 and / or the prompt structure generator system 290 in FIG. 2, for example. The CSL 804 may be used to implement the SQL database 170 in FIG. 1 , the first database 271 and / or the second database 272 in FIG. 2, for example. The FL 806 may be used to provide back-end services in the system 200 of FIG. 2, for example.

[0089] CSL 804 provides an abstraction for accessing cloud or network stored patient data in corresponding healthcare data schema format. For example, some standard data formats prevail in a substantial portion of the healthcare industry and the stores provided by CSL 804 offer aready-to-go storage layer of repositories that can be accessed by, for example and without imputing limitation, a healthcare establishment to store and retrieve image data or patient records. In some examples, CSL 804 may include specialized stores for Fast Healthcare Interoperability Resources (FHIR) format 852, Digital Imaging and Communications in Medicine (DICOM) format 854, Health Level 7 (HL7) format 856, or other healthcare data schema formats 858 as new standards emerge or customized to particular hospitals as needed.

[0090] In one example, FL 806 includes modules and processes for funneling identity and access management through a single preferred layer. As a result, a single identity, assigned to an individual level user or an entity level user, may log into multiple applications across multiple types of devices, for example clinical workstations, healthcare application endpoints, and others.

[0091] REST APIs and / or HTTPS access points 850 may allow for applications to provide to user interfaces for configuration and user preferences 836 and / or receiving or viewing alerts and / or notices from various processes in PL 802. A logging and auditing 832 module provides for operational records of events and processes within FL 806, which may be later retrieved and reviewed. Notably, in some examples, IAM 838 may integrate across FL 806 and CPL 808, such as, for example and without imputing undue limitation, when a product or service includes direct instantiation of a cloud managed services 830 or other CPL 808 feature.

[0092] According to FIG. 8, PL 802 is the primary layer of the supporting infrastructure in which hosted business logic and platform services are conducted. As depicted in FIG. 8, PL 802 includes a managed platform services 810 module, an orchestration 826 module, and a PaaS manager 828 (platform as a service manager) module, though it is to be understood that PL 802 may include additional modules and / or processes relevant to product and service offerings operating upon the backend infrastructure.

[0093] Orchestration 826 is responsible for managing workloads for various instances of the PL 802 modules, such as managed platform services 810 module, that are delegated to CPL 808. In particular, orchestration 826 interfaces with cloud managed services 830, which are exposed by CPL 808 to facilitate various cloud provisioning utilities, such as on demand data processing, virtual server instantiation, certain database processes, and the like.

[0094] PaaS manager 828 likewise intersects with CPL 808 via cloud managed services 830. PaaS manager 828 handles platform-level workload orchestration, such as in the case of a hosted singular platform that serves customers rather than separate applications serving individualcustomers. In addition, PaaS manager 828 performs orchestration operations for managed platform services 810.

[0095] In FIG. 8, managed platform services 810 is depicted as including a module group made of data ingestion 818, data storage 820, data processing frameworks 822, and data processing 824 processes, a module group made of device management module 812 and device integrations module 814 processes, and an IO bridge 816 module. However, it is to be understood that other modules, processes, and the like may be included among managed platform services 810.

[0096] Device management module 812 and device integrations module 814 form a module group that integrates networked devices, such as diagnostic scanner arrays, across a hospital information system (HIS) and the like, into the supporting infrastructure. Device integrations module 814 incorporates communication endpoints and mappings by which devices may integrate. In some examples, device integrations module 814 may extend into or interoperate with control and / or data planes of a hospital or clinical network. Device management module 812 includes processes and access points facilitating visibility and access to devices connected to the supporting infrastructure via device integrations module 814.

[0097] Aspects of the teachings herein may be supported by supporting infrastructure as in FIG. 8, including various information technology (IT) backends, including either or both local architectures, either as monoliths, networked, or a combination thereof, and hosted architectures, such as a software as a service (SaaS), platform as a service (PaaS), and / or infrastructure as a service (laaS), or the like. In an example, a supporting infrastructure includes multiple interconnected layers respectively hosting, as an abstraction, various IT processes, services, accounts, and other management components.

[0098] The layers of the supporting infrastructure may be divided into, for example and without imputing limitation, a Foundation Layer (FL), a Clinical Storage Layer (CSL), a Platform Layer (PL), and a Cloud Provision Layer (CPL). Each layer is generally structured to support a designated aspect of the IT backend supporting software product offerings and is made of various components that each may include any or all of libraries, functions, application programming interfaces (APIs), data stores, and more.

[0099] The FL includes hosting management utilities enabling management of accounts related to the CPL, either directly by an end user or by a software product IT management. An additional Device Layer (DL) may serve as an environment for testing and development software productsprovidable over the IT backend. The CPL is a foundational layer “upon” which the FL, DL, and PL operate and with which components of each of the other layers interact to varying degrees. The CPL provides direct access to remote server infrastructure supporting the IT backend which supports the infrastructural hosting (e.g., spinning up of virtual machines, containers, etc.) for supported applications. The PL includes several components for providing platform support for software products. Platform support may include functions such as orchestration, layer integrations, software product operational functions, data repository access and / or management, and more.

[0100] The CPL includes various processes for performing infrastructural services supporting the layers which operate upon it. In some examples, these processes may include virtual server instantiation, hardware (e.g., physical server) orchestration and management, redundancy and backup operations, security and logging, routing operations, and more.

[0101] The PL is the primary location of the hosted product(s) and / or service offering(s), as well as the process(es) that support these offerings, including, but not limited to, providing services to one or more users. Among the various processes and modules that reside in, or originate from, the PL, orchestration and management modules typically interconnect the PL and the CPL.

[0102] FIG. 9 illustrates a module group of a platform service for conversational interaction with databases, in accordance with a representative embodiment.

[0103] FIG. 9 depicts a module group 900 that is one example of the module group of managed platform services 810 made of data ingestion 818, data storage 820, data processing frameworks 822, and data processing 824, as discussed above. It is to be understood that module group 900 is one example of a data insights and analytics module group provided by managed platform services 810, and variations, modifications, and additions to the system depicted may fall within the scope of the present disclosure. To the extent that descriptions of the module group 900 are consistent with the system 100 in FIG. 1 and the system 200 in FIG. 2, the descriptions of the module group 900 may be incorporated into the functionality and devices shown in and described with respect to FIG. 1 and FIG. 2. In general, data processing frameworks 902, data processing 904, data ingestion 906, and data storage 960 maintain continuous communications and integrations with each other, only a portion of which are depicted and discussed here.

[0104] In particular, data processing frameworks 902 maintains processes for retrieving ingestion frameworks 920, processing frameworks 922, and access frameworks 924. Frameworks arepurpose-built preconfigured loadouts for data processing tasks and may include references to data stored in stores for models 966, data 968, and / or other data 970.

[0105] Data processing 904 includes a gateway service 908 for enabling access and view to data processing tasks and results. Gateway service 908 includes an access and security process 910 which, in some examples, may interface with IAM 838 described above. Logging 914 maintains records of tasks of and interactions with data processing 904. Interface 912 provides the mechanism by which users may access, view and interact with aspects of data processing 904 as described above, through processes such as configuration 916 and query 918, respectively for performing configuration operations and reviewing logs and records.

[0106] Data processing 904 also includes managed services 901 for performing specific tasks on a product and / or service basis. It is also where the primary processing of data processing 904 is performed or orchestrated. Examples of managed services 901 may include, without imputing undue limitation, hosted development environments and workbenches, analytics services such as machine learning (ML) and / or natural language processing (NLP) services, runtime services such as worker or service scoring, services supporting experiments, and others.

[0107] Where data processing 904 must retrieve data for processing, such as in the case of training a ML model, data ingestion 906 provides support such activities. Data ingestion 906 includes an ingestion framework 930, provenance 932, and PaaS manager 934. Here, PaaS manager 934 is an integration with PaaS manager 228 discussed above for orchestrating ingestion operations over, in part, CPL 808.

[0108] Provenance 932 maintains tracking on activities of data ingestion 906, such as data pipeline operations and the like. In some examples, pipeline executions or tasks can be added to ingestion framework 930 and provenance 932 may track the additional pipeline task as well as its results. This tracking may then be stored in a bookkeeping or log service for later review and / or auditing.

[0109] Ingestion framework 930 receives a framework for data ingestion from ingestion frameworks 920 and operationalizes the framework on processing 962. In some examples, ingestion framework 930 may intake data from data 968 storing data within the supporting infrastructure, or from an external data source 950, which is mapped according to a separate framework retrieved from access frameworks 924. External data sources may be external data stores 952, networked access points 954 (e.g., HIS services, etc.), and / or directly provided files 956 (e.g., supplied JSON, zip, csv, etc., files and the like).

[0110] Data storage 960 encompasses data hosting processes for both real-time activities and longterm storage. Processing 962 and serving 964 respectively provide in-use storage of data for processing or training tasks and deployed processes, such as a deployed model, respectively. Models 966 is a repository for processing models, such as a trained ML model. Data 968 stores hosted datasets for, as an example, use in training a ML model. Other data 970 stores other data relevant to a particular managed service of the managed services 901 or other activity, such as configuration files and the like. The stores and repositories of data storage 960 may be dedicated physical databases or may be virtual or instantiated databases.

[0111] Any steps described in relation to examples and / or training herein can be performed by a specific-purpose computer system or general-purpose computer system, or a computer-readable medium, or data carrier system configured to carry out any of the steps described previously. The computer system can include a set of software instructions that can be executed to cause the computer system to perform any of the methods or computer-based functions disclosed herein. The computer system may operate as a standalone device or may be connected, for example using a network, to other computer systems or peripheral devices. As an example, a computer system performs logical processing based on digital signals received via an analogue-to-digital converter.

[0112] FIG. 10 illustrates an IO bridge module for conversational interaction with databases, in accordance with a representative embodiment.

[0113] FIG. 10 depicts an IO bridge module 1000, such as IO bridge 816 discussed above. IO bridge module 1000 integrates HIS network endpoints with various components of PL 802. Here, IO bridge 1002 also integrates a clinical data repository 1004 for information storage and later retrieval. The HIS network endpoints each constitute an HIS access point 1008, such as, for example and without imputing limitation, access terminals, diagnostic devices, patient monitoring devices, hospital network computers, etc. Each PL 802 may be a PL process 1006 and include any of the process, modules, and / or module groups of PL 802 discussed above.

[0114] A process performed by the systems herein begins with interpreting user speech, which is converted to text. The text is then translated into a query such as an SQL query using a SQL agent. The SQL agent formulates the query to retrieve relevant data from connected databases, ensuring that the data output matches the user’s intent as described earlier. Once the data is obtained, the LLM may be instructed to, for example, generate Python language code, leveraging libraries such as Plotly or Matplotlib, to create visualizations from the query results. In other embodiments theLLM may be instructed to generate JavaScript language code to create visualizations from the query results. Python or JavaScript code can be executed by the systems to generate visualizations.

[0115] The automated approach of transforming a query to a dashboard not only provides a seamless transition from raw data to meaningful visuals but also allows users to interact with the data in an easily interpretable format, significantly enhancing data exploration and insight generation. Users can create dashboards on-demand and customize the dashboards dynamically. These features allow for real-time visualization of data, tailored to the specific query or metric of interest to the user.

[0116] In an embodiment, dedicated hardware implementations, such as application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays and other hardware components, are constructed to implement one or more of the methods described herein. One or more embodiments described herein may implement functions using two or more specific interconnected hardware modules or devices with related control and data signals that can be communicated between and through the modules. Accordingly, the present disclosure encompasses software, firmware, and hardware implementations. Nothing in the present application should be interpreted as being implemented or implementable solely with software and not hardware such as a tangible non-transitory processor and / or memory.

[0117] In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes software programs. Further, in an exemplary, non-limited embodiment, implementations can include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing may implement one or more of the methods or functionalities as described herein, and a processor described herein may be used to support a virtual processing environment.

[0118] Accordingly, conversational interaction with databases enables users to carry on a conversation with an embedded artificial intelligence (Al) agent that translates the user’s questions, expressed in natural language, into appropriate database queries such that the resulting data may be transformed into appropriate text or visual elements that answer the user’s question. The users may include, for example, clinicians, department heads, administrators, supporting staff, researchers, quality and safety officials, and executives. The teachings above enable users to timely access metrics computed from such databases insofar as hospitals and similar institutions have such databases that store large amounts of healthcare information including contain patientrecords, clinical, operational, and financial data. Elements of the systems described herein may provide an intuitive, natural language interface powered by a generative Al model, real-time query generation and execution capabilities, dynamic dashboards, session memory, and multilingual support. Together, some or all of these features solve the core problems of data accessibility, usability, and flexibility, enabling transformation of healthcare data interaction.

[0119] Although conversational interaction with databases has been described with reference to several exemplary embodiments, it is understood that the words that have been used are words of description and illustration, rather than words of limitation. Changes may be made within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of conversational interaction with databases in its aspects. Although conversational interaction with databases has been described with reference to particular means, materials and embodiments, conversational interaction with databases is not intended to be limited to the particulars disclosed; rather conversational interaction with databases extends to all functionally equivalent structures, methods, and uses such as are within the scope of the appended claims.

[0120] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of the various embodiments. The illustrations are not intended to serve as a complete description of all of the elements and features of the disclosure described herein. Many other embodiments may be apparent to those of skill in the art upon reviewing the disclosure. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Additionally, the illustrations are merely representational and may not be drawn to scale. Certain proportions within the illustrations may be exaggerated, while other proportions may be minimized. Accordingly, the disclosure and the figures are to be regarded as illustrative rather than restrictive.

[0121] One or more embodiments of the disclosure may be referred to herein, individually and / or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any particular invention or inventive concept. Moreover, although specific embodiments have been illustrated and described herein, it should be appreciated that any subsequent arrangement designed to achieve the same or similar purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover anyand all subsequent adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the description. For example, it is to be understood that there may be variations of this disclosure, such as around the areas of prompt generation, session history, configuration of databases (e.g., relational, document, graph, etc.), configuration of agents (e.g., SQL, LLM, Chart, etc.), user interface in speech, text, chart, and other forms, and implementation choices (e.g., local LLM, cloud LLM, local and cloud data sources, etc.). Nevertheless, it is understood that such variations fall within the spirit and scope of the invention described herein.

[0122] The Abstract of the Disclosure is provided to comply with 37 C.F.R. § 1.72(b) and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, various features may be grouped together or described in a single embodiment for the purpose of streamlining the disclosure. This disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all of the features of any of the disclosed embodiments. Thus, the following claims are incorporated into the Detailed Description, with each claim standing on its own as defining separately claimed subject matter.

[0123] The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to practice the concepts described in the present disclosure. As such, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents and shall not be restricted or limited by the foregoing detailed description.

Claims

CLAIMS:

1. A system, comprising: one or more memory that stores instructions; and one or more processors that executes the instructions, wherein, when executed by the one or more processors, the instructions cause the system to: interrogate a first database; extract, based on interrogating the first database, a first schema of the first database; and generate, based on the first schema of the first database, a first system prompt structure to be accessed by a large language model to generate a database query for input to search the first database.

2. The system of claim 1, wherein the first schema of the first database includes at least one of table names, column attributes, relationships between tables, or relationships between columns.

3. The system of claim 1, wherein, when executed by the one or more processors, the instructions cause the system to: interrogate a second database different from the first database; extract, based on interrogating the second database, a second schema of the second database; and generate, based on the second schema of the second database, a second system prompt structure to be accessed by the large language model to generate a database query for input to search the second database.

4. The system of claim 3, wherein the first system prompt structure is customized for the first database and wherein the second system prompt structure is customized for the second database.

5. The system of claim 1, wherein, when executed by the one or more processors, the instructions cause the system to:29re-interrogate the first database; re-extract, based on re-interrogating the first database, an updated first schema of the first database; and re-generate, based on the updated first schema of the first database, an updated first system prompt structure to be accessed by the large language model to generate a database query for input to search the first database.

6. A system, comprising: one or more memory that stores instructions, and one or more processors that executes the instructions, wherein, when executed by the one or more processors, the instructions cause the system to: receive input via a user interface; select a first system prompt structure using a large language model that receives the input from the user interface; generate, by the large language model based on the first system prompt structure and the input, a query to search a first database; send the query to the first database to search the first database; receive results of the search of the first database; and generate output of the results of the search of the first database.

7. The system of claim 6, wherein, when executed by the one or more processors, the instructions cause the system to: convert the input into the query based on the first system prompt structure.

8. The system of claim 7, wherein, when executed by the one or more processors, the instructions cause the system further to: convert voice input as the input into text input and display the text input via a user interface for confirmation of the text input.

309. The system of claim 7, wherein the large language model is configured to convert the input into the query for the first database in accordance with formatting required for the first database.

10. The system of claim 6, wherein, when executed by the one or more processors, the instructions cause the system further to: generate a visualization for data stored in the first database for the output of the results of the search of the first database.

11. The system of claim 6, wherein the first system prompt structure is based on a mapping of the first database.

12. The system of claim 6, wherein, when executed by the one or more processors, the instructions cause the system further to: create a session history for a session in which the input is received, wherein the query is generated further based on the session history.

13. The system of claim 6, wherein the instructions comprise an access program to access a remote instantiation of the large language model, and wherein the access program is configured to provide to the remote instantiation of the large language model the input, the first system prompt structure, and a session history for a session in which the input is received.

14. A computer- implemented method of querying clinical data repositories, the method comprising: receiving a natural language query as input via a user interface; selecting a first system prompt structure using a large language model that receives the input from the user interface; generating, by the large language model based on the first system prompt structure and the input, a query to search a first database; sending the query to the first database to search the first database; receiving results of the search of the first database; andgenerating output of the results of the search of the first database, and receiving a response with results for the query from a language model.

15. The computer-implemented method of claim 14, further comprising: augmenting the first system prompt structure with instructions for rendering aspects of the results in a graphical chart; wherein a portion of the received response is processed by a graph generating module to render contextual data visualizations.

16. The computer-implemented method of claim 14, wherein includes an interpretation of a schema of each of the one or more databases.

17. The computer-implemented method of claim 16, wherein the first system prompt structure is based on a mapping of the first database.

18. The computer-implemented method of claim 17, wherein the mapping is generated by an automated framework comprising a discovery module, interpretation module, and schema learning module.

Images